Got more than one item. - unable to add/modify machines in the UI

[mahmoh]

Added a machine manually with the wrong architecture, tried to commission it, realized my mistake, tried to change it from amd64 to arm64/xgene-uboot-mustang (from daily) and it switches back but dumps the trace below:

2015-11-09 17:22:25 [-] 127.0.0.1 - - [10/Nov/2015:01:22:24 +0000] "GET /MAAS/rpc/ HTTP/1.1" 200 400 "-" "provisioningserver.rpc.clusterservice.ClusterClientService"
2015-11-09 17:22:36 [-] Error on request (64) node.update: Got more than one item.
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 783, in __bootstrap
     self.__bootstrap_inner()
   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 763, in run
     self.__target(*self.__args, **self.__kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 404, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 300, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/twisted.py", line 154, in wrapper
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 399, in call_within_transaction
     return func_within_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/websockets/base.py", line 332, in execute
     return m(params)
   File "/usr/lib/python2.7/dist-packages/maasserver/websockets/handlers/node.py", line 463, in update
     data = super(NodeHandler, self).update(params)
   File "/usr/lib/python2.7/dist-packages/maasserver/websockets/base.py", line 402, in update
     obj = form.save()
   File "/usr/lib/python2.7/dist-packages/maasserver/forms.py", line 1086, in save
     node.add_mac_address(mac)
   File "/usr/lib/python2.7/dist-packages/maasserver/models/node.py", line 845, in add_mac_address
     lease = get_one(leases)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 96, in get_one
     raise get_exception_class(items)("Got more than one item.")
 maasserver.models.dhcplease.MultipleObjectsReturned: Got more than one item.

ubuntu@maas-apm-atc:/var/log/maas$ dpkg -l | grep maas
ii maas 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server all-in-one metapackage
ii maas-cli 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS command line API tool
ii maas-cluster-controller 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server cluster controller
ii maas-common 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server common files
ii maas-dhcp 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS DHCP server
ii maas-dns 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS DNS server
ii maas-proxy 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS Caching Proxy
ii maas-region-controller 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server complete region controller
ii maas-region-controller-min 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS Server minimum region controller
ii python-django-maas 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server Django web framework
ii python-maas-client 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS python API client
ii python-maas-provisioningserver 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server provisioning libraries

[mahmoh (mahmoh)]

So I'm seeing this now when I try and add a new machine, this is what I think is happening:

1) ARM64 node DHCP and PXE
2) MAC is logged
3) PXE responds with AMD64 kernel/initrd by default and fails to boot (of course)
4) I try and add/modify this machine in the UI and the get the "Got more than one item." traceback (but that's my best guess)

[Andres Rodriguez (andreserl)]

Hi Massimo,

Can you please let us know what MAAS version are you using?

Thanks!

[mahmoh (mahmoh)]

Hi Andres,

Per above:

ii maas 1.8.3+bzr4053-0ubuntu1~trusty1 all MAAS server all-in-one metapackage

Also, it appears the problem goes away when the lease expires possibly, since after an overnight I was able to add the mac back without the error.

Thank you.

[Launchpad Janitor (janitor)]

[Expired for MAAS because there has been no activity for 60 days.]

[Chad Smith (chad.smith)]

Just hit this again on cisco hardware, first time I have seen it on this hardware

This occurred after successfully creating a first node C220-M4-2, out setup script fell over issuing a nearly identical command for C220-M4-1.

https://pastebin.canonical.com/148773/

dpkg -l | grep maas
ii maas 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server all-in-one metapackage
ii maas-cli 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS command line API tool
ii maas-cluster-controller 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server cluster controller
ii maas-common 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server common files
ii maas-dhcp 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS DHCP server
ii maas-dns 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS DNS server
ii maas-proxy 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS Caching Proxy
ii maas-region-controller 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server complete region controller
ii maas-region-controller-min 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS Server minimum region controller
ii python-django-maas 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server Django web framework
ii python-maas-client 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS python API client
ii python-maas-provisioningserver 1.8.2+bzr4041-0ubuntu1~trusty1 all MAAS server provisioning libraries

[Chad Smith (chad.smith)]

This failure is intermittent at best. I have never seen it before on this hardware. Our automated script adds up to 8 nodes to maas via nearly identical calls to:

maas admin nodes new [k=v pairs].

The first call for C220-M4-2 succeeded, the second call to the api for C220-M4-1 returns an error with the message "Got more than one item".

The logs are below.

[Chad Smith (chad.smith)]

Traceback from /var/log/maas/regiond.log for the maas nodes new failure

2016-01-29 21:14:52 [-] 127.0.0.1 - - [29/Jan/2016:21:14:51 +0000] "POST /MAAS/api/1.0/nodes/?op=new HTTP/1.1" 200 449 "-" "Python-httplib2/0.8 (gzip)"
2016-01-29 21:14:52 [maasserver] ERROR: ################################ Exception: Got more than one item. ################################
2016-01-29 21:14:52 [maasserver] ERROR: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api/support.py", line 52, in __call__
    response = upcall(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api/support.py", line 200, in dispatch
    return function(self, request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/maasserver/api/nodes.py", line 919, in new
    node = create_node(request)
  File "/usr/lib/python2.7/dist-packages/maasserver/api/nodes.py", line 783, in create_node
    node = form.save()
  File "/usr/lib/python2.7/dist-packages/maasserver/forms.py", line 1086, in save
    node.add_mac_address(mac)
  File "/usr/lib/python2.7/dist-packages/maasserver/models/node.py", line 845, in add_mac_address
    lease = get_one(leases)
  File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 96, in get_one
    raise get_exception_class(items)("Got more than one item.")
MultipleObjectsReturned: Got more than one item.

[Chad Smith (chad.smith)]

MAAS Logs for failure during node creation via api maas admin nodes new

[Andres Rodriguez (andreserl)]

MAAS 1.8 has now been deprecated and as such we are no longer fixing bugs. If you believe this bug stills exists in any newer versions of MAAS, please re-open and provide additional information as to what release is affected.

Thanks!