Quick bug, need to head out.
Error [1] seen during cleaning while adding traits to the nodes in devstack.
Looks to be an issue with attempting JSON serialisation of trait RPC objects.
Traceback:
Feb 16 16:36:43.889351 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: /usr/local/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py:192: UserWarning: Cannot convert Trait(created_at=2018-02-16T16:34:14Z,node_id='1',trait='CUSTOM_GOLD',updated_at=None) to primitive, will raise ValueError instead of warning in version 3.0 Feb 16 16:36:43.889664 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: "instead of warning in version 3.0" % (value,)) Feb 16 16:36:43.894193 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor [-] Asynchronous exception: Node failed to start the first cleaning step. Exception: Circular reference detected for node 6a420cf7-6b98-4fc2-a70b-f78deb64e4a4: ValueError: Circular reference detected Feb 16 16:36:43.894467 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last): Feb 16 16:36:43.894717 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/opt/stack/new/ironic/ironic/drivers/modules/agent_base_vendor.py", line 328, in heartbeat Feb 16 16:36:43.895005 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor self.refresh_clean_steps(task) Feb 16 16:36:43.895255 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/local/lib/python2.7/dist-packages/ironic_lib/metrics.py", line 61, in wrapped Feb 16 16:36:43.895497 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor result = f(*args, **kwargs) Feb 16 16:36:43.895798 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/opt/stack/new/ironic/ironic/drivers/modules/agent_base_vendor.py", line 405, in refresh_clean_steps Feb 16 16:36:43.896022 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor agent_result = self._client.get_clean_steps(node, task.ports).get( Feb 16 16:36:43.896256 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/local/lib/python2.7/dist-packages/ironic_lib/metrics.py", line 61, in wrapped Feb 16 16:36:43.896464 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor result = f(*args, **kwargs) Feb 16 16:36:43.896672 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/opt/stack/new/ironic/ironic/drivers/modules/agent_client.py", line 179, in get_clean_steps Feb 16 16:36:43.896886 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor wait=True) Feb 16 16:36:43.897122 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/local/lib/python2.7/dist-packages/ironic_lib/metrics.py", line 61, in wrapped Feb 16 16:36:43.897330 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor result = f(*args, **kwargs) Feb 16 16:36:43.897550 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/opt/stack/new/ironic/ironic/drivers/modules/agent_client.py", line 58, in _command Feb 16 16:36:43.897766 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor body = self._get_command_body(method, params) Feb 16 16:36:43.897985 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/opt/stack/new/ironic/ironic/drivers/modules/agent_client.py", line 52, in _get_command_body Feb 16 16:36:43.898197 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor 'params': params, Feb 16 16:36:43.898406 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/local/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py", line 213, in dumps Feb 16 16:36:43.898619 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor return json.dumps(obj, default=default, **kwargs) Feb 16 16:36:43.898829 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/json/__init__.py", line 251, in dumps Feb 16 16:36:43.899033 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor sort_keys=sort_keys, **kw).encode(obj) Feb 16 16:36:43.899244 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/json/encoder.py", line 207, in encode Feb 16 16:36:43.899457 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor chunks = self.iterencode(o, _one_shot=True) Feb 16 16:36:43.899675 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode Feb 16 16:36:43.899890 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor return _iterencode(o, 0) Feb 16 16:36:43.900099 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor ValueError: Circular reference detected Feb 16 16:36:43.900309 ubuntu-xenial-rax-iad-0002580271 ironic-conductor[20042]: ERROR ironic.drivers.modules.agent_base_vendor
[1] http://logs.openstack.org/70/545370/1/check/ironic-tempest-dsvm-ipa-partition-uefi-pxe_ipmitool-tinyipa/d5486a9/logs/screen-ir-cond.txt.gz#_Feb_16_16_36_43_889351
Quick bug, need to head out.
Error [1] seen during cleaning while adding traits to the nodes in devstack.
Looks to be an issue with attempting JSON serialisation of trait RPC objects.
Traceback:
Feb 16 16:36:43.889351 ubuntu- xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: /usr/local/ lib/python2. 7/dist- packages/ oslo_serializat ion/jsonutils. py:192: UserWarning: Cannot convert Trait(created_ at=2018- 02-16T16: 34:14Z, node_id= '1',trait= 'CUSTOM_ GOLD',updated_ at=None) to primitive, will raise ValueError instead of warning in version 3.0 xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: "instead of warning in version 3.0" % (value,)) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor [-] Asynchronous exception: Node failed to start the first cleaning step. Exception: Circular reference detected for node 6a420cf7- 6b98-4fc2- a70b-f78deb64e4 a4: ValueError: Circular reference detected xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor Traceback (most recent call last): xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/opt/stack/ new/ironic/ ironic/ drivers/ modules/ agent_base_ vendor. py", line 328, in heartbeat xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor self.refresh_ clean_steps( task) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/local/ lib/python2. 7/dist- packages/ ironic_ lib/metrics. py", line 61, in wrapped xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor result = f(*args, **kwargs) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/opt/stack/ new/ironic/ ironic/ drivers/ modules/ agent_base_ vendor. py", line 405, in refresh_clean_steps xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor agent_result = self._client. get_clean_ steps(node, task.ports).get( xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/local/ lib/python2. 7/dist- packages/ ironic_ lib/metrics. py", line 61, in wrapped xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor result = f(*args, **kwargs) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/opt/stack/ new/ironic/ ironic/ drivers/ modules/ agent_client. py", line 179, in get_clean_steps xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor wait=True) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/local/ lib/python2. 7/dist- packages/ ironic_ lib/metrics. py", line 61, in wrapped xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor result = f(*args, **kwargs) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/opt/stack/ new/ironic/ ironic/ drivers/ modules/ agent_client. py", line 58, in _command xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor body = self._get_ command_ body(method, params) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/opt/stack/ new/ironic/ ironic/ drivers/ modules/ agent_client. py", line 52, in _get_command_body xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor 'params': params, xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/local/ lib/python2. 7/dist- packages/ oslo_serializat ion/jsonutils. py", line 213, in dumps xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor return json.dumps(obj, default=default, **kwargs) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/lib/ python2. 7/json/ __init_ _.py", line 251, in dumps xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor sort_keys= sort_keys, **kw).encode(obj) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/lib/ python2. 7/json/ encoder. py", line 207, in encode xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor chunks = self.iterencode(o, _one_shot=True) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor File "/usr/lib/ python2. 7/json/ encoder. py", line 270, in iterencode xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor return _iterencode(o, 0) xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor ValueError: Circular reference detected xenial- rax-iad- 0002580271 ironic- conductor[ 20042]: ERROR ironic. drivers. modules. agent_base_ vendor
Feb 16 16:36:43.889664 ubuntu-
Feb 16 16:36:43.894193 ubuntu-
Feb 16 16:36:43.894467 ubuntu-
Feb 16 16:36:43.894717 ubuntu-
Feb 16 16:36:43.895005 ubuntu-
Feb 16 16:36:43.895255 ubuntu-
Feb 16 16:36:43.895497 ubuntu-
Feb 16 16:36:43.895798 ubuntu-
Feb 16 16:36:43.896022 ubuntu-
Feb 16 16:36:43.896256 ubuntu-
Feb 16 16:36:43.896464 ubuntu-
Feb 16 16:36:43.896672 ubuntu-
Feb 16 16:36:43.896886 ubuntu-
Feb 16 16:36:43.897122 ubuntu-
Feb 16 16:36:43.897330 ubuntu-
Feb 16 16:36:43.897550 ubuntu-
Feb 16 16:36:43.897766 ubuntu-
Feb 16 16:36:43.897985 ubuntu-
Feb 16 16:36:43.898197 ubuntu-
Feb 16 16:36:43.898406 ubuntu-
Feb 16 16:36:43.898619 ubuntu-
Feb 16 16:36:43.898829 ubuntu-
Feb 16 16:36:43.899033 ubuntu-
Feb 16 16:36:43.899244 ubuntu-
Feb 16 16:36:43.899457 ubuntu-
Feb 16 16:36:43.899675 ubuntu-
Feb 16 16:36:43.899890 ubuntu-
Feb 16 16:36:43.900099 ubuntu-
Feb 16 16:36:43.900309 ubuntu-
[1] http:// logs.openstack. org/70/ 545370/ 1/check/ ironic- tempest- dsvm-ipa- partition- uefi-pxe_ ipmitool- tinyipa/ d5486a9/ logs/screen- ir-cond. txt.gz# _Feb_16_ 16_36_43_ 889351