test_baremetal_server_ops times out waiting to terminate instance

[Matt Riedemann]

Seen here:

http://logs.openstack.org/43/257243/3/gate/gate-tempest-dsvm-ironic-pxe_ipa/4e1437a/console.html#_2015-12-22_10_40_17_635

2015-12-22 10:40:17.635 | Captured traceback:
2015-12-22 10:40:17.635 | ~~~~~~~~~~~~~~~~~~~
2015-12-22 10:40:17.635 | Traceback (most recent call last):
2015-12-22 10:40:17.635 | File "tempest/test.py", line 113, in wrapper
2015-12-22 10:40:17.635 | return f(self, *func_args, **func_kwargs)
2015-12-22 10:40:17.635 | File "tempest/scenario/test_baremetal_basic_ops.py", line 145, in test_baremetal_server_ops
2015-12-22 10:40:17.636 | self.terminate_instance()
2015-12-22 10:40:17.636 | File "tempest/scenario/manager.py", line 1357, in terminate_instance
2015-12-22 10:40:17.636 | timeout=CONF.baremetal.unprovision_timeout)
2015-12-22 10:40:17.636 | File "tempest/scenario/manager.py", line 1273, in wait_provisioning_state
2015-12-22 10:40:17.636 | target_states=state, timeout=timeout)
2015-12-22 10:40:17.636 | File "tempest/scenario/manager.py", line 1265, in _node_state_timeout
2015-12-22 10:40:17.636 | check_state, timeout, interval):
2015-12-22 10:40:17.637 | File "tempest/test.py", line 851, in call_until_true
2015-12-22 10:40:17.637 | time.sleep(sleep_for)
2015-12-22 10:40:17.637 | File "/opt/stack/new/tempest/.tox/all/local/lib/python2.7/site-packages/fixtures/_fixtures/timeout.py", line 52, in signal_handler
2015-12-22 10:40:17.637 | raise TimeoutException()
2015-12-22 10:40:17.637 | fixtures._fixtures.timeout.TimeoutException

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22File%20%5C%5C%5C%22tempest/scenario/manager.py%5C%5C%5C%22%5C%22%20AND%20message:%5C%22in%20_node_state_timeout%5C%22%20AND%20tags:%5C%22console%5C%22%20AND%20voting:%5C%221%5C%22

162 hits in 7 days, check and gate, all branches, mostly the slow OVH nodes. There are no errors in the ironic logs from what I can see.

[Jim Rollenhagen (jim-rollenhagen)]

Related to https://bugs.launchpad.net/openstack-ci/+bug/1393099

[Dmitry Tantsur (divius)]

can we merge this into https://bugs.launchpad.net/openstack-ci/+bug/1393099 then? I'd also add IPA to one of these

[Matt Riedemann (mriedem)]

The failure in the tests is slightly different I think. In this one the test is timing out waiting for the node instance to be terminated, in the other I think it's waiting to get active. The underlying issue might be the same though, I'm not sure.

[Matt Riedemann (mriedem)]

We haven't seen this in months from what I can tell, so marking it invalid.