test_image_create_delete race fails in gate-horizon-dsvm-integration with "AttributeError: 'NoneType' object has no attribute 'cells'"

[Matt Riedemann]

http://logs.openstack.org/47/275747/3/gate/gate-horizon-dsvm-integration/b83e62a/console.html#_2016-02-09_01_25_30_551

2016-02-09 01:25:30.551 | 2016-02-09 01:25:30.522 | Traceback (most recent call last):
2016-02-09 01:25:30.555 | 2016-02-09 01:25:30.525 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/tests/test_image_create_delete.py", line 32, in test_image_create_delete
2016-02-09 01:25:30.560 | 2016-02-09 01:25:30.531 | self.assertTrue(images_page.is_image_active(self.IMAGE_NAME))
2016-02-09 01:25:30.562 | 2016-02-09 01:25:30.533 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/pages/project/compute/imagespage.py", line 103, in is_image_active
2016-02-09 01:25:30.569 | 2016-02-09 01:25:30.540 | self._wait_till_text_present_in_element(cell_getter, 'Active')
2016-02-09 01:25:30.572 | 2016-02-09 01:25:30.543 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/basewebobject.py", line 107, in _wait_till_text_present_in_element
2016-02-09 01:25:30.575 | 2016-02-09 01:25:30.546 | self._wait_until(predicate, timeout)
2016-02-09 01:25:30.577 | 2016-02-09 01:25:30.548 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/basewebobject.py", line 91, in _wait_until
2016-02-09 01:25:30.580 | 2016-02-09 01:25:30.551 | predicate)
2016-02-09 01:25:30.582 | 2016-02-09 01:25:30.553 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 71, in until
2016-02-09 01:25:30.585 | 2016-02-09 01:25:30.556 | value = method(self._driver)
2016-02-09 01:25:30.588 | 2016-02-09 01:25:30.559 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/basewebobject.py", line 104, in predicate
2016-02-09 01:25:30.592 | 2016-02-09 01:25:30.562 | elt = element() if hasattr(element, '__call__') else element
2016-02-09 01:25:30.599 | 2016-02-09 01:25:30.569 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/pages/project/compute/imagespage.py", line 101, in cell_getter
2016-02-09 01:25:30.602 | 2016-02-09 01:25:30.573 | return row.cells[self.IMAGES_TABLE_STATUS_COLUMN]
2016-02-09 01:25:30.604 | 2016-02-09 01:25:30.575 | AttributeError: 'NoneType' object has no attribute 'cells'

Looks like this started around 2/8:

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22AttributeError%3A%20'NoneType'%20object%20has%20no%20attribute%20'cells'%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20build_name%3A%5C%22gate-horizon-dsvm-integration%5C%22&from=7d

[Matt Riedemann (mriedem)]

There is a 404 in the glance-api logs:

http://logs.openstack.org/47/275747/3/gate/gate-horizon-dsvm-integration/b83e62a/logs/screen-g-api.txt.gz?level=TRACE

2016-02-09 01:09:02.136 17904 ERROR swiftclient [req-a32254c0-a71e-4bc4-a61d-e5e432cbc8fc 31d36a78e9f7457bbafc73a009695d30 df4780c1cf25498fbf233b5ab931d948 - - -] Container HEAD failed: http://127.0.0.1:8080/v1/AUTH_22c1cc47a75a4bb9887885274acf02a5/glance 404 Not Found
2016-02-09 01:09:02.136 17904 ERROR swiftclient Traceback (most recent call last):
2016-02-09 01:09:02.136 17904 ERROR swiftclient File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 1425, in _retry
2016-02-09 01:09:02.136 17904 ERROR swiftclient service_token=self.service_token, **kwargs)
2016-02-09 01:09:02.136 17904 ERROR swiftclient File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py", line 792, in head_container
2016-02-09 01:09:02.136 17904 ERROR swiftclient http_response_content=body)
2016-02-09 01:09:02.136 17904 ERROR swiftclient ClientException: Container HEAD failed: http://127.0.0.1:8080/v1/AUTH_22c1cc47a75a4bb9887885274acf02a5/glance 404 Not Found
2016-02-09 01:09:02.136 17904 ERROR swiftclient

That's at a different time from the AttributeError failure though, so probably unrelated.

[Doug Fish (drfish)]

We had some discussion about this in IRC
http://eavesdrop.openstack.org/irclogs/%23openstack-dev/latest.log.html#t2016-02-11T15:18:08

my initial impression is that this is an intermittent timeout where glance takes longer than Horizon expects to create a new image.

Given that it's quite infrequent, and seems to be a test issue rather than a product issue, marking as "low" priority.

[Gary W. Smith (gary-w-smith)]

Integration tests have been abandoned all together by Horizon. This no longer applies.