test_floatingip, test_keypair tests, test_create_delete_user are consistently failing

[Luigi Toscano]

The test above are failing with different reasons. Please note that they were working until yesterday (2015-06-22).

test_floatingip:

2015-06-23 12:30:20.815 | 2015-06-23 12:30:20.797 | Traceback (most recent call last):
2015-06-23 12:30:20.817 | 2015-06-23 12:30:20.799 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/tests/test_floatingip.py", line 25, in test_floatingip
2015-06-23 12:30:20.819 | 2015-06-23 12:30:20.801 | floating_ip = floatingip_page.allocate_floatingip()
2015-06-23 12:30:20.821 | 2015-06-23 12:30:20.803 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/pages/project/compute/access_and_security/floatingipspage.py", line 62, in allocate_floatingip
2015-06-23 12:30:20.822 | 2015-06-23 12:30:20.804 | self.floatingips_table.allocate_ip_to_project.click()
2015-06-23 12:30:20.824 | 2015-06-23 12:30:20.806 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/regions/baseregion.py", line 59, in __getattr__
2015-06-23 12:30:20.826 | 2015-06-23 12:30:20.808 | return self._dynamic_properties[name]()
2015-06-23 12:30:20.827 | 2015-06-23 12:30:20.810 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/regions/baseregion.py", line 78, in __call__
2015-06-23 12:30:20.829 | 2015-06-23 12:30:20.811 | return result if self.index is None else result[self.index]
2015-06-23 12:30:20.830 | 2015-06-23 12:30:20.813 | IndexError: list index out of range

Most likely the page was changed.

test_keypair:
2015-06-23 12:30:22.598 | 2015-06-23 12:30:22.581 | Traceback (most recent call last):
2015-06-23 12:30:22.600 | 2015-06-23 12:30:22.582 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/tests/test_keypair.py", line 28, in test_keypair
2015-06-23 12:30:22.601 | 2015-06-23 12:30:22.583 | self.assertTrue(keypair_page.is_keypair_present(self.KEYPAIR_NAME))
2015-06-23 12:30:22.603 | 2015-06-23 12:30:22.585 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue
2015-06-23 12:30:22.604 | 2015-06-23 12:30:22.586 | raise self.failureException(msg)
2015-06-23 12:30:22.606 | 2015-06-23 12:30:22.588 | AssertionError: False is not true

Either changed page, or an active polling of the "ready" state is needed.

See for example http://logs.openstack.org/72/193072/1/gate/gate-horizon-dsvm-integration/dd294c6/console.html#_2015-06-23_12_30_20_817

The bug is to track the skip patch which will follow and the future investigations of the real issues, that needs to be fixed.

[Matthias Runge (mrunge)]

Related patch: https://review.openstack.org/194686

[Luigi Toscano (ltoscano)]

Also test_create_delete_user started to fail consistently.
2015-06-23 17:25:58.307 | 2015-06-23 17:25:58.306 | Traceback (most recent call last):
2015-06-23 17:25:58.309 | 2015-06-23 17:25:58.308 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/tests/test_user_create_delete.py", line 25, in test_create_delete_user
2015-06-23 17:25:58.311 | 2015-06-23 17:25:58.310 | project='admin', role='admin')
2015-06-23 17:25:58.313 | 2015-06-23 17:25:58.312 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/pages/identity/userspage.py", line 74, in create_user
2015-06-23 17:25:58.315 | 2015-06-23 17:25:58.314 | self._wait_till_spinner_disappears()
2015-06-23 17:25:58.317 | 2015-06-23 17:25:58.316 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/pages/basepage.py", line 70, in _wait_till_spinner_disappears
2015-06-23 17:25:58.319 | 2015-06-23 17:25:58.318 | self._wait_till_element_disappears(spinner)
2015-06-23 17:25:58.321 | 2015-06-23 17:25:58.320 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/basewebobject.py", line 96, in _wait_till_element_disappears
2015-06-23 17:25:58.323 | 2015-06-23 17:25:58.322 | timeout)
2015-06-23 17:25:58.325 | 2015-06-23 17:25:58.324 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/basewebobject.py", line 84, in _wait_until
2015-06-23 17:25:58.327 | 2015-06-23 17:25:58.326 | predicate)
2015-06-23 17:25:58.329 | 2015-06-23 17:25:58.328 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 78, in until
2015-06-23 17:25:58.331 | 2015-06-23 17:25:58.330 | raise TimeoutException(message, screen, stacktrace)
2015-06-23 17:25:58.333 | 2015-06-23 17:25:58.332 | selenium.common.exceptions.TimeoutException: Message:

Updating the bug and the patch.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/kilo
Review: https://review.openstack.org/195657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/195657
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=fcff73c4a050346abb718107d5389eaac2f344f7
Submitter: Jenkins
Branch: stable/kilo

commit fcff73c4a050346abb718107d5389eaac2f344f7
Author: Luigi Toscano <email address hidden>
Date: Thu Jun 25 18:08:26 2015 +0200

    Skip test_{floatingip,image_register_unregister,create_delete_user} for now

    The tests are failing for different reasons (see the referenced bug),
    so skip them for now.

    The patch has been changed:
    - test_floatingip does not exist in stable/kilo
    - test_image_register_unregister needs to be disabled as well for now;
      it will be fixed with another backported patch, but this patch should
      first fix the test results.

    Change-Id: I9622ed1ba7444089fa1ee6d0c915b8d31c752af6
    Related-Bug: #1467950
    (cherry picked from commit b0c62463c74c9cd984ff123a7c2b303f4b231563)

[Alan Pevec (apevec)]

Only test skipping was merged to stable/kilo. Thanks Tosky for noticing this!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/214238

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/214670

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/215101

[OpenStack Infra (hudson-openstack)]

Change abandoned by svasheka (<email address hidden>) on branch: master
Review: https://review.openstack.org/215106
Reason: made in scope of a different patch

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/214670
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=964342fa384d217b611da67cfeefdc656fd8b32b
Submitter: Jenkins
Branch: master

commit 964342fa384d217b611da67cfeefdc656fd8b32b
Author: Timur Sufiev <email address hidden>
Date: Thu Aug 20 14:36:37 2015 +0300

    Adapt integration tests to recent devstack and page layout changes

    This patch:
    * re-enables 3 previously disabled integration tests for users
      crate/delete, keypairs and floating ips
    * updates selectors for sidebar menu and user context menu
    * adds some selenium specific classes to different sidebar menus in
      order integration tests could check open/closed state of
      dashboard/panel group sections (horizon.selenium.js)
    * updates default path to horizon (which is now /dashboard at
      devstack)
    * temporarily disables Sahara integration tests

    Related-Bug: #1467950
    Implements blueprint: integration-tests-hardening
    Change-Id: I688eb7d72d5504cb55b977d85435019e3f69323f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/215101
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=636376468d06bdd9cb3dbdd133b375c3c1eaadc1
Submitter: Jenkins
Branch: master

commit 636376468d06bdd9cb3dbdd133b375c3c1eaadc1
Author: Timur Sufiev <email address hidden>
Date: Thu Aug 20 14:41:49 2015 +0300

    Reduce the chance of Selenium timeouts in integration tests

    Avoid situations when we trying to take something out of a destroyed
    box (contents of an element that is no longer there). Plus other
    improvements to a couple of element getter functions/predicates that
    might had been causing timeouts.

    Related-Bug: #1467950
    Implements blueprint: integration-tests-hardening

    Change-Id: Ia17cb2134eaa5dd14a04384d1b703a8a648f952d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/214238
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=59717143bcc3c5823ca262b2cc3239a8ad6608a4
Submitter: Jenkins
Branch: master

commit 59717143bcc3c5823ca262b2cc3239a8ad6608a4
Author: Timur Sufiev <email address hidden>
Date: Wed Aug 19 18:18:31 2015 +0300

    Integration tests no longer use form's fields ordering for matching

    Relying on suggestion that some predefined list of field names in test
    will always be the same in actual forms rendered by Horizon is
    perilous. It leads to a test failure every time the order of fields in
    some form being tested changes or a new field appears at the beginning
    of the form. Tests which are using 'name' attribute for matching
    predefined fields to the actual form fields are much more durable
    since field names change less frequently.

    Closes-Bug: #1467950
    Related-Bug: #1456294
    Related-Bug: #1447432
    Implements blueprint: integration-tests-hardening

    Change-Id: Ia39c003a628f5abbb73ff824329b2a31a0b6e179