Integration tests sometime fail with "WebDriverException: Message: The browser appears to have exited before we could connect."

Bug #1412469 reported by Julie Pichon
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Dashboard (Horizon)
Fix Released
Medium
Tomáš Nováčik

Bug Description

For example on test here ( http://logs.openstack.org/04/148204/4/check/gate-horizon-dsvm-integration/5b55890/console.html )

2015-01-19 10:04:02.134 | 2015-01-19 10:04:02.118 | ERROR: openstack_dashboard.test.integration_tests.tests.test_flavors.TestFlavors.test_flavor_create
2015-01-19 10:04:02.136 | 2015-01-19 10:04:02.119 | ----------------------------------------------------------------------
2015-01-19 10:04:02.168 | 2015-01-19 10:04:02.121 | _StringException: Traceback (most recent call last):
2015-01-19 10:04:02.168 | 2015-01-19 10:04:02.122 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/helpers.py", line 72, in setUp
2015-01-19 10:04:02.169 | 2015-01-19 10:04:02.124 | super(AdminTestCase, self).setUp()
2015-01-19 10:04:02.169 | 2015-01-19 10:04:02.125 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/helpers.py", line 33, in setUp
2015-01-19 10:04:02.169 | 2015-01-19 10:04:02.127 | self.driver = webdriver.WebDriverWrapper()
2015-01-19 10:04:02.169 | 2015-01-19 10:04:02.128 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
2015-01-19 10:04:02.170 | 2015-01-19 10:04:02.130 | self.binary, timeout),
2015-01-19 10:04:02.170 | 2015-01-19 10:04:02.131 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
2015-01-19 10:04:02.170 | 2015-01-19 10:04:02.132 | self.binary.launch_browser(self.profile)
2015-01-19 10:04:02.171 | 2015-01-19 10:04:02.134 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 66, in launch_browser
2015-01-19 10:04:02.171 | 2015-01-19 10:04:02.135 | self._wait_until_connectable()
2015-01-19 10:04:02.171 | 2015-01-19 10:04:02.137 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 100, in _wait_until_connectable
2015-01-19 10:04:02.171 | 2015-01-19 10:04:02.138 | raise WebDriverException("The browser appears to have exited "
2015-01-19 10:04:02.172 | 2015-01-19 10:04:02.140 | WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.

3 tests with that error in http://logs.openstack.org/47/147047/5/gate/gate-horizon-dsvm-integration/2e044be/console.html :

2015-01-19 14:09:49.343 | 2015-01-19 14:09:49.309 | ======================================================================
2015-01-19 14:09:49.343 | 2015-01-19 14:09:49.310 | ERROR: openstack_dashboard.test.integration_tests.tests.test_keypair.TestKeypair.test_keypair
2015-01-19 14:09:49.343 | 2015-01-19 14:09:49.311 | ----------------------------------------------------------------------
2015-01-19 14:09:49.343 | 2015-01-19 14:09:49.313 | _StringException: Traceback (most recent call last):
2015-01-19 14:09:49.343 | 2015-01-19 14:09:49.314 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/helpers.py", line 59, in setUp
2015-01-19 14:09:49.344 | 2015-01-19 14:09:49.315 | super(TestCase, self).setUp()
2015-01-19 14:09:49.344 | 2015-01-19 14:09:49.316 | File "/opt/stack/new/horizon/openstack_dashboard/test/integration_tests/helpers.py", line 33, in setUp
2015-01-19 14:09:49.344 | 2015-01-19 14:09:49.318 | self.driver = webdriver.WebDriverWrapper()
2015-01-19 14:09:49.344 | 2015-01-19 14:09:49.319 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 59, in __init__
2015-01-19 14:09:49.344 | 2015-01-19 14:09:49.320 | self.binary, timeout),
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.321 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/extension_connection.py", line 47, in __init__
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.323 | self.binary.launch_browser(self.profile)
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.324 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 66, in launch_browser
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.325 | self._wait_until_connectable()
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.327 | File "/opt/stack/new/horizon/.tox/py27integration/local/lib/python2.7/site-packages/selenium/webdriver/firefox/firefox_binary.py", line 100, in _wait_until_connectable
2015-01-19 14:09:49.345 | 2015-01-19 14:09:49.329 | raise WebDriverException("The browser appears to have exited "
2015-01-19 14:09:49.346 | 2015-01-19 14:09:49.330 | WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.

Maybe we should also set that log_file attribute and add it to the list of collected logs to help debug gate job failures.

Revision history for this message
Julie Pichon (jpichon) wrote :
Revision history for this message
Tomáš Nováčik (tnovacik-6) wrote :

I believe this is caused by xvfb memory bug http://blog.jeffterrace.com/2012/07/xvfb-memory-leak-workaround.html. Supplying -noreset parameter should fix this issue.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to horizon (master)

Fix proposed to branch: master
Review: https://review.openstack.org/148300

Changed in horizon:
assignee: nobody → Tomáš Nováčik (tnovacik-6)
status: New → In Progress
Revision history for this message
Julie Pichon (jpichon) wrote :

Thanks for the link and taking this one on Tomas! Seems worth doing if there are no other side-effects.

Revision history for this message
Lin Hua Cheng (lin-hua-cheng) wrote :

Interesting, good find! Thanks!

Changed in horizon:
milestone: none → kilo-2
importance: Undecided → Low
Revision history for this message
Julie Pichon (jpichon) wrote :

Raising to medium since jobs failing because of this cause patches to fail the gate, and there's been 52 more hits over the last 24h.

Changed in horizon:
importance: Low → Medium
Revision history for this message
Julie Pichon (jpichon) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to horizon (master)

Reviewed: https://review.openstack.org/148300
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=2ebb804c05b776b31e65f814df5226287cd939b2
Submitter: Jenkins
Branch: master

commit 2ebb804c05b776b31e65f814df5226287cd939b2
Author: Tomáš Nováčik <email address hidden>
Date: Mon Jan 19 16:51:50 2015 +0100

    Adding -ac and -noreset arguments to Xvfb

    There is outstanding memory leak issue with Xvfb, that should be fixed
    by -noreset argument.

    -ac disables X access control - which i believe is not necessary for
                                    testing

    Partially implements blueprint: selenium-integration-testing
    Closes-Bug: #1412469

    Change-Id: I5f603266f891728bb8e83331f5a4e57799f07f79

Changed in horizon:
status: In Progress → Fix Committed
Revision history for this message
Julie Pichon (jpichon) wrote :

0 hits so far since this merged, looks like that was the correct fix! Well debugged Tomas, thanks :)

Thierry Carrez (ttx)
Changed in horizon:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in horizon:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.