Jenkins fails patch verification on: ConnectionError: HTTPConnectionPool(host='public.nova.example.com', port=8774): Max retries exceeded

[Daniel Korn]

Jenkins fails on three tests:
======================

gate-horizon-python26
gate-horizon-python27
gate-horizon-python27-django14

The error that repeats in the log files is:
================================

14-06-22 20:18:42.483 | ERROR: test_change_password_shows_message_on_login_page (openstack_dashboard.dashboards.settings.password.tests.ChangePasswordTests)
| ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/test/helpers.py", line 81, in instance_stub_out
    return fn(self, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/dashboards/settings/password/tests.py", line 65, in test_change_password_shows_message_on_login_page
    res = self.client.post(INDEX_URL, formData, follow=True)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/test/client.py", line 485, in post
    response = self._handle_redirects(response, **extra)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/test/client.py", line 617, in _handle_redirects
     response = self.get(url.path, QueryDict(url.query), follow=False, **extra)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/test/client.py", line 473, in get
    response = super(Client, self).get(path, data=data, **extra)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/test/client.py", line 280, in get
    return self.request(**r)
  File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/test/client.py", line 444, in request
     six.reraise(*exc_info)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/decorators.py", line 36, in dec
    return view_func(request, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/decorators.py", line 52, in dec
     return view_func(request, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/decorators.py", line 36, in dec
     return view_func(request, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
     return self.dispatch(request, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/tables/views.py", line 152, in get
     handled = self.construct_tables()
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/tables/views.py", line 143, in construct_tables
     handled = self.handle_table(table)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/tables/views.py", line 116, in handle_table
     data = self._get_data_dict()
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/tables/views.py", line 179, in _get_data_dict
     self._data = {self.table_class._meta.name: self.get_data()}
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/dashboards/project/overview/views.py", line 55, in get_data
     super(ProjectOverview, self).get_data()
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/usage/views.py", line 43, in get_data
     self.usage.summarize(*self.usage.get_date_range())
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/usage/base.py", line 195, in summarize
     if not api.nova.extension_supported('SimpleTenantUsage', self.request):
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/utils/memoized.py", line 90, in wrapped
     value = cache[key] = func(*args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/api/nova.py", line 750, in extension_supported
     extensions = list_extensions(request)
   File "/home/jenkins/workspace/gate-horizon-python27/horizon/utils/memoized.py", line 90, in wrapped
    value = cache[key] = func(*args, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/openstack_dashboard/api/nova.py", line 741, in list_extensions
     return nova_list_extensions.ListExtManager(novaclient(request)).show_all()
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/v1_1/contrib/list_extensions.py", line 37, in show_all
     return self._list("/extensions", 'extensions')
  File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/base.py", line 64, in _list
     _resp, body = self.api.client.get(url)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/client.py", line 283, in get
      return self._cs_request(url, 'GET', **kwargs)
    File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/client.py", line 260, in _cs_request
     **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/client.py", line 242, in _time_request
     resp, body = self.request(url, method, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/novaclient/client.py", line 213, in request
     **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
     resp = self.send(prep, **send_kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
     r = adapter.send(request, **kwargs)
   File "/home/jenkins/workspace/gate-horizon-python27/.tox/py27/local/lib/python2.7/site-packages/requests/adapters.py", line 375, in send
     raise ConnectionError(e, request=request)
 ConnectionError: HTTPConnectionPool(host='public.nova.example.com', port=8774): Max retries exceeded with url: /v2/extensions (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

[Julie Pichon (jpichon)]

All Horizon patches are currently failing due to this.

[Julie Pichon (jpichon)]

This is related to django_openstack_auth 1.1.6, I can reproduce the issue locally and the test passes with 1.1.4.

[Julie Pichon (jpichon)]

The problem appears due to 1.1.6 and bug 1308637. However as far as I can tell that patch is not causing the problem itself but only surfacing an issue that already existed within the Horizon test. At this point when we redirect the user to /auth/logout/ in a unit test the user is not properly logged out and the django session is not terminated. (It works fine testing in a real environment.)

To fix the test we need to mock the logout method properly, though it doesn't seem totally straightforward because of the way the redirection is handled. I think the test is trying to do too much and would fit better as an integration test.

My suggestion for now would be to disable the test to get the horizon gate going again. I'll propose a second patch later today to change it so it has a reduced scope.

This also affects Icehouse.

[Julie Pichon (jpichon)]

Initial patch to disable the test at https://review.openstack.org/#/c/101857/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/101857
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=5455b8f6f595633513b6521449ebc1985d5b9070
Submitter: Jenkins
Branch: master

commit 5455b8f6f595633513b6521449ebc1985d5b9070
Author: Julie Pichon <email address hidden>
Date: Mon Jun 23 11:58:09 2014 +0100

    Disable broken unit test (related to "Change Password")

    The new django_openstack_auth 1.1.6 library surfaced a defect in this
    test, whereby the test client does not properly log out the user. This
    only affects the test, the functionality otherwise works fine.

    This temporarily disables the test in order to repair the gate, while
    waiting to replace it with a new unit test more reduced in scope and/or
    an integration test.

    Change-Id: I8779666c053a4835e22038a39a530fe5fc84b6a3
    Partial-Bug: #1333144

[Julie Pichon (jpichon)]

Lowering severity now that the gate isn't blocked anymore.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/102274

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/110054

[Julie Pichon (jpichon)]

Setting Icehouse bug importance to "medium" as we're not on fire yet - without this patch though, the icehouse branch will fail once a new django_openstack_auth release is out.

[Julie Pichon (jpichon)]

Opening a Havana task as well since this is when the test was added and the django_openstack_auth requirement is open ended, https://github.com/openstack/horizon/blob/stable/havana/requirements.txt#L5

[Julie Pichon (jpichon)]

Actually the fix isn't necessary for Havana (and we didn't backport the !=1.1.6 requirement restriction either) because these tests are skipped already, due to them not working with Keystone v3. Closing the task.

https://github.com/openstack/horizon/blob/stable/havana/openstack_dashboard/dashboards/settings/password/tests.py#L66
https://github.com/openstack/horizon/blob/stable/havana/openstack_dashboard/test/settings.py#L83

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/110054
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=785a5266b9c58538ad9b8524d48b967975ab8644
Submitter: Jenkins
Branch: stable/icehouse

commit 785a5266b9c58538ad9b8524d48b967975ab8644
Author: Julie Pichon <email address hidden>
Date: Mon Jun 23 11:58:09 2014 +0100

    Disable broken unit test (related to "Change Password")

    The new django_openstack_auth 1.1.6 library surfaced a defect in this
    test, whereby the test client does not properly log out the user. This
    only affects the test, the functionality otherwise works fine.

    This temporarily disables the test in order to repair the gate, while
    waiting to replace it with a new unit test more reduced in scope and/or
    an integration test.

    Change-Id: I8779666c053a4835e22038a39a530fe5fc84b6a3
    Closes-Bug: #1333144
    (cherry picked from commit 5455b8f6f595633513b6521449ebc1985d5b9070)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/102274
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=11c21626c0f63a4fa72c5df2a840cdf6c00307ab
Submitter: Jenkins
Branch: master

commit 11c21626c0f63a4fa72c5df2a840cdf6c00307ab
Author: Julie Pichon <email address hidden>
Date: Tue Jun 24 16:19:22 2014 +0100

    Integration test for showing logout message (password change)

    This replaces the unit test that had to be disabled.

    Change-Id: I0921bd1164a10d0d33a8f00e522ae47ebdf4887e
    Closes-Bug: #1333144