OpenStack Keystone Charm

Keystone SSL Exception

Bug #1723892 reported by Chris MacNaughton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Keystone Charm
Fix Released
High
Unassigned
OpenStack Keystone Charm 17.11
Openstack Mojo Testing
Invalid
Undecided
Unassigned

Bug Description

In the OSCI mojo run at http://10.245.162.49:8080/job/mojo_runner/1193, there wan an SSL exception connecting to Keystone to authenticate, pate of the traceback is at http://pastebin.ubuntu.com/25751778/

Tags: serverstack
Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :
Download full text (4.6 KiB)

04:29:24 2017-10-14 04:29:23 [DEBUG] Starting new HTTPS connection (1): 172.17.107.33
04:29:24 Traceback (most recent call last):
04:29:24 File "/tmp/tmp.2rwjEGZfch/mojo-openstack-specs/trusty/osci-mojo/spec/specs/full_stack/next_deploy_ssl/icehouse/network_setup.py", line 103, in <module>
04:29:24 sys.exit(main(sys.argv))
04:29:24 File "/tmp/tmp.2rwjEGZfch/mojo-openstack-specs/trusty/osci-mojo/spec/specs/full_stack/next_deploy_ssl/icehouse/network_setup.py", line 99, in main
04:29:24 setup_sdn(net_topology, net_info)
04:29:24 File "/tmp/tmp.2rwjEGZfch/mojo-openstack-specs/trusty/osci-mojo/spec/specs/full_stack/next_deploy_ssl/icehouse/network_setup.py", line 18, in setup_sdn
04:29:24 api_version=overcloud_novarc['API_VERSION']
04:29:24 File "/tmp/tmp.2rwjEGZfch/mojo-openstack-specs/trusty/osci-mojo/spec/helper/setup/utils/mojo_os_utils.py", line 286, in get_project_id
04:29:24 all_projects = ks_client.projects.list(domain=domain_id)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/positional/__init__.py", line 108, in inner
04:29:24 return wrapped(*args, **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneclient/v3/projects.py", line 119, in list
04:29:24 **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneclient/base.py", line 75, in func
04:29:24 return f(*args, **new_kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneclient/base.py", line 390, in list
04:29:24 self.collection_key)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneclient/base.py", line 125, in _list
04:29:24 resp, body = self.client.get(url, **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 304, in get
04:29:24 return self.request(url, 'GET', **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 463, in request
04:29:24 resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 189, in request
04:29:24 return self.session.request(url, method, **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 573, in request
04:29:24 auth_headers = self.get_auth_headers(auth)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 900, in get_auth_headers
04:29:24 return auth.get_headers(self, **kwargs)
04:29:24 File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/plugin.py", line 90, in get_headers
04:29:24 token = self.get_token(se...

Read more...

Revision history for this message
David Ames (thedac) wrote :

This bug is confirmed. Apache is not being restarted after openstack_https_frontend.conf is rendered which causes all attempted connections to fail. A subsequent restart of apache2 works around the issue.

Chris, correct me if I am wrong this is just on icehouse and kilo, correct?

That should give us a hint where to look.

Changed in openstack-mojo-specs:
status: New → Confirmed
importance: Undecided → High
Changed in charm-keystone:
status: New → Confirmed
importance: Undecided → High
Changed in openstack-mojo-specs:
status: Confirmed → Invalid
importance: High → Undecided
Changed in charm-keystone:
milestone: none → 17.11
Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote :

Correct, it's just happening on trusty-[icehouse, kilo]

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-keystone (master)

Reviewed: https://review.openstack.org/518602
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=7c065062d292b3f87254deb4333623a5b967e276
Submitter: Zuul
Branch: master

commit 7c065062d292b3f87254deb4333623a5b967e276
Author: David Ames <email address hidden>
Date: Wed Nov 8 18:40:12 2017 +0000

    Ensure HTTPS configuration completes

    There was a race where the https apache2 site,
    openstack_https_frontend.conf, would be rendered in one hook, then
    subsequently the config-changed hook would run and enable that site.
    However, the subsequent config-changed hook would see the template as
    having not changed and therefore it would fail to restart apache2.
    This lead to apache2 failing to listen on the correct ports.

    This was due to CONFIGS.write_all() being called but a2ensite not
    being called. This change fixes this race and adds a call to
    configure_https() to ensure the configuration completes and apache2
    is restarted.

    Change-Id: I229d25c707a0630c9d609fd20a962a0de2e42c77
    Closes-Bug: #1723892

Changed in charm-keystone:
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-keystone (stable/17.08)

Fix proposed to branch: stable/17.08
Review: https://review.openstack.org/518742

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-keystone (stable/17.08)

Reviewed: https://review.openstack.org/518742
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=02b8db2e6e2ca370992e813fef7e2d09c69ed5d1
Submitter: Zuul
Branch: stable/17.08

commit 02b8db2e6e2ca370992e813fef7e2d09c69ed5d1
Author: David Ames <email address hidden>
Date: Wed Nov 8 18:40:12 2017 +0000

    Ensure HTTPS configuration completes

    There was a race where the https apache2 site,
    openstack_https_frontend.conf, would be rendered in one hook, then
    subsequently the config-changed hook would run and enable that site.
    However, the subsequent config-changed hook would see the template as
    having not changed and therefore it would fail to restart apache2.
    This lead to apache2 failing to listen on the correct ports.

    This was due to CONFIGS.write_all() being called but a2ensite not
    being called. This change fixes this race and adds a call to
    configure_https() to ensure the configuration completes and apache2
    is restarted.

    Change-Id: I229d25c707a0630c9d609fd20a962a0de2e42c77
    Closes-Bug: #1723892
    (cherry picked from commit 7c065062d292b3f87254deb4333623a5b967e276)

James Page (james-page)
Changed in charm-keystone:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.