OpenStack-Ansible

Missing insecure flag for [neutron] section of nova.conf

Bug #1566629 reported by Jordan Callicoat on 2016-04-06

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack-Ansible	Fix Released	High	Ala Raddaoui	OpenStack-Ansible newton-1

Bug Description

When setting `keystone_service_internaluri_insecure: true` in user_variables.yml, the generated nova.conf file is setting `insecure = True` in the [keystone_auth] section, but not in the [neutron] section. In an environment where publicURL is using https with a self-signed cert, you get the following trace in nova-compute.log:

2016-04-06 03:07:57.640 28418 ERROR oslo_messaging.rpc.dispatcher [req-96f7de0a-fe8e-420f-8d7c-411da4a7f2de fa05ffd2ddc440bb88ca8c96424d6f2c 351448c1c0af460a81cf9c579d4dd44d - - -] Exception during message handling: SSL exception connecting to https://72.32.112.255:5000/v3/auth/tokens: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 6936, in start_instance
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return self.manager.start_instance(ctxt, instance)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher payload)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 304, in decorated_function
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 383, in decorated_function
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 361, in decorated_function
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 349, in decorated_function
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 2852, in start_instance
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher self._power_on(context, instance)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 2820, in _power_on
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher network_info = self._get_instance_nw_info(context, instance)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/compute/manager.py", line 1383, in _get_instance_nw_info
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return self.network_api.get_instance_nw_info(context, instance)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 747, in get_instance_nw_info
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher preexisting_port_ids)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 763, in _get_instance_nw_info
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher preexisting_port_ids)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1515, in _build_network_info_model
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher client = get_client(context, admin=True)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 205, in get_client
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher auth_token = _ADMIN_AUTH.get_token(_SESSION)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 104, in get_token
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return self.get_access(session).auth_token
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py", line 144, in get_access
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher self.auth_ref = self.get_auth_ref(session)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/generic/base.py", line 176, in get_auth_ref
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return self._plugin.get_auth_ref(session, **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py", line 173, in get_auth_ref
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher authenticated=False, log=False, **rkwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 499, in post
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return self.request(url, 'POST', **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher return func(*args, **kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 384, in request
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher resp = send(**kwargs)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 422, in _send_request
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher raise exceptions.SSLError(msg)
2016-04-06 03:07:57.640 28418 TRACE oslo_messaging.rpc.dispatcher SSLError: SSL exception connecting to https://72.32.112.255:5000/v3/auth/tokens: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Adding `insecure = True` to the [neutron] section resolves this issue. This can be set via an override:

nova_nova_conf_overrides:
neutron:
insecure: true

However, given that we are setting the option in the [keystone_auth] section, we should probably be adding it to the [neutron] section as well.

Ala Raddaoui (raddaoui-ala) on 2016-04-08

Changed in openstack-ansible:
assignee:	nobody → Ala Raddaoui (raddaoui-ala)

Jesse Pretorius (jesse-pretorius) on 2016-04-12

Changed in openstack-ansible:
status:	New → Confirmed
importance:	Undecided → High
milestone:	none → newton-1

Revision history for this message

Ala Raddaoui (raddaoui-ala) wrote on 2016-04-15:

Hi Jordan can you pass me your nova.conf configurations under the two sections [keystone_authtoken] and [neutron].

Revision history for this message

Major Hayden (rackerhacker) wrote on 2016-04-18:

We're dealing with three API's here:

  1) Internal URL
  2) Admin URL (which is also "internal")
  3) Public URL

It would make sense to me that if someone had the `insecure` flag set on their internal URL, they would want the same flag set on the admin URL. It sounds like we want `keystone_service_adminuri_insecure` to become True whenever `keystone_service_internaluri_insecure` is set to True. Is that right?

Also -- are your nova and neutron nodes communicating over public IP addresses, Jordan?

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-18: Fix proposed to openstack-ansible (master)

Fix proposed to branch: master
Review: https://review.openstack.org/307512

Changed in openstack-ansible:
status:	Confirmed → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-21: Fix merged to openstack-ansible (master)

Reviewed: https://review.openstack.org/307512
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=2ffb77638f48d75a444c35e619355cd6f1339076
Submitter: Jenkins
Branch: master

commit 2ffb77638f48d75a444c35e619355cd6f1339076
Author: Ala Raddaoui <email address hidden>
Date: Mon Apr 18 22:22:25 2016 +0000

Change keystone admin/internal insecure flags

    Make `keystone_service_internaluri_insecure` and
    `keystone_service_adminuri_insecure` flag default values
    derived from facts like protocol each endpoint is using and
    kind of certificates being used to reach that endpoint
    (user-provided or self-signed).

Change-Id: I921031c676776884cb121a7914fcd9c505345e67
Closes-Bug: 1566629

Changed in openstack-ansible:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-21: Fix proposed to openstack-ansible (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/308988

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-13: Change abandoned on openstack-ansible (stable/mitaka)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/308988
Reason: The documentation has already substancially changed in master and this patch is not essential for stable/mitaka.

Revision history for this message

Thierry Carrez (ttx) wrote on 2016-06-15: Fix included in openstack/openstack-ansible 14.0.0.0b1

This issue was fixed in the openstack/openstack-ansible 14.0.0.0b1 development milestone.

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2016-07-28: Fix included in openstack/openstack-ansible 14.0.0.0b2

This issue was fixed in the openstack/openstack-ansible 14.0.0.0b2 development milestone.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.