Sahara

Neutron client should use user defined endpoint type

Bug #1564805 reported by Frédéric Gaudet
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Sahara
Fix Released
High
Vitalii Gridnev
Sahara newton-1 "n1"

Bug Description

Currently, neutron client doesn't use the endpoint type, falling back to publicURL. It could be an issue if a firewall prevent api access from the management network.

Step to reproduce :
1/ Use namespace
2/ Use neutron
3/ No floating IP
4/ Define a sahara cluster
5/ Launch a sahara cluster

The cluster is stuck on the 'Waiting' state. Neutron client tries to reach the neutron server to get the router id via the publicURL instead of the internal one.

Here is some output :

Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/ops.py", line 164, in wrapper
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] f(cluster_id, *args, **kwds)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/ops.py", line 259, in _provision_cluster
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] INFRA.create_cluster(cluster)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/heat/heat_engine.py", line 63, in create_cluster
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] self._launch_instances(cluster, target_count, CREATE_STAGES)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/heat/heat_engine.py", line 240, in _launch_instances
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] volumes.mount_to_instances(instances)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/volumes.py", line 217, in mount_to_instances
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] use_xfs = _can_use_xfs(instances)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/volumes.py", line 101, in _can_use_xfs
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] if not _check_installed_xfs(instance):
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/service/volumes.py", line 80, in _check_installed_xfs
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] with instance.remote() as r:
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 562, in __enter__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] _release_remote_semaphore()
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] six.reraise(self.type_, self.value, self.tb)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 558, in __enter__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] self.bulk = BulkInstanceInteropHelper(self.instance)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 810, in __init__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] procutils.shutdown_subprocess(self.proc, _cleanup)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] six.reraise(self.type_, self.value, self.tb)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 807, in __init__
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] self._get_conn_params())
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 645, in _get_conn_params
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] info=None, rootwrap_command=rootwrap)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/ssh_remote.py", line 601, in _build_proxy_command
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] keywords['router_id'] = client.get_router()
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/sahara/utils/openstack/neutron.py", line 76, in get_router
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] routers = self.neutron.list_routers()['routers']
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 102, in with_params
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] ret = self.function(instance, *args, **kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 653, in list_routers
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] **_params)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 307, in list
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] for r in self._pagination(collection, path, **params):
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 320, in _pagination
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] res = self.get(path, params=params)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 293, in get
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] headers=headers, params=params)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 270, in retry_request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] headers=headers, params=params)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 200, in do_request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] content_type=self.content_type())
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 306, in do_request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] return self.request(url, method, **kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 294, in request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] resp = super(SessionClient, self).request(*args, **kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 95, in request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] return self.session.request(url, method, **kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] return func(*args, **kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 386, in request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] resp = send(**kwargs)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 430, in _send_request
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] raise exceptions.ConnectionRefused(msg)
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8] ConnectionRefused: Unable to establish connection to https://api.isima.fr:9696/v2.0/routers.json
Mar 31 19:38:29 net000 sahara-all: 2016-03-31 19:38:29.483 27271 ERROR sahara.service.ops [instance: none, cluster: ade7ee7d-a15f-4cfe-a465-e89b57d134a8]

A simple fix about that is to add the endpoint type in NeutronClient class (neutron.py)

Revision history for this message
Frédéric Gaudet (frgaudet) wrote :
Revision history for this message
Frédéric Gaudet (frgaudet) wrote :

Forgot to write that endpoint_type is set to internalURL in the neutron section of /etc/sahara/sahara.conf

That's the thing, its being ignored.

Vitalii Gridnev (vgridnev)
Changed in sahara:
importance: Undecided → High
assignee: nobody → Vitaly Gridnev (vgridnev)
milestone: none → newton-1
Vitalii Gridnev (vgridnev)
Changed in sahara:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to sahara (master)

Fix proposed to branch: master
Review: https://review.openstack.org/308757

Changed in sahara:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (master)

Reviewed: https://review.openstack.org/308757
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=beba6e7528f839c503933600525603563014ea24
Submitter: Jenkins
Branch: master

commit beba6e7528f839c503933600525603563014ea24
Author: Vitaly Gridnev <email address hidden>
Date: Thu Apr 21 08:48:00 2016 +0300

    use the only method to initialize client

    using separate methods will produce a confusion and
    issues, like not using endpoint_type during client creation.

    Change-Id: I01f911da2eb9e8fb3157a71f68ea4ad79437b70f
    Closes-bug: 1564805

Changed in sahara:
status: In Progress → Fix Released
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/sahara 5.0.0.0b1

This issue was fixed in the openstack/sahara 5.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.