security group list error in py34

Bug #1455313 reported by Dean Troyer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-openstackclient
Invalid
Low
Unassigned

Bug Description

This is probably not specific to the security group list command but that is the one that reliably times out for me on the DreamHost cloud.

OS/X 10.10.3
Python 3.4 (installed via homebrew)
OSC master e9159d6f1e3a39bec4aabc27ea439eeb858710f8

The call to get a list of projects so we can show project names instead of IDs is designed to fail when the caller does not have an administrative role. On the DreamHost cloud this call times out because the Identity client is attempting to do version discovery and the DreamHost cloud does not respond on Keystone's root route ('/'). When this times out (here comes this particular bug!) this is the timeout traceback:

DEBUG: openstackclient.identity.client Instantiating identity client: <class 'openstackclient.identity.client.IdentityClientv2'>
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://keystone-admin.dream.io:35357/ -H "Accept: application/json" -H "User-Agent: python-openstackclient"
INFO: requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): keystone-admin.dream.io
WARNING: keystoneclient.auth.identity.base Failed to contact the endpoint at https://keystone-admin.dream.io:35357/v2.0 for discovery. Fallback to using that endpoint as the base url.
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://keystone-admin.dream.io:35357/v2.0/tenants -H "Accept: application/json" -H "X-Auth-Token: {SHA1}6b882ccf0d48a383db20dc24bb923da29ed15a08" -H "User-Agent: python-keystoneclient"
INFO: requests.packages.urllib3.connectionpool Starting new HTTPS connection (2): keystone-admin.dream.io
ERROR: openstack Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
    self._validate_conn(conn)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
    conn.connect()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    raise err
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
    self._validate_conn(conn)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
    conn.connect()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    raise err
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)

requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 416, in _send_request
    resp = self.session.request(method, url, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 310, in run_subcommand
    result = cmd.run(parsed_args)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/display.py", line 92, in run
    column_names, data = self.take_action(parsed_args)
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/compute/v2/security_group.py", line 164, in take_action
    projects = self.app.client_manager.identity.projects.list()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/v2_0/tenants.py", line 123, in list
    tenant_list = self._list('/tenants%s' % query, 'tenants')
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/base.py", line 113, in _list
    resp, body = self.client.get(url, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 170, in get
    return self.request(url, 'GET', **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 206, in request
    resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 95, in request
    return self.session.request(url, method, **kwargs)
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/common/session.py", line 40, in request
    resp = super(TimingSession, self).request(url, method, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 382, in request

    resp = send(**kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 426, in _send_request
    raise exceptions.ConnectionRefused(msg)
keystoneclient.openstack.common.apiclient.exceptions.ConnectionRefused: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
DEBUG: openstackclient.shell clean_up ListSecurityGroup: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
    self._validate_conn(conn)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
    conn.connect()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    raise err
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
    self._validate_conn(conn)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
    conn.connect()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
    conn = self._new_conn()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
    raise err
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
    sock.connect(sa)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 416, in _send_request
    resp = self.session.request(method, url, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 152, in run
    return super(OpenStackShell, self).run(argv)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 242, in run
    result = self.run_subcommand(remainder)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 310, in run_subcommand
    result = cmd.run(parsed_args)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/display.py", line 92, in run
    column_names, data = self.take_action(parsed_args)
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/compute/v2/security_group.py", line 164, in take_action
    projects = self.app.client_manager.identity.projects.list()
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/v2_0/tenants.py", line 123, in list
    tenant_list = self._list('/tenants%s' % query, 'tenants')
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/base.py", line 113, in _list
    resp, body = self.client.get(url, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 170, in get
    return self.request(url, 'GET', **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 206, in request
    resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 95, in request
    return self.session.request(url, method, **kwargs)
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/common/session.py", line 40, in request
    resp = super(TimingSession, self).request(url, method, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/utils.py", line 318, in inner
    return func(*args, **kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 382, in request
    resp = send(**kwargs)
  File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 426, in _send_request
    raise exceptions.ConnectionRefused(msg)
keystoneclient.openstack.common.apiclient.exceptions.ConnectionRefused: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dtroyer/lib/osc/bin/openstack", line 10, in <module>
    sys.exit(main())
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 353, in main
    return OpenStackShell().run(argv)
  File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 157, in run
    self.log.error(traceback.format_exc(e))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 256, in format_exc
    return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 181, in format_exception
    return list(_format_exception_iter(etype, value, tb, limit, chain))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 153, in _format_exception_iter
    yield from _format_list_iter(_extract_tb_iter(tb, limit=limit))
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 18, in _format_list_iter
    for filename, lineno, name, line in extracted_list:
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 58, in _extract_tb_or_stack_iter
    while curr is not None and (limit is None or n < limit):
TypeError: unorderable types: int() < ConnectionRefused()

Yes sports fans, 7 (count 'em, 7!) nested exceptions within exception handlers. I really have no ides where the real bug lies yet, but OSC needs to be a bit smarter about catching this and reporting something intelligent with debugging turned off.

FWIW, I can't test this on py27 because the TLS handshake fails

Changed in python-openstackclient:
importance: Undecided → Low
jiaxi (tjxiter)
Changed in python-openstackclient:
assignee: nobody → jiaxi (tjxiter)
jiaxi (tjxiter)
Changed in python-openstackclient:
assignee: jiaxi (tjxiter) → nobody
Revision history for this message
Dean Troyer (dtroyer) wrote :

Current attempts to duplicate this are succeeding, er, failing...its working. It could be a py34 update or any one of a number of things...closing

Changed in python-openstackclient:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.