This is probably not specific to the security group list command but that is the one that reliably times out for me on the DreamHost cloud.
OS/X 10.10.3
Python 3.4 (installed via homebrew)
OSC master e9159d6f1e3a39bec4aabc27ea439eeb858710f8
The call to get a list of projects so we can show project names instead of IDs is designed to fail when the caller does not have an administrative role. On the DreamHost cloud this call times out because the Identity client is attempting to do version discovery and the DreamHost cloud does not respond on Keystone's root route ('/'). When this times out (here comes this particular bug!) this is the timeout traceback:
DEBUG: openstackclient.identity.client Instantiating identity client: <class 'openstackclient.identity.client.IdentityClientv2'>
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://keystone-admin.dream.io:35357/ -H "Accept: application/json" -H "User-Agent: python-openstackclient"
INFO: requests.packages.urllib3.connectionpool Starting new HTTPS connection (1): keystone-admin.dream.io
WARNING: keystoneclient.auth.identity.base Failed to contact the endpoint at https://keystone-admin.dream.io:35357/v2.0 for discovery. Fallback to using that endpoint as the base url.
DEBUG: keystoneclient.session REQ: curl -g -i -X GET https://keystone-admin.dream.io:35357/v2.0/tenants -H "Accept: application/json" -H "X-Auth-Token: {SHA1}6b882ccf0d48a383db20dc24bb923da29ed15a08" -H "User-Agent: python-keystoneclient"
INFO: requests.packages.urllib3.connectionpool Starting new HTTPS connection (2): keystone-admin.dream.io
ERROR: openstack Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
raise value.with_traceback(tb)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 416, in _send_request
resp = self.session.request(method, url, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 310, in run_subcommand
result = cmd.run(parsed_args)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/display.py", line 92, in run
column_names, data = self.take_action(parsed_args)
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/compute/v2/security_group.py", line 164, in take_action
projects = self.app.client_manager.identity.projects.list()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/v2_0/tenants.py", line 123, in list
tenant_list = self._list('/tenants%s' % query, 'tenants')
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/base.py", line 113, in _list
resp, body = self.client.get(url, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 170, in get
return self.request(url, 'GET', **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 206, in request
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 95, in request
return self.session.request(url, method, **kwargs)
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/common/session.py", line 40, in request
resp = super(TimingSession, self).request(url, method, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/utils.py", line 318, in inner
return func(*args, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 382, in request
resp = send(**kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 426, in _send_request
raise exceptions.ConnectionRefused(msg)
keystoneclient.openstack.common.apiclient.exceptions.ConnectionRefused: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
DEBUG: openstackclient.shell clean_up ListSecurityGroup: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
raise value.with_traceback(tb)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 204, in connect
conn = self._new_conn()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 134, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 88, in create_connection
raise err
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 78, in create_connection
sock.connect(sa)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 416, in _send_request
resp = self.session.request(method, url, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(60, 'Operation timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 152, in run
return super(OpenStackShell, self).run(argv)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 242, in run
result = self.run_subcommand(remainder)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/app.py", line 310, in run_subcommand
result = cmd.run(parsed_args)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/cliff/display.py", line 92, in run
column_names, data = self.take_action(parsed_args)
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/compute/v2/security_group.py", line 164, in take_action
projects = self.app.client_manager.identity.projects.list()
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/v2_0/tenants.py", line 123, in list
tenant_list = self._list('/tenants%s' % query, 'tenants')
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/base.py", line 113, in _list
resp, body = self.client.get(url, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 170, in get
return self.request(url, 'GET', **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 206, in request
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/adapter.py", line 95, in request
return self.session.request(url, method, **kwargs)
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/common/session.py", line 40, in request
resp = super(TimingSession, self).request(url, method, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/utils.py", line 318, in inner
return func(*args, **kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 382, in request
resp = send(**kwargs)
File "/Users/dtroyer/lib/osc/lib/python3.4/site-packages/keystoneclient/session.py", line 426, in _send_request
raise exceptions.ConnectionRefused(msg)
keystoneclient.openstack.common.apiclient.exceptions.ConnectionRefused: Unable to establish connection to https://keystone-admin.dream.io:35357/v2.0/tenants
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/dtroyer/lib/osc/bin/openstack", line 10, in <module>
sys.exit(main())
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 353, in main
return OpenStackShell().run(argv)
File "/Users/dtroyer/src/openstack/python-openstackclient/openstackclient/shell.py", line 157, in run
self.log.error(traceback.format_exc(e))
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 256, in format_exc
return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 181, in format_exception
return list(_format_exception_iter(etype, value, tb, limit, chain))
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 153, in _format_exception_iter
yield from _format_list_iter(_extract_tb_iter(tb, limit=limit))
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 18, in _format_list_iter
for filename, lineno, name, line in extracted_list:
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 58, in _extract_tb_or_stack_iter
while curr is not None and (limit is None or n < limit):
TypeError: unorderable types: int() < ConnectionRefused()
Yes sports fans, 7 (count 'em, 7!) nested exceptions within exception handlers. I really have no ides where the real bug lies yet, but OSC needs to be a bit smarter about catching this and reporting something intelligent with debugging turned off.
FWIW, I can't test this on py27 because the TLS handshake fails
Current attempts to duplicate this are succeeding, er, failing...its working. It could be a py34 update or any one of a number of things...closing