Activity log for bug #1834565

Date Who What changed Old value New value Message
2019-06-28 05:49:09 Chris MacNaughton bug added bug
2019-06-28 05:50:30 Chris MacNaughton bug task added charm-neutron-gateway
2019-06-28 05:51:14 Chris MacNaughton summary Services not running that should be: cinder-scheduler Services not running that should be: cinder-scheduler, neutron-lbaasv2-agent, neutron-dhcp-agent, neutron-metering-agent
2019-06-28 05:51:43 Chris MacNaughton description IN an SSL enabled OpenStack deploy, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service IN an SSL enabled OpenStack deploy, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Additionally, charm-neutron-gateway has the same stacktrace in the mentioned services.
2019-06-28 06:01:34 Chris MacNaughton attachment added juju-crashdump-eecb87f2-6514-4a7c-b8d3-4401820bdbe1.tar.gz https://bugs.launchpad.net/charm-neutron-gateway/+bug/1834565/+attachment/5273861/+files/juju-crashdump-eecb87f2-6514-4a7c-b8d3-4401820bdbe1.tar.gz
2019-06-28 13:41:43 Chris MacNaughton description IN an SSL enabled OpenStack deploy, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Additionally, charm-neutron-gateway has the same stacktrace in the mentioned services. In an SSL enabled OpenStack Stein deploy, deployed on Disco, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Additionally, charm-neutron-gateway has the same stacktrace in the mentioned services.
2019-07-01 09:18:03 James Page bug task added python-eventlet (Ubuntu)
2019-07-01 09:32:31 James Page bug watch added https://github.com/eventlet/eventlet/issues/526
2019-07-01 09:46:10 James Page nominated for series Ubuntu Eoan
2019-07-01 09:46:10 James Page bug task added python-eventlet (Ubuntu Eoan)
2019-07-01 09:46:10 James Page nominated for series Ubuntu Disco
2019-07-01 09:46:10 James Page bug task added python-eventlet (Ubuntu Disco)
2019-07-01 09:46:18 James Page python-eventlet (Ubuntu Eoan): status New Fix Released
2019-07-01 09:46:22 James Page python-eventlet (Ubuntu Eoan): importance Undecided High
2019-07-01 09:46:25 James Page python-eventlet (Ubuntu Disco): status New In Progress
2019-07-01 09:46:26 James Page python-eventlet (Ubuntu Disco): importance Undecided High
2019-07-01 09:46:28 James Page python-eventlet (Ubuntu Disco): assignee James Page (james-page)
2019-07-01 09:46:41 James Page bug task added cloud-archive
2019-07-01 09:46:50 James Page nominated for series cloud-archive/train
2019-07-01 09:46:50 James Page bug task added cloud-archive/train
2019-07-01 09:46:50 James Page nominated for series cloud-archive/stein
2019-07-01 09:46:50 James Page bug task added cloud-archive/stein
2019-07-01 09:46:56 James Page cloud-archive/train: status New Fix Released
2019-07-01 09:46:59 James Page cloud-archive/stein: status New In Progress
2019-07-01 09:47:01 James Page cloud-archive/stein: importance Undecided High
2019-07-01 09:47:02 James Page cloud-archive/train: importance Undecided High
2019-07-01 09:47:07 James Page charm-cinder: status New Invalid
2019-07-01 09:47:08 James Page charm-neutron-gateway: status New Invalid
2019-07-01 09:51:43 James Page description In an SSL enabled OpenStack Stein deploy, deployed on Disco, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Additionally, charm-neutron-gateway has the same stacktrace in the mentioned services. [Impact] Unable to enable TLS termination of any endpoints for OpenStack with Python 3.7. [Test Case] sudo apt install python3-eventlet python3 << EOF import eventlet eventlet.monkey_patch() import socket import ssl sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) wrappedSocket = ssl.wrap_socket(sock) EOF Traceback (most recent call last): File "<stdin>", line 8, in <module> File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket return GreenSSLSocket(sock, *a, **kw) File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ *args, **kw File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket suppress_ragged_eofs=suppress_ragged_eofs File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket return GreenSSLSocket(sock, *a, _context=self, **kw) File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ *args, **kw TypeError: wrap_socket() got an unexpected keyword argument '_context' [Regression Potential] Low; codepaths are all Python 3.7 specific due to internal changes in the SSL implementation. [Original bug report] In an SSL enabled OpenStack Stein deploy, deployed on Disco, the cinder-scheduler service is crashing with the below stacktrace: 2019-06-27 18:35:07.593 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: juju-20eff5-mojo-0-7@LVM) 2019-06-27 18:35:07.594 5427 WARNING cinder.scheduler.host_manager [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] volume service is down. (host: cinder@cinder-ceph) 2019-06-27 18:35:07.610 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume objects version 1.37 as minimum service version. 2019-06-27 18:35:07.619 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-volume RPC version 3.16 as minimum service version. 2019-06-27 18:35:07.638 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler objects version 1.37 as minimum service version. 2019-06-27 18:35:07.647 5427 INFO cinder.rpc [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Automatically selected cinder-scheduler RPC version 3.11 as minimum service version. 2019-06-27 18:35:07.702 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.703 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "signing_dir" from group "keystone_authtoken" is deprecated for removal (PKI token format is no longer supported.). Its value may be silently ignored in the future. 2019-06-27 18:35:07.705 5427 WARNING oslo_config.cfg [req-9c5d17cd-1414-482a-bfc9-a6bef2369637 - - - - -] Deprecated: Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2019-06-27 18:35:07.726 5427 INFO cinder.service [-] Starting cinder-scheduler node (version 14.0.0) 2019-06-27 18:35:07.736 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Initiating service 4 cleanup 2019-06-27 18:35:07.745 5427 INFO cinder.manager [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Service 4 cleanup completed. 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service [req-e208c3e5-83e5-4c5b-85a3-1f5e09c86da9 - - - - -] Error starting thread.: TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Traceback (most recent call last): 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_service/service.py", line 796, in run_service 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service service.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/cinder/service.py", line 236, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.rpcserver.start() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 270, in wrapper 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service log_after, timeout_timer) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 190, in run_once 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service post_fn = fn() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 269, in <lambda> 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service states[state].run_once(lambda: fn(self, *args, **kwargs), 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/server.py", line 414, in start 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.listener = self._create_listener() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 151, in _create_listener 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.transport._listen(self._target, 1, None) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 143, in _listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service batch_timeout) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 652, in listen 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in _get_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service purpose=purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/common.py", line 407, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection = connection_pool.create(purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/pool.py", line 144, in create 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection_cls(self.conf, self.url, purpose) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 531, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.ensure_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 646, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.connection.ensure_connection(errback=on_error) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 405, in ensure_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service callback) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/utils/functional.py", line 332, in retry_over_time 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return fun(*args, **kwargs) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 261, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self.connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 802, in connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._connection = self._establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/connection.py", line 757, in _establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn = self.transport.establish_connection() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/kombu/transport/pyamqp.py", line 130, in establish_connection 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service conn.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/connection.py", line 307, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.transport.connect() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 83, in connect 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.socket_settings, self.read_timeout, self.write_timeout, 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 187, in _init_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self._setup_transport() 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 302, in _setup_transport 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service self.sock = self._wrap_socket(self.sock, **self.sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 309, in _wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return self._wrap_socket_sni(sock, **sslopts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/amqp/transport.py", line 344, in _wrap_socket_sni 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service sock = ssl.wrap_socket(**opts) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 403, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3.7/ssl.py", line 1222, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service suppress_ragged_eofs=suppress_ragged_eofs 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 426, in wrap_socket 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service return GreenSSLSocket(sock, *a, _context=self, **kw) 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service File "/usr/lib/python3/dist-packages/eventlet/green/ssl.py", line 80, in __init__ 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service *args, **kw 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service TypeError: wrap_socket() got an unexpected keyword argument '_context' 2019-06-27 18:35:07.776 5427 ERROR oslo_service.service Additionally, charm-neutron-gateway has the same stacktrace in the mentioned services.
2019-07-01 09:51:51 James Page bug added subscriber Ubuntu Stable Release Updates Team
2019-07-02 20:45:34 Brian Murray python-eventlet (Ubuntu Disco): status In Progress Fix Committed
2019-07-02 20:45:39 Brian Murray bug added subscriber SRU Verification
2019-07-02 20:45:43 Brian Murray tags verification-needed verification-needed-disco
2019-07-03 07:49:41 James Page cloud-archive/stein: status In Progress Fix Committed
2019-07-03 07:49:44 James Page tags verification-needed verification-needed-disco verification-needed verification-needed-disco verification-stein-needed
2019-07-03 07:59:15 James Page tags verification-needed verification-needed-disco verification-stein-needed verification-done-disco verification-needed
2019-07-03 07:59:36 James Page tags verification-done-disco verification-needed verification-done verification-done-disco
2019-07-11 08:44:03 Łukasz Zemczak tags verification-done verification-done-disco verification-done verification-needed-disco
2019-07-11 08:44:09 Łukasz Zemczak tags verification-done verification-needed-disco verification-needed verification-needed-disco
2019-10-10 19:29:16 Brian Murray tags verification-needed verification-needed-disco removal-candidate verification-needed verification-needed-disco
2019-11-02 02:41:07 Mathew Hodson tags removal-candidate verification-needed verification-needed-disco removal-candidate verification-needed verification-needed-disco verification-stein-needed
2019-11-02 02:52:01 Mathew Hodson summary Services not running that should be: cinder-scheduler, neutron-lbaasv2-agent, neutron-dhcp-agent, neutron-metering-agent python 3.7: wrap_socket() got an unexpected keyword argument '_context'
2019-11-02 02:53:35 Mathew Hodson affects charm-cinder eventlet
2019-11-02 02:53:53 Mathew Hodson eventlet: importance Undecided Unknown
2019-11-02 02:53:53 Mathew Hodson eventlet: status Invalid Unknown
2019-11-02 02:53:53 Mathew Hodson eventlet: remote watch github.com/eventlet/eventlet/issues #526
2019-11-02 02:54:49 Mathew Hodson affects charm-neutron-gateway ubuntu-translations
2019-11-02 02:55:07 Mathew Hodson bug task deleted ubuntu-translations
2019-11-02 02:55:14 Mathew Hodson bug added subscriber Mathew Hodson
2019-11-02 10:35:01 Liam Young tags removal-candidate verification-needed verification-needed-disco verification-stein-needed removal-candidate verification-done-disco verification-needed verification-stein-needed
2019-11-02 20:51:47 Mathew Hodson tags removal-candidate verification-done-disco verification-needed verification-stein-needed verification-done-disco verification-needed verification-stein-needed
2019-11-04 09:47:28 James Page cloud-archive: status Fix Committed Fix Released
2019-11-04 13:41:14 Launchpad Janitor python-eventlet (Ubuntu Disco): status Fix Committed Fix Released
2019-11-04 13:41:18 Łukasz Zemczak removed subscriber Ubuntu Stable Release Updates Team
2019-11-08 01:53:30 Mathew Hodson removed subscriber Mathew Hodson
2020-01-13 15:25:47 Corey Bryant cloud-archive/stein: status Fix Committed Fix Released
2020-01-13 15:28:50 Corey Bryant tags verification-done-disco verification-needed verification-stein-needed verification-done-disco verification-needed verification-stein-done
2020-10-08 17:28:11 Bug Watch Updater eventlet: status Unknown Fix Released