Charm cs:cinder-276, xenial
When I upgraded the cloud from Ocata to Pike, the cinder-scheduler service fails to start with:
2018-12-20 05:08:29.901 759617 INFO cinder.service [-] Starting cinder-scheduler node (version 11.1.1)
2018-12-20 05:08:29.905 759617 INFO cinder.manager [req-f2a0c7b7-88d8-48d3-8a45-c112e930254b - - - - -] Initiating service 13 cleanup
2018-12-20 05:08:29.908 759617 INFO cinder.manager [req-f2a0c7b7-88d8-48d3-8a45-c112e930254b - - - - -] Service 13 cleanup completed.
2018-12-20 05:08:29.909 759617 DEBUG cinder.service [req-f2a0c7b7-88d8-48d3-8a45-c112e930254b - - - - -] Creating RPC server for service cinder-scheduler start /usr/lib/python2.7/dist-packages/cinder/service.py:219
2018-12-20 05:08:29.915 759617 DEBUG cinder.service [req-f2a0c7b7-88d8-48d3-8a45-c112e930254b - - - - -] Pinning object versions for RPC server serializer to 1.3 start /usr/lib/python2.7/dist-packages/cinder/service.py:226
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service [req-f2a0c7b7-88d8-48d3-8a45-c112e930254b - - - - -] Error starting thread.: RPCVersionCapError: Requested message version, 3.0 is incompatible. It needs to be equal in major version and less than or equal in minor version as the specified version cap 2.0.
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service Traceback (most recent call last):
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 721, in run_service
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service service.start()
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 259, in start
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service self.manager.init_host_with_rpc()
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/cinder/scheduler/manager.py", line 87, in init_host_with_rpc
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service self.request_service_capabilities(ctxt)
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/cinder/scheduler/manager.py", line 205, in request_service_capabilities
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service volume_rpcapi.VolumeAPI().publish_service_capabilities(context)
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/cinder/volume/rpcapi.py", line 239, in publish_service_capabilities
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service cctxt.cast(ctxt, 'publish_service_capabilities')
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 144, in cast
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service self._check_version_cap(msg.get('version'))
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 121, in _check_version_cap
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service version_cap=self.version_cap)
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service RPCVersionCapError: Requested message version, 3.0 is incompatible. It needs to be equal in major version and less than or equal in minor version as the specified version cap 2.0.
2018-12-20 05:08:29.959 759617 ERROR oslo_service.service
2018-12-20 05:08:29.977 759617 DEBUG oslo_concurrency.lockutils [req-ebb18cc4-69ec-45f4-99d6-95958322449e - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:215
2018-12-20 05:08:29.978 759617 DEBUG oslo_concurrency.lockutils [req-ebb18cc4-69ec-45f4-99d6-95958322449e - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:228
There were some entries in the Cinder db, in the 'services' table, which were elderly and should have been cleaned up, which were blocking the start of cinder-scheduler. When I deleted them using cinder-manage service remove cinder-volume cinder@LVM (etc) that fixed the issue.
https:/ /pastebin. canonical. com/p/hYXqFwttk 2/