No volume service started successfully - Versioned Objects in DB are capped

[Moises Emilio Benzan Mora]

2 out of the 3 Cinder units started successfully while the third one failed with the bellow traceback. We believe it could be a problem with the cinder-scheduler selecting a wrong version.

Test run: https://solutions.qa.canonical.com/v2/testruns/04e65b69-4465-494b-bcb5-84233f454cd0/

Test artifacts: https://oil-jenkins.canonical.com/artifacts/04e65b69-4465-494b-bcb5-84233f454cd0/index.html

----------

Openstack crashdump: https://oil-jenkins.canonical.com/artifacts/04e65b69-4465-494b-bcb5-84233f454cd0/generated/generated/openstack/juju-crashdump-openstack-2022-10-24-18.11.21.tar.gz

2022-10-24 18:18:55.911 266060 INFO cinder.rpc [req-e14b6f7c-2233-4037-8756-b5f9df1e9364 - - - - -] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume [req-e14b6f7c-2233-4037-8756-b5f9df1e9364 - - - - -] Volume service cinder@cinder-ceph failed to start.: cinder.exception.CappedVersionUnknown: Unrecoverable Error: Versioned Objects in DB are capped to unknown version 1.39. Most likely your environment contains only new services and you're trying to start an older one. Use `cinder-manage service list` to check that and upgrade this service.
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume Traceback (most recent call last):
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/cmd/volume.py", line 100, in _launch_service
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume server = service.Service.create(host=host,
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/service.py", line 394, in create
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume service_obj = cls(host, binary, topic, manager,
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/service.py", line 152, in __init__
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume self.manager = manager_class(host=self.host,
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 214, in __init__
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume super(VolumeManager, self).__init__(service_name='volume',
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/manager.py", line 180, in __init__
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume self.scheduler_rpcapi = scheduler_rpcapi.SchedulerAPI()
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/rpc.py", line 211, in __init__
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume serializer = base.CinderObjectSerializer(obj_version_cap)
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume File "/usr/lib/python3/dist-packages/cinder/objects/base.py", line 556, in __init__
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume raise exception.CappedVersionUnknown(version=version_cap)
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume cinder.exception.CappedVersionUnknown: Unrecoverable Error: Versioned Objects in DB are capped to unknown version 1.39. Most likely your environment contains only new services and you're trying to start an older one. Use `cinder-manage service list` to check that and upgrade this service.
2022-10-24 18:18:55.912 266060 ERROR cinder.cmd.volume
2022-10-24 18:18:55.918 266060 ERROR cinder.cmd.volume [req-e14b6f7c-2233-4037-8756-b5f9df1e9364 - - - - -] No volume service(s) started successfully, terminating.