The following packages were updated on Wallaby compute nodes:
python3-nova:amd64 (3:23.2.2-0ubuntu1~cloud1, 3:23.2.2-0ubuntu1~cloud2), python3-os-brick:amd64 (4.3.3-0ubuntu1~cloud0, 4.3.3-0ubuntu1~cloud1), nova-compute-libvirt:amd64 (3:23.2.2-0ubuntu1~cloud1, 3:23.2.2-0ubuntu1~cloud2), nova-common:amd64 (3:23.2.2-0ubuntu1~cloud1, 3:23.2.2-0ubuntu1~cloud2), os-brick-common:amd64 (4.3.3-0ubuntu1~cloud0, 4.3.3-0ubuntu1~cloud1), nova-compute-kvm:amd64 (3:23.2.2-0ubuntu1~cloud1, 3:23.2.2-0ubuntu1~cloud2), nova-compute:amd64 (3:23.2.2-0ubuntu1~cloud1, 3:23.2.2-0ubuntu1~cloud2)
nova-compute is now unable to detach volumes from instances:
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server [req-470d3e0e-e59c-40c5-9597-6649c08add16 046191f8ebfd4695b3387a5ead3a9a55 85945271df8b4a6f9d37c37e4e52958d - default default] Exception during message handling: TypeError: disconnect_volume() got an unexpected keyword argument 'force' 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/exception_wrapper.py", line 71, in wrapped 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server _emit_versioned_exception_notification( 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server raise self.value 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/exception_wrapper.py", line 63, in wrapped 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/utils.py", line 1434, in decorated_function 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 211, in decorated_function 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server compute_utils.add_instance_fault_from_exc(context, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server raise self.value 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 200, in decorated_function 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 7195, in detach_volume 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server do_detach_volume(context, volume_id, instance, attachment_id) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py", line 360, in inner 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 7192, in do_detach_volume 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self._detach_volume(context, bdm, instance, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 7143, in _detach_volume 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server driver_bdm.detach(context, instance, self.volume_api, self.driver, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 476, in detach 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self._do_detach(context, instance, volume_api, virt_driver, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 408, in _do_detach 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self.driver_detach(context, instance, volume_api, virt_driver) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 347, in driver_detach 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server volume_api.roll_detaching(context, volume_id) 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server raise self.value 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/block_device.py", line 328, in driver_detach 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server virt_driver.detach_volume(context, connection_info, instance, mp, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 2592, in detach_volume 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server self._disconnect_volume(context, connection_info, instance, 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 1862, in _disconnect_volume 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server vol_driver.disconnect_volume( 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server TypeError: disconnect_volume() got an unexpected keyword argument 'force' 2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.rpc.server
Looks like it doesn't know about the "force" keyword that's being passed.
The following packages were updated on Wallaby compute nodes:
python3-nova:amd64 (3:23.2. 2-0ubuntu1~ cloud1, 3:23.2. 2-0ubuntu1~ cloud2) , os-brick: amd64 (4.3.3- 0ubuntu1~ cloud0, 4.3.3-0ubuntu1~ cloud1) , libvirt: amd64 (3:23.2. 2-0ubuntu1~ cloud1, 3:23.2. 2-0ubuntu1~ cloud2) , 2-0ubuntu1~ cloud1, 3:23.2. 2-0ubuntu1~ cloud2) , common: amd64 (4.3.3- 0ubuntu1~ cloud0, 4.3.3-0ubuntu1~ cloud1) , kvm:amd64 (3:23.2. 2-0ubuntu1~ cloud1, 3:23.2. 2-0ubuntu1~ cloud2) , 2-0ubuntu1~ cloud1, 3:23.2. 2-0ubuntu1~ cloud2)
python3-
nova-compute-
nova-common:amd64 (3:23.2.
os-brick-
nova-compute-
nova-compute:amd64 (3:23.2.
nova-compute is now unable to detach volumes from instances:
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging. rpc.server [req-470d3e0e- e59c-40c5- 9597-6649c08add 16 046191f8ebfd469 5b3387a5ead3a9a 55 85945271df8b4a6 f9d37c37e4e5295 8d - default default] Exception during message handling: TypeError: disconnect_volume() got an unexpected keyword argument 'force' rpc.server Traceback (most recent call last): rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/server. py", line 165, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/dispatcher. py", line 309, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_messaging/ rpc/dispatcher. py", line 229, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/exception_ wrapper. py", line 71, in wrapped rpc.server _emit_versioned _exception_ notification( rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 227, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 200, in force_reraise rpc.server raise self.value rpc.server File "/usr/lib/ python3/ dist-packages/ nova/exception_ wrapper. py", line 63, in wrapped rpc.server return f(self, context, *args, **kw) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ utils.py" , line 1434, in decorated_function rpc.server return function(self, context, *args, **kwargs) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ manager. py", line 211, in decorated_function rpc.server compute_ utils.add_ instance_ fault_from_ exc(context, rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 227, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 200, in force_reraise rpc.server raise self.value rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ manager. py", line 200, in decorated_function rpc.server return function(self, context, *args, **kwargs) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ manager. py", line 7195, in detach_volume rpc.server do_detach_ volume( context, volume_id, instance, attachment_id) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_concurrenc y/lockutils. py", line 360, in inner rpc.server return f(*args, **kwargs) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ manager. py", line 7192, in do_detach_volume rpc.server self._detach_ volume( context, bdm, instance, rpc.server File "/usr/lib/ python3/ dist-packages/ nova/compute/ manager. py", line 7143, in _detach_volume rpc.server driver_ bdm.detach( context, instance, self.volume_api, self.driver, rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ block_device. py", line 476, in detach rpc.server self._do_ detach( context, instance, volume_api, virt_driver, rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ block_device. py", line 408, in _do_detach rpc.server self.driver_ detach( context, instance, volume_api, virt_driver) rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ block_device. py", line 347, in driver_detach rpc.server volume_ api.roll_ detaching( context, volume_id) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 227, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python3/ dist-packages/ oslo_utils/ excutils. py", line 200, in force_reraise rpc.server raise self.value rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ block_device. py", line 328, in driver_detach rpc.server virt_driver. detach_ volume( context, connection_info, instance, mp, rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ libvirt/ driver. py", line 2592, in detach_volume rpc.server self._disconnec t_volume( context, connection_info, instance, rpc.server File "/usr/lib/ python3/ dist-packages/ nova/virt/ libvirt/ driver. py", line 1862, in _disconnect_volume rpc.server vol_driver. disconnect_ volume( rpc.server TypeError: disconnect_volume() got an unexpected keyword argument 'force' rpc.server
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
2023-05-13 05:53:00.128 3219193 ERROR oslo_messaging.
Looks like it doesn't know about the "force" keyword that's being passed.