Activity log for bug #1891547

Date Who What changed Old value New value Message
2020-08-13 18:09:07 Lee Yarwood bug added bug
2020-08-13 18:09:54 OpenStack Infra nova: status New In Progress
2020-08-13 18:11:40 Lee Yarwood description Description =========== I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced support for the libvirt hot unplug error code VIR_ERR_DEVICE_MISSING that was itself introduced in libvirt v4.3.0. The change did not however cover versions < v4.3.0 such as v4.0.0 installed in our bionic based CI test envs, causing attribute errors when we attempt to reference. Steps to reproduce ================== * Attempt to detach a busy or missing device from an instance with libvirt < v4.3.0 installed. Expected result =============== The correct error codes are referenced and checked to confirm what happened. Actual result ============= AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING' as the error code is not available prior to libvirt v4.3.0. Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ master 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? libvirt + * 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? N/A 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) N/A Logs & Configs ============== https://zuul.opendev.org/t/openstack/build/2d57acc8c90741e6ba5a6795195e3ffd/log/controller/logs/screen-n-cpu.txt?severity=4 Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server Traceback (most recent call last): Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 273, in dispatch Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 193, in _do_dispatch Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 78, in wrapped Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server function_name, call_dict, binary) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.force_reraise() Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server raise value Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 69, in wrapped Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10199, in external_instance_event Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server event.tag) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10004, in _process_instance_vif_deleted_event Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.driver.detach_interface(context, instance, vif) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2233, in detach_interface Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server alternative_device_name=self.vif_driver.get_vif_devname(vif)) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 452, in detach_device_with_retry Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server _try_detach_device(conf, persistent, live) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 420, in _try_detach_device Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server libvirt.VIR_ERR_DEVICE_MISSING): Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING' Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server Description =========== I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced support for the libvirt hot unplug error code VIR_ERR_DEVICE_MISSING that was itself introduced in libvirt v4.1.0. The change did not however cover versions < v4.1.0 such as v4.0.0 installed in our bionic based CI test envs, causing attribute errors when we attempt to reference. Steps to reproduce ================== * Attempt to detach a busy or missing device from an instance with libvirt < v4.1.0 installed. Expected result =============== The correct error codes are referenced and checked to confirm what happened. Actual result ============= AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING' as the error code is not available prior to libvirt v4.1.0. Environment =========== 1. Exact version of OpenStack you are running. See the following   list for all releases: http://docs.openstack.org/releases/    master 2. Which hypervisor did you use?    (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)    What's the version of that?    libvirt + * 2. Which storage type did you use?    (For example: Ceph, LVM, GPFS, ...)    What's the version of that?    N/A 3. Which networking type did you use?    (For example: nova-network, Neutron with OpenVSwitch, ...)    N/A Logs & Configs ============== https://zuul.opendev.org/t/openstack/build/2d57acc8c90741e6ba5a6795195e3ffd/log/controller/logs/screen-n-cpu.txt?severity=4 Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server Traceback (most recent call last): Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 273, in dispatch Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 193, in _do_dispatch Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 78, in wrapped Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server function_name, call_dict, binary) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.force_reraise() Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server raise value Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 69, in wrapped Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10199, in external_instance_event Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server event.tag) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10004, in _process_instance_vif_deleted_event Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.driver.detach_interface(context, instance, vif) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2233, in detach_interface Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server alternative_device_name=self.vif_driver.get_vif_devname(vif)) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 452, in detach_device_with_retry Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server _try_detach_device(conf, persistent, live) Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 420, in _try_detach_device Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server libvirt.VIR_ERR_DEVICE_MISSING): Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING' Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server
2020-08-17 20:59:33 melanie witt nova: importance Undecided High
2020-08-17 21:00:57 melanie witt nominated for series nova/ussuri
2020-08-17 21:00:57 melanie witt bug task added nova/ussuri
2020-08-21 02:45:43 OpenStack Infra nova: status In Progress Fix Released
2020-08-21 08:24:10 OpenStack Infra nova/ussuri: status New In Progress
2020-08-21 08:24:10 OpenStack Infra nova/ussuri: assignee Lee Yarwood (lyarwood)
2020-08-26 11:50:03 OpenStack Infra nova/ussuri: status In Progress Fix Committed
2020-08-28 09:40:19 OpenStack Infra tags libvirt in-stable-train libvirt
2020-09-02 01:52:55 melanie witt nominated for series nova/queens
2020-09-02 01:52:55 melanie witt bug task added nova/queens
2020-09-02 01:52:55 melanie witt nominated for series nova/train
2020-09-02 01:52:55 melanie witt bug task added nova/train
2020-09-02 01:52:55 melanie witt nominated for series nova/rocky
2020-09-02 01:52:55 melanie witt bug task added nova/rocky
2020-09-02 01:52:55 melanie witt nominated for series nova/stein
2020-09-02 01:52:55 melanie witt bug task added nova/stein
2020-09-02 01:53:32 melanie witt nova/train: importance Undecided High
2020-09-02 01:53:32 melanie witt nova/train: status New Fix Committed
2020-09-02 01:53:32 melanie witt nova/train: assignee Lee Yarwood (lyarwood)
2020-09-02 01:53:43 melanie witt nova/rocky: importance Undecided High
2020-09-02 01:53:45 melanie witt nova/queens: importance Undecided High
2020-09-02 01:53:48 melanie witt nova/ussuri: importance Undecided High
2020-09-02 01:53:50 melanie witt nova/stein: importance Undecided High
2020-09-04 20:30:30 OpenStack Infra nova/stein: status New Fix Committed
2020-09-11 09:56:43 OpenStack Infra nova/rocky: status New In Progress
2020-09-11 09:56:43 OpenStack Infra nova/rocky: assignee Lee Yarwood (lyarwood)
2020-09-24 20:01:10 OpenStack Infra nova/rocky: status In Progress Fix Committed
2020-11-24 13:32:15 Elod Illes nova/stein: status Fix Committed Fix Released
2020-11-24 13:34:28 Elod Illes nova/train: status Fix Committed Fix Released
2020-11-24 14:25:18 Elod Illes nova/ussuri: status Fix Committed Fix Released
2022-11-11 18:09:57 OpenStack Infra nova/queens: status New Fix Released
2022-11-11 18:18:03 OpenStack Infra nova/rocky: status Fix Committed Fix Released