OpenStack Compute (nova)

AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING'

  1. Series ussuri
  2. Bug #1891547
Bug #1891547 reported by Lee Yarwood
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Lee Yarwood
Queens
Fix Released
High
Unassigned
Rocky
Fix Released
High
Lee Yarwood
Stein
Fix Released
High
Unassigned
Train
Fix Released
High
Lee Yarwood
Ussuri
Fix Released
High
Lee Yarwood

Bug Description

Description
===========

I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced support for the libvirt hot unplug error code VIR_ERR_DEVICE_MISSING that was itself introduced in libvirt v4.1.0. The change did not however cover versions < v4.1.0 such as v4.0.0 installed in our bionic based CI test envs, causing attribute errors when we attempt to reference.

Steps to reproduce
==================

* Attempt to detach a busy or missing device from an instance with libvirt < v4.1.0 installed.

Expected result
===============
The correct error codes are referenced and checked to confirm what happened.

Actual result
=============
AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING' as the error code is not available prior to libvirt v4.1.0.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  list for all releases: http://docs.openstack.org/releases/

   master

2. Which hypervisor did you use?
   (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
   What's the version of that?

   libvirt + *

2. Which storage type did you use?
   (For example: Ceph, LVM, GPFS, ...)
   What's the version of that?

   N/A

3. Which networking type did you use?
   (For example: nova-network, Neutron with OpenVSwitch, ...)

   N/A

Logs & Configs
==============

https://zuul.opendev.org/t/openstack/build/2d57acc8c90741e6ba5a6795195e3ffd/log/controller/logs/screen-n-cpu.txt?severity=4

Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 273, in dispatch
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_messaging/rpc/dispatcher.py", line 193, in _do_dispatch
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 78, in wrapped
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.force_reraise()
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server raise value
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/exception_wrapper.py", line 69, in wrapped
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10199, in external_instance_event
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server event.tag)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/compute/manager.py", line 10004, in _process_instance_vif_deleted_event
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server self.driver.detach_interface(context, instance, vif)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2233, in detach_interface
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server alternative_device_name=self.vif_driver.get_vif_devname(vif))
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 452, in detach_device_with_retry
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server _try_detach_device(conf, persistent, live)
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 420, in _try_detach_device
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server libvirt.VIR_ERR_DEVICE_MISSING):
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server AttributeError: module 'libvirt' has no attribute 'VIR_ERR_DEVICE_MISSING'
Aug 13 13:50:50.038271 ubuntu-bionic-inap-mtl01-0019255828 nova-compute[32162]: ERROR oslo_messaging.rpc.server

See original description
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/746210

Changed in nova:
status: New → In Progress
Lee Yarwood (lyarwood)
description: updated
melanie witt (melwitt)
Changed in nova:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/746210
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=bc96af565937072c04dea31781d86d2073b77ed4
Submitter: Zuul
Branch: master

commit bc96af565937072c04dea31781d86d2073b77ed4
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 13 16:56:35 2020 +0100

    libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0

    I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced
    VIR_ERR_DEVICE_MISSING into the hot unplug libvirt error code list
    within detach_device_with_retry. While the change correctly referenced
    that the error code was introduced in v4.1.0 it made no attempt to
    handle versions prior to this. With MIN_LIBVIRT_VERSION currently pinned
    to v4.0.0 we need to handle libvirt < v4.1.0 to avoid referencing the
    non-existent error code within the libvirt module.

    Closes-Bug: #1891547
    Change-Id: I32908b77c18f8ec08211dd67be49bbf903611c34

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/747357

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/747358

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/747360

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/747361

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/747362

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ussuri)

Reviewed: https://review.opendev.org/747357
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3f3b889f4e7e204a140d32d71201c4f23dd54c24
Submitter: Zuul
Branch: stable/ussuri

commit 3f3b889f4e7e204a140d32d71201c4f23dd54c24
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 13 16:56:35 2020 +0100

    libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0

    I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced
    VIR_ERR_DEVICE_MISSING into the hot unplug libvirt error code list
    within detach_device_with_retry. While the change correctly referenced
    that the error code was introduced in v4.1.0 it made no attempt to
    handle versions prior to this. With MIN_LIBVIRT_VERSION currently pinned
    to v4.0.0 we need to handle libvirt < v4.1.0 to avoid referencing the
    non-existent error code within the libvirt module.

    Closes-Bug: #1891547
    Change-Id: I32908b77c18f8ec08211dd67be49bbf903611c34
    (cherry picked from commit bc96af565937072c04dea31781d86d2073b77ed4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/train)

Reviewed: https://review.opendev.org/747358
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c61f4c8e20d712ba84a8965cbe0cba90c7d27d0b
Submitter: Zuul
Branch: stable/train

commit c61f4c8e20d712ba84a8965cbe0cba90c7d27d0b
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 13 16:56:35 2020 +0100

    libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0

    I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced
    VIR_ERR_DEVICE_MISSING into the hot unplug libvirt error code list
    within detach_device_with_retry. While the change correctly referenced
    that the error code was introduced in v4.1.0 it made no attempt to
    handle versions prior to this. With MIN_LIBVIRT_VERSION currently pinned
    to v4.0.0 we need to handle libvirt < v4.1.0 to avoid referencing the
    non-existent error code within the libvirt module.

    NOTE(lyarwood): Conflict as I8e8035dcf508f5215bba9b7575c5c6abfe41da31
    isn't present in stable/train.

    Conflicts:
        nova/virt/libvirt/driver.py

    Closes-Bug: #1891547
    Change-Id: I32908b77c18f8ec08211dd67be49bbf903611c34
    (cherry picked from commit bc96af565937072c04dea31781d86d2073b77ed4)
    (cherry picked from commit 3f3b889f4e7e204a140d32d71201c4f23dd54c24)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/stein)

Reviewed: https://review.opendev.org/747360
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=334a479ae2f4ce3d48dcc4c1b9e14d0cb9822272
Submitter: Zuul
Branch: stable/stein

commit 334a479ae2f4ce3d48dcc4c1b9e14d0cb9822272
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 13 16:56:35 2020 +0100

    libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0

    I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced
    VIR_ERR_DEVICE_MISSING into the hot unplug libvirt error code list
    within detach_device_with_retry. While the change correctly referenced
    that the error code was introduced in v4.1.0 it made no attempt to
    handle versions prior to this. With MIN_LIBVIRT_VERSION currently pinned
    to v4.0.0 we need to handle libvirt < v4.1.0 to avoid referencing the
    non-existent error code within the libvirt module.

    NOTE(lyarwood): Conflict as I2830ccfc81cfa9654cfeac7ad5effc294f523552
    and Idd49b0c70caedfcd42420ffa2ac926a6087d406e are not present in
    stable/stein.

    Conflicts:
        nova/virt/libvirt/driver.py

    Closes-Bug: #1891547
    Change-Id: I32908b77c18f8ec08211dd67be49bbf903611c34
    (cherry picked from commit bc96af565937072c04dea31781d86d2073b77ed4)
    (cherry picked from commit 3f3b889f4e7e204a140d32d71201c4f23dd54c24)
    (cherry picked from commit c61f4c8e20d712ba84a8965cbe0cba90c7d27d0b)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.opendev.org/747361
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9c885b67a9e6c30570084f1f78218defa0278d83
Submitter: Zuul
Branch: stable/rocky

commit 9c885b67a9e6c30570084f1f78218defa0278d83
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 13 16:56:35 2020 +0100

    libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0

    I7eb86edc130d186a66c04b229d46347ec5c0b625 introduced
    VIR_ERR_DEVICE_MISSING into the hot unplug libvirt error code list
    within detach_device_with_retry. While the change correctly referenced
    that the error code was introduced in v4.1.0 it made no attempt to
    handle versions prior to this. With MIN_LIBVIRT_VERSION currently pinned
    to v4.0.0 we need to handle libvirt < v4.1.0 to avoid referencing the
    non-existent error code within the libvirt module.

    NOTE(lyarwood): Conflict as Ia3f1d8e83cbc574ce5cb440032e12bbcb1e10e98
    landed ahead of this change in stable/rocky while it was behind it on
    stable/stein.

    Conflicts:
        nova/virt/libvirt/driver.py

    Closes-Bug: #1891547
    Change-Id: I32908b77c18f8ec08211dd67be49bbf903611c34
    (cherry picked from commit bc96af565937072c04dea31781d86d2073b77ed4)
    (cherry picked from commit 3f3b889f4e7e204a140d32d71201c4f23dd54c24)
    (cherry picked from commit c61f4c8e20d712ba84a8965cbe0cba90c7d27d0b)
    (cherry picked from commit 334a479ae2f4ce3d48dcc4c1b9e14d0cb9822272)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova queens-eol

This issue was fixed in the openstack/nova queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova rocky-eol

This issue was fixed in the openstack/nova rocky-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.