Broken ubuntu-binary-nova-compute:pike No module named libvirt

[Markus Adam]

Hello, I'm trying to deploy Kolla Ansible based on Ubuntu Xenial, all-in-one mode.
However nova_compute container refuses to start.
nova compute process reports issue "ImportError: No module named libvirt"

Detailed error message:
# tail -20 /var/lib/docker/volumes/kolla_logs/_data/nova/nova-compute.log
2018-02-16 13:03:33.059 8 ERROR nova.virt.driver libvirt = importutils.import_module('libvirt')
2018-02-16 13:03:33.059 8 ERROR nova.virt.driver File "/usr/lib/python2.7/dist-packages/oslo_utils/importutils.py", line 73, in import_module
2018-02-16 13:03:33.059 8 ERROR nova.virt.driver __import__(import_str)
2018-02-16 13:03:33.059 8 ERROR nova.virt.driver ImportError: No module named libvirt
2018-02-16 13:03:33.059 8 ERROR nova.virt.driver
2018-02-16 13:07:02.823 7 INFO os_vif [-] Loaded VIF plugins: ovs, linux_bridge
2018-02-16 13:07:02.824 7 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2018-02-16 13:07:03.190 7 INFO nova.virt.driver [req-cf81278e-ad5a-4290-be00-40315726587b - - - - -] Loading compute driver 'libvirt.LibvirtDriver'
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver [req-cf81278e-ad5a-4290-be00-40315726587b - - - - -] Unable to load the virtualization driver: ImportError: No module named libvirt
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver Traceback (most recent call last):
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver File "/usr/lib/python2.7/dist-packages/nova/virt/driver.py", line 1638, in load_compute_driver
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver virtapi)
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver File "/usr/lib/python2.7/dist-packages/oslo_utils/importutils.py", line 44, in import_object
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver return import_class(import_str)(*args, **kwargs)
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 312, in __init__
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver libvirt = importutils.import_module('libvirt')
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver File "/usr/lib/python2.7/dist-packages/oslo_utils/importutils.py", line 73, in import_module
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver __import__(import_str)
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver ImportError: No module named libvirt
2018-02-16 13:07:03.594 7 ERROR nova.virt.driver

Environment details:
PIP output:
kolla (5.0.1)
kolla-ansible (5.0.1)

Docker inspect output:

           "Env": [
                "KOLLA_SERVICE_NAME=nova-compute",
                "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/lib/udev",
                "PIP_INDEX_URL=http://mirror.mtl01.inap.openstack.org:8080/pypi/simple",
                "PIP_TRUSTED_HOST=mirror.mtl01.inap.openstack.org",
                "KOLLA_BASE_DISTRO=ubuntu",
                "KOLLA_INSTALL_TYPE=binary",
                "KOLLA_INSTALL_METATYPE=rdo",
                "PS1=$(tput bold)($(printenv KOLLA_SERVICE_NAME))$(tput sgr0)[$(id -un)@$(hostname -s) $(pwd)]$ ",
                "DEBIAN_FRONTEND=noninteractive"
            ],
            "Cmd": [
                "kolla_start"
            ],
            "Image": "kolla/ubuntu-binary-nova-compute:pike",

[Markus Adam (markus.adam)]

Looks like this error is similar to this one:
https://bugs.launchpad.net/kolla/+bug/1642640
https://review.openstack.org/#/c/399145/

Dockerfile is missing python-libvirt package.

https://github.com/openstack/kolla/blob/stable/pike/docker/nova/nova-compute/Dockerfile.j2

[Eduardo Gonzalez (egonzalez90)]

Hi, in libvirt should be a dependency of nova packages, in source installs we define such package, but in binary it was always a dependency of nova packages.

A quick look at nova packages, nova-compute (installed) have a nova-compute-kvm as dependency which have python-libvirt as dependency itself.

IMO is a ubuntu bug.

When were the images built?

[Markus Adam (markus.adam)]

Hi Eduardo, agree that having nova-compute-kvm one may expect to have python-libvirt installed too. However looks this is not the case :(
Images were fetched from Docker.io repo, not built by myself.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[loonatic (albert-u)]

I can confirm that the (binary) Docker.io repo nova-compute image from 3 days ago fails on account of the missing libvirt package. Rebuilding nova-compute from master kolla-ansible with python-libvirt in Dockerfile.j2, then redeploying will fix the issue.

Just curious, shouldn't a CI/CD run have caught these kind of obvious errors ?

[James Page (james-page)]

nova-compute-libvirt Depends on python-libvirt
nova-compute-kvm Depends on nova-compute-libvirt
nova-compute Depends on nova-compute-kvm | nova-compute-hypervisor

so unless you have another package installed that provides nova-compute-hypervisor, you should get nova-compute-kvm by default

[Corey Bryant (corey.bryant)]

The following binary packages provide the nova-compute-hypervisor virtual package. Please can you let us know which ones are installed?

nova-compute-kvm, nova-compute-qemu, nova-compute-vmware, nova-compute-xen, nova-compute-lxc, nova-compute-lxd

[Randy (randymartini)]

The Pike version of Dockerfile.j2 adds these files to the libvirt container.

        'ceph-common',
        'ebtables',
        'libvirt-clients',
        'libvirt-daemon-system',
        'openvswitch-switch',
        'pm-utils',
        'qemu',
        'qemu-block-extra'
You can view the file here: https://github.com/openstack/kolla/blob/stable/pike/docker/nova/nova-libvirt/Dockerfile.j2

The apt install of libvirt-daemon-system files to install the libvirt-daemon. This is using the Ubuntu Pike PPA.

[Bo Quan (bo-quan)]

We install the following package:

ubuntu-source-nova-compute

[Corey Bryant (corey.bryant)]

It'd be useful if we could get a full list of installed packages. ubuntu-source-nova-compute isn't an Ubuntu package and I can't tell what nova-compute-* packages are installed. I think you can capture this info automatically by running 'apport-collect 1749944'.

[Launchpad Janitor (janitor)]

[Expired for nova (Ubuntu) because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for kolla because there has been no activity for 60 days.]