[nova] Command 'sudo nova-rootwrap /etc/nova/rootwrap.conf kpartx -a /dev/nbd9' failed

[Vladimir Khlyunev]

CI job failed: https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.bvt_ubuntu_bootstrap/74/
ISO 8.0-478
Error is not related to test scenario and hard-reproducible
Snapshot is attached
OSTF test "Launch instance with file injection" failed while building an instance. In nova-compute.log I found the following message:
http://paste.openstack.org/show/484868/

MOS-Nova team, please help with investigation - this is very annoying error (the second run of this tests always pass)

[Roman Podoliaka (rpodolyaka)]

So based on the feedback from

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1271205

and

https://bugs.launchpad.net/nova/+bug/1024586

this is a long standing issue of device-mapper/kpartx.

Curiously, this only affects cirros qcow2 images:

http://paste.openstack.org/show/484902/

While Ubuntu cloud image (also qcow2) works just fine:

http://paste.openstack.org/show/484903/

as well as cirros image converted to raw format:

http://paste.openstack.org/show/484904/

FWIW, we should not be using nbd for file injection and should use libguestfs instead, although it does not work out of the box on Ubuntu Trusty either and we have to config it properly (the most annoying part is to change access mode on host kernel images).

The bottom line is, this is a valid issue, I'm not convinced it's important enough to be fixed in 8.0 based on the user impact described above.

At the same time I agree we should somehow fix this for the sake of CI. The only part I don't currently understand is how this works in Nova in the second test run - on my ubuntu machine kpartx call fails every single time on cirros qcow2 images.

[Roman Podoliaka (rpodolyaka)]

Nothing changed in the code, but looks like we no longer see it on CI. Move to 10.0 just in case, but nothing should be done in 9.0.

[Roman Podoliaka (rpodolyaka)]

This is effectively fixed in https://review.openstack.org/#/c/322132/ (we explicitly disable file injection by the means of nbd/libguestfs and rely on config-drive + cloud-init for that)