I've been digging into this issue
- with other images (official cirros release)
- with downgraded qemu packages
and got the same situation - time by time not all files were injected.
I guess that root cause is in nbd module.
On compute node /var/log/sudo.log or /var/log/nova/nova-compute.log (with debug=true) have sequence of commands
with return code 0 for all, but at the same time we get errors in /var/log/syslog - attempts to send on closed socket.
So socket is closing earlier than "blockdev" and "umoutn" flush buffers into the disk.
But, now this test is 100% passed on MOS 9.0 (build 376) and even old ISOs (where it failed before) with different images.
So as we can't reproduce this issue - I moved to Incomplite. Please, reopen ticket if issue is reproduced again.
I've been digging into this issue
- with other images (official cirros release)
- with downgraded qemu packages
and got the same situation - time by time not all files were injected.
I guess that root cause is in nbd module. nova/nova- compute. log (with debug=true) have sequence of commands
On compute node /var/log/sudo.log or /var/log/
qemu-nbd -c /dev/nbd12 /var/lib/ nova/instances/ 16e7efb2- ccbb-411d- 83f4-c629dffcc1 52/disk vfs-localfsXC7I qB vfs-localfsXC7I qB/etc vfs-localfsXC7I qB/etc vfs-localfsXC7I qB/etc/ test0.txt vfs-localfsXC7I qB/etc/ test0.txt vfs-localfsXC7I qB/etc vfs-localfsXC7I qB/etc
mount /dev/nbd12p1 /tmp/openstack-
readlink -nm /tmp/openstack-
readlink -e /tmp/openstack-
readlink -nm /tmp/openstack-
tee /tmp/openstack-
readlink -nm /tmp/openstack-
readlink -e /tmp/openstack-
...
blockdev --flushbufs /dev/nbd12
umount /dev/nbd12p1
qemu-nbd -d /dev/nbd12
with return code 0 for all, but at the same time we get errors in /var/log/syslog - attempts to send on closed socket.
So socket is closing earlier than "blockdev" and "umoutn" flush buffers into the disk.
But, now this test is 100% passed on MOS 9.0 (build 376) and even old ISOs (where it failed before) with different images.
So as we can't reproduce this issue - I moved to Incomplite. Please, reopen ticket if issue is reproduced again.