Activity log for bug #1906266

Date Who What changed Old value New value Message
2020-11-30 15:21:31 Peter Sabaini bug added bug
2020-11-30 15:22:25 Peter Sabaini summary libvirt.libvirtError: Requested operation is not valid: format of backing image %s of image %s was not specified After upgrade: "libvirt.libvirtError: Requested operation is not valid: format of backing image %s of image %s was not specified"
2020-11-30 15:26:01 Felipe Reyes bug added subscriber Felipe Reyes
2020-11-30 15:36:16 Peter Sabaini bug added subscriber Canonical IS BootStack
2020-11-30 22:32:12 Dominique Poulain bug added subscriber Dominique Poulain
2020-12-15 17:42:19 Stephen Finucane tags libvirt
2020-12-17 11:41:01 Stephen Finucane nova: status New Won't Fix
2020-12-21 14:51:13 Peter Sabaini bug task added nova (Ubuntu)
2021-01-04 18:18:08 Felipe Reyes bug task added libvirt (Ubuntu)
2021-01-04 18:18:49 Felipe Reyes nominated for series Ubuntu Focal
2021-01-04 18:18:49 Felipe Reyes bug task added libvirt (Ubuntu Focal)
2021-01-04 18:18:49 Felipe Reyes bug task added nova (Ubuntu Focal)
2021-01-04 18:18:49 Felipe Reyes nominated for series Ubuntu Groovy
2021-01-04 18:18:49 Felipe Reyes bug task added libvirt (Ubuntu Groovy)
2021-01-04 18:18:49 Felipe Reyes bug task added nova (Ubuntu Groovy)
2021-01-04 18:19:10 Felipe Reyes libvirt (Ubuntu Groovy): status New Invalid
2021-01-05 12:32:12 Christian Ehrhardt  libvirt (Ubuntu): status New Fix Released
2021-01-05 12:51:00 Launchpad Janitor merge proposal linked https://code.launchpad.net/~paelzer/ubuntu/+source/libvirt/+git/libvirt/+merge/395778
2021-01-05 12:53:46 Christian Ehrhardt  libvirt (Ubuntu Focal): status New Triaged
2021-01-05 13:51:08 Christian Ehrhardt  bug added subscriber Christian Ehrhardt 
2021-06-24 14:53:27 Christian Ehrhardt  tags libvirt libvirt server-next
2021-06-29 07:26:35 Christian Ehrhardt  description In a site upgraded to Ussuri we are getting faults starting instances 2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server libvirt.libvirtError: Requested operation is not valid: format of backing image '/var/lib/nova/instances/_base/xxx' of image '/var/lib/nova/instances/xxx' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting) Bug #1864020 reports similar symptoms, where due to an upstream change in Libvirt v6.0.0+ images need the backing format specified. The fix for Bug #1864020 handles the case for new instances. However, for upgraded instances we're hitting the same problem, as those still don't have backing format specified. [Impact] * New libvirt got more strict in regard to file format specification. While this is generally the right approach it causes some issues for upgraders that have old image chains now failing. * Upstream has added code to relax those checks under a set of conditions which will allow to go forward with stricter conditions as planned but at the same time not break/block upgrades. [Test Plan] * TBD [Where problems could occur] * Of the many things that qemu/libvirt do this changes only the format probing. So issues (hopefully not) would be expected to appear mostly around complex scenarios of image files. We've had a look at image files and image file chains, and so far all were good. But there are more obscure (and not supported) cases like image backed by real-disk that might misbehave. But still it would fix Focal to be the outlier as the past was ok (didn't care) and the future (relaxed check) and only focal is left broken in between. [Other Info] * A lot has changes in that area, but instead of pulling in a vast set of changes a smaller set was identified to suite the SRU needs. It was so far found not found regressing anything and OTOH fixed the issue (tested form PPA) for affected people. ---- In a site upgraded to Ussuri we are getting faults starting instances 2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server libvirt.libvirtError: Requested operation is not valid: format of backing image '/var/lib/nova/instances/_base/xxx' of image '/var/lib/nova/instances/xxx' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting) Bug #1864020 reports similar symptoms, where due to an upstream change in Libvirt v6.0.0+ images need the backing format specified. The fix for Bug #1864020 handles the case for new instances. However, for upgraded instances we're hitting the same problem, as those still don't have backing format specified.
2021-06-29 07:28:42 Christian Ehrhardt  description [Impact] * New libvirt got more strict in regard to file format specification. While this is generally the right approach it causes some issues for upgraders that have old image chains now failing. * Upstream has added code to relax those checks under a set of conditions which will allow to go forward with stricter conditions as planned but at the same time not break/block upgrades. [Test Plan] * TBD [Where problems could occur] * Of the many things that qemu/libvirt do this changes only the format probing. So issues (hopefully not) would be expected to appear mostly around complex scenarios of image files. We've had a look at image files and image file chains, and so far all were good. But there are more obscure (and not supported) cases like image backed by real-disk that might misbehave. But still it would fix Focal to be the outlier as the past was ok (didn't care) and the future (relaxed check) and only focal is left broken in between. [Other Info] * A lot has changes in that area, but instead of pulling in a vast set of changes a smaller set was identified to suite the SRU needs. It was so far found not found regressing anything and OTOH fixed the issue (tested form PPA) for affected people. ---- In a site upgraded to Ussuri we are getting faults starting instances 2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server libvirt.libvirtError: Requested operation is not valid: format of backing image '/var/lib/nova/instances/_base/xxx' of image '/var/lib/nova/instances/xxx' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting) Bug #1864020 reports similar symptoms, where due to an upstream change in Libvirt v6.0.0+ images need the backing format specified. The fix for Bug #1864020 handles the case for new instances. However, for upgraded instances we're hitting the same problem, as those still don't have backing format specified. [Impact]  * New libvirt got more strict in regard to file format specification.    While this is generally the right approach it causes some issues for    upgraders that have old image chains now failing.  * Upstream has added code to relax those checks under a set of conditions    which will allow to go forward with stricter conditions as planned but    at the same time not break/block upgrades. [Test Plan]  * Thanks to Brett Milford for sharing his test steps for this sudo apt-get update sudo apt-get install libvirt-daemon-system cloud-image-utils virtinst -y IMG="focal-server-cloudimg-amd64.img" IMG_PATH="/var/lib/libvirt/images/base/$IMG" INSTANCE_NAME=testinst [ -f $IMG_PATH ] || { sudo mkdir -p /var/lib/libvirt/images/base sudo wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img \ -O $IMG_PATH } sudo mkdir -p /var/lib/libvirt/images/$INSTANCE_NAME sudo qemu-img convert -O raw $IMG_PATH ${IMG_PATH%.*} sudo qemu-img create -f qcow2 -o backing_file=${IMG_PATH%.*} /var/lib/libvirt/images/$INSTANCE_NAME/root.img sudo qemu-img resize /var/lib/libvirt/images/$INSTANCE_NAME/root.img 5G virt-install --connect qemu:///system --name $INSTANCE_NAME --cpu host --os-type linux --os-variant generic --graphics vnc --console pty,target_type=serial --disk path=/var/lib/libvirt/images/$INSTANCE_NAME/root.img,bus=virtio,format=qcow2 --network default,model=virtio --noautoconsole --vcpus 1 --memory 1024 --import [Where problems could occur]  * Of the many things that qemu/libvirt do this changes only the format    probing. So issues (hopefully not) would be expected to appear mostly    around complex scenarios of image files.    We've had a look at image files and image file chains, and so far all    were good. But there are more obscure (and not supported) cases like    image backed by real-disk that might misbehave. But still it would    fix Focal to be the outlier as the past was ok (didn't care) and the    future (relaxed check) and only focal is left broken in between. [Other Info]  * A lot has changes in that area, but instead of pulling in a vast set    of changes a smaller set was identified to suite the SRU needs. It    was so far found not found regressing anything and OTOH fixed the issue    (tested form PPA) for affected people. ---- In a site upgraded to Ussuri we are getting faults starting instances 2020-11-30 13:41:40.586 232871 ERROR oslo_messaging.rpc.server libvirt.libvirtError: Requested operation is not valid: format of backing image '/var/lib/nova/instances/_base/xxx' of image '/var/lib/nova/instances/xxx' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting) Bug #1864020 reports similar symptoms, where due to an upstream change in Libvirt v6.0.0+ images need the backing format specified. The fix for Bug #1864020 handles the case for new instances. However, for upgraded instances we're hitting the same problem, as those still don't have backing format specified.
2021-06-30 14:25:07 Christian Ehrhardt  libvirt (Ubuntu Groovy): status Invalid Fix Released
2021-06-30 14:25:10 Christian Ehrhardt  libvirt (Ubuntu Focal): status Triaged In Progress
2021-06-30 14:33:52 Robie Basak libvirt (Ubuntu Focal): status In Progress Fix Committed
2021-06-30 14:33:56 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2021-06-30 14:33:59 Robie Basak bug added subscriber SRU Verification
2021-06-30 14:34:06 Robie Basak tags libvirt server-next libvirt server-next verification-needed verification-needed-focal
2021-07-12 23:44:41 Brett Milford tags libvirt server-next verification-needed verification-needed-focal libvirt server-next verification-done-focal verification-needed
2021-07-13 08:36:11 Christian Ehrhardt  tags libvirt server-next verification-done-focal verification-needed libvirt server-next verification-done verification-done-focal
2021-07-13 08:43:09 Brett Milford bug task added cloud-archive
2021-07-13 08:43:39 Brett Milford nominated for series cloud-archive/ussuri
2021-07-13 08:43:39 Brett Milford bug task added cloud-archive/ussuri
2021-07-13 16:02:32 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2021-07-13 16:18:41 Launchpad Janitor libvirt (Ubuntu Focal): status Fix Committed Fix Released
2021-07-14 12:14:31 Corey Bryant cloud-archive/ussuri: status New Fix Committed
2021-07-14 12:14:34 Corey Bryant tags libvirt server-next verification-done verification-done-focal libvirt server-next verification-done verification-done-focal verification-ussuri-needed
2021-07-28 23:16:17 Brian Murray nova (Ubuntu Groovy): status New Won't Fix
2021-07-31 07:13:13 Brett Milford tags libvirt server-next verification-done verification-done-focal verification-ussuri-needed libvirt server-next verification-done verification-done-focal verification-ussuri-done
2021-09-15 15:16:30 Christian Ehrhardt  tags libvirt server-next verification-done verification-done-focal verification-ussuri-done libvirt verification-done verification-done-focal verification-ussuri-done
2021-09-24 02:08:44 Erwin Dwight Paler bug added subscriber SBI BITS