Ephemeral with vfat format fails inspection

[Dan Smith]

When configured to format ephemerals as vfat, we get this failure:

Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.358 2 DEBUG oslo_utils.imageutils.format_inspector [None req-fcf3a278-3417-4a6d-8b10-66e91ca1677d 60ed4d3e522640b6ad19633b28c5b5bb ae43aec9c3c242a785c8256abdda1747 - - default default] Format inspector failed, aborting: Signature KDMV not found: b'\xebX\x90m' _process_chunk /usr/lib/python3.9/site-packages/oslo_utils/imageutils/format_inspector.py:1302
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.365 2 DEBUG oslo_utils.imageutils.format_inspector [None req-fcf3a278-3417-4a6d-8b10-66e91ca1677d 60ed4d3e522640b6ad19633b28c5b5bb ae43aec9c3c242a785c8256abdda1747 - - default default] Format inspector failed, aborting: Region signature not found at 30000 _process_chunk /usr/lib/python3.9/site-packages/oslo_utils/imageutils/format_inspector.py:1302
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.366 2 WARNING oslo_utils.imageutils.format_inspector [None req-fcf3a278-3417-4a6d-8b10-66e91ca1677d 60ed4d3e522640b6ad19633b28c5b5bb ae43aec9c3c242a785c8256abdda1747 - - default default] Safety check mbr on gpt failed because GPT MBR has no partitions defined: oslo_utils.imageutils.format_inspector.SafetyViolation: GPT MBR has no partitions defined
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.366 2 WARNING nova.virt.libvirt.imagebackend [None req-fcf3a278-3417-4a6d-8b10-66e91ca1677d 60ed4d3e522640b6ad19633b28c5b5bb ae43aec9c3c242a785c8256abdda1747 - - default default] Base image /var/lib/nova/instances/_base/ephemeral_1_0706d66 failed safety check: Safety checks failed: mbr: oslo_utils.imageutils.format_inspector.SafetyCheckFailed: Safety checks failed: mbr
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [None req-fcf3a278-3417-4a6d-8b10-66e91ca1677d 60ed4d3e522640b6ad19633b28c5b5bb ae43aec9c3c242a785c8256abdda1747 - - default default] [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] Instance failed to spawn: nova.exception.InvalidDiskInfo: Disk info file is invalid: Base image failed safety check
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] Traceback (most recent call last):
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/imagebackend.py", line 685, in create_image
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] inspector.safety_check()
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/oslo_utils/imageutils/format_inspector.py", line 430, in safety_check
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] raise SafetyCheckFailed(failures)
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] oslo_utils.imageutils.format_inspector.SafetyCheckFailed: Safety checks failed: mbr
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a]
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] During handling of the above exception, another exception occurred:
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a]
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] Traceback (most recent call last):
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 2894, in _build_resources
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] yield resources
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 2641, in _build_and_run_instance
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] self.driver.spawn(context, instance, image_meta,
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 4734, in spawn
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] created_instance_dir, created_disks = self._create_image(
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line 5198, in _create_image
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] disk_image.cache(fetch_func=fn,
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/imagebackend.py", line 301, in cache
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] self.create_image(fetch_func_sync, base, size,
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] File "/usr/lib/python3.9/site-packages/nova/virt/libvirt/imagebackend.py", line 691, in create_image
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] raise exception.InvalidDiskInfo(
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a] nova.exception.InvalidDiskInfo: Disk info file is invalid: Base image failed safety check
Sep 03 17:34:28 compute-2 nova_compute[133243]: 2024-09-03 17:34:28.367 2 ERROR nova.compute.manager [instance: 263ccd01-10b1-46a6-9f81-a6fc27c7177a]

There are a number of problems here:

1. Nova shouldn't need to inspect blank disks it has created by itself, but because of the nature of the imagebackend, that code can't tell the difference
2. FAT actually starts with a VBR which looks like an MBR because it's the same data structure and signature, but without any partition entries. This leftover from the 1980s is inconvenient for us, but the format_inspector should try to avoid detecting these FAT images as GPT/MBR

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/928462

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/928310
Committed: https://opendev.org/openstack/nova/commit/e310049eab7dc3d05a25a37c68fec3b34d621c33
Submitter: "Zuul (22348)"
Branch: master

commit e310049eab7dc3d05a25a37c68fec3b34d621c33
Author: Sean Mooney <email address hidden>
Date: Fri Sep 6 03:00:54 2024 +0100

    Add functional repoducer for ephemeral disks

    This commit add a functional repoducer for a regression
    in ephmeral disk image format validation

    When we changed to use the oslo.utils verison of the
    format_inspector we aquired a new inspector which can
    detect if a file has a partition table.

    As the qcow 2 backend does not know that ephemeral disk
    or swap disk are not boot able it requires that they
    also have partition tables.

    Related-Bug: #2079850
    Change-Id: I26c6e8d2efbf7403542e99b8ac45e2a31f08afd1

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/928829
Committed: https://opendev.org/openstack/nova/commit/8de15e9a276dc4261dd0656e26ca5a917825f441
Submitter: "Zuul (22348)"
Branch: master

commit 8de15e9a276dc4261dd0656e26ca5a917825f441
Author: Sean Mooney <email address hidden>
Date: Tue Sep 10 14:41:15 2024 +0100

    only safety check bootable files created from glance

    For blank files that are created by nova such as swap
    disks and ephemeral disks we do not need need to safety
    check them as they always are just bare filesystems.

    In the future we should refactor the qcow imagebackend to
    not require backing files for swap and ephemeral disks
    but for now we simply disable the check to workaround
    the addition of the gpt image inspector and the incompatiblity
    with vfat. future versions of oslo will account for vfat boot
    recored. this is a minimal patch to avoid needing a new oslo
    release for 2024.2

    Closes-Bug: #2079850
    Change-Id: I7df3d9859aa4be3a012ff919f375a7a3d9992af4

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 30.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/oslo.utils/+/928448
Committed: https://opendev.org/openstack/oslo.utils/commit/3c33e37d64e44addc9a818bd556f5919ed2e9002
Submitter: "Zuul (22348)"
Branch: master

commit 3c33e37d64e44addc9a818bd556f5919ed2e9002
Author: Dan Smith <email address hidden>
Date: Fri Sep 6 07:51:13 2024 -0700

    Avoid detecting FAT VBR as an MBR

    The 1980s FAT filesystem has a VBR in the first sector, which looks
    almost exactly like an MBR with zero partitions. To avoid detecting
    these as MBRs, look for some extra attributes that indicate that the
    structure is a VBR and avoid matching it as a GPT/MBR in that case.

    We can add an inspector for this as a separate thing, but at the
    moment we don't have that immediate need.

    Closes-Bug: #2079850
    Change-Id: Ibad87743b5a3b6469bd708d4caafe7911b045855

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.utils 7.4.0 release.