Failure to detach volume during Tempest test cleanup due to APIC related kernel panic within the guest OS

Bug #1939108 reported by Lee Yarwood
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
High
Lee Yarwood
devstack
Fix Released
Undecided
Lee Yarwood

Bug Description

Creating a fresh bug for this as we haven't seen it in a while since the libvirt event rewrite landed but it did just cause a gate failure...

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5d6/801990/6/gate/tempest-integrated-compute/5d6a7ec/testr_results.html

13 Aug 05 16:01:08.964854 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR nova.virt.libvirt.driver [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempe st-AttachVolumeNegativeTest-1260221999-project] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f is timed out.
47114 Aug 05 16:01:08.969793 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: DEBUG nova.virt.libvirt.driver [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempe st-AttachVolumeNegativeTest-1260221999-project] Failed to detach device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Libvirt did not re port any error but the device is still in the config. {{(pid=104299) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2394}}
47115 Aug 05 16:01:08.970117 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR nova.virt.libvirt.driver [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempe st-AttachVolumeNegativeTest-1260221999-project] Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. De vice is still attached to the guest.
47116 Aug 05 16:01:08.971007 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: WARNING nova.virt.block_device [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempe st-AttachVolumeNegativeTest-1260221999-project] [instance: ce4b444f-1587-43e9-b708-d2ae2c9cd59f] Guest refused to detach volume ff8d0867-1e92-4f04-8e52-01bf05f52ac3: nova.exception.DeviceDetachFailed: Devi ce detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.
47117 Aug 05 16:01:09.024848 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: DEBUG oslo_concurrency.lockutils [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tem pest-AttachVolumeNegativeTest-1260221999-project] Lock "ce4b444f-1587-43e9-b708-d2ae2c9cd59f" released by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 201.180s {{(p id=104299) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}}
47118 Aug 05 16:01:09.095134 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR oslo_messaging.rpc.server [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 temp est-AttachVolumeNegativeTest-1260221999-project] Exception during message handling: nova.exception.DeviceDetachFailed: Device detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.

$ logsearch storedsearch bug-xxx-detach
Running stored search:
  bug-xxx-detach:
    branches:
    - master
    files:
    - controller/logs/screen-n-cpu.txt
    job-groups:
    - nova-devstack
    limit: 100
    project: openstack/nova
    regex: from the live domain config. Device is still attached to the guest.
    result: FAILURE
[..]
5d6a7ec36f104f10a1f49fe44909aa23:.logsearch/5d6a7ec36f104f10a1f49fe44909aa23/controller/logs/screen-n-cpu.txt:47115:Aug 05 16:01:08.970117 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR nova.virt.libvirt.driver [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempest-AttachVolumeNegativeTest-1260221999-project] Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.
5d6a7ec36f104f10a1f49fe44909aa23:.logsearch/5d6a7ec36f104f10a1f49fe44909aa23/controller/logs/screen-n-cpu.txt:47116:Aug 05 16:01:08.971007 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: WARNING nova.virt.block_device [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempest-AttachVolumeNegativeTest-1260221999-project] [instance: ce4b444f-1587-43e9-b708-d2ae2c9cd59f] Guest refused to detach volume ff8d0867-1e92-4f04-8e52-01bf05f52ac3: nova.exception.DeviceDetachFailed: Device detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.
5d6a7ec36f104f10a1f49fe44909aa23:.logsearch/5d6a7ec36f104f10a1f49fe44909aa23/controller/logs/screen-n-cpu.txt:47118:Aug 05 16:01:09.095134 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR oslo_messaging.rpc.server [None req-54714613-ee7e-4c9f-8e60-8251a77a526e tempest-AttachVolumeNegativeTest-1260221999 tempest-AttachVolumeNegativeTest-1260221999-project] Exception during message handling: nova.exception.DeviceDetachFailed: Device detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.
5d6a7ec36f104f10a1f49fe44909aa23:.logsearch/5d6a7ec36f104f10a1f49fe44909aa23/controller/logs/screen-n-cpu.txt:47170:Aug 05 16:01:09.101298 ubuntu-focal-ovh-gra1-0025782813 nova-compute[104299]: ERROR oslo_messaging.rpc.server nova.exception.DeviceDetachFailed: Device detach failed for vdb: Run out of retry while detaching device vdb with device alias virtio-disk1 from instance ce4b444f-1587-43e9-b708-d2ae2c9cd59f from the live domain config. Device is still attached to the guest.

Builds with matching logs 1/100:
+----------------------------------+---------------------+-----------------------------------+----------+----------------------------+
| uuid | finished | review | pipeline | job |
+----------------------------------+---------------------+-----------------------------------+----------+----------------------------+
| 5d6a7ec36f104f10a1f49fe44909aa23 | 2021-08-05T16:25:36 | https://review.opendev.org/801990 | gate | tempest-integrated-compute |
+----------------------------------+---------------------+-----------------------------------+----------+----------------------------+

Revision history for this message
Lee Yarwood (lyarwood) wrote :
Download full text (7.5 KiB)

[ 0.000000] Linux version 5.3.0-26-generic (buildd@lgw01-amd64-039) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #28~18.04.1-Ubuntu SMP Wed Dec 18 16:40:14 UTC 2019 (Ubuntu 5.3.0-26.28~18.04.1-generic 5.3.13)
[ 0.000000] Command line: LABEL=cirros-rootfs ro console=tty1 console=ttyS0
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] zhaoxin Shanghai
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000007fdcfff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000007fdd000-0x0000000007ffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: OpenStack Foundation OpenStack Nova, BIOS 1.13.0-1ubuntu1.1 04/01/2014
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 2394.452 MHz processor
[ 0.026481] last_pfn = 0x7fdd max_arch_pfn = 0x400000000
[ 0.028543] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.059293] found SMP MP-table at [mem 0x000f5c90-0x000f5c9f]
[ 0.080002] check: Scanning 1 areas for low memory corruption
[ 0.087928] RAMDISK: [mem 0x0798e000-0x07fccfff]
[ 0.088855] ACPI: Early table checksum verification disabled
[ 0.089662] ACPI: RSDP 0x00000000000F5AA0 000014 (v00 BOCHS )
[ 0.090108] ACPI: RSDT 0x0000000007FE153C 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.091295] ACPI: FACP 0x0000000007FE1418 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.092403] ACPI: DSDT 0x0000000007FE0040 0013D8 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.092593] ACPI: FACS 0x0000000007FE0000 000040
[ 0.092735] ACPI: APIC 0x0000000007FE148C 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.092800] ACPI: HPET 0x0000000007FE1504 000038 (v01 BOCHS BXPCHPET 00000001 BXPC 00000001)
[ 0.096602] No NUMA configuration found
[ 0.096673] Faking a node at [mem 0x0000000000000000-0x0000000007fdcfff]
[ 0.097804] NODE_DATA(0) allocated [mem 0x07963000-0x0798dfff]
[ 0.104051] Zone ranges:
[ 0.104153] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.104229] DMA32 [mem 0x0000000001000000-0x0000000007fdcfff]
[ 0.104250] Normal empty
[ 0.104279] Device empty
[ 0.104302] Movable zone start for each node
[ 0.104370] Early memory node ranges
[ 0.104422] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.104679] node 0: [mem 0x0000000000100000-0x0000000007fdcfff]
[ 0.105220] Zeroed struct page in unavailable ranges: 98 pages
[ 0.105486] Initmem setup node 0 [mem 0x0000000000001000-0x0000000007fdcfff]
[ 0.113353] ACPI: PM-Timer IO Port: 0x608
[ 0.11...

Read more...

Revision history for this message
Lee Yarwood (lyarwood) wrote :
Changed in nova:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Lee Yarwood (lyarwood)
tags: added: volumes
Revision history for this message
Lee Yarwood (lyarwood) wrote :
Lee Yarwood (lyarwood)
summary: - tempest.api.compute.volumes.test_attach_volume_negative.AttachVolumeNegativeTest
- fails due to detach failure during cleanup
+ Failure to detach volume during Tempest test cleanup due to APIC related
+ kernel panic within the guest OS
Revision history for this message
Lee Yarwood (lyarwood) wrote :
Revision history for this message
Lee Yarwood (lyarwood) wrote :

https://github.com/cirros-dev/cirros/issues/69 - raised to see if there's any interest in adding the kernel command line arg by default to Cirros.

Revision history for this message
Lee Yarwood (lyarwood) wrote :
Lee Yarwood (lyarwood)
Changed in devstack:
assignee: nobody → Lee Yarwood (lyarwood)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.opendev.org/c/openstack/nova/+/766043
Committed: https://opendev.org/openstack/nova/commit/ec48e1523dfdd0f1031e1a70c28c56962db66e8a
Submitter: "Zuul (22348)"
Branch: master

commit ec48e1523dfdd0f1031e1a70c28c56962db66e8a
Author: Sean Mooney <email address hidden>
Date: Tue Dec 8 16:42:52 2020 +0000

    workarounds: Add libvirt_disable_apic

    This change adds a new workaround config option to allow disabling the
    guest apic.

    Co-authored-by: Lee Yarwood <email address hidden>
    Related-Bug: #1939108
    Change-Id: I2da867f2734b590a884b1fe1200c402cbf7e9e1c

Changed in devstack:
status: New → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to devstack (master)

Reviewed: https://review.opendev.org/c/openstack/devstack/+/805219
Committed: https://opendev.org/openstack/devstack/commit/1e86a25cc28e34d7f73a4c6ccbbc3fc667598d50
Submitter: "Zuul (22348)"
Branch: master

commit 1e86a25cc28e34d7f73a4c6ccbbc3fc667598d50
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 19 14:24:28 2021 +0100

    nova: Enable apic removal workaround for bug #1939108 when using QEMU

    This change enables [workarounds]libvirt_disable_apic when devstack is
    deployed using the libvirt virt driver and qemu virt type in an effort
    to avoid issues outlined in bug #1939108 caused by the older kernel
    currently used in Cirros 0.5.2.

    Depends-On: https://review.opendev.org/c/openstack/nova/+/766043
    Closes-Bug: #1939108
    Change-Id: Ibb6c34133bb1c95ef11cc59d9b12a0f65502c61b

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/805628

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/805629

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/805630

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/805631

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to devstack (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/devstack/+/805632

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to devstack (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/devstack/+/805633

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to devstack (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/devstack/+/805634

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to devstack (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/devstack/+/805636

Revision history for this message
Lee Yarwood (lyarwood) wrote :

Still seeing the odd hit of this but nothing like we were a few weeks ago:

$ logsearch storedsearch bug-apic
Running stored search:
  bug-apic:
    branches:
    - master
    files:
    - testr_results.html
    job-groups:
    - nova-devstack
    limit: 100
    project: openstack/nova
    regex: 'Kernel panic - not syncing: IO-APIC'
    result: FAILURE
[..]
Searching logs:
30946b8e90fb4aeaa200d7847b253780:.logsearch/30946b8e90fb4aeaa200d7847b253780/testr_results.html:514:[ 1.834866] Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option.
30946b8e90fb4aeaa200d7847b253780:.logsearch/30946b8e90fb4aeaa200d7847b253780/testr_results.html:529:[ 1.841148] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. ]---

Builds with matching logs 1/100:
+----------------------------------+---------------------+-----------------------------------+----------+-----------+
| uuid | finished | review | pipeline | job |
+----------------------------------+---------------------+-----------------------------------+----------+-----------+
| 30946b8e90fb4aeaa200d7847b253780 | 2021-08-31T14:18:58 | https://review.opendev.org/792356 | check | nova-next |
+----------------------------------+---------------------+-----------------------------------+----------+-----------+

Revision history for this message
Lee Yarwood (lyarwood) wrote :

$ logsearch storedsearch bug-apic
[..]
Builds with matching logs 2/100:
+----------------------------------+---------------------+-----------------------------------+----------+-------------------+
| uuid | finished | review | pipeline | job |
+----------------------------------+---------------------+-----------------------------------+----------+-------------------+
| ea382e2e76d34d0e9f960f8e2b0e2889 | 2021-10-07T12:14:03 | https://review.opendev.org/812473 | check | tempest-ipv6-only |
| 547649e867374c04bad3b29a82cb3132 | 2021-10-07T12:49:41 | https://review.opendev.org/812392 | check | nova-multi-cell |
+----------------------------------+---------------------+-----------------------------------+----------+-------------------+

Revision history for this message
Attila Fazekas (afazekas) wrote :

Kernel probably has blind spto between pci enumeration and hotplug enable.
Initiation of pci rescan after the device addition can make the device show up.
https://stackoverflow.com/questions/32334870/how-to-do-a-true-rescan-of-pcie-bus

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/nova/+/805628
Committed: https://opendev.org/openstack/nova/commit/80572cef67d691ff3f2de50ea2214f0d0c4725c4
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 80572cef67d691ff3f2de50ea2214f0d0c4725c4
Author: Sean Mooney <email address hidden>
Date: Tue Dec 8 16:42:52 2020 +0000

    workarounds: Add libvirt_disable_apic

    This change adds a new workaround config option to allow disabling the
    guest apic.

    NOTE(lyarwood): Conflict caused by I66a8e94f6d04cc89785646aa9c3e138f22a51f60
    not being present in stable/wallaby (test_driver.py), and
    Ie904d1513b5cf76d6d5f6877545e8eb378dd5499 merged in different order in
    different branches (workarounds.py).

    Conflicts:
        nova/tests/unit/virt/libvirt/test_driver.py
        nova/conf/workarounds.py

    Co-authored-by: Lee Yarwood <email address hidden>
    Related-Bug: #1939108
    Change-Id: I2da867f2734b590a884b1fe1200c402cbf7e9e1c
    (cherry picked from commit ec48e1523dfdd0f1031e1a70c28c56962db66e8a)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to devstack (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/devstack/+/805632
Committed: https://opendev.org/openstack/devstack/commit/566896e22cd70b9c946d05a68106db8c4fd821b7
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 566896e22cd70b9c946d05a68106db8c4fd821b7
Author: Lee Yarwood <email address hidden>
Date: Thu Aug 19 14:24:28 2021 +0100

    nova: Enable apic removal workaround for bug #1939108 when using QEMU

    This change enables [workarounds]libvirt_disable_apic when devstack is
    deployed using the libvirt virt driver and qemu virt type in an effort
    to avoid issues outlined in bug #1939108 caused by the older kernel
    currently used in Cirros 0.5.2.

    Depends-On: https://review.opendev.org/c/openstack/nova/+/805628
    Closes-Bug: #1939108
    Change-Id: Ibb6c34133bb1c95ef11cc59d9b12a0f65502c61b
    (cherry picked from commit 1e86a25cc28e34d7f73a4c6ccbbc3fc667598d50)

Revision history for this message
LIU Yulong (dragon889) wrote :
Download full text (13.5 KiB)

Hi there:

We got same error after disable APIC for upstream cloud image: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-2111.qcow2

OpenStack version: rocky

host kernel: 3.10.0-1160.53.1.el7.x86_64

VM console log:
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.10.0-1160.45.1.el7.x86_64 (<email address hidden>) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Wed Oct 13 17:20:51 UTC 2021
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.10.0-1160.45.1.el7.x86_64 root=UUID=44a6a613-4e21-478b-a909-ab653c9d39df ro console=tty0 console=ttyS0,115200n8 crashkernel=auto net.ifnames=0 console=ttyS0 LANG=en_US.UTF-8
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffdbfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000007ffdc000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: OpenStack Foundation OpenStack Nova, BIOS 1.11.0-2.el7 04/01/2014
[ 0.000000] e820: last_pfn = 0x7ffdc max_arch_pfn = 0x400000000
[ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC
[ 0.000000] found SMP MP-table at [mem 0x000f63a0-0x000f63af] mapped at [ffffffffff2003a0]
[ 0.000000] RAMDISK: [mem 0x32419000-0x35204fff]
[ 0.000000] Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 00000000000f6170 00014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 000000007ffe14f7 0002C (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 000000007ffe140b 00074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 000000007ffe0040 013CB (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 000000007ffe0000 00040
[ 0.000000] ACPI: APIC 000000007ffe147f 00078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000007ffdbfff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x7ffb5000-0x7ffdbfff]
[ 0.000000] Reserving 161MB of memory at 640MB for crashkernel (System RAM: 2047MB)
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00001000-0x0009efff]
[ 0.000000] node 0: [mem 0x00100000-0x7ffdbfff]
[ 0.000000] Initmem setup node 0 [mem 0x00001000-0x7ffdbfff]
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] ACPI: IOAPIC (id[0x00] ad...

Revision history for this message
norman shen (jshen28) wrote :

Hi, what qemu version is used? Is there any reliable way to reproduce it? thank you.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/train)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/nova/+/805631
Reason: stable/train branch of nova projects' have been tagged as End of Life. All open patches have to be abandoned in order to be able to delete the branch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on devstack (stable/train)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/train
Review: https://review.opendev.org/c/openstack/devstack/+/805636
Reason: Train is about to transition to End of Life. Open patches needs to be abandoned before branch deletion.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/ussuri)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/nova/+/805630
Reason: stable/ussuri branch of openstack/nova transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on devstack (stable/ussuri)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/devstack/+/805634
Reason: stable/ussuri branch of openstack/devstack transitioned to End of Life and is about to be deleted. To be able to do that, all open patches need to be abandoned.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/victoria)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/nova/+/805629
Reason: stable/victoria branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/victoria if you want to further work on this patch.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.