live snapshot of a paused instance hangs

[Matt Riedemann]

Seeing this on this CI job where the Pike UCA is not enabled so libvirt 1.3.1 is being used, and I don't think we really support libvirt live snapshot in CI until at least libvirt 3.6.0.

http://logs.openstack.org/86/531386/5/check/tempest-full/2932954/job-output.txt.gz#_2018-01-06_03_19_12_728469

In this case, the live snapshot on the paused instance just hangs. If you trace req-f7805820-c671-487f-8043-a1fe30dd0372 through the n-cpu logs you'll see it just hangs:

http://logs.openstack.org/86/531386/5/check/tempest-full/2932954/controller/logs/screen-n-cpu.txt

Jan 06 02:41:44.636751 ubuntu-xenial-infracloud-vanilla-0001712754 nova-compute[10798]: WARNING nova.compute.manager [None req-f7805820-c671-487f-8043-a1fe30dd0372 tempest-ImagesTestJSON-1310987708 tempest-ImagesTestJSON-1310987708] [instance: 8c15c0d7-667d-40f8-b2d8-b6adb6a321e7] trying to snapshot a non-running instance: (state: 3 expected: 1)

We should probably not even attempt a live snapshot on a paused instance since that doesn't really make sense.

[Matt Riedemann (mriedem)]

Also saw this back when we were testing out the change for disable_libvirt_livesnapshot=False:

https://review.openstack.org/#/c/454323/2/nova/virt/libvirt/driver.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/532214

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/532214
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=af326fd6f64cb331e53e87400330fffe509f0461
Submitter: Zuul
Branch: master

commit af326fd6f64cb331e53e87400330fffe509f0461
Author: Matt Riedemann <email address hidden>
Date: Tue Jan 9 10:16:21 2018 -0500

    libvirt: don't attempt to live snapshot paused instances

    When we changed the default value of the
    workarounds.disable_libvirt_livesnapshot config option value
    to False in 980d0fcd75c2b15ccb0af857a9848031919c6c7d earlier
    in Queens, we were testing against the Pike UCA packages which
    has libvirt 3.6.0 and qemu 2.10. Live snapshots of a paused
    instance work with those package versions as shown by the
    test_create_image_from_paused_server test in Tempest.

    However, if you just use the Ubuntu 16.04 packages for libvirt
    (1.3.1) and qemu (2.5), that test fails and the live snapshot hangs
    on the paused instance.

    This change adds PAUSED to a list of power states that aren't
    valid for live snapshot. We can eventually remove this when we
    require (or add a conditional check for) libvirt>=3.6.0 and
    qemu>=2.10.

    Change-Id: If6c4dd6890ad6e2d00b186c6a9aa85f507b354e0
    Closes-Bug: #1741667

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/740334

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/740334
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=279f89e320eb3fe9d4b236dc9a97fb6bc08d7421
Submitter: Zuul
Branch: master

commit 279f89e320eb3fe9d4b236dc9a97fb6bc08d7421
Author: Stephen Finucane <email address hidden>
Date: Thu Jul 9 16:22:27 2020 +0100

    libvirt: Re-enable live snapshot for paused instances

    In change If6c4dd6890ad6e2d00b186c6a9aa85f507b354e0, we added PAUSED to
    a list of power states that aren't valid for live snapshot. This was
    necessary to workaround issues with older versions of libvirt and QEMU,
    such as those found on Ubuntu 16.04. Our versions of libvirt are now new
    enough that this is no longer necessary, so stop doing it.

    A confusing (read: wrong) note is removed.

    Change-Id: Ife37c2920f2a85eca3401f4b698bc9260277d5a1
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1741667