Activity log for bug #1764668

Date Who What changed Old value New value Message
2018-04-17 08:10:17 Christian Ehrhardt  bug added bug
2018-04-17 08:27:42 Janåke Rönnblom bug added subscriber Janåke Rönnblom
2018-04-17 09:15:30 Christian Ehrhardt  libvirt (Ubuntu): status New In Progress
2018-04-17 09:19:51 Christian Ehrhardt  nominated for series Ubuntu Xenial
2018-04-17 09:19:51 Christian Ehrhardt  bug task added libvirt (Ubuntu Xenial)
2018-04-17 09:19:51 Christian Ehrhardt  nominated for series Ubuntu Bionic
2018-04-17 09:19:51 Christian Ehrhardt  bug task added libvirt (Ubuntu Bionic)
2018-04-17 09:19:51 Christian Ehrhardt  nominated for series Ubuntu Artful
2018-04-17 09:19:51 Christian Ehrhardt  bug task added libvirt (Ubuntu Artful)
2018-04-17 09:22:22 Launchpad Janitor libvirt (Ubuntu Xenial): status New Confirmed
2018-04-17 09:22:22 Launchpad Janitor libvirt (Ubuntu Artful): status New Confirmed
2018-04-20 08:35:58 Dariusz Gadomski bug added subscriber Dariusz Gadomski
2018-04-24 19:17:04 Launchpad Janitor libvirt (Ubuntu Bionic): status In Progress Fix Released
2018-04-25 08:26:43 Christian Ehrhardt  description The recent fix to libvirt-guests.sh.in works for what it intended to fix (variable scope) but failed to adapt the loop in check_guests_shutdown correctly. Due to that it currently detects all guests as "Failed to determine state of guest". It all works, but guests are just assumed dead and logs are spilled with false-positive warnings. A suggested fix is at: https://git.launchpad.net/~paelzer/libvirt/commit/?id=2ccca91678cd1091163490fd0b466b58c8be2b79 [Impact] * Due to a bug in a recent fix guests now are immediately considered shut down. Due to that the helper to shut down guests cleanly might not wait long enough. * backporting the upstream fix to the affected releases. [Test Case] * Spawn a few KVM guests through libvirt, e.g. $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b1 arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b2 arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b3 arch=amd64 release=bionic label=daily * A) Shut down the computer and watch how it handles guest shutdown. * B) You can better track that if instead of a shutdown you call what the shutdown would via: $ sudo /usr/lib/libvirt/libvirt-guests.sh stop * It will consider guests as shut down no matter what, so you might want to modify guests to not shut down or if you have a system to do so use huge guests with plenty of dirty memory which will slow it down. [Regression Potential] * The fix "only" affects this shutdown handling and no other part of libvirt which limits the ranges of potential regressions. * The current issue is that we don't wait enough for the geusts, we can't wait "less" so we won't regress to be less forgiving slow-shutdowns, but if a mistake would be in the code the regression risk would be to slow down shutdowns - tests did not see that, but this section is about regression POTENTIAL so I thought I add it. [Other Info] * n/a --- The recent fix to libvirt-guests.sh.in works for what it intended to fix (variable scope) but failed to adapt the loop in check_guests_shutdown correctly. Due to that it currently detects all guests as "Failed to determine state of guest". It all works, but guests are just assumed dead and logs are spilled with false-positive warnings. A suggested fix is at: https://git.launchpad.net/~paelzer/libvirt/commit/?id=2ccca91678cd1091163490fd0b466b58c8be2b79
2018-04-25 08:38:08 Christian Ehrhardt  description [Impact] * Due to a bug in a recent fix guests now are immediately considered shut down. Due to that the helper to shut down guests cleanly might not wait long enough. * backporting the upstream fix to the affected releases. [Test Case] * Spawn a few KVM guests through libvirt, e.g. $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b1 arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b2 arch=amd64 release=bionic label=daily $ uvt-kvm create --memory 128 --password ubuntu b3 arch=amd64 release=bionic label=daily * A) Shut down the computer and watch how it handles guest shutdown. * B) You can better track that if instead of a shutdown you call what the shutdown would via: $ sudo /usr/lib/libvirt/libvirt-guests.sh stop * It will consider guests as shut down no matter what, so you might want to modify guests to not shut down or if you have a system to do so use huge guests with plenty of dirty memory which will slow it down. [Regression Potential] * The fix "only" affects this shutdown handling and no other part of libvirt which limits the ranges of potential regressions. * The current issue is that we don't wait enough for the geusts, we can't wait "less" so we won't regress to be less forgiving slow-shutdowns, but if a mistake would be in the code the regression risk would be to slow down shutdowns - tests did not see that, but this section is about regression POTENTIAL so I thought I add it. [Other Info] * n/a --- The recent fix to libvirt-guests.sh.in works for what it intended to fix (variable scope) but failed to adapt the loop in check_guests_shutdown correctly. Due to that it currently detects all guests as "Failed to determine state of guest". It all works, but guests are just assumed dead and logs are spilled with false-positive warnings. A suggested fix is at: https://git.launchpad.net/~paelzer/libvirt/commit/?id=2ccca91678cd1091163490fd0b466b58c8be2b79 [Impact]  * Due to a bug in a recent fix guests now are immediately considered shut    down. Due to that the helper to shut down guests cleanly might not wait    long enough.  * backporting the upstream fix to the affected releases. [Test Case]  * Spawn a few KVM guests through libvirt, e.g.    $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily    $ uvt-kvm create --memory 128 --password ubuntu b1 arch=amd64 release=bionic label=daily    $ uvt-kvm create --memory 128 --password ubuntu b2 arch=amd64 release=bionic label=daily    $ uvt-kvm create --memory 128 --password ubuntu b3 arch=amd64 release=bionic label=daily  * A) Shut down the computer and watch how it handles guest shutdown.  * B) You can better track that if instead of a shutdown you call what the    shutdown would via:    $ sudo /usr/lib/libvirt/libvirt-guests.sh stop  * It will consider guests as shut down no matter what, so you might want    to modify guests to not shut down or if you have a system to do so use    huge guests with plenty of dirty memory which will slow it down. One good trick is to start the guests right before stopping. Guests are not yet responding to shutdown signals at that time, so the issue is revealed (not waiting on shutdown) To do so you could stop the guests and then do: $ virsh start b1; virsh start b2; virsh start b3; $ /usr/lib/libvirt/libvirt-guests.sh stop The bad case looks like: Running guests on default URI: b1, b2, b3 Shutting down guests on default URI... Starting shutdown on guest: b1 Starting shutdown on guest: b2 Starting shutdown on guest: b3 Waiting for 3 guests to shut down, 120 seconds left Shutdown of guest b1 complete. Shutdown of guest b2 complete. Shutdown of guest b3 complete. But you can check with virsh list they are still alive. The improved case at least properly waits like: ... Waiting for 3 guests to shut down, 120 seconds left Waiting for 3 guests to shut down, 115 seconds left Waiting for 3 guests to shut down, 110 seconds left ... [Regression Potential]  * The fix "only" affects this shutdown handling and no other part of    libvirt which limits the ranges of potential regressions.  * The current issue is that we don't wait enough for the geusts, we can't    wait "less" so we won't regress to be less forgiving slow-shutdowns,    but if a mistake would be in the code the regression risk would    be to slow down shutdowns - tests did not see that, but this section is    about regression POTENTIAL so I thought I add it. [Other Info]  * n/a --- The recent fix to libvirt-guests.sh.in works for what it intended to fix (variable scope) but failed to adapt the loop in check_guests_shutdown correctly. Due to that it currently detects all guests as "Failed to determine state of guest". It all works, but guests are just assumed dead and logs are spilled with false-positive warnings. A suggested fix is at: https://git.launchpad.net/~paelzer/libvirt/commit/?id=2ccca91678cd1091163490fd0b466b58c8be2b79
2018-04-26 11:47:49 Robie Basak tags regression-update
2018-04-26 12:18:12 Robie Basak libvirt (Ubuntu Xenial): status Confirmed Fix Committed
2018-04-26 12:18:15 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2018-04-26 12:18:18 Robie Basak bug added subscriber SRU Verification
2018-04-26 12:18:24 Robie Basak tags regression-update regression-update verification-needed verification-needed-xenial
2018-04-26 12:18:58 Robie Basak libvirt (Ubuntu Artful): status Confirmed Fix Committed
2018-04-26 12:19:04 Robie Basak tags regression-update verification-needed verification-needed-xenial regression-update verification-needed verification-needed-artful verification-needed-xenial
2018-04-30 07:12:39 Christian Ehrhardt  tags regression-update verification-needed verification-needed-artful verification-needed-xenial regression-update verification-done verification-done-artful verification-done-xenial
2018-05-03 16:21:30 Launchpad Janitor libvirt (Ubuntu Xenial): status Fix Committed Fix Released
2018-05-03 16:21:36 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2018-05-03 16:22:47 Launchpad Janitor libvirt (Ubuntu Artful): status Fix Committed Fix Released