| 2017-05-05 10:04:05 |
Christoph Wolff |
bug |
|
|
added bug |
| 2017-05-08 09:04:18 |
Christian Ehrhardt |
libvirt (Ubuntu): status |
New |
Triaged |
|
| 2017-05-08 09:04:20 |
Christian Ehrhardt |
libvirt (Ubuntu): importance |
Undecided |
Medium |
|
| 2017-05-08 09:04:28 |
Christian Ehrhardt |
bug |
|
|
added subscriber Ubuntu Server Team |
| 2017-05-08 09:08:23 |
Christian Ehrhardt |
tags |
|
need-upstream-report |
|
| 2017-05-08 09:21:13 |
Christian Ehrhardt |
attachment added |
|
suggested fix against upstream https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1688508/+attachment/4873404/+files/0001-tools-avoid-text-spilling-into-variables.patch |
|
| 2017-05-08 09:21:31 |
Christian Ehrhardt |
tags |
need-upstream-report |
need-upstream-report server-next |
|
| 2017-05-08 12:26:14 |
Ubuntu Foundations Team Bug Bot |
tags |
need-upstream-report server-next |
need-upstream-report patch server-next |
|
| 2017-05-11 16:31:09 |
Christoph Wolff |
bug watch added |
|
https://bugzilla.redhat.com/show_bug.cgi?id=1450141 |
|
| 2017-05-12 07:08:37 |
Christian Ehrhardt |
bug |
|
|
added subscriber ChristianEhrhardt |
| 2017-05-12 07:09:01 |
Christian Ehrhardt |
bug task added |
|
libvirt |
|
| 2017-06-13 13:46:17 |
Christian Ehrhardt |
tags |
need-upstream-report patch server-next |
need-upstream-report patch |
|
| 2017-07-05 10:20:08 |
Norbert R. |
bug |
|
|
added subscriber Norbert R. |
| 2017-10-27 04:52:14 |
Bug Watch Updater |
libvirt: status |
Unknown |
In Progress |
|
| 2017-10-27 04:52:14 |
Bug Watch Updater |
libvirt: importance |
Unknown |
Undecided |
|
| 2017-10-31 16:00:34 |
Jorge Niedbalski |
nominated for series |
|
Ubuntu Zesty |
|
| 2017-10-31 16:00:34 |
Jorge Niedbalski |
nominated for series |
|
Ubuntu Artful |
|
| 2017-10-31 16:00:34 |
Jorge Niedbalski |
nominated for series |
|
Ubuntu Xenial |
|
| 2017-10-31 16:01:30 |
Eric Desrochers |
bug task added |
|
libvirt (Ubuntu Artful) |
|
| 2017-10-31 16:01:35 |
Eric Desrochers |
bug task added |
|
libvirt (Ubuntu Xenial) |
|
| 2017-10-31 16:01:43 |
Eric Desrochers |
bug task added |
|
libvirt (Ubuntu Zesty) |
|
| 2017-10-31 16:43:33 |
Jorge Niedbalski |
libvirt (Ubuntu): status |
Triaged |
Confirmed |
|
| 2017-10-31 16:43:38 |
Jorge Niedbalski |
libvirt (Ubuntu): assignee |
|
Jorge Niedbalski (niedbalski) |
|
| 2017-10-31 16:43:44 |
Jorge Niedbalski |
libvirt (Ubuntu Xenial): status |
New |
In Progress |
|
| 2017-10-31 16:43:48 |
Jorge Niedbalski |
libvirt (Ubuntu Zesty): status |
New |
In Progress |
|
| 2017-10-31 16:43:52 |
Jorge Niedbalski |
libvirt (Ubuntu Artful): status |
New |
In Progress |
|
| 2017-10-31 16:43:57 |
Jorge Niedbalski |
libvirt (Ubuntu Xenial): importance |
Undecided |
Medium |
|
| 2017-10-31 16:43:59 |
Jorge Niedbalski |
libvirt (Ubuntu Zesty): importance |
Undecided |
Medium |
|
| 2017-10-31 16:44:05 |
Jorge Niedbalski |
libvirt (Ubuntu Artful): importance |
Undecided |
Medium |
|
| 2017-10-31 16:44:07 |
Jorge Niedbalski |
libvirt (Ubuntu Xenial): assignee |
|
Jorge Niedbalski (niedbalski) |
|
| 2017-10-31 16:44:09 |
Jorge Niedbalski |
libvirt (Ubuntu Zesty): assignee |
|
Jorge Niedbalski (niedbalski) |
|
| 2017-10-31 16:44:14 |
Jorge Niedbalski |
libvirt (Ubuntu Artful): assignee |
|
Jorge Niedbalski (niedbalski) |
|
| 2017-10-31 18:00:25 |
Jorge Niedbalski |
description |
On a brand new installation of Ubuntu Server 16.04.2 LTS the libvirt-guests.sh script fails to shutdown my VMs if run with PARALLEL_SHUTDOWN > 0, instead running forever thereby preventing me from shutting down my system. I debugged the script with 'set -x' and found out that the problem lies in the `check_guests_shutdown()` function (line 337). If a guest is shutdown, it runs eval_gettext() to print that the state of the guest cannot be determined. This is used as a return value on line 428, causing the script to interpret the error message as a list of VM IDs and trying to shutdown these. This of course fails again and the cycle continues. For now I have commented out the eval_gettext line and the script works fine again.
My libvirt version is 1.3.1-1ubuntu10.8. |
[Environment]
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
[Impact]
There is a bug/race condition on libvirt-guests.service, that prevents the shutdown of guests to happen in parallel and by consequence locking
the machine shutdown during the entire SHUTDOWN_TIMEOUT.
The critical chain for this service is:
libvirt-guests.service +20ms
└─libvirt-bin.service @2.784s +140ms
└─remote-fs.target @2.777s
└─remote-fs-pre.target @2.775s
└─open-iscsi.service @2.554s +116ms
└─iscsid.service @2.525s +18ms
└─network-online.target @2.502s
└─network.target @1.955s
└─networking.service @1.625s +299ms
└─network-pre.target @1.601s
└─cloud-init-local.service @405ms +1.072s
└─systemd-remount-fs.service @232ms +64ms
└─systemd-journald.socket @178ms
└─-.slice @117ms
As an example, I have the following kvm host with 42 virtual
machines.
ubuntu@xenial-base:~$ virsh list --all
Id Name State
----------------------------------------------------
12 locked-trusty-2 running
13 locked-trusty-3 running
[...]
41 locked-trusty-42 running
After rebooting the machine:
[ 250.999516] libvirt-guests.sh[4215]: Running guests on default URI: locked-trusty-2, locked-trusty-4, locked-trusty-12, locked-trusty-3, locked-trusty-5, locked-trusty-11, locked-trusty-10, locked-trusty-8, locked-trusty-9, locked-trusty-7, locked-trusty-6, locked-trusty-13, locked-trusty-14, locked-trusty-15, locked-trusty-16, locked-trusty-17, locked-trusty-18, locked-trusty-19, locked-trusty-20, locked-trusty-21, locked-trusty-22, locked-trusty-23, locked-trusty-24, locked-trusty-25, locked-trusty-26, locked-trusty-27, locked-trusty-28, locked-trusty-29, locked-trusty-30, locked-trusty-31, locked-trusty-32, locked-trusty-33, locked-trusty-34, locked-trusty-35, locked-trusty-36, locked-trusty-37, locked-trusty-38, locked-trusty-39, locked-trusty-40, locked-trusty-41, locked-trusty-42
[ 251.011367] libvirt-guests.sh[4215]: Shutting down guests on default URI...
[ 251.027072] libvirt-guests.sh[4215]: Starting shutdown on guest: locked-trusty-2
[...]
[ 391.949941] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 10 seconds left
[ 398.074405] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 5 seconds left
[ 403.020479] libvirt-guests.sh[4215]: Timeout expired while shutting down domains
[ OK ] Stopped Suspend Active Libvirt Guests.
[ OK ] Stopped target System Time Synchronized.
[Test Case]
* Make sure the following variables are set in /etc/default/libvirt-guests (which are all default options):
ON_SHUTDOWN=shutdown
PARALLEL_SHUTDOWN=10
SHUTDOWN_TIMEOUT=120
* Create over 20 virtual machines (in my case, using uvt-kvm).
$ for f in $(seq 0 40); do uvt-kvm create --memory 2000 --cpu 1 locked-trusty-$f release=xenial arch=amd64 ; done
* Reboot the machine and monitor the systemd service stop sequence
or console output.
(With systemd: systemctl start debug-shell and jumpt to ctrl+alt+f9)
* Error message "Timeout expired while shutting down domains" should
be displayed.
[Regression Potential]
* None identified.
[Other Info]
* There is a proposed patch in upstream already that has been already
linked to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1450141 |
|
| 2017-10-31 18:25:59 |
Jorge Niedbalski |
description |
[Environment]
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
[Impact]
There is a bug/race condition on libvirt-guests.service, that prevents the shutdown of guests to happen in parallel and by consequence locking
the machine shutdown during the entire SHUTDOWN_TIMEOUT.
The critical chain for this service is:
libvirt-guests.service +20ms
└─libvirt-bin.service @2.784s +140ms
└─remote-fs.target @2.777s
└─remote-fs-pre.target @2.775s
└─open-iscsi.service @2.554s +116ms
└─iscsid.service @2.525s +18ms
└─network-online.target @2.502s
└─network.target @1.955s
└─networking.service @1.625s +299ms
└─network-pre.target @1.601s
└─cloud-init-local.service @405ms +1.072s
└─systemd-remount-fs.service @232ms +64ms
└─systemd-journald.socket @178ms
└─-.slice @117ms
As an example, I have the following kvm host with 42 virtual
machines.
ubuntu@xenial-base:~$ virsh list --all
Id Name State
----------------------------------------------------
12 locked-trusty-2 running
13 locked-trusty-3 running
[...]
41 locked-trusty-42 running
After rebooting the machine:
[ 250.999516] libvirt-guests.sh[4215]: Running guests on default URI: locked-trusty-2, locked-trusty-4, locked-trusty-12, locked-trusty-3, locked-trusty-5, locked-trusty-11, locked-trusty-10, locked-trusty-8, locked-trusty-9, locked-trusty-7, locked-trusty-6, locked-trusty-13, locked-trusty-14, locked-trusty-15, locked-trusty-16, locked-trusty-17, locked-trusty-18, locked-trusty-19, locked-trusty-20, locked-trusty-21, locked-trusty-22, locked-trusty-23, locked-trusty-24, locked-trusty-25, locked-trusty-26, locked-trusty-27, locked-trusty-28, locked-trusty-29, locked-trusty-30, locked-trusty-31, locked-trusty-32, locked-trusty-33, locked-trusty-34, locked-trusty-35, locked-trusty-36, locked-trusty-37, locked-trusty-38, locked-trusty-39, locked-trusty-40, locked-trusty-41, locked-trusty-42
[ 251.011367] libvirt-guests.sh[4215]: Shutting down guests on default URI...
[ 251.027072] libvirt-guests.sh[4215]: Starting shutdown on guest: locked-trusty-2
[...]
[ 391.949941] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 10 seconds left
[ 398.074405] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 5 seconds left
[ 403.020479] libvirt-guests.sh[4215]: Timeout expired while shutting down domains
[ OK ] Stopped Suspend Active Libvirt Guests.
[ OK ] Stopped target System Time Synchronized.
[Test Case]
* Make sure the following variables are set in /etc/default/libvirt-guests (which are all default options):
ON_SHUTDOWN=shutdown
PARALLEL_SHUTDOWN=10
SHUTDOWN_TIMEOUT=120
* Create over 20 virtual machines (in my case, using uvt-kvm).
$ for f in $(seq 0 40); do uvt-kvm create --memory 2000 --cpu 1 locked-trusty-$f release=xenial arch=amd64 ; done
* Reboot the machine and monitor the systemd service stop sequence
or console output.
(With systemd: systemctl start debug-shell and jumpt to ctrl+alt+f9)
* Error message "Timeout expired while shutting down domains" should
be displayed.
[Regression Potential]
* None identified.
[Other Info]
* There is a proposed patch in upstream already that has been already
linked to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1450141 |
[Environment]
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
[Impact]
There is a bug/race condition on libvirt-guests.service, that prevents the shutdown of guests to happen in parallel.
The critical chain for this service is:
libvirt-guests.service +20ms
└─libvirt-bin.service @2.784s +140ms
└─remote-fs.target @2.777s
└─remote-fs-pre.target @2.775s
└─open-iscsi.service @2.554s +116ms
└─iscsid.service @2.525s +18ms
└─network-online.target @2.502s
└─network.target @1.955s
└─networking.service @1.625s +299ms
└─network-pre.target @1.601s
└─cloud-init-local.service @405ms +1.072s
└─systemd-remount-fs.service @232ms +64ms
└─systemd-journald.socket @178ms
└─-.slice @117ms
As an example, I have the following kvm host with 42 virtual
machines.
ubuntu@xenial-base:~$ virsh list --all
Id Name State
----------------------------------------------------
12 locked-trusty-2 running
13 locked-trusty-3 running
[...]
41 locked-trusty-42 running
After rebooting the machine:
[ 250.999516] libvirt-guests.sh[4215]: Running guests on default URI: locked-trusty-2, locked-trusty-4, locked-trusty-12, locked-trusty-3, locked-trusty-5, locked-trusty-11, locked-trusty-10, locked-trusty-8, locked-trusty-9, locked-trusty-7, locked-trusty-6, locked-trusty-13, locked-trusty-14, locked-trusty-15, locked-trusty-16, locked-trusty-17, locked-trusty-18, locked-trusty-19, locked-trusty-20, locked-trusty-21, locked-trusty-22, locked-trusty-23, locked-trusty-24, locked-trusty-25, locked-trusty-26, locked-trusty-27, locked-trusty-28, locked-trusty-29, locked-trusty-30, locked-trusty-31, locked-trusty-32, locked-trusty-33, locked-trusty-34, locked-trusty-35, locked-trusty-36, locked-trusty-37, locked-trusty-38, locked-trusty-39, locked-trusty-40, locked-trusty-41, locked-trusty-42
[ 251.011367] libvirt-guests.sh[4215]: Shutting down guests on default URI...
[ 251.027072] libvirt-guests.sh[4215]: Starting shutdown on guest: locked-trusty-2
[...]
[ 391.949941] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 10 seconds left
[ 398.074405] libvirt-guests.sh[4215]: Waiting for 28 guests to shut down, 5 seconds left
[ 403.020479] libvirt-guests.sh[4215]: Timeout expired while shutting down domains
[ OK ] Stopped Suspend Active Libvirt Guests.
[ OK ] Stopped target System Time Synchronized.
[Test Case]
* Make sure the following variables are set in /etc/default/libvirt-guests (which are all default options):
ON_SHUTDOWN=shutdown
PARALLEL_SHUTDOWN=10
SHUTDOWN_TIMEOUT=120
* Create over 20 virtual machines (in my case, using uvt-kvm).
$ for f in $(seq 0 40); do uvt-kvm create --memory 2000 --cpu 1 locked-trusty-$f release=xenial arch=amd64 ; done
* Reboot the machine and monitor the systemd service stop sequence
or console output.
(With systemd: systemctl start debug-shell and jumpt to ctrl+alt+f9)
* Error message "Timeout expired while shutting down domains" should
be displayed.
[Regression Potential]
* None identified.
[Other Info]
* There is a proposed patch in upstream already that has been already
linked to this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1450141 |
|
| 2018-02-23 15:45:08 |
Christian Ehrhardt |
libvirt (Ubuntu): status |
Confirmed |
In Progress |
|
| 2018-02-23 15:45:11 |
Christian Ehrhardt |
libvirt (Ubuntu): assignee |
Jorge Niedbalski (niedbalski) |
ChristianEhrhardt (paelzer) |
|
| 2018-02-23 18:21:16 |
Launchpad Janitor |
libvirt (Ubuntu): status |
In Progress |
Fix Released |
|
| 2018-02-25 10:49:33 |
Bug Watch Updater |
libvirt: status |
In Progress |
Fix Released |
|
| 2018-02-26 13:28:05 |
Dariusz Gadomski |
libvirt (Ubuntu Zesty): status |
In Progress |
Won't Fix |
|
| 2018-02-26 14:47:41 |
Dariusz Gadomski |
attachment added |
|
artful_libvirt_3.6.0-1ubuntu6.4.debdiff https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1688508/+attachment/5063489/+files/artful_libvirt_3.6.0-1ubuntu6.4.debdiff |
|
| 2018-02-26 14:48:30 |
Dariusz Gadomski |
attachment added |
|
xenial_libvirt_1.3.1-1ubuntu10.20.debdiff https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1688508/+attachment/5063490/+files/xenial_libvirt_1.3.1-1ubuntu10.20.debdiff |
|
| 2018-02-26 14:49:04 |
Dariusz Gadomski |
bug |
|
|
added subscriber STS Sponsors |
| 2018-02-26 14:49:11 |
Dariusz Gadomski |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2018-02-26 22:50:51 |
Dan Streetman |
tags |
need-upstream-report patch |
need-upstream-report patch sts-sponsor-ddstreet |
|
| 2018-02-27 17:33:09 |
Dan Streetman |
tags |
need-upstream-report patch sts-sponsor-ddstreet |
need-upstream-report patch |
|
| 2018-02-28 13:19:43 |
Janåke Rönnblom |
bug |
|
|
added subscriber Janåke Rönnblom |
| 2018-02-28 14:16:17 |
Chris J Arges |
libvirt (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2018-02-28 14:16:20 |
Chris J Arges |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-02-28 14:16:21 |
Chris J Arges |
bug |
|
|
added subscriber SRU Verification |
| 2018-02-28 14:16:24 |
Chris J Arges |
tags |
need-upstream-report patch |
need-upstream-report patch verification-needed verification-needed-xenial |
|
| 2018-02-28 14:16:28 |
Chris J Arges |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2018-02-28 14:16:55 |
Chris J Arges |
libvirt (Ubuntu Artful): status |
In Progress |
Fix Committed |
|
| 2018-02-28 14:16:59 |
Chris J Arges |
tags |
need-upstream-report patch verification-needed verification-needed-xenial |
need-upstream-report patch verification-needed verification-needed-artful verification-needed-xenial |
|
| 2018-03-02 13:00:11 |
Eric Desrochers |
removed subscriber STS Sponsors |
|
|
|
| 2018-03-13 15:19:33 |
Dariusz Gadomski |
bug |
|
|
added subscriber Dariusz Gadomski |
| 2018-03-13 15:20:22 |
Dariusz Gadomski |
attachment added |
|
xenial_libvirt-guests.log.gz https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1688508/+attachment/5078252/+files/libvirt-guests.log.gz |
|
| 2018-03-14 15:11:45 |
Christian Ehrhardt |
tags |
need-upstream-report patch verification-needed verification-needed-artful verification-needed-xenial |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-needed verification-needed-artful verification-needed-xenial |
|
| 2018-03-14 15:14:00 |
Christian Ehrhardt |
tags |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-needed verification-needed-artful verification-needed-xenial |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-failed verification-failed-artful verification-failed-xenial |
|
| 2018-03-14 15:14:06 |
Christian Ehrhardt |
libvirt (Ubuntu Artful): status |
Fix Committed |
Triaged |
|
| 2018-03-14 15:14:11 |
Christian Ehrhardt |
libvirt (Ubuntu Xenial): status |
Fix Committed |
Triaged |
|
| 2018-03-14 15:14:15 |
Christian Ehrhardt |
libvirt (Ubuntu): status |
Fix Released |
In Progress |
|
| 2018-03-21 16:20:52 |
Launchpad Janitor |
libvirt (Ubuntu): status |
In Progress |
Fix Released |
|
| 2018-04-05 06:20:32 |
Christian Ehrhardt |
libvirt (Ubuntu Artful): status |
Triaged |
In Progress |
|
| 2018-04-05 06:20:35 |
Christian Ehrhardt |
libvirt (Ubuntu Xenial): status |
Triaged |
In Progress |
|
| 2018-04-05 09:46:51 |
Łukasz Zemczak |
libvirt (Ubuntu Artful): status |
In Progress |
Fix Committed |
|
| 2018-04-05 09:46:58 |
Łukasz Zemczak |
tags |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-failed verification-failed-artful verification-failed-xenial |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-failed-xenial verification-needed verification-needed-artful |
|
| 2018-04-05 09:57:17 |
Łukasz Zemczak |
libvirt (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2018-04-05 09:57:24 |
Łukasz Zemczak |
tags |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-failed-xenial verification-needed verification-needed-artful |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-needed verification-needed-artful verification-needed-xenial |
|
| 2018-04-06 15:25:20 |
Dariusz Gadomski |
tags |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-needed verification-needed-artful verification-needed-xenial |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-done-artful verification-done-xenial verification-needed |
|
| 2018-04-06 15:25:45 |
Dariusz Gadomski |
tags |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-done-artful verification-done-xenial verification-needed |
4.0.0-1ubuntu6 need-upstream-report needs-upstreaming patch verification-done-artful verification-done-xenial |
|
| 2018-04-12 07:45:11 |
Launchpad Janitor |
libvirt (Ubuntu Artful): status |
Fix Committed |
Fix Released |
|
| 2018-04-12 07:45:25 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2018-04-12 07:46:05 |
Launchpad Janitor |
libvirt (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
