VM will not auto-start automatically when server is rebooted after virsh domrename
- Bionic (18.04)
- Bug #1921325
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
libvirt (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
Triaged
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
OS: Ubuntu 18.04.5
uname -r
4.15.0-118-generic
/usr/bin/
QEMU emulator version 2.11.1(Debian 1:2.11+
$ virsh version
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.1
VM running on KVM on different servers won't start automatically when the server is rebooted after changing the VM domain name.
Issue Replication:
1. Set the VM to autostart
2. Reboot the server
3. VM will start automatically
4. Changed the VM domain name virsh domrename VM.old VM.new
5. Set the VM.new domain to autostart
6. Reboot the server
7. VM.new won't start automatically.
Can you please assist?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #1 |
Changed in libvirt (Ubuntu): | |
status: | New → Fix Released |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #2 |
I tried Bionic and Focal.
Setting autostart as well as seeing the effect on reboot worked just the same there:
oot@b:~# virsh autostart h-test
Domain h-test marked as autostarted
root@b:~# ll /etc/libvirt/
total 2
drwxr-xr-x 2 root root 3 Mar 26 08:56 ./
drwxr-xr-x 4 root root 5 Mar 26 08:56 ../
lrwxrwxrwx 1 root root 28 Mar 26 08:56 h-test.xml -> /etc/libvirt/
root@b:~# virsh shutdown h-test
Domain h-test is being shutdown
root@b:~# reboot
root@b:~# [✗=129]
$ lxc exec b bash
root@b:~# virsh list
Id Name State
-------
1 h-test running
root@f:~# virsh autostart h-test
Domain h-test marked as autostarted
root@f:~# ll /etc/libvirt/
total 2
drwxr-xr-x 2 root root 3 Mar 26 08:56 ./
drwxr-xr-x 4 root root 5 Mar 26 08:56 ../
lrwxrwxrwx 1 root root 28 Mar 26 08:56 h-test.xml -> /etc/libvirt/
root@f:~# virsh shutdown h-test
Domain h-test is being shutdown
root@f:~# reboot
root@f:~# [✗=129]
$ lxc exec f bash
root@f:~# virsh list
Id Name State
-------
1 h-test running
Now trying the rename ...
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #3 |
Focal is ok
root@f:~# virsh domrename h-test h-test-newname
Domain successfully renamed
root@f:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 09:13 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
lrwxrwxrwx 1 root root 36 Mar 26 09:13 h-test-newname.xml -> /etc/libvirt/
root@f:~# reboot
root@f:~# [✗=129]
$ lxc exec f bash
root@f:~# virsh list
Id Name State
-------
1 h-test-newname running
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #4 |
Bionic seems indeed broken in this use case
root@b:~# virsh domrename h-test h-test-newname
Domain successfully renamed
root@b:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 08:56 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
lrwxrwxrwx 1 root root 28 Mar 26 08:56 h-test.xml -> /etc/libvirt/
You already mentioned that you issue "virsh autostart" again after the rename, which in newer releases isn't needed anymore.
Indeed this will now fail to create the matching link
root@b:~# virsh autostart h-test-newname
Domain h-test-newname marked as autostarted
root@b:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 08:56 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
lrwxrwxrwx 1 root root 28 Mar 26 08:56 h-test.xml -> /etc/libvirt/
Until I've found the fix (if it is backportable at all) this will be an easy workaround:
root@b:~# ln -s /etc/libvirt/
root@b:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 09:15 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
lrwxrwxrwx 1 root root 36 Mar 26 09:15 h-test-newname.xml -> /etc/libvirt/
Changed in libvirt (Ubuntu Bionic): | |
status: | New → Confirmed |
Changed in libvirt (Ubuntu Focal): | |
status: | New → Fix Released |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #5 |
Interestingly after manually creating the symlink it works again.
root@b:~# virsh autostart --disable h-test-newname
Domain h-test-newname unmarked as autostarted
root@b:~# ll /etc/libvirt/
total 2
drwxr-xr-x 2 root root 2 Mar 26 09:16 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
root@b:~# virsh autostart h-test-newname
Domain h-test-newname marked as autostarted
root@b:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 09:17 ./
drwxr-xr-x 4 root root 5 Mar 26 09:13 ../
lrwxrwxrwx 1 root root 36 Mar 26 09:17 h-test-newname.xml -> /etc/libvirt/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #6 |
Upstream issue: https:/
Upstream fix: https:/
I have a build for you to try in this PPA: https:/
Please let me know if that would work fine for you, then we can consider to start the SRU process
Changed in libvirt (Ubuntu Bionic): | |
status: | Confirmed → Triaged |
tags: | added: sever-next |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : Re: [Bug 1921325] Re: VM will not auto-start automatically when server is rebooted after virsh domrename | #7 |
Hi Christian,
Thank you very much for your help.
I tried creating the symlink manually and that worked. However, I tried to
install the build from the launchpad
https:/
not available anymore.
Can you please make it available again? I need to install it on a couple of
production servers.
Thanks.
Francisco Menchaca
On Fri, 26 Mar 2021 at 20:30, Christian Ehrhardt <
<email address hidden>> wrote:
> Upstream issue: https:/
> Upstream fix:
> https:/
> I have a build for you to try in this PPA:
> https:/
>
> Please let me know if that would work fine for you, then we can consider
> to start the SRU process
>
> ** Bug watch added: Red Hat Bugzilla #1594985
> https:/
>
> ** Changed in: libvirt (Ubuntu Bionic)
> Status: Confirmed => Triaged
>
> ** Tags added: sever-next
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> VM will not auto-start automatically when server is rebooted after
> virsh domrename
>
> Status in libvirt package in Ubuntu:
> Fix Released
> Status in libvirt source package in Bionic:
> Triaged
> Status in libvirt source package in Focal:
> Fix Released
>
> Bug description:
>
> OS: Ubuntu 18.04.5
> uname -r
> 4.15.0-118-generic
>
>
> /usr/bin/
> QEMU emulator version 2.11.1(Debian 1:2.11+
>
>
> $ virsh version
> Compiled against library: libvirt 4.0.0
> Using library: libvirt 4.0.0
> Using API: QEMU 4.0.0
> Running hypervisor: QEMU 2.11.1
>
> VM running on KVM on different servers won't start automatically when
> the server is rebooted after changing the VM domain name.
>
> Issue Replication:
>
> 1. Set the VM to autostart
> 2. Reboot the server
> 3. VM will start automatically
> 4. Changed the VM domain name virsh domrename VM.old VM.new
> 5. Set the VM.new domain to autostart
> 6. Reboot the server
> 7. VM.new won't start automatically.
>
> Can you please assist?
>
> To manage notifications about this bug go to:
>
> https:/
>
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #8 |
Hi Francisco,
I thought I had lost your activity on this case - glad to see you back on it.
Yeah the old PPA is gone - you fell victim to my bi-yearly PPA cleanup :-)
At some point I have to give launchpad all its gigabytes back ...
I rebased the branch (since it aged anyway) and have thrown a new build into
this new PPA/ticket:
https:/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #9 |
Thank you very much Christian.
I apologise for not responding earlier.
Quick question do PPA's are integrated into future patches?
Thanks in advance for your help.
Regards,
Francisco
On Fri, 16 Apr 2021, 4:15 pm Christian Ehrhardt , <
<email address hidden>> wrote:
> Hi Francisco,
> I thought I had lost your activity on this case - glad to see you back on
> it.
> Yeah the old PPA is gone - you fell victim to my bi-yearly PPA cleanup :-)
> At some point I have to give launchpad all its gigabytes back ...
>
> I rebased the branch (since it aged anyway) and have thrown a new build
> into
> this new PPA/ticket:
>
> https:/
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> VM will not auto-start automatically when server is rebooted after
> virsh domrename
>
> Status in libvirt package in Ubuntu:
> Fix Released
> Status in libvirt source package in Bionic:
> Triaged
> Status in libvirt source package in Focal:
> Fix Released
>
> Bug description:
>
> OS: Ubuntu 18.04.5
> uname -r
> 4.15.0-118-generic
>
>
> /usr/bin/
> QEMU emulator version 2.11.1(Debian 1:2.11+
>
>
> $ virsh version
> Compiled against library: libvirt 4.0.0
> Using library: libvirt 4.0.0
> Using API: QEMU 4.0.0
> Running hypervisor: QEMU 2.11.1
>
> VM running on KVM on different servers won't start automatically when
> the server is rebooted after changing the VM domain name.
>
> Issue Replication:
>
> 1. Set the VM to autostart
> 2. Reboot the server
> 3. VM will start automatically
> 4. Changed the VM domain name virsh domrename VM.old VM.new
> 5. Set the VM.new domain to autostart
> 6. Reboot the server
> 7. VM.new won't start automatically.
>
> Can you please assist?
>
> To manage notifications about this bug go to:
>
> https:/
>
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #10 |
Hi Christian,
We tried installing the build you supplied:
https:/
In two different systems running Ubuntu 18.04.5 and we received this error:
pladmin@
ppa:ci-
[sudo] password for pladmin:
Cannot add PPA: 'ppa:~ci-
ERROR: '~ci-train-
Also , is there a way to install the ppa manually??
Thanks in advance.
FRANCISCO MENCHACASALES ENGINEER | https:/
<email address hidden> | +61 406 204 483
<http://
On Fri, 16 Apr 2021 at 16:15, Christian Ehrhardt <
<email address hidden>> wrote:
> Hi Francisco,
> I thought I had lost your activity on this case - glad to see you back on
> it.
> Yeah the old PPA is gone - you fell victim to my bi-yearly PPA cleanup :-)
> At some point I have to give launchpad all its gigabytes back ...
>
> I rebased the branch (since it aged anyway) and have thrown a new build
> into
> this new PPA/ticket:
>
> https:/
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> VM will not auto-start automatically when server is rebooted after
> virsh domrename
>
> Status in libvirt package in Ubuntu:
> Fix Released
> Status in libvirt source package in Bionic:
> Triaged
> Status in libvirt source package in Focal:
> Fix Released
>
> Bug description:
>
> OS: Ubuntu 18.04.5
> uname -r
> 4.15.0-118-generic
>
>
> /usr/bin/
> QEMU emulator version 2.11.1(Debian 1:2.11+
>
>
> $ virsh version
> Compiled against library: libvirt 4.0.0
> Using library: libvirt 4.0.0
> Using API: QEMU 4.0.0
> Running hypervisor: QEMU 2.11.1
>
> VM running on KVM on different servers won't start automatically when
> the server is rebooted after changing the VM domain name.
>
> Issue Replication:
>
> 1. Set the VM to autostart
> 2. Reboot the server
> 3. VM will start automatically
> 4. Changed the VM domain name virsh domrename VM.old VM.new
> 5. Set the VM.new domain to autostart
> 6. Reboot the server
> 7. VM.new won't start automatically.
>
> Can you please assist?
>
> To manage notifications about this bug go to:
>
> https:/
>
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #11 |
Hi Christian,
An update I tried reinstalling the ppa again and received the following
error:
Can you please advise what I'm doing wrong?
pladmin@
Hit:1 http://
Err:3 http://
InRelease
403 Forbidden [IP: 91.189.95.85 80]
Get:4 http://
Get:2 http://
Get:5 http://
kB]
Get:6 http://
InRelease [20.8 kB]
Reading package lists... Done
E: Failed to fetch
http://
403 Forbidden [IP: 91.189.95.85 80]
E: The repository 'http://
bionic InRelease' is no longer signed.
N: Updating from such a repository can't be done securely, and is therefore
disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration
details.
E: Release file for
http://
valid yet (invalid for another 13d 20h 8min 41s). Updates for this
repository will not be applied.
E: Release file for
http://
valid yet (invalid for another 13d 20h 9min 42s). Updates for this
repository will not be applied.
E: Release file for
http://
valid yet (invalid for another 13d 22h 14min 55s). Updates for this
repository will not be applied.
E: Release file for
http://
is not valid yet (invalid for another 9d 3h 57min 50s). Updates for this
repository will not be applied.
FRANCISCO MENCHACASALES ENGINEER | https:/
<email address hidden> | +61 406 204 483
<http://
<http://
On Tue, 20 Apr 2021 at 01:41, Paco Menchaca <email address hidden> wrote:
> Hi Christian,
>
> We tried installing the build you supplied:
>
> https:/
>
> In two different systems running Ubuntu 18.04.5 and we received this error:
>
> pladmin@
> ppa:ci-
>
> [sudo] password for pladmin:
>
> Cannot add PPA: 'ppa:~ci-
>
> ERROR: '~ci-train-
>
>
>
>
>
>
>
>
>
> Also , is there a way to install the ppa manually??
>
> Thanks in advance.
>
> FRANCISCO MENCHACASALES ENGINEER | https:/
> <email address hidden> | +61 406 204 483
> <http://
>
>
> On Fri, 16 Apr 2021 at 16:15, Christian Ehrhardt <
> <email address hidden>> wrote:
>
>> Hi Francisco,
>> I thought I had lost your activity on this case - glad to see you back on
>> it.
>> Yeah the old PPA is gone - you fell ...
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #12 |
I see two problems
#1
The new PPA is 4529 which seems to work in your output.
But you still have the old (deleted) one configured as well.
You'll probably want to clear that old one out:
$ sudo rm /etc/apt/
$ sudo apt update
(I have guessed the filenme, but it will be that or very similar)
#2 it seems your system is very much "out of time"
" is not valid yet (invalid for another 13d 22h 14min 55s)."
You'd have to have a valid system time for most things to work (cryptography, updates, ...)
I'm not sure but could it be so easy as "set your time right"?
Please check
$ date
if it is anywhere close to real time
Changed in libvirt (Ubuntu): | |
status: | Fix Released → In Progress |
status: | In Progress → Fix Released |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #13 |
Hi Christian,
I fixed our time synch issue and install the ppa in our dev server where it seemed to work.
I can reboot the VMs and they will auto start automatically.
However last night we tried to install the ppa's on two production Ubuntu 18.04.5 servers and the patch did not seem to work. We also tried creating a symlink pointed to /etc/libvirt/
pladmin@
total 8
drwxr-xr-x 2 root root 4096 May 6 16:12 ./
drwxr-xr-x 4 root root 4096 May 6 16:03 ../
lrwxrwxrwx 1 root root 37 May 6 16:12 EQX-ESM-vPRE-01.xml -> /etc/libvirt/
See below libvirt log on one of the servers:
pladmin@
2021-05-06 16:01:56.444+0000: shutting down, reason=failed
2021-05-06 16:10:14.867+0000: starting up libvirt version: 4.0.0, package: 1ubuntu8.
LC_ALL=C PATH=/usr/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #14 |
Hi Francisco,
I have re-checked but I don't see any other related code for your issue.
The PPA contains the one fix we know about.
I could think of old guests (that existed and were configured autostart before the upgrade) to maybe stay in their odd state. But disabling and enabling autostart for them should make it work.
Is at least that working for you?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #15 |
Hi Christian,
Thanks for your response.
Do you mean disabling and enabling autostart to the old guests?
In our dev lab worked straight away after installing the first PPA you
bundled together, I didn't have to disable and enable autostart.
Kind Regards,
FRANCISCO MENCHACASALES ENGINEER | https:/
<email address hidden> | +61 406 204 483
<http://
<http://
On Mon, 10 May 2021 at 22:31, Christian Ehrhardt <
<email address hidden>> wrote:
> Hi Francisco,
> I have re-checked but I don't see any other related code for your issue.
> The PPA contains the one fix we know about.
>
> I could think of old guests (that existed and were configured autostart
> before the upgrade) to maybe stay in their odd state. But disabling and
> enabling autostart for them should make it work.
> Is at least that working for you?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> VM will not auto-start automatically when server is rebooted after
> virsh domrename
>
> Status in libvirt package in Ubuntu:
> Fix Released
> Status in libvirt source package in Bionic:
> Triaged
> Status in libvirt source package in Focal:
> Fix Released
>
> Bug description:
>
> OS: Ubuntu 18.04.5
> uname -r
> 4.15.0-118-generic
>
>
> /usr/bin/
> QEMU emulator version 2.11.1(Debian 1:2.11+
>
>
> $ virsh version
> Compiled against library: libvirt 4.0.0
> Using library: libvirt 4.0.0
> Using API: QEMU 4.0.0
> Running hypervisor: QEMU 2.11.1
>
> VM running on KVM on different servers won't start automatically when
> the server is rebooted after changing the VM domain name.
>
> Issue Replication:
>
> 1. Set the VM to autostart
> 2. Reboot the server
> 3. VM will start automatically
> 4. Changed the VM domain name virsh domrename VM.old VM.new
> 5. Set the VM.new domain to autostart
> 6. Reboot the server
> 7. VM.new won't start automatically.
>
> Can you please assist?
>
> To manage notifications about this bug go to:
>
> https:/
>
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #16 |
On Tue, May 11, 2021 at 1:35 AM Francisco Menchaca
<email address hidden> wrote:
>
> Hi Christian,
>
> Thanks for your response.
>
> Do you mean disabling and enabling autostart to the old guests?
>
> In our dev lab worked straight away after installing the first PPA you
> bundled together, I didn't have to disable and enable autostart.
I didn't have to do it in any of my tests either, all I was saying was
"whatever it is maybe it helps there".
As I said I'm lost on options a bit, I could get the fix prepared to
get into Ubuntu as-is, but since you seem to have a more complex issue
underneath in your prod environment we need to find if there the fix
a) doesn't help at all
b) only helps after guest autostart off/on
c) only helps after (b) plus service or even system restart
d) ??? something else ???
Since only your environment is affected by this new special behavior I
can't really debug/help much until we know what the difference is :-/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Francisco Menchaca (piscesiscariot) wrote : | #17 |
Hi Chhristian.
I am going to try to install the ppa on a different Ubuntu 18.04 server and will let you know my findings.
The only difference from the preduction servers and my test server is the following:
Production servers were upgraded from Ubuntu 16.04 to Ubuntu 18.04 and the production servers are sitting behind a proxy server.
Cheers.
Francisco
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Christian Ehrhardt (paelzer) wrote : | #18 |
> Hi Chhristian.
>
> I am going to try to install the ppa on a different Ubuntu 18.04 server
> and will let you know my findings.
Thanks in advance
> The only difference from the preduction servers and my test server is
> the following:
>
> Production servers were upgraded from Ubuntu 16.04 to Ubuntu 18.04 and
> the production servers are sitting behind a proxy server.
Thanks for letting me know, but in this case that should (tm) not make
a difference.
For qemu/kvm domains that comes down to the function qemuDomainSetAu tostart and that just creates a symlink.
So for example:
$ virsh autostart h-test1
Domain 'h-test1' marked as autostarted
Actually creates: qemu/autostart/ qemu/h- test1.xml
root@h:~# ll /etc/libvirt/
total 3
drwxr-xr-x 2 root root 3 Mar 26 08:24 ./
drwxr-xr-x 4 root root 6 Mar 26 08:24 ../
lrwxrwxrwx 1 root root 29 Mar 26 08:24 h-test1.xml -> /etc/libvirt/
Now if after that I rename (which only works if it is inactive) it automatically also renames the symlink.
root@h:~# virsh domrename h-test1 h-test1-newname
error: Requested operation is not valid: cannot rename active domain
root@h:~# virsh shutdown h-test1
Domain 'h-test1' is being shutdown
root@h:~# virsh domrename h-test1 h-test1-newname
Domain successfully renamed
root@h:~# ll /etc/libvirt/ qemu/autostart/ qemu/h- test1-newname. xml
total 3
drwxr-xr-x 2 root root 3 Mar 26 08:32 ./
drwxr-xr-x 4 root root 6 Mar 26 08:32 ../
lrwxrwxrwx 1 root root 37 Mar 26 08:32 h-test1-newname.xml -> /etc/libvirt/
I've tested that with the latest libvirt in Ubuntu which is 7.0.0-2ubuntu1
Now chances are still that this was broken in the past ... let me try that as well