Comment 24 for bug 1941706

Revision history for this message
ignazio (cassano) wrote : Re: [Bug 1941706] Re: kolla wallaby: stopping (or restarting) nova_libvirt causes vm shutdown

Hello,
it was not necessary to thank me. Plus I must say that I have never found
such reactivity in the community.
Thank you very much
Ignazio

Il Sab 28 Ago 2021, 19:15 Radosław Piliszek <email address hidden> ha
scritto:

> I have sent a relevant notice to the openstack-discuss mailing list to
> notify other operators. Inserting it here:
>
> Dear Operators of Kolla-based deployments,
>
> There is a critical regression in current Kolla Ansible Wallaby code
> that results in an environment that shuts down VMs on each libvirtd
> container stop or restart on non-cgroupsv2 distros (so CentOS, Ubuntu
> and Debian Buster but not Debian Bullseye). [1]
>
> The fix is already available. [2]
> Please apply it to your Kolla Ansible installation if you are using
> Wallaby.
> Do note the fix only applies after redeploying which means
> redeployment action will still trigger the buggy behaviour that once!
>
> What to do if you have already deployed Wallaby?
> First of all, make sure you don't accidentally take an action that
> stops nova_libvirt (including restarts: both manual and those applied
> by Kolla Ansible due to user-requested changes).
> Please apply the patch above but don't rush with redeploying!
> Redeploy each compute node separately (or in batches if you prefer) -
> using --limit commandline parameter - and always make sure you have
> first migrated relevant VMs out of the nodes that are going to get
> nova_libvirt restarted.
> This way you can safely fix an existing deployment.
>
> We will be working on improving the testing to avoid such issues in the
> future.
>
> Acknowledgements
> Thanks to Ignazio Cassano for noticing and reporting the issue.
> I have triaged and analysed it, proposing a fix afterwards.
>
> [1] https://bugs.launchpad.net/kolla-ansible/+bug/1941706
> [2] https://review.opendev.org/c/openstack/kolla-ansible/+/806476
>
> -yoctozepto
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1941706
>
> Title:
> kolla wallaby: stopping (or restarting) nova_libvirt causes vm
> shutdown
>
> Status in kolla-ansible:
> In Progress
> Status in kolla-ansible wallaby series:
> In Progress
> Status in kolla-ansible xena series:
> In Progress
>
> Bug description:
> Hello, I tried to update kolla wallaby because new images has been
> released.
> During the update nova_libvirt container restarted and all vm went down.
> So I verified that every time I restart nova_libvirt container, all vm
> restarted and nova_compute reports lost connection with libvirt.
> In openstack installations where docker is not used, the libvirt restart
> does not cause vm shutdown.
> Ignazio
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/kolla-ansible/+bug/1941706/+subscriptions
>
>