upgrading nova-libvirt causes KVM processes to exit

Bug #1494060 reported by Steven Dake
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Steven Dake

Bug Description

Reproducer:
Deploy an AIO
modify libvirt's start.sh to echo "this is a test" somewhere in the code before the CMD is executed
redeploy

All KVM processes are killed, meaning the VMs are shut down during an upgrade. The VMs should remain active during an upgrade ideally (I have prototyped this working in the past!) or a weaker alternative is to atleast get the VMs back into a running state.

Steven Dake (sdake)
Changed in kolla:
status: New → Triaged
importance: Undecided → Critical
milestone: none → liberty-rc1
assignee: nobody → Steven Dake (sdake)
Revision history for this message
Steven Dake (sdake) wrote :

From nova-compute:
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(c, e, tb)
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher rv = meth(*args, **kwargs)
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/site-packages/libvirt.py", line 996, in createWithFlags
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher libvirtError: Unable to pre-create chardev file '/var/lib/nova/instances/bf1411d5-d87a-48bb-a581-0983b8dcaf5f/console.log': No such file or directory
2015-09-09 23:39:02.091 1 ERROR oslo_messaging.rpc.dispatcher

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/222014

Changed in kolla:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/222014
Committed: https://git.openstack.org/cgit/stackforge/kolla/commit/?id=ca4db1d965aca2111c25f1141bac9f7612a7f6fa
Submitter: Jenkins
Branch: master

commit ca4db1d965aca2111c25f1141bac9f7612a7f6fa
Author: Steven Dake <email address hidden>
Date: Wed Sep 9 18:56:30 2015 -0700

    Enable nova-libvirt to be updated

    Libvirt shuts down existing virtual machines. To solve this problem,
    the /var/lib/libvirt directory needs to be persisted. By persisting
    this directory in the nova data container, libvirt can communicate
    with the sockets setup by the previous data container.

    Change-Id: I362c533f8493984123686fe6e3f8e43d97434ca6
    Closes-Bug: #1494060

Changed in kolla:
status: In Progress → Fix Committed
Sam Yaple (s8m)
Changed in kolla:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.