Rolling upgrade M to N: DBDeadlock Error when create instance during sync database
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Expired
|
Undecided
|
Unassigned |
Bug Description
I have 3 controller nodes running HA active/active, using KVM hypervisor and Maria cluster as shared database. The system was deployed by Devstack Mitaka version on virtual machines which was created by virt-manager.
I have upgraded Keytone to N version, then I tried to Rolling Upgrade Nova from M to N version folowed:
http://
The document said that:
Using the newly installed nova code, run the DB sync. (nova-manage db sync; nova-manage api_db sync). These schema change operations should have minimal or no effect on performance, and should not cause any operations to fail.
However, during the sync database, I cannot create the VM. Nova raise that:
ERROR (ClientException): Unexpected API Error. Please report this at http://
<class 'oslo_db.
Full log here: http://
After finishing the sync process, I can create VM as well.
==Reproduce==
# Controller1:
1. Stop all nova service, except nova api (n-api).
2. Upgrade source code:
$ cd /opt/stack/nova/
$ git checkout remotes/
$ git checkout -b stable/newton remotes/
$ git pull
$ sudo -E pip install -r requirements.txt --upgrade
3. Downgrade some packages dependency (because I used --upgrade as above)
$ sudo pip uninstall oslo.messaging
$ sudo pip uninstall kombu
$ sudo pip uninstall cffi
$ sudo -E pip install oslo.messaging=
$ sudo -E pip install kombu==3.0.35
$ sudo -E pip install cffi==1.5.2
4. Update /etc/nova/
[upgrade_levels]
compute = auto
5. Sync DB
$ nova-manage db sync
$ nova-manage api_db sync
6. During the Sync DB, try to create VM, execute on controller 2 and 3 (not concurrency):
$ nova boot --flavor m1.nano --image 21ffa33b-
ERROR (ClientException): Unexpected API Error. Please report this at http://
<class 'oslo_db.
tags: | added: upgrades |
Changed in nova: | |
assignee: | nobody → Sujitha (sujitha-neti) |
Changed in nova: | |
status: | New → In Progress |
Changed in nova: | |
assignee: | Sujitha (sujitha-neti) → nobody |
Changed in nova: | |
status: | In Progress → Won't Fix |
status: | Won't Fix → New |
Hi Sujitha, do you still continue fixing this bug? Could you please share with me some information about this? :D I hope this bug can be fix soon :)