Instance's cleaned filed was set 1 after rollback at destination when do live-migration failed

Bug #1598353 reported by Charlotte Han
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
Medium
Charlotte Han

Bug Description

Description
===========
I did live-migration for an instance, error occured by libvirt process.
Then I found field of 'cleaned' was '1' in instances table, but this instance was not deleted.

Steps to reproduce
==================
* I did live-migrate instance:
  $ nova live-migration --block fa0b2b1e-4e00-43bf-bb5d-13ba86277846

* then I found this instance's vm_state was active and instance's host was still on source compute node.
log is as follow:
2016-07-02 12:06:19.656 4078 ERROR nova.virt.libvirt.driver [req-29a088d7-e52c-43bf-b872-92c34175b8d4 0b78e7b5c15c49769d4f5af63628164c e948821962cd4509a6c687c7b7dcb96f - - -] [instance: fa0b2b1e-4e00-43bf-bb5d-13ba86277846] Live Migration failure: failed to connect to monitor socket: No such process
2016-07-02 12:06:20.044 4078 ERROR nova.virt.libvirt.driver [req-29a088d7-e52c-43bf-b872-92c34175b8d4 0b78e7b5c15c49769d4f5af63628164c e948821962cd4509a6c687c7b7dcb96f - - -] [instance: fa0b2b1e-4e00-43bf-bb5d-13ba86277846] Migration operation has aborted

* then I query database
MariaDB [nova]> select uuid,updated_at,deleted,cleaned from instances where deleted=0 and cleaned=1;
+--------------------------------------+---------------------+---------+---------+
| uuid | updated_at | deleted | cleaned |
+--------------------------------------+---------------------+---------+---------+
| fa0b2b1e-4e00-43bf-bb5d-13ba86277846 | 2016-07-02 04:06:22 | 0 | 1 |
+--------------------------------------+---------------------+---------+---------+
1 row in set (0.00 sec)

MariaDB [nova]> select * from migrations where instance_uuid='fa0b2b1e-4e00-43bf-bb5d-13ba86277846';
+---------------------+---------------------+------------+------+----------------+---------------------+-------------+---------------+--------------------------------------+----------------------+----------------------+-------------+---------------------+---------+----------------+--------+
| created_at | updated_at | deleted_at | id | source_compute | dest_compute | dest_host | status | instance_uuid | old_instance_type_id | new_instance_type_id | source_node | dest_node | deleted | migration_type | hidden |
+---------------------+---------------------+------------+------+----------------+---------------------+-------------+---------------+--------------------------------------+----------------------+----------------------+-------------+---------------------+---------+----------------+--------+
| 2016-07-02 04:06:02 | 2016-07-02 04:06:20 | NULL | 1912 | SBCRslot2 | SBCR-chenling-slot4 | 10.43.239.3 | error | fa0b2b1e-4e00-43bf-bb5d-13ba86277846 | 207 | 207 | NULL | SBCR-chenling-slot4 | 0 | live-migration | 0 |
+---------------------+---------------------+------------+------+----------------+---------------------+-------------+---------------+--------------------------------------+----------------------+----------------------+-------------+---------------------+---------+----------------+--------+
1 row in set (0.00 sec)

Expected result
===============
I hope instance's cleaned is 0.

Actual result
=============
+--------------------------------------+---------------------+---------+---------+
| uuid | updated_at | deleted | cleaned |
+--------------------------------------+---------------------+---------+---------+
| fa0b2b1e-4e00-43bf-bb5d-13ba86277846 | 2016-07-02 04:06:22 | 0 | 1 |
+--------------------------------------+---------------------+---------+---------+

deleted == 0 and cleaned == 1, which is not consistent.

Environment
===========
1. Exact version of OpenStack you are running. See the following
  Mitaka

2. Which hypervisor did you use?
   Libvirt + KVM

3. Which storage type did you use?
   Local storage

3. Which networking type did you use?
   Neutron with OpenVSwitch

Charlotte Han (hanrong)
Changed in nova:
assignee: nobody → Charlotte Han (hanrong)
Revision history for this message
Charlotte Han (hanrong) wrote :

I found when live-migration successfully, the value of field 'cleaned' of migrated instance was correct too.

deleted == 0 and cleaned == 1, which is not consistent.

Changed in nova:
status: New → In Progress
Charlotte Han (hanrong)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
Sean Dague (sdague) wrote :

There are no currently open reviews on this bug, changing the status back to the previous state and unassigning. If there are active reviews related to this bug, please include links in comments.

Changed in nova:
status: In Progress → New
assignee: Charlotte Han (hanrong) → nobody
Revision history for this message
Sean Dague (sdague) wrote :

Found open reviews for this bug in gerrit, setting to In Progress.

review: https://review.openstack.org/336771 in branch: master

Changed in nova:
status: New → In Progress
assignee: nobody → Charlotte Han (hanrong)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Stephen Finucane (<email address hidden>) on branch: master
Review: https://review.openstack.org/336771
Reason: Been sitting here for > 1 year with -1s. Time to abandon this.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.