Allocation audit error during live migration

Bug #2044331 reported by Haidong Pang
34
This bug affects 8 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
Undecided
Haidong Pang

Bug Description

Description
===========
In live migration,
creates destination node allocation which can get accidentally deleted by audit.

Steps to reproduce
==================
()[root@busybox-openstack-db97db44f-vddg8 /]# openstack server list --all-project --long
+--------------------------------------+---------+--------+------------+-------------+-------------------------+------------+----------+-------------------+-------------------+------------+
| ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Availability Zone | Host | Properties |
+--------------------------------------+---------+--------+------------+-------------+-------------------------+------------+----------+-------------------+-------------------+------------+
| 31af6b71-df56-4f56-87fb-64d75d321285 | test-vm | ACTIVE | None | Running | share_net=192.168.111.9 | | | default-az | node-6.domain.tld | |
+--------------------------------------+---------+--------+------------+-------------+-------------------------+------------+----------+-------------------+-------------------+------------+

()[root@busybox-openstack-db97db44f-vddg8 /]# openstack compute service list --service nova-compute
+-----+--------------+-------------------+------------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+-----+--------------+-------------------+------------+---------+-------+----------------------------+
| 133 | nova-compute | node-3.domain.tld | default-az | enabled | up | 2023-11-23T06:22:15.000000 |
| 148 | nova-compute | node-1.domain.tld | ddd | enabled | up | 2023-11-23T06:22:15.000000 |
| 151 | nova-compute | node-2.domain.tld | ddd | enabled | up | 2023-11-23T06:22:15.000000 |
| 226 | nova-compute | node-6.domain.tld | default-az | enabled | up | 2023-11-23T06:22:15.000000 |
| 587 | nova-compute | node-7.domain.tld | ddd | enabled | up | 2023-11-23T06:22:15.000000 |
+-----+--------------+-------------------+------------+---------+-------+----------------------------+

()[root@busybox-openstack-db97db44f-vddg8 /]# openstack server migrate 31af6b71-df56-4f56-87fb-64d75d321285 --live node-3.domain.tld

()[root@nova-maintenance-77f7cf548f-p6rrv /]# nova-manage placement audit --verbose --delete
Deprecated: Option "notification_format" from group "DEFAULT" is deprecated. Use option "notification_format" from group "notifications".
Allocations were set against consumer UUID 31af6b71-df56-4f56-87fb-64d75d321285 but no existing instances or active migrations are related.
2023-11-23 14:23:49.356 144 INFO nova.scheduler.client.report [req-99bf4509-367b-480c-97d9-0c99676a93be - - - - -] Deleted allocation for unknown 31af6b71-df56-4f56-87fb-64d75d321285
Deleted allocations for consumer UUID 31af6b71-df56-4f56-87fb-64d75d321285 on Resource Provider 301d0960-9bc4-4a88-9860-44446286b525: {'MEMORY_MB': 1024, 'VCPU': 1}
Processed 1 allocation.

Expected result
===============
`nova-manage placement audit` can detect active migrations and skip them.

Actual result
=============
`nova-manage placement audit` considers the allocation created on the target node during live migration as an orphaned allocation and will delete it.(The reason for this is that the nova database currently lists the original node as the host for this virtual machine instance. This record will be updated to reflect the target node only upon the completion of the live migration process.)

Environment
===========
1. Exact version of OpenStack you are running. See the following
   Openstack Wallaby

2. Which hypervisor did you use?
   Libvirt + KVM

2. Which storage type did you use?
   Ceph

3. Which networking type did you use?
   Neutron

Changed in nova:
assignee: nobody → Haidong Pang (haidong-pang)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/901721

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.