Comment 16 for bug 1824858

Revision history for this message
chen haochuan (martin1982) wrote :

My reproduce step

1, on active controller
$ openstack --os-username 'admin' --os-password 'Local.123' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server create --image 9da41b71-075a-4f28-990c-6fcd570747cd --flavor 39e233dd-acd4-4dfa-bb85-e9f520792ba3 --nic net-id=external-net0 vm0

$ openstack --os-username 'admin' --os-password 'Local.123' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server stop 7dd1d8ef-a473-42fd-be61-e2f859f0656a

2, from compute-0 copy folder to /var/lib/nova/instance/<instance uuid>/ compute-1 with same location

3, update database
$ kubectl exec -n openstack -it mariadb-server-0 -- grep password /etc/mysql/admin_user.cnf
to get password
$ kubectl exec -n openstack -it mariadb-server-0 bash
in mariadb container
$ mysql -u root -p <last got mariadb password>
    MariaDB [(none)]> use nova;
    MariaDB [nova]> update instances set host='compute-1', node='compute-1' where uuid='7dd1d8ef-a473-42fd-be61-e2f859f0656a
';
    MariaDB [(none)]> quit;

4, launch vm
openstack --os-username 'admin' --os-password 'Local.123' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server start 7dd1d8ef-a473-42fd-be61-e2f859f0656a

After above stop check nova-compute log
$ kubectl -n openstack logs nova-compute-compute-0-75ea0372-c4l2n -c nova-compute

Got such log nova-compute log
2019-07-19 08:02:50.083 2239894 WARNING nova.compute.manager [req-3df620d7-e138-4259-a98f-687e6c71291d - - - - -] While synchronizing instance power states, found 0 instances in the database and 1 instances on the hypervisor.
2019-07-19 08:03:11.962 2239894 WARNING nova.compute.resource_tracker [req-3df620d7-e138-4259-a98f-687e6c71291d - - - - -] Instance 7dd1d8ef-a473-42fd-be61-e2f859f0656a has been moved to another host compute-1(compute-1). There are allocations remaining against the source host that might need to be removed: {u'resources': {u'MEMORY_MB': 512, u'VCPU': 1, u'DISK_GB': 1}}.

Firstly, please check my reproduce step correct or not. If correct, I think left instances remnants is nova expect behavior.

Nova-compute already could detect instance file in /var/lib/nova/<instance UUID>, so it prompt " 1 instances on the hypervisor" and also find no record in database, "0 instances in the database". Study nova/compute/manager.py, in such case, nova-compute just only promote a WARNING log and do nothing, so if think it is expected behavior.