Resize on vCenter failed because of _VM_REFS_CACHE

[Feng Xi Yan]

This bug is for nova/master branch.

The resize action in vmware environment always fails.

The reason is that nova resized the ****-orign rather than the new cloned vm.

It is caused by the outdated vm_ref in _VM_REFS_CACHE.

In nova/virt/vmwareapi/vmops.py:

def finish_migration(self, context, migration, instance, disk_info,
                         network_info, image_meta, resize_instance=False,
                         block_device_info=None, power_on=True):
        """Completes a resize, turning on the migrated instance."""
        if resize_instance:
            client_factory = self._session._get_vim().client.factory
            vm_ref = vm_util.get_vm_ref(self._session, instance)
            vm_resize_spec = vm_util.get_vm_resize_spec(client_factory,
                                                        instance)
            reconfig_task = self._session._call_method(
                                            self._session._get_vim(),
                                            "ReconfigVM_Task", vm_ref,
                                            spec=vm_resize_spec)
           .......

From this code, we can see we get vm_ref by vm_util.get_vm_ref.

In nova/virt/vmwareapi/vm_util.py

@vm_ref_cache_from_instance
def get_vm_ref(session, instance):
    """Get reference to the VM through uuid or vm name."""
    uuid = instance['uuid']
    vm_ref = (_get_vm_ref_from_vm_uuid(session, uuid) or
                  _get_vm_ref_from_extraconfig(session, uuid) or
                  _get_vm_ref_from_uuid(session, uuid) or
                  _get_vm_ref_from_name(session, instance['name']))
    if vm_ref is None:
        raise exception.InstanceNotFound(instance_id=uuid)
    return vm_ref

The "get_vm_ref" method is decorated by "vm_ref_cache_from_instance".
"vm_ref_cache_from_instance" will firstly check cache variable _VM_REFS_CACHE. But _VM_REFS_CACHE contains a outdated vm_ref(The original one) keyed by our instance_uuid, because the virtual machine's name is changed.

[Feng Xi Yan (yanfengxi)]

Fix proposed to branch: master
Review: https://review.openstack.org/#/c/79833/

[Feng Xi Yan (yanfengxi)]

Commit a new fix.
Abandoned the former method "get_vm_ref_for_resize" and try to update cache during resize.

Review: https://review.openstack.org/#/c/79833/

[Feng Xi Yan (yanfengxi)]

Add unit test:
https://review.openstack.org/#/c/79833/

[Shawn Hartsock (hartsock)]

This bug arises due to changes for https://bugs.launchpad.net/nova/+bug/1258179 see https://review.openstack.org/#/c/60259/ which describes a caching solution to fix a Python IO problem.

Other solutions could have been to reduce the page size (number of VMs returned per result object) to a lower number (like 10) instead of introducing the cache.

The cache is manipulated through a set of decorators. Try to keep consistent and instead add a decorator to manipulate the cache as well.

I suggest use of names like

def vm_ref_cache_update_by_name(func):

def vm_ref_cache_update_by_instance(func):

def vm_ref_cache_invalidate_by_name(func):

def vm_ref_cache_invalidate_by_instance(func):

[Tracy Jones (tjones-i)]

this bug could be pushed to icehouse-rc-potential if not merged by 2/24 12pm UTC

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/82104

[Shawn Hartsock (hartsock)]

See https://review.openstack.org/#/c/79833/

[Russell Bryant (russellb)]

per Tracy, moving to icehouse-rc-potential