VMwareVCDriver and VMwareESXDriver doesn't update memory, disk and cpu usage to account for externally allocated VMs

[Maithem]

The ESX driver doesn't account for resources that have been allocated without nova-compute. When nova-compute first launches, it polls the esx hypervisor for available resources and initializes the disk/memory/vcpu usage metrics with incorrect values (assumes all resources are available). Which can result in over allocation errors when provisioning new virtual machines.

How to reproduce:
1. Create a virtual machine on the ESX hypervisor(using vsphere client)
2. Launch Nova
3. Check the "Hypervisor Summary" page on the dashboard
4. Notice that the disk/memory/vcpu usage doesn't reflect the virtual machine's(from step 1) allocated resources

[Sabari Murugesan (smurugesan)]

Hi

Can you check for the following lines in nova-compute log and post them here :-
>Auditing locally available compute resources
>Hypervisor: free ram (MB): XYZ
>Hypervisor: free disk (GB): XY
>Hypervisor: free VCPUs: X

This is the hypervisor's view of resource usage that is reported by the drivers to the compute manager. The usage stats here would account for the existing instances on the ESX host/ VC Cluster. The usage stats on the Hypervisor page is the usage based on instances.

[Gary Kotton (garyk)]

This bug needs to be broken up into a number of bugs:
1. Disk usage:
    i. If a datastore_regex is configured then this is not used. This leads us to reporting incorrect disk usage as the first datastore will be selected and not the actual datastore in question (I have opened bug https://bugs.launchpad.net/nova/+bug/1242638)
  ii. the resource tracking on nova does not take into account the actual used disk space on the disk. It estimates the used diskspace (https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L587). So for example this may not take into account (https://bugs.launchpad.net/nova/+bug/1242645)
2. Memory usage:
I think that this is the same as above. My concern here is that we may report the incorrect amount of total memory. I hope that Sabari can help out here.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/126237

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/126237
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

[Davanum Srinivas (DIMS) (dims-v)]

Removing "In Progress" status and assignee as change is abandoned.

[Davanum Srinivas (DIMS) (dims-v)]

Not sure why this is marked high. Anyone interested, please chime in

[Markus Zoeller (markus_z) (mzoeller)]

This is an automated cleanup. This bug report has been closed because it
is older than 18 months and there is no open code change to fix this.
After this time it is unlikely that the circumstances which lead to
the observed issue can be reproduced.

If you can reproduce the bug, please:
* reopen the bug report (set to status "New")
* AND add the detailed steps to reproduce the issue (if applicable)
* AND leave a comment "CONFIRMED FOR: <RELEASE_NAME>"
  Only still supported release names are valid (LIBERTY, MITAKA, OCATA, NEWTON).
  Valid example: CONFIRMED FOR: LIBERTY