Stopped instance's disk sizes are not calculated for disk_available_least

[Rikimaru Honjo]

Description
===========
disk_available_least is a free disk size information of hypervisors.
This is calculated by the following formula:

disk_available_least = <free disk size> - <Total gap between virtual disk size and actual disk size for all instances>

But stopped instance's virtual disk sizes are not calculated now.
So disk_available_least will be larger than actual free disk size.
As a result, instances will be scheduled beyond the actual free disk size if stopped instances are on a host.

I think that this is a bug.
Because stopped instances are on a host unlike shelved instances.

Steps to reproduce
==================
1. Call hyper visor show API for any hypervisor.
   And, check the value of disk_available_least.
2. Create a instance with qcow2 image on 1's hypervisor.
3. Wait for over 1 minute.
4. Call hyper visor show API.
   And, check that disk_available_least is smaller than step 1's value.
5. Call Stop Server API for the instance.
6. Wait until instance's state is changed to STOPPED.
7. Wait for over 1 minute.
8. Call hyper visor show API.
   And, check the value of disk_available_least.

Expected result
===============
disk_available_least value is same as step 8.
Because stopped instance is still on the host.

Actual result
=============
disk_available_least value is bigger than step4's value in step 8.

Environment
===========
* I used latest devstack.
* I used libvirt + kvm.
* I used qcow2 image.

Logs & Configs
==============
I think that this bug affects for all settings.

When was this bug made?
=======================
Following patch made this bug:
https://review.openstack.org/#/c/105127/

Stopped instance's disk sizes were calculated until merging the above patch in Juno cycle.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/468269

[Rikimaru Honjo (honjo-rikimaru-c6)]

I added "When was this bug made?" section to Bug Description.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/468269
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3342215034c7a7a938c497c39f6405763201a261
Submitter: Jenkins
Branch: master

commit 3342215034c7a7a938c497c39f6405763201a261
Author: Rikimaru Honjo <email address hidden>
Date: Fri May 26 14:04:44 2017 +0900

    Calculate stopped instance's disk sizes for disk_available_least

    disk_available_least is a free disk size information of hypervisors.
    This is calculated by the following fomula:

    disk_available_least = <free disk size> - <Total gap between virtual
    disk size and actual disk size for all instances>

    But stopped instance's virtual disk sizes were not calculated
    after merging following patch in Juno cycle:
    https://review.openstack.org/#/c/105127

    So disk_available_least might be larger than actual free disk size.
    As a result, instances might be scheduled beyond the actual free
    disk size if stopped instances were on a host.

    This patch fix it.
    Stopped instance's disks will be calculated after merging this patch.

    Change-Id: I8abf6edfa80e3920539e4f6d4906c573f9543b91
    Closes-Bug: #1693679

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/474029

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/474034

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/474029
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2a113159c00b51215f3506a9e9834fa7cc0d5ffb
Submitter: Jenkins
Branch: stable/ocata

commit 2a113159c00b51215f3506a9e9834fa7cc0d5ffb
Author: Rikimaru Honjo <email address hidden>
Date: Fri May 26 14:04:44 2017 +0900

    Calculate stopped instance's disk sizes for disk_available_least

    disk_available_least is a free disk size information of hypervisors.
    This is calculated by the following fomula:

    disk_available_least = <free disk size> - <Total gap between virtual
    disk size and actual disk size for all instances>

    But stopped instance's virtual disk sizes were not calculated
    after merging following patch in Juno cycle:
    https://review.openstack.org/#/c/105127

    So disk_available_least might be larger than actual free disk size.
    As a result, instances might be scheduled beyond the actual free
    disk size if stopped instances were on a host.

    This patch fix it.
    Stopped instance's disks will be calculated after merging this patch.

    Change-Id: I8abf6edfa80e3920539e4f6d4906c573f9543b91
    Closes-Bug: #1693679
    (cherry-picked from commit 3342215034c7a7a938c497c39f6405763201a261)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.0.6 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/474034
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6ceb92524a9fac6e7cffcc05b1377e3cd99c1cb8
Submitter: Jenkins
Branch: stable/newton

commit 6ceb92524a9fac6e7cffcc05b1377e3cd99c1cb8
Author: Rikimaru Honjo <email address hidden>
Date: Fri May 26 14:04:44 2017 +0900

    Calculate stopped instance's disk sizes for disk_available_least

    disk_available_least is a free disk size information of hypervisors.
    This is calculated by the following fomula:

    disk_available_least = <free disk size> - <Total gap between virtual
    disk size and actual disk size for all instances>

    But stopped instance's virtual disk sizes were not calculated
    after merging following patch in Juno cycle:
    https://review.openstack.org/#/c/105127

    So disk_available_least might be larger than actual free disk size.
    As a result, instances might be scheduled beyond the actual free
    disk size if stopped instances were on a host.

    This patch fix it.
    Stopped instance's disks will be calculated after merging this patch.

    Change-Id: I8abf6edfa80e3920539e4f6d4906c573f9543b91
    Closes-Bug: #1693679
    (cherry-picked from commit 3342215034c7a7a938c497c39f6405763201a261)
    (cherry-picked from commit 2a113159c00b51215f3506a9e9834fa7cc0d5ffb)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 14.0.8 release.