max_over_subscription_ratio is miscalculated for drivers reporting list [True, False] for thin_provisioning

[Goutham Pacha Ravi]

For a pool that supports thin_provisioning, the capacity weigher uses the configuration of max_over_subscription_ratio to figure out the expected pool capacity as against the reported capacity.

However, currently :https://github.com/openstack/manila/blob/af2e103/manila/scheduler/weighers/capacity.py#L66 lists resolve to 'True' and if thick_provisioned volumes eat up capacity, the calculation goes awry.

[Goutham Pacha Ravi (gouthamr)]

Looking at the way the current filter is implemented, this is a fail point for backends that can support both thick/thin shares. If we intend to support both thick/thin, the following changes are necessary: (one way of solving the problem, open for discussion)

1) Make every share_type have a required extra_spec - 'thin_provisioning' (False by default)
2) Copy this extra_spec into the share model, i.e brand a share as being 'thick' or 'thin'
3) In the Capacity Filter, calculate the sum of sizes of 'thick' shares that have already been accommodated on your pool that supports both 'thick' and 'thin' shares (simple DB call), subtract this from the advertised 'free_space' before multiplying this free_space with the max_over_subscription_ratio and inflating it as 'adjusted_free_virtual' in case you are provisioning a 'thin' share. If you are provisioning a thick share, a simple free space check should suffice.

Thoughts?

[Goutham Pacha Ravi (gouthamr)]

The only driver that supports both thick/thin for all pools currently is the NetApp driver.

[Goutham Pacha Ravi (gouthamr)]

Fix proposed to branch: master
Review: https://review.openstack.org/315266

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/367466

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/315266
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=83c93c776707d5da9c89cde81c03022178127261
Submitter: Jenkins
Branch: master

commit 83c93c776707d5da9c89cde81c03022178127261
Author: xing-yang <email address hidden>
Date: Wed May 11 15:23:31 2016 -0400

    Check 'thin_provisioning' in extra specs

    Currently in the capacity filter and weigher of the scheduler,
    we use the logic to evaluate whether there is enough capacity
    to thin provision a share on a backend if the driver reports
    thin_provisioning to be True. However, a driver may be able to
    support both thin and thick provisioning. The logic does not
    check whether the user wants the share to be provisioned as thin
    or not.

    In this patch, we check 'thin_provisioning' in extra specs of the
    share type and decide whether to use the logic for thin or thick.
    In the following two cases, we will use the thin logic:
    1) 'thin_provisioning' is not set in extra specs. This is to
       provide backward compatibility.
    2) 'thin_provisioning' is set in extra specs and it is
       '<is> True' or 'True'.

    To provision a thick share on a backend that supports both
    thin and thick, set one of the following in extra specs:
        {'thin_provisioning': 'False'}
        {'thin_provisioning': '<is> False'}
        {'capabilities:thin_provisioning': 'False'}
        {'capabilities:thin_provisioning': '<is> False'}

    DocImpact
    Change-Id: I238a7962425ea35c356c5ed2e31b8f68462b3769
    Closes-Bug: #1578718

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/367466
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=2d957f389086bb1499dacf5c76a1fba21f99cc54
Submitter: Jenkins
Branch: master

commit 2d957f389086bb1499dacf5c76a1fba21f99cc54
Author: xing-yang <email address hidden>
Date: Thu Jul 21 09:14:04 2016 -0400

    Documentation changes for thin/thick provisioning

    This patch adds documentation changes for the logic changed
    to address thin/thick provisioning in the following patch:
        https://review.openstack.org/#/c/315266/

    Change-Id: I6a15a572abd7ccec59917bd58db5e3cca32d77a3
    Partial-Bug: #1578718

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 3.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/manila 3.0.0 release.