using percentage for hugepage overcommits pages

[Krzysztof Klimonda]

Trying to define hugepage based on a percentage of total memory results in Juju assigning most of the memory for hugepages. That's due to the fact that hugepage_size is assumed to be in KB, but psutil.virtual_memory() outputs bytes.

[James Page (james-page)]

def get_hugepage_number():
    # TODO: defaults to 2M - this should probably be configurable
    # and support multiple pool sizes - e.g. 2M and 1G.
    hugepage_size = 2048
    hugepage_config = config('hugepages')
    hugepages = None
    if hugepage_config:
        if hugepage_config.endswith('%'):
            import psutil
            mem = psutil.virtual_memory()
            hugepage_config_pct = hugepage_config.strip('%')
            hugepage_multiplier = float(hugepage_config_pct) / 100
            hugepages = int((mem.total * hugepage_multiplier) / hugepage_size)
        else:
            hugepages = int(hugepage_config)
    return hugepages

[James Page (james-page)]

Correction should be:

  hugepage_size = 2048 *1024

however do note that the preferred/reliable way to allocate hugepages is todo it via kernel boot options set via MAAS tags.

[Krzysztof Klimonda (kklimonda)]

Yes, but the preferred way works only with systemd-enabled systems according to the documentation, and we are not yet ready to update our base system to 16.04. Your patch is how I've fixed it locally.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/453051

[James Page (james-page)]

https://review.openstack.org/453051

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/453051
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=b794b4a6f751c2cb26cece61ffe0977c7157abad
Submitter: Jenkins
Branch: master

commit b794b4a6f751c2cb26cece61ffe0977c7157abad
Author: James Page <email address hidden>
Date: Tue Apr 4 08:42:10 2017 +0100

    Correct hugepage percentage memory calculation

    psutil.virtual_memory() returns values in bytes, not kilobytes.

    Ensure 2MB hugepage size is calculated in the same units, ensuring
    that % calculations are correct.

    Resync of charmhelpers to fixup amulet test issues due to newer
    client versions.

    Change-Id: I95c431edcf316d194de752048ec1503d8fe996f1
    Closes-Bug: 1678185