Absolute Limits fails when returned unknown limits

[Byron McCollum]

Running `nova absolute-limits` or `nova limits` can fail when unaccounted for limits are returned, with the error:

DEBUG (shell:894) Field names must be unique!
Traceback (most recent call last):
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/shell.py", line 892, in main
    OpenStackComputeShell().main(argv)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/shell.py", line 819, in main
    args.func(self.cs, args)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3200, in do_absolute_limits
    _print_absolute_limits(limits)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3264, in _print_absolute_limits
    utils.print_list(limit_list, columns)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/utils.py", line 170, in print_list
    pt = prettytable.PrettyTable([f for f in fields], caching=False)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 122, in __init__
    self.field_names = field_names
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 412, in _set_field_names
    self._validate_option("field_names", val)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 258, in _validate_option
    self._validate_field_names(val)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 302, in _validate_field_names
    raise Exception("Field names must be unique!")
Exception: Field names must be unique!
ERROR (Exception): Field names must be unique!

The way absolute limits are display was refactored to combine a limit type's "Used" and "Max" values into a single pretty table row.

Old Format:

+--------------------------+--------+
| Name | Value |
+--------------------------+--------+
| maxServerMeta | 40 |
| maxPersonality | 5 |
| totalPrivateNetworksUsed | 2 |
| maxImageMeta | 40 |
| maxPersonalitySize | 1000 |
| maxTotalRAMSize | 131072 |
| maxSecurityGroupRules | -1 |
| maxTotalKeypairs | 100 |
| totalCoresUsed | 55 |
| totalRAMUsed | 56320 |
| maxSecurityGroups | -1 |
| totalFloatingIpsUsed | 0 |
| totalInstancesUsed | 15 |
| totalSecurityGroupsUsed | 0 |
| maxTotalPrivateNetworks | 10 |
| maxTotalFloatingIps | -1 |
| maxTotalInstances | 100 |
| maxTotalCores | -1 |
+--------------------------+--------+

New Format:

+--------------------+--------+--------+
| Name | Used | Max |
+--------------------+--------+--------+
| Cores | 103 | -1 |
| FloatingIps | 0 | -1 |
| ImageMeta | - | 40 |
| Instances | 27 | 250 |
| Keypairs | - | 100 |
| Personality | - | 5 |
| Personality Size | - | 1000 |
| PrivateNetworks | 2 | 30 |
| RAM | 108544 | 512000 |
| SecurityGroupRules | - | -1 |
| SecurityGroups | 0 | -1 |
| Server Meta | - | 40 |
+--------------------+--------+--------+

This is accomplished by a static limit_map that looks like the following:

    limit_map = {
        'maxServerMeta': {'name': 'Server Meta', 'type': 'max'},
        'maxPersonality': {'name': 'Personality', 'type': 'max'},
        'maxPersonalitySize': {'name': 'Personality Size', 'type': 'max'},
        'maxImageMeta': {'name': 'ImageMeta', 'type': 'max'},
        'maxTotalKeypairs': {'name': 'Keypairs', 'type': 'max'},
        'totalCoresUsed': {'name': 'Cores', 'type': 'used'},
        'maxTotalCores': {'name': 'Cores', 'type': 'max'},
        'totalRAMUsed': {'name': 'RAM', 'type': 'used'},
        'maxTotalRAMSize': {'name': 'RAM', 'type': 'max'},
        'totalInstancesUsed': {'name': 'Instances', 'type': 'used'},
        'maxTotalInstances': {'name': 'Instances', 'type': 'max'},
        'totalFloatingIpsUsed': {'name': 'FloatingIps', 'type': 'used'},
        'maxTotalFloatingIps': {'name': 'FloatingIps', 'type': 'max'},
        'totalSecurityGroupsUsed': {'name': 'SecurityGroups', 'type': 'used'},
        'maxSecurityGroups': {'name': 'SecurityGroups', 'type': 'max'},
        'maxSecurityGroupRules': {'name': 'SecurityGroupRules', 'type': 'max'},
        'maxServerGroups': {'name': 'ServerGroups', 'type': 'max'},
        'totalServerGroupsUsed': {'name': 'ServerGroups', 'type': 'used'},
        'maxServerGroupMembers': {'name': 'ServerGroupMembers', 'type': 'max'},
    }

This works OK for a standard Vanilla OpenStack deployment today, but is quite fragile if new limit types are introduced in the future, or if a deployer sends back other custom limits.

For instance, my limits include `maxTotalPrivateNetworks` and `totalPrivateNetworksUsed` limits, which do not map accordingly, and thus result in an error.

There seems to be a well established naming convention for these limits:

max[limit-type]
total[limit-type]Used

The Used / Max mapping could be done in a more dynamic manner, which will allow deployers to return custom limits, and future proofing the Nova client from any newly introduced limits going forward.

[Bhargavi (challa-bhargavi1)]

Can you please provide the steps for reproduce this issue.

[Byron McCollum (byron-mccollum)]

@Bhargavi

It's in the description: Running `nova absolute-limits` or `nova limits` can fail when unaccounted for limits are returned. See the bug description for the error returned.

This was using python-novaclient 3.2.0 against Rackspace Public Cloud. I do not know which release of OpenStack Rackspace runs in its public cloud, so I can't answer that.

If the logic for mapping is refactored instead of using the static mapping, then it everything should just work...

Cheers
Byron

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/527922
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=43f093623a766b743daca8ba467bb857611a2c64
Submitter: Zuul
Branch: master

commit 43f093623a766b743daca8ba467bb857611a2c64
Author: Rahul <email address hidden>
Date: Thu Dec 14 15:55:43 2017 +0530

    nova limits ERROR (Exception): Field names must be unique

    Running `nova absolute-limits` or `nova limits` can fail
    when unaccounted for limits are returned

    This works OK for a standard Vanilla OpenStack deployment
    today, but it is quite fragile if new limit types are
    introduced, or if a deployer sends back
    other custom limits.
    For instance, my limits include `maxTotalPrivateNetworks`
    and `totalPrivateNetworksUsed` limits, which do not map
    accordingly, and thus result in an error.

    This happens when 'Others' field(custom limit type) is
    occuring more than once which is not handled in code
    causing multiple columns having same name.
    Which throws exception and does not pretty prints
    absolute limit.

    This fix should also fix any future custom limits.

    Closes-bug: #1546767

    Change-Id: I1d3cf707722fc71c20cf4ec517b3f4f4875480e0

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-novaclient 10.2.0 release.