Absolute Limits fails when returned unknown limits

Bug #1546767 reported by Byron McCollum
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python-novaclient
Fix Released
Undecided
rahul bardia

Bug Description

Running `nova absolute-limits` or `nova limits` can fail when unaccounted for limits are returned, with the error:

DEBUG (shell:894) Field names must be unique!
Traceback (most recent call last):
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/shell.py", line 892, in main
    OpenStackComputeShell().main(argv)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/shell.py", line 819, in main
    args.func(self.cs, args)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3200, in do_absolute_limits
    _print_absolute_limits(limits)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/v2/shell.py", line 3264, in _print_absolute_limits
    utils.print_list(limit_list, columns)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/novaclient/utils.py", line 170, in print_list
    pt = prettytable.PrettyTable([f for f in fields], caching=False)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 122, in __init__
    self.field_names = field_names
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 412, in _set_field_names
    self._validate_option("field_names", val)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 258, in _validate_option
    self._validate_field_names(val)
  File "/Users/bgmccollum/Documents/public-cloud-creds/.venv/lib/python2.7/site-packages/prettytable.py", line 302, in _validate_field_names
    raise Exception("Field names must be unique!")
Exception: Field names must be unique!
ERROR (Exception): Field names must be unique!

The way absolute limits are display was refactored to combine a limit type's "Used" and "Max" values into a single pretty table row.

Old Format:

+--------------------------+--------+
| Name | Value |
+--------------------------+--------+
| maxServerMeta | 40 |
| maxPersonality | 5 |
| totalPrivateNetworksUsed | 2 |
| maxImageMeta | 40 |
| maxPersonalitySize | 1000 |
| maxTotalRAMSize | 131072 |
| maxSecurityGroupRules | -1 |
| maxTotalKeypairs | 100 |
| totalCoresUsed | 55 |
| totalRAMUsed | 56320 |
| maxSecurityGroups | -1 |
| totalFloatingIpsUsed | 0 |
| totalInstancesUsed | 15 |
| totalSecurityGroupsUsed | 0 |
| maxTotalPrivateNetworks | 10 |
| maxTotalFloatingIps | -1 |
| maxTotalInstances | 100 |
| maxTotalCores | -1 |
+--------------------------+--------+

New Format:

+--------------------+--------+--------+
| Name | Used | Max |
+--------------------+--------+--------+
| Cores | 103 | -1 |
| FloatingIps | 0 | -1 |
| ImageMeta | - | 40 |
| Instances | 27 | 250 |
| Keypairs | - | 100 |
| Personality | - | 5 |
| Personality Size | - | 1000 |
| PrivateNetworks | 2 | 30 |
| RAM | 108544 | 512000 |
| SecurityGroupRules | - | -1 |
| SecurityGroups | 0 | -1 |
| Server Meta | - | 40 |
+--------------------+--------+--------+

This is accomplished by a static limit_map that looks like the following:

    limit_map = {
        'maxServerMeta': {'name': 'Server Meta', 'type': 'max'},
        'maxPersonality': {'name': 'Personality', 'type': 'max'},
        'maxPersonalitySize': {'name': 'Personality Size', 'type': 'max'},
        'maxImageMeta': {'name': 'ImageMeta', 'type': 'max'},
        'maxTotalKeypairs': {'name': 'Keypairs', 'type': 'max'},
        'totalCoresUsed': {'name': 'Cores', 'type': 'used'},
        'maxTotalCores': {'name': 'Cores', 'type': 'max'},
        'totalRAMUsed': {'name': 'RAM', 'type': 'used'},
        'maxTotalRAMSize': {'name': 'RAM', 'type': 'max'},
        'totalInstancesUsed': {'name': 'Instances', 'type': 'used'},
        'maxTotalInstances': {'name': 'Instances', 'type': 'max'},
        'totalFloatingIpsUsed': {'name': 'FloatingIps', 'type': 'used'},
        'maxTotalFloatingIps': {'name': 'FloatingIps', 'type': 'max'},
        'totalSecurityGroupsUsed': {'name': 'SecurityGroups', 'type': 'used'},
        'maxSecurityGroups': {'name': 'SecurityGroups', 'type': 'max'},
        'maxSecurityGroupRules': {'name': 'SecurityGroupRules', 'type': 'max'},
        'maxServerGroups': {'name': 'ServerGroups', 'type': 'max'},
        'totalServerGroupsUsed': {'name': 'ServerGroups', 'type': 'used'},
        'maxServerGroupMembers': {'name': 'ServerGroupMembers', 'type': 'max'},
    }

This works OK for a standard Vanilla OpenStack deployment today, but is quite fragile if new limit types are introduced in the future, or if a deployer sends back other custom limits.

For instance, my limits include `maxTotalPrivateNetworks` and `totalPrivateNetworksUsed` limits, which do not map accordingly, and thus result in an error.

There seems to be a well established naming convention for these limits:

max[limit-type]
total[limit-type]Used

The Used / Max mapping could be done in a more dynamic manner, which will allow deployers to return custom limits, and future proofing the Nova client from any newly introduced limits going forward.

description: updated
Changed in python-novaclient:
assignee: nobody → Mohammed Ashraf (mohammed-asharaf)
status: New → In Progress
Changed in python-novaclient:
assignee: Mohammed Ashraf (mohammed-asharaf) → nobody
Changed in python-novaclient:
assignee: nobody → Mohammed Ashraf (mohammed-asharaf)
Changed in python-novaclient:
assignee: Mohammed Ashraf (mohammed-asharaf) → Bhargavi (challa-bhargavi1)
Revision history for this message
Bhargavi (challa-bhargavi1) wrote :

Can you please provide the steps for reproduce this issue.

Revision history for this message
Byron McCollum (byron-mccollum) wrote :

@Bhargavi

It's in the description: Running `nova absolute-limits` or `nova limits` can fail when unaccounted for limits are returned. See the bug description for the error returned.

This was using python-novaclient 3.2.0 against Rackspace Public Cloud. I do not know which release of OpenStack Rackspace runs in its public cloud, so I can't answer that.

If the logic for mapping is refactored instead of using the static mapping, then it everything should just work...

Cheers
Byron

Bhargavi (bhargavi-c81)
Changed in python-novaclient:
assignee: Bhargavi (challa-bhargavi1) → nobody
Changed in python-novaclient:
assignee: nobody → rahul bardia (rahulbardia)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-novaclient (master)

Reviewed: https://review.openstack.org/527922
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=43f093623a766b743daca8ba467bb857611a2c64
Submitter: Zuul
Branch: master

commit 43f093623a766b743daca8ba467bb857611a2c64
Author: Rahul <email address hidden>
Date: Thu Dec 14 15:55:43 2017 +0530

    nova limits ERROR (Exception): Field names must be unique

    Running `nova absolute-limits` or `nova limits` can fail
    when unaccounted for limits are returned

    This works OK for a standard Vanilla OpenStack deployment
    today, but it is quite fragile if new limit types are
    introduced, or if a deployer sends back
    other custom limits.
    For instance, my limits include `maxTotalPrivateNetworks`
    and `totalPrivateNetworksUsed` limits, which do not map
    accordingly, and thus result in an error.

    This happens when 'Others' field(custom limit type) is
    occuring more than once which is not handled in code
    causing multiple columns having same name.
    Which throws exception and does not pretty prints
    absolute limit.

    This fix should also fix any future custom limits.

    Closes-bug: #1546767

    Change-Id: I1d3cf707722fc71c20cf4ec517b3f4f4875480e0

Changed in python-novaclient:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-novaclient 10.2.0

This issue was fixed in the openstack/python-novaclient 10.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.