TypeError: unorderable types: int() < NoneType() possible during quota limit check

[melanie witt]

Saw this on a recent py35 unit test run [1]:

Traceback (most recent call last):
  File "/home/jenkins/workspace/gate-nova-python35/nova/tests/unit/network/test_manager.py", line 1234, in test_allocate_floating_ip
    self.network.allocate_floating_ip(ctxt, ctxt.project_id)
  File "/home/jenkins/workspace/gate-nova-python35/nova/network/floating_ips.py", line 223, in allocate_floating_ip
    project_id)
  File "/home/jenkins/workspace/gate-nova-python35/.tox/py35/lib/python3.5/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
    result = fn(cls, context, *args, **kwargs)
  File "/home/jenkins/workspace/gate-nova-python35/nova/objects/quotas.py", line 201, in check_deltas
    cls.limit_check_project_and_user(context, **check_kwargs)
  File "/home/jenkins/workspace/gate-nova-python35/.tox/py35/lib/python3.5/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
    result = fn(cls, context, *args, **kwargs)
  File "/home/jenkins/workspace/gate-nova-python35/nova/objects/quotas.py", line 136, in limit_check_project_and_user
    project_id=project_id, user_id=user_id)
  File "/home/jenkins/workspace/gate-nova-python35/nova/quota.py", line 1639, in limit_check_project_and_user
    user_values=user_values, project_id=project_id, user_id=user_id)
  File "/home/jenkins/workspace/gate-nova-python35/nova/quota.py", line 628, in limit_check_project_and_user
    overs = [key for key, val in merged_values.items()
  File "/home/jenkins/workspace/gate-nova-python35/nova/quota.py", line 629, in <listcomp>
    if merged_quotas[key] >= 0 and merged_quotas[key] < val]
TypeError: unorderable types: int() < NoneType()

The problem is some logic that tries to .get(key) a value from one of two dicts that contains the key [2] and if the found value is 0, it won't take precedence over a default not found value of None. In the case where None is picked, we end up with a merged value of None, which blows up (thankfully) in Python 3.

[1] http://logs.openstack.org/42/446242/18/check/gate-nova-python35/fdf57fc/testr_results.html.gz
[2] https://github.com/openstack/nova/blob/c7afbb9/nova/quota.py#L613-L614

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/471973

[Sylvain Bauza (sylvain-bauza)]

Looks to me like a regression.

[Sylvain Bauza (sylvain-bauza)]

The regression could be due to https://github.com/openstack/nova/commit/832cc4709f3c4389caf49f54c5f5f6a4e8ec32ea

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/472287

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/471973
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b1647af2906db7eedc93063557497aa6204255a6
Submitter: Jenkins
Branch: master

commit b1647af2906db7eedc93063557497aa6204255a6
Author: melanie witt <email address hidden>
Date: Wed Jun 7 23:58:25 2017 +0000

    Default to 0 when merging values in limit check

    When merging project_values and user_values after taking the symmetric
    difference of their keys, we should use a default value of 0 for the
    .get(key) calls because a default value of None will not take
    precedence over 0 if 0 is the value in project_values or user_values.
    In the buggy case, we end up with a merged value of None instead of 0
    because: 'None or 0' is 0 but '0 or None' is None.

    Closes-Bug: #1696605

    Change-Id: I54a3a52cf5e9b7826dad37c6d1e053ba92cd4fd9

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/472287
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=08e6b8eab0ef07ba5449eda8d37f2a912510d622
Submitter: Jenkins
Branch: master

commit 08e6b8eab0ef07ba5449eda8d37f2a912510d622
Author: Matt Riedemann <email address hidden>
Date: Thu Jun 8 11:11:53 2017 -0400

    Add docstring for test_limit_check_project_and_user_zero_values

    This came up in review for I54a3a52cf5e9b7826dad37c6d1e053ba92cd4fd9.

    The test was a bit confusing since it doesn't explicitly assert
    anything, so a docstring is added to clarify intent.

    Change-Id: I1346f67686e2196af056a1f1702702144b823ed9
    Related-Bug: #1696605

[melanie witt (melwitt)]

Just to clarify: this isn't a regression as limit_check_project_and_user() isn't used anywhere yet. This bug was noticed in the unmerged patch series that begins using limit_check_project_and_user().

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b2 development milestone.

[Deepa (dpaclt)]

Can you please let us know when this will be available publicly