Comment 28 for bug 1353962

Revision history for this message
Matt Riedemann (mriedem) wrote : Re: Test job failes with FixedIpLimitExceeded with nova network

Here we go, we have better logging in the DB API when this fails now:

http://logs.openstack.org/01/167101/5/check/check-tempest-dsvm-full/9f94b8c/logs/screen-n-cond.txt.gz#_2015-03-25_19_19_37_678

Raise OverQuota exception because: project_quotas: {'fixed_ips': -1}, user_quotas: {'fixed_ips': 99L}, deltas: {u'fixed_ips': 1}, overs: [u'fixed_ips'], project_usages: {u'ram': {'total': 128L, 'reserved': 0L, 'in_use': 128L}, u'floating_ips': {'total': 1L, 'reserved': 0L, 'in_use': 1L}, u'instances': {'total': 2L, 'reserved': 0L, 'in_use': 2L}, u'cores': {'total': 2L, 'reserved': 0L, 'in_use': 2L}, u'fixed_ips': {'total': 1L, 'reserved': 0L, 'in_use': 1L}, u'security_groups': {'total': 1L, 'reserved': 0L, 'in_use': 1L}}, user_usages: {u'ram': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedba50>, u'floating_ips': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedb5d0>, u'instances': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedbc90>, u'cores': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedb750>, u'fixed_ips': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedb510>, u'security_groups': <nova.db.sqlalchemy.models.QuotaUsage object at 0x7faaccedbf90>}

So project_quotas and user_quotas are different:

project_quotas: {'fixed_ips': -1}, user_quotas: {'fixed_ips': 99L}

The user_quotas are what's passed back to the network manager which is why the log message is confusing:

http://git.openstack.org/cgit/openstack/nova/tree/nova/db/sqlalchemy/api.py?id=2015.1.0b3#n3578

So we see this in the n-net logs:

http://logs.openstack.org/01/167101/5/check/check-tempest-dsvm-full/9f94b8c/logs/screen-n-net.txt.gz?level=TRACE#_2015-03-25_19_19_37_685

2015-03-25 19:19:37.685 WARNING nova.network.manager [req-0f01e070-e2ac-4058-a7cb-9d619bb557d6 FloatingIPsTestJSON-1260806534 FloatingIPsTestJSON-109816172] [instance: c90d1a08-4be7-4f88-8a73-36424d70acd6] Quota exceeded for project e3bba3eb15c844cda054b8c73ec68c0b, tried to allocate fixed IP. 1 of 99 are in use or are already reserved.

So from project_usages, we have this:

u'fixed_ips': {'total': 1L, 'reserved': 0L, 'in_use': 1L}

Which is why we go over-quota, because it thinks we have 1 total fixed_ip to use and it's already in use.