nova-manage out of memory error creating many floating IPs

[David Ripton]

https://bugzilla.redhat.com/show_bug.cgi?id=918761

Summarizing the downstream bug report, if we do something silly like CONFIG_NOVA_NETWORK_FLOATRANGE=192.168.2.224/2 , which creates an unreasonably huge number of floating IP addresses, nova-manage can run out of memory.

The workaround is to loop over the range and not create more than BIG_NOT_HUGE_NUMBER of floating IP addresses in one call to floating_ip_bulk_create. (At which point trying to create a silly number of floating IPs would probably still fail, but with a better exception related to exhausting IP address, rather than an OutOfMemoryError.)

[David Ripton (dripton)]

floating_ip_bulk_create reads all existing floating IPs out of the database and shoves them into a dict. So fixing nova-manage to work in chunks will actually just move the problem, and we'll run out of memory in floating_ip_bulk_create instead.

So I think the real fix is to make nova-manage FloatingIpCommands.address_to_hosts raise an exception up front if we give it silly input like "192.168.2.224/2", rather than having it attempt to create millions of IPs then blow up later.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/25918

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/25918
Committed: http://github.com/openstack/nova/commit/34de8d1529fb9a2e7fad096a6967b9a483ab0cb0
Submitter: Jenkins
Branch: master

commit 34de8d1529fb9a2e7fad096a6967b9a483ab0cb0
Author: David Ripton <email address hidden>
Date: Tue Apr 2 12:19:17 2013 -0400

    Raise rather than generating millions of IPs.

    Fixes bug #1163394

    Formerly if we tried to create many millions of floating IPs with a
    range like "192.168.2.224/2", address_to_hosts would happily iterate
    over all of them (which would take a long time) and then nova-manage
    would eventually crash with an OutOfMemoryError when we tried putting
    all of them in a list. Or, if the initial list wasn't quite big
    enough to use all memory, it would blow up later (even more slowly)
    when we tried to put them in the database via SQLAlchemy.

    Now, raise exception.InvalidInput if the number of IPs is a million
    or more. (A million is erring on the side of caution.)

    Change-Id: Ifc6b6a8faadc2e97e09f9f6c975e52229f705789

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/27854

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/27854
Committed: http://github.com/openstack/nova/commit/df5d13c29eae64d9c6340aa1cd54de7b3d9b2dcb
Submitter: Jenkins
Branch: stable/grizzly

commit df5d13c29eae64d9c6340aa1cd54de7b3d9b2dcb
Author: David Ripton <email address hidden>
Date: Tue Apr 2 12:19:17 2013 -0400

    Raise rather than generating millions of IPs.

    Fixes bug #1163394

    Formerly if we tried to create many millions of floating IPs with a
    range like "192.168.2.224/2", address_to_hosts would happily iterate
    over all of them (which would take a long time) and then nova-manage
    would eventually crash with an OutOfMemoryError when we tried putting
    all of them in a list. Or, if the initial list wasn't quite big
    enough to use all memory, it would blow up later (even more slowly)
    when we tried to put them in the database via SQLAlchemy.

    Now, raise exception.InvalidInput if the number of IPs is a million
    or more. (A million is erring on the side of caution.)

    Change-Id: Ifc6b6a8faadc2e97e09f9f6c975e52229f705789
    (cherry picked from commit 34de8d1529fb9a2e7fad096a6967b9a483ab0cb0)