Seeing incomplete modifications of the floating_ips table when deleting long-lived instances in a multi-host, auto_assign_floating_ip environment.
Here is record in the floating_ips table for the floating IP in question AFTER terminating the associated instance
"2012-06-20 21:46:30.253886";"2012-06-26 21:57:29.50243";"";FALSE;"105";"16.125.108.105";;"d7127ed3027044c4942fc2f94549489e";"";TRUE;"nova";"eth0"
The project_id column is not properly updated, nor is the auto_assigned column. The network manager code looks like it should be nulling out project_id, host, and fixed_ip_id, as well as setting auto_assigned to False. fixed_ip_id and host are updated properly.
In addition, I was finally was able to locate the related traceback that is logged when this occurs:
2012-06-26 15:57:29 ERROR nova.rpc.amqp [req-d094a9f3-a0b6-4d8e-adf7-6d9e8eb5117f e4a51bc329a74001806ce7820b784313 d7127ed3027044c4942fc2f94549489e] Exception during message handling
2012-06-26 15:57:29 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/rpc/amqp.py", line 253, in _process_data
2012-06-26 15:57:29 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 257, in wrapped
2012-06-26 15:57:29 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 374, in deallocate_for_instance
2012-06-26 15:57:29 TRACE nova.rpc.amqp affect_auto_assigned=True)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 257, in wrapped
2012-06-26 15:57:29 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 425, in deallocate_floating_ip
2012-06-26 15:57:29 TRACE nova.rpc.amqp raise exception.FloatingIpAssociated(address=floating_address)
2012-06-26 15:57:29 TRACE nova.rpc.amqp FloatingIpAssociated: Floating ip 16.125.108.105 is associated.
Seeing incomplete modifications of the floating_ips table when deleting long-lived instances in a multi-host, auto_assign_ floating_ ip environment.
Here is record in the floating_ips table for the floating IP in question AFTER terminating the associated instance 253886" ;"2012- 06-26 21:57:29. 50243"; "";FALSE; "105";" 16.125. 108.105" ;;"d7127ed30270 44c4942fc2f9454 9489e"; "";TRUE; "nova"; "eth0"
"2012-06-20 21:46:30.
The project_id column is not properly updated, nor is the auto_assigned column. The network manager code looks like it should be nulling out project_id, host, and fixed_ip_id, as well as setting auto_assigned to False. fixed_ip_id and host are updated properly.
In addition, I was finally was able to locate the related traceback that is logged when this occurs:
2012-06-26 15:57:29 ERROR nova.rpc.amqp [req-d094a9f3- a0b6-4d8e- adf7-6d9e8eb511 7f e4a51bc329a7400 1806ce7820b7843 13 d7127ed3027044c 4942fc2f9454948 9e] Exception during message handling python2. 6/site- packages/ nova/rpc/ amqp.py" , line 253, in _process_data context= ctxt, **node_args) python2. 6/site- packages/ nova/network/ manager. py", line 257, in wrapped python2. 6/site- packages/ nova/network/ manager. py", line 374, in deallocate_ for_instance auto_assigned= True) python2. 6/site- packages/ nova/network/ manager. py", line 257, in wrapped python2. 6/site- packages/ nova/network/ manager. py", line 425, in deallocate_ floating_ ip FloatingIpAssoc iated(address= floating_ address) iated: Floating ip 16.125.108.105 is associated.
2012-06-26 15:57:29 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/
2012-06-26 15:57:29 TRACE nova.rpc.amqp rval = node_func(
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/
2012-06-26 15:57:29 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/
2012-06-26 15:57:29 TRACE nova.rpc.amqp affect_
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/
2012-06-26 15:57:29 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-06-26 15:57:29 TRACE nova.rpc.amqp File "/usr/lib/
2012-06-26 15:57:29 TRACE nova.rpc.amqp raise exception.
2012-06-26 15:57:29 TRACE nova.rpc.amqp FloatingIpAssoc