network_rpcapi.allocate_for_instance timing out under load.

[Joe Gordon]

Environment:

* devstack with nova Grizzly-RC1
* 'compute_driver = nova.virt.fake.FakeDriver' in nova.conf
* following nova branch https://github.com/jogo/nova/commits/perf_logging (commit c55372908de3e66bc000ebb9dd17f688b4914101)
* no quantum

when trying to run 50 VMs in devstack using 'euca-run-instances ami-00000003 -t m1.micro -n 100'

I see network_rpcapi.allocate_for_instance occasionally RPC timing out (call from nova-compute to nova-network).

nova-compute log http://paste.openstack.org/show/34252/
nova-network log http://paste.openstack.org/show/34255/

[Vish Ishaya (vishvananda)]

I suspect you are running into iptables lock contention. I'm not sure if there is a good fix for this besides updating the rpc timeout.

[Joe Gordon (jogo)]

I am surprised we are seeing this break with just 50 VMs.

[Matt Riedemann (mriedem)]

Maybe this is a duplicate of bug 1311778?

[Joe Gordon (jogo)]

This has been fixed.