Adding a random lease_time value for dhcp-agent in large scale environment

[siyingchun]

In our large scale environment, we sometimes found it can't be guaranteed when booting a large number of new instances at the same time. Meanwhile, the lease time from all these instances will also age simultaneously. In addition, it will cause a burst of the network traffic of dhcp broadcast for a while.

According to dhcp-agent, it's simply dealt with by the key word "dhcp_lease_duration" in the configuration, which is fixed in /etc/neutron/neutron.conf, with a value 600.

So our team modified this issue by adding another value, which is called dhcp_lease_random, with a random number. And it's used by dnsmasq for being plus the value when the dhcp server gives client a real lease time.

Here we use the modulo(%) operator with part of the network_id, and the modulus is the dhcp_lease_random.

* conditions:
You'd better have a large scale environment which hosts over around 300 VMs and create or delete them at the same time. Or creating them and watching them after the lease time.

* Version:
Openstack Newton, deployed with Fuel 10.0
Ubuntu Ubuntu 16.04.1 LTS, running kernel 4.4.0-57-generic
Neutron version 5.1.0
Dnsmasq version 2.75

[Armando Migliaccio (armando-migliaccio)]

I fear that this is a bit of an edge case, in reality leases won't get renewed exactly at the same time due to the distributed nature of the boot process for every single VM; adding a splay can increase the randomness, but I wonder if that comes at the expenses of complicating troubleshooting.

[Brian Haley (brian-haley)]

If this is for a benchmark, why can't you set the lease for a longer period of time, or infinite, so it doesn't interfere? Plus, any lease renewal should be unicast to the server, the initial request will be broadcast though.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/434663
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Bug closed due to lack of activity, please feel free to reopen if needed.