Replying to questions in #2 below. It is clear that we require a spec, and carefully think trough this before merging any implementation.

> 1. In case which You described with Ironic instance,
>    each of those 3 dhcp clients will send dhcp request
>    with different DUID/IAID so we should send them 3
>    different IP addresses?

Yes that is the idea. It would be left to the DHCP server to allocate and address from the pool.

> 2. If we will do that per port, how neutron will
>    actually know which IPs are already taken and what
>    IPs can be allocated to "standard" VMs?

The idea is that neutron won't know exactly which IPs are already taken, it will only know that pool(s) of addresses is outside it's contol. This pool of addresses should never be allocated to "standard" VMs. The dhcp server would have to be the one keeping track of the leased addresses in the dynamic pool.

In the Ironic example, a dynamic address will be used during instance provisioning. Once Ironic is done provisioning, a port with a static address, i.e a "standard" neutron port with a neutron ipam address allocation etc is bound to the final baremetal instance.

There is a HA challenge here, I did'nt think about this before today. Only one dhcp-agent should configure it's dhcp server instance with the dynamic pool. We don't want two dhcp servers handing out addresses from one dynamic pool, we would end up with conflicts. Unless there is a way to keep leased addresses in sync. I don't think dnsmasq can do that. For IPv6 there is a lot of addresses, so we might be able to split a pool into separate a pool into per-agent sub-pools.

We may be able to sync back the used IP addresses to neutron, the dhcp-agent could read the lease file and report back to neutron. But in the dhcpv6 case it's extra tricky since there is no data in the leasefile to identify the host. For dhcpv4 the MAC address is in the lease file. (Ref docstring: https://github.com/openstack/neutron/blob/master/neutron/agent/linux/dhcp.py#L810) Another option would be to use the --dhcp-script option in dnsmasq to do a callback to the neutron dhcp-agent when an address is leased, the environment of the script would include "DNSMASQ_MAC - containing the MAC address of the client, if known."




Other options:
1. This patch to dnsmasq completely solves the problem:
 http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2018q4/012707.html

Several people have asked on the list to consider merging this patch, but to date the dnsmasq maintainer did not respond. The maintainer is not a fan of the idea of ignoring the changing IAID in this mail thread: http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2017q1/011267.html


2. If UEFI firmware bug https://bugzilla.tianocore.org/show_bug.cgi?id=1518 is fixed, we could push for iPXE to either support doing a DHCPv6 RELEASE before chainloading. (Or at least make it possible to pass the lease information to the chainloaded OS so that it can take-over or do a RELEASE. ref: http://lists.ipxe.org/pipermail/ipxe-devel/2019-November/006805.html and http://lists.ipxe.org/pipermail/ipxe-devel/2019-December/006828.html

3. It is possible a different DHCP server behaves differently. (I know that ISC-DHCP dhcpd6 ignores the changed IAID when the server uses the client's DUID as identifier for static DHCP.) ISC-DHCP however cannot operate with MAC address as the key for static addressing unless the ISC-DHCP client is also used ... They expect the MAC in an option of the DHCPv6 SOLICIT, which is not according to RFC ... I did'nt try Kea DHCP yet.