dhcp not respecting reserved ranges

[james beedy]

I define a dynamic and reserved range on my subnet, all node interfaces are set to DHCP. Somehow some of my nodes are still getting ip addresses inside my reserved range.

Notice nodes 0 and 5 http://paste.ubuntu.com/26092132/

See screenshot of dhcp configuration attached.

[Andres Rodriguez (andreserl)]

@james,

Can you please clarify,

1. Were the ranges created before you started deploying with juju?
2. Can you provide MAAS logs?
3. Can you provide juju debug logs, specially when the co gainers are created with MAAS and ask for IP’s.

Thanks

[Andres Rodriguez (andreserl)]

Also, please attach your dhcpd.conf

[Andres Rodriguez (andreserl)]

actually, two errors on my part. There are no containers and they are machines.

So, the only way they could be getting ip's from that range is if:

1. The subnet is *unmanaged* instead of *managed*
2. MAAS is giving IP's automatically even when "DHCP" is the assignment
3. There's a previous lease and the machine is just asking for that.
4. The dhcpd.conf has that reserved range.

Can you confirm a couple things then:

1. What is the subnet, is it unmanaged or managed?
2. What does the dhcpd.conf look like?
3. What does the lease file look like?

[Mike Pontillo (mpontillo)]

The MAAS API allows reserved ranges to be used for allocation, but only if specifically requested. MAAS on its own should not allocate from a reserved range as shown.

That said, is it possible that a static static lease is present for the MACs of those machines? (Maybe another DHCP server?) Can you confirm the presence of leases on the MAAS DHCP server for the machines that have IP addresses on the reserved range? In addition to the debugging that Andres has requested, it might also be helpful to see logs from the DHCP server, and the lease file.

In addition, I would like to see the output of the following CLI commands:

    maas $PROFILE subnet statistics $CIDR include_ranges=True

    sudo -u postgres psql maasdb -P pager=off \
        -c 'select * from maas_support__ip_allocation'

[james beedy (jamesbeedy)]

@andreserl @mpontillo

I'm willing to bet the machines that have ip addresses from the reserved range got a lease (from MAAS) before the reserved range was created. I had possibly configured the node interfaces to DHCP prior to creating the reserved range, and the lease has stuck around - even after deletion/reenlistment.

$ juju status | pastebinit - (SIDE NOTE: juju should display the ip address for the default space for the machine instead of .... well I can't tell how juju chooses what ip address to display, but I feel it would be more aesthetically pleasing if the ips were consistent - possibly this just isnt worth putting the effort into though)
http://paste.ubuntu.com/26106223/

$ for i in {0..8}; do echo "$i - "; juju run --machine $i "ip a" | pastebinit; done
0 -
http://paste.ubuntu.com/26106262/
1 -
http://paste.ubuntu.com/26106263/
2 -
http://paste.ubuntu.com/26106266/
3 -
http://paste.ubuntu.com/26106268/
4 -
http://paste.ubuntu.com/26106270/
5 -
http://paste.ubuntu.com/26106271/
6 -
http://paste.ubuntu.com/26106273/
7 -
http://paste.ubuntu.com/26106274/
8 -
http://paste.ubuntu.com/26106275/

Notice machine 0 ip from the reserved range - the reserved range was configured long before bootstrapping MAAS with juju and deploy what is machine-0^.

@mpontillo

$ sudo maas admin subnet statistics 10.10.0.0/16 include_ranges=True | pastebinit
sudo: unable to resolve host maas-region-rack-00
http://paste.ubuntu.com/26106335/

$ psql -U juju_maas-region-rack -h 10.10.0.110 maasdb -P pager=off -c 'select * from maas_support__ip_allocation' | pastebinit
Password for user juju_maas-region-rack:
http://paste.ubuntu.com/26106349/

[james beedy (jamesbeedy)]

$ juju status --format yaml | pastebinit
http://paste.ubuntu.com/26106373/

[james beedy (jamesbeedy)]

dhcp leases and conf

$ sudo cat /var/snap/maas/current/var/lib/maas/dhcp/dhcpd.leases | pastebinit
http://paste.ubuntu.com/26106395/

$ sudo cat /var/snap/maas/current/var/lib/maas/dhcpd.conf | pastebinit
http://paste.ubuntu.com/26106398/

[james beedy (jamesbeedy)]

just sent off a new deploy, and whats odd is that I see the dhcp error in the logs, but this time, my lxd containers all deployed successfully

debug-log http://paste.ubuntu.com/26113536/

$ juju status | pastebinit
http://paste.ubuntu.com/26113564/

$ juju status --format yaml | pastebinit
http://paste.ubuntu.com/26113566/

[james beedy (jamesbeedy)]

geh, ignore #9, wrong bug

[Mike Pontillo (mpontillo)]

The data presented in comment #6 is interesting, but still inconclusive.

The MAAS database shows no record of the 10.10.200.19 and 10.10.200.11 IPs being assigned to any node, nor does the current lease database.

If you were to "juju ssh" to those nodes, how is /etc/network/interfaces configured? If it's set to DHCP, it's unlikely to ever be able to renew that lease from MAAS. (Possibly they were set to auto-assign before the reservation was created, or possibly the subnet was accidentally switched into "Unmanaged" mode for a short time, whereby it would have allocated directly from the reserved ranges on purpose?) But that theory can be eliminated if you consider that the MAAS database has no record of those IP assignments.

So your theory about the machines getting those IPs before you finalized the range configuration sounds likely, but I doesn't explain everything about this situation.

[Alberto Donato (ack)]

Is this still happening? If so what version of the snap is currently being used?

[Test User (whisperx)]

This bug is still affecting MAAS 2.6 where if I enable DHCP on a custom range in a subnet (e.g. 10.0.30.180 - 10.0.30.254) before powering on any compute nodes to be added and commissioned, they will take an IP from the reserved range the first time they run for commissioning (like 10.0.30.180 10.0.30.181 etc.), but when deploying they will get an IP outside the range (e.g. 10.0.30.3 10.0.30.4).

I have been able to reproduce in many test installations.

[Launchpad Janitor (janitor)]

[Expired for MAAS because there has been no activity for 60 days.]