Enlisted Nodes Cannot Access Metadata IPv6 URL

[Jacky Chu]

In an IPv6-only environment, enlisted nodes of MAAS are unable to parse the metadata url provided to cloud-init after a lookup to the internal DNS if it is an IPv6 address. This ultimately blocks nodes from being commissioned as cloud-init fails with a DatasourceNotFoundException. This causes the following error (or see attached screenshot):

In this example, the MAAS node has an IPv6 address at xxxx:xxxx:a9be:0305::2:0.

[WARNING]: retrieving url "http://xxxx:xxxx:a9be:0305::2:0:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed" failed: Failed to parse: http://xxxx:xxxx:a9be:0305::2:0:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed

Looking at the nginx http logs indicate that the request never made it to MAAS. This might be an issue with the url_helper.read_file_or_url method in the cloudinit main.py file where the conversion of the internal MAAS DNS record to an IPv6 address omits the square brackets required to make a valid http address. The address http://[xxxx:xxxx:a9be:0305::2:0]:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed can be successfully accessed from curl on the IPv6 network. On a side note, all the configuration files including /etc/maas/preseed/enlist wrap their IPv6 addresses with square brackets.

Build:
DEB packages maas/focal,now 1:3.2.6-12016-g.19812b4da-0ubuntu1-20.04.1

Steps to Reproduce:

1. Create an IPv6-only network to attach one MAAS node and one node to commission.

2. Install MAAS onto Focal using the 3.2 DEB packages and create an admin.

3. Go to the controller VLAN and enable DHCPv6 on a dynamic reserved range for the IPv6 subnet. Leave the gateway blank. Modify the IPv6 subnet to allow internal DNS resolution and set the DNS server to the MAAS-node's global IPv6 address.

4. Attempt to PXE-boot with IPv6 from a node on the same IPv6 network. Boot should succeed in loading Ubuntu image but fails when it attempts to access metadata url at the MAAS node's IPv6 address.

Note: one weird non-related issue with step 4 is that PXE booting appears to only work when the last octet of the MAAS IPv6 address ends with all zeros, e.g. xxxx:xxxx:a9be:a305::2:0 in the provided screenshot. This can be confirmed by running tcpdump on the MAAS node where the commissioned node sends Neighbor Solicitations for the MAAS node's IPv6 address but with the last octet of all zeros when searching for the tftp server.

Logs:
/var/log/maas/maas.log - nothing relevant
/var/log/maas/regiond.log - nothing relevant
/var/log/maas/rackd.log - shows that boot loader, grub, and squashfs are successfully requested by the enlisting node
/var/log/maas/rsyslog/ - no logs as machines do not finish commissioning

[Federico Bosi (rhxto)]

I think this is related to https://bugs.launchpad.net/maas/+bug/2045020, which has been fixed.