Azure multiple ips prevent access to metadata service

[Ryan Harper]

On an instance with a single interface and many private address, the "primary" address, is not used for outgoing communication as the source_ipaddr in packets; this prevents communicating with the IMDS service.

Example metadata:

https://paste.ubuntu.com/p/2y2NBck6VJ/

Generated netplan yaml:

https://paste.ubuntu.com/p/Zn9tgYzTMb/

[Ryan Harper (raharper)]

ip addr show output:

https://paste.ubuntu.com/p/3gHdFzkrrQ/

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Ryan Harper (raharper)]

OK, this is easily reproduced by adding a single static IP in the same subnet as the IP that comes from dhcp'ing on the interface.

For example, if you launch an instance with a 10.0.0./24 subnet, the dhcp IP is something like 10.0.0.4

You can login and

curl -H Metadata:true "http://169.254.169.254/metadata/instance/network?api-version=2017-08-01"

is successful.

If you append:

addresses:
- 10.0.0.20/24

To /etc/netplan/50-cloud-init.yaml, like this:

network:
    ethernets:
        eth0:
            addresses:
            - 10.0.0.20/24
            dhcp4: true
            match:
                macaddress: 00:0d:3a:4e:c5:66
            set-name: eth0
    version: 2

And netplan apply;

then the static ip is 'primary' on the interface
and the dhcp ip is 'secondary' which results it
the wrong source ip when connecting to the metadata services.

[Chad Smith (chad.smith)]

filed a related bug against netplan for discussion about how best to express this type of configuration https://bugs.launchpad.net/netplan/+bug/1834956

[Chad Smith (chad.smith)]

This issue is resolved by the following WIP branch: https://code.launchpad.net/~ddstreet/ubuntu/+source/systemd/+git/systemd/+merge/370152

Expectation is that this branch will be included in a systemd SRU: LP:#1835581.

[Joshua Powers (powersj)]

A systemd SRU [1] has started with the branch referenced in comment 6 included in it. It would be good to verify this actually fixes this issue and reply on the SRU bug.

[1] https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1835581

[Chad Smith (chad.smith)]

I have verified that Bionic and Disco work with the aforementioned SRU (and have attached logs to https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1835581). Eoan SRU is not yet queued for SRU verification with a fix because an existing SRU is already in the pipeline there and needs to land first.

[Chad Smith (chad.smith)]

This will require no work from cloud-init as it is a systemd related fix. Cloud-init just needs to be aware of when the existing systemd SRUs land in Bionic, Disco and Eoan

[Chad Smith (chad.smith)]

The systemd SRU has landed in Bionic and later.

This bug can be closed as fix released once cloud-init v.19.2.10 or later.

[thinktwo (jan-ridabu)]

From what understand the fix in systemd was release with 18.04.3, but i still seem to hit this with cloud-init 19.2-36-g059d049c-0ubuntu2~18.04.1 on 18.04.3. Are something still missing in cloud-init to resolve this?

[Ryan Harper (raharper)]

@thinktwo nothing missing, the fix is entirely in systemd. Please open a new bug and put in your steps so we can track down what's going wrong.

[Chad Smith (chad.smith)]

This bug is believed to be fix released in systemd 242-7ubuntu3.2 or later and cloud-init 19.2 or later. If it is still an issue, please re-open this bug.