cloud-init crash on EC2 datasources when IMDS returns an error

[Maxime DUFOUR]

Hello,

We are using the EC2 datasource for crawling the metadata and in our cloud provider, if the IMDS returns an 404 error on one metadata resource, cloud-init crashes and the setup fails.

Here is the configuration
```/etc/cloud/cloud.cfg.d/99_metadata.cfg
disable-ec2-metadata: false

datasource_list: [ Ec2 ]
datasource:
  Ec2:
    strict_id: false
    metadata_urls: [ 'http://169.254.169.254:80' ]
    timeout: 5
    max_wait: 10
```

Here is the log of the error
```
[ 23.223228] cloud-init[576]: Cloud-init v. 21.4-0ubuntu1~20.04.1 running 'init' at Tue, 30 Aug 2022 11:43:36 +0000. Up 15.96 seconds.
[ 23.224719] cloud-init[576]: ci-info: +++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
[ 23.226427] cloud-init[576]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[ 23.228137] cloud-init[576]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
[ 23.230390] cloud-init[576]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[ 23.232965] cloud-init[576]: ci-info: | eth0 | True | 10.9.42.189 | 255.255.255.0 | global | aa:03:94:21:c3:a1 |
[ 23.235247] cloud-init[576]: ci-info: | eth0 | True | fe80::a803:94ff:fe21:c3a1/64 | . | link | aa:03:94:21:c3:a1 |
[ 23.250295] cloud-init[576]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
[ 23.255043] cloud-init[576]: ci-info: | lo | True | ::1/128 | . | host | . |
[ 23.256681] cloud-init[576]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[ 23.258318] cloud-init[576]: ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
[ 23.259755] cloud-init[576]: ci-info: +-------+-------------+-----------+-----------------+-----------+-------+
[ 23.261224] cloud-init[576]: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
[ 23.262683] cloud-init[576]: ci-info: +-------+-------------+-----------+-----------------+-----------+-------+
[ 23.264190] cloud-init[576]: ci-info: | 0 | 0.0.0.0 | 10.9.42.3 | 0.0.0.0 | eth0 | UG |
[ 23.265660] cloud-init[576]: ci-info: | 1 | 10.9.42.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U |
[ 23.267324] cloud-init[576]: ci-info: | 2 | 10.9.42.3 | 0.0.0.0 | 255.255.255.255 | eth0 | UH |
[ 23.277516] cloud-init[576]: ci-info: +-------+-------------+-----------+-----------------+-----------+-------+
[ 23.279090] cloud-init[576]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
[ 23.280396] cloud-init[576]: ci-info: +-------+-------------+---------+-----------+-------+
[ 23.281694] cloud-init[576]: ci-info: | Route | Destination | Gateway | Interface | Flags |
[ 23.283008] cloud-init[576]: ci-info: +-------+-------------+---------+-----------+-------+
[ 23.284347] cloud-init[576]: ci-info: | 1 | fe80::/64 | :: | eth0 | U |
[ 23.285660] cloud-init[576]: ci-info: | 3 | local | :: | eth0 | U |
[ 23.286969] cloud-init[576]: ci-info: | 4 | multicast | :: | eth0 | U |
[ 23.288342] cloud-init[576]: ci-info: +-------+-------------+---------+-----------+-------+
[ 23.289631] cloud-init[576]: 2022-08-30 11:43:44,207 - util.py[WARNING]: Failed fetching meta-data/ from url http://169.254.169.254:80/2016-09-02/meta-data/
[ 23.291570] cloud-init[576]: 2022-08-30 11:43:44,216 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceEc2.DataSourceEc2'> failed
```

[Maxime DUFOUR (outscale-mdr)]

This issue is discussed in Github PR: https://github.com/canonical/cloud-init/pull/1686

[Chad Smith (chad.smith)]

This upstream commit has landed with a detection of 3DS Outscale cloud and the ability to skip IMDS meta-data/tag routes that return 404s.

[Brett Holman (holmanb)]

This is expected to be released in upstream in 22.4.

[shixuantong (sxt1001)]

The problem seems to have been resolved, version 22.4 has also been released，we can close it.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/4012