EC2: cloud-init network fails in ipv6-only subnet with multi-NICs

[James Falcon]

[ Impact ]

Cloud-init recently added policy based routing for multi-nic setups in EC2. The added code assumed that "subnet-ipv4-cidr-block" would be present in the metadata obtained from EC2's IMDS. However, on ipv6-only instances, this is not true. The assumption leads to a traceback when attempting to access the key, resulting in cloud-init failing to properly render the network configuration and leaving the instance unaccessable.

The fix is to check for required keys before configuring policy based routing, and if the keys do not exist, skip configuration.

[ Test Plan ]

Launch an EC2 instance with at least two NICs in an ipv6-only subnet.
Verify that the instance can be accessed via SSH.
Verify that there are no warnings of tracebacks in the logs related to network configuration.
Verify that /etc/netplan/50-cloud-init.yaml contains the relevant routing information and that `ip route` shows the routes as active.

Run tests/integration_tests/modules/test_hotplug.py and
tests/integration_tests/test_networking.py
to ensure no regression to existing multinic rendering behavior.

[ Where problems could occur ]

Since we've added a guard to the policy-based routing rendering, any error in this code would result in the same functionality as now, or result in rendering all of the network except for the policy based routing for secondary NICs.

[ Other Info ]
Upstream bug: https://github.com/canonical/cloud-init/issues/5283
Upstream fix: https://github.com/canonical/cloud-init/pull/5287
Upstream change causing issue: https://github.com/canonical/cloud-init/pull/4799/files#diff-66c4d0c34e626a49273eda62819c3068f5501520b86f28e89e0f998cf4cf2027R1058

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.2~3ge68c8c74-0ubuntu1

---------------
cloud-init (24.2~3ge68c8c74-0ubuntu1) oracular; urgency=medium

  * Upstream snapshot based on upstream/main at e68c8c74.
    - Bugs fixed in this snapshot: (LP: #2066979, #2066985)

 -- James Falcon <email address hidden> Fri, 24 May 2024 12:08:51 -0500

[Andreas Hasenack (ahasenack)]

Hello James, or anyone else affected,

Accepted cloud-init into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.1.3-0ubuntu3.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello James, or anyone else affected,

Accepted cloud-init into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.1.3-0ubuntu1~23.10.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello James, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.1.3-0ubuntu1~20.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello James, or anyone else affected,

Accepted cloud-init into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.1.3-0ubuntu1~22.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Falcon (falcojr)]

Verification results attached.

`verify_2066979.txt` contains results of manual verification as laid out in the bug description.
`multinic-2066979-template.json` contains one of the supporting files used in those tests.
`<series>_integration_tests.txt` contains the automated regression integration test results for each series.

Explanations of warnings of hiccups are in comments at the top of files.

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.1.3-0ubuntu3.3

---------------
cloud-init (24.1.3-0ubuntu3.3) noble; urgency=medium

  * Upstream bug fix release based on 24.1.7
    + functional fixes in debian/patches:
      - cpick-417ee551: fix(ec2): Ensure metadata exists before configuring PBR.
        (LP: #2066979)
      - cpick-d6776632: fix: Check renderer for netplan-specific code (#5321)
      - cpick d771d1f4: fix(ec2): Correctly identify netplan renderer (#5361)
        (LP: #2066985)
    + test fixes in debian/patches:
      - cpick-74dc7cce: test: Fix failing test_ec2.py test (#5324)

 -- James Falcon <email address hidden> Wed, 05 Jun 2024 12:37:32 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.1.3-0ubuntu1~23.10.6

---------------
cloud-init (24.1.3-0ubuntu1~23.10.6) mantic; urgency=medium

  * Upstream bug fix release based on 24.1.7
    + functional fixes in debian/patches:
      - cpick-417ee551: fix(ec2): Ensure metadata exists before configuring PBR.
        (LP: #2066979)
      - cpick-d6776632: fix: Check renderer for netplan-specific code (#5321)
      - cpick d771d1f4: fix(ec2): Correctly identify netplan renderer (#5361)
        (LP: #2066985)
    + test fixes in debian/patches:
      - cpick-74dc7cce: test: Fix failing test_ec2.py test (#5324)

 -- James Falcon <email address hidden> Wed, 05 Jun 2024 12:34:41 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.1.3-0ubuntu1~22.04.5

---------------
cloud-init (24.1.3-0ubuntu1~22.04.5) jammy; urgency=medium

  * Upstream bug fix release based on 24.1.7
    + functional fixes in debian/patches:
      - cpick-417ee551: fix(ec2): Ensure metadata exists before configuring PBR.
        (LP: #2066979)
      - cpick-d6776632: fix: Check renderer for netplan-specific code (#5321)
      - cpick-d771d1f4: fix(ec2): Correctly identify netplan renderer (#5361)
        (LP: #2066985)
    + test fixes in debian/patches:
      - cpick-74dc7cce: test: Fix failing test_ec2.py test (#5324)

 -- James Falcon <email address hidden> Wed, 05 Jun 2024 12:32:23 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.1.3-0ubuntu1~20.04.5

---------------
cloud-init (24.1.3-0ubuntu1~20.04.5) focal; urgency=medium

  * Upstream bug fix release based on 24.1.7
    + functional fixes in debian/patches:
      - cpick-417ee551: fix(ec2): Ensure metadata exists before configuring PBR.
        (LP: #2066979)
      - cpick-d6776632: fix: Check renderer for netplan-specific code (#5321)
        (LP: #2066985)
      - cpick d771d1f4: fix(ec2): Correctly identify netplan renderer (#5361)
        (LP: #2066985)
    + test fixes in debian/patches:
      - cpick-74dc7cce: test: Fix failing test_ec2.py test (#5324)

 -- James Falcon <email address hidden> Wed, 05 Jun 2024 12:40:38 -0500