ec2metadata does not speak EC2 IMDSv2

[Noah Meyerhans]

Forwarded from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=952563

The ec2metadata command queries a well-known link-local endpoint
(169.254.169.254 in Amazon EC2) to obtain information about the instance
on which it runs. Last year, AWS released "IMDSv2" in an effort to
protect customers against some potentially severe information leaks
related to accidentally proxying this local data to the network. Details
at
https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/

IMDSv2 makes use of a session-based protocol, requiring clients to first
retrieve a time-limited session token, and then to include that token with
subsequent requests.

Because the intended purpose of IMDSv2 is to provide an additional layer
of defense against network abuses, customers utilizing it may choose to
disable IMDSv1. Doing so today causes ec2metadata to fail.

[Noah Meyerhans (noahm)]

I've attached a patch the implements IMDSv2 support. Flake8 reports no issues. Tested on EC2 with the following configurations:

1. IMDSv1 support enabled (token optional).
2. IMDSv2 support enabled (token required).
3. IMDS endpoint disabled (raises an exception due to a 403 response from the metadata endpoint).

In the attached patch, if the token retrieval endpoint returns a 404, we assume we're running on some other cloud (not EC2) that presents an EC2-compatible IMDS. In that case, we continue operation as normal, but without using the token. So essentially we've fallen back to IMDSv1 mode, and functionality is not impacted. In theory, some implementation may return something other than a 404, and we should handle its response in a similar way. I don't specifically know of any implementation where this happens.

[Paride Legovini (paride)]

Patch by Fred Vogt submitted in LP: #1882389:

https://github.com/fred-vogt/cloud-utils/commit/ef6afb6dc4357386561a417d1da4276c41ba8421#diff-e917d6e1831a92f1330ac63d774a6d67

[Paride Legovini (paride)]

https://github.com/canonical/cloud-utils/pull/1/commits/ef6afb6dc4357386561a417d1da4276c41ba8421

[Paride Legovini (paride)]

This is Fix Released in version 0.31-22-g37d4e32a-0ubuntu1 (Groovy). LP didn't automatically mark this as Fix Released as the bug is a project bug and and not an Ubuntu bug.

[Thayne (thayne-u)]

Any chance this could be backported to focal?

[Paride Legovini (paride)]

This should be fairly simple to SRU to Bionic and Focal. I added bug tasks for those and assigned myself.

[Reed Loden (reed)]

Would it also make sense to backport to all still-supported LTS releases?
That way companies still using LTS can take advantage of this important AWS
security feature without first having to upgrade all their instances. Just
a thought…

On Mon, Sep 6, 2021 at 8:00 AM Paride Legovini <email address hidden>
wrote:

> This should be fairly simple to SRU to Focal and Hirsute. I added bug
> tasks for those and assigned myself.
>
> ** Also affects: cloud-utils (Ubuntu)
> Importance: Undecided
> Status: New
>
> ** Also affects: cloud-utils (Ubuntu Focal)
> Importance: Undecided
> Status: New
>
> ** Also affects: cloud-utils (Ubuntu Hirsute)
> Importance: Undecided
> Status: New
>
> ** Changed in: cloud-utils (Ubuntu Focal)
> Status: New => Triaged
>
> ** Changed in: cloud-utils (Ubuntu Hirsute)
> Status: New => Triaged
>
> ** Changed in: cloud-utils (Ubuntu)
> Status: New => Fix Released
>
> ** Changed in: cloud-utils (Ubuntu Focal)
> Assignee: (unassigned) => Paride Legovini (paride)
>
> ** Changed in: cloud-utils (Ubuntu Hirsute)
> Assignee: (unassigned) => Paride Legovini (paride)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1870244
>
> Title:
> ec2metadata does not speak EC2 IMDSv2
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-utils/+bug/1870244/+subscriptions
>
>

[Paride Legovini (paride)]

I put some more thought on the possibility of SRUing this to Bionic and Focal, and my conclusion is that it's probably not a good idea to attempt a SRU. The rationale is:

1. This doesn't clearly fit into the SRU requirements [1], as we're not fixing a bug, but enabling a new feature. I agree it's a fairly important security features, and it may fit under the "we sometimes want to introduce new features" case, however I'm not *really* sure [2] is a totally safe change, as it touches the currently used code path.

2. Now we SRU support for IMDSv2. Then, for example, how long before before we also want support for the "new" ec2metadata fields [3]?

I understand the clouds are a moving target, and maybe cloud-utils should be granted a SRU exception, like cloud-init has.

Another solution consists in publishing the newer versions of cloud-utils to the -backports pockets (bionic-backports and focal-backports in this case). This is easy to do and will enable users to use the new features when needed, even if without the guarantees given by the SRU process [1].

Any opinion for or against proceeding with backports is welcome!

[1] https://wiki.ubuntu.com/StableReleaseUpdates
[2] https://github.com/canonical/cloud-utils/commit/2dabc1a177a18525da07983c1365673e7f91efbb
[3] https://github.com/canonical/cloud-utils/commit/558b5d005c801b9b3c8b8603f0144de82997ed17

[Karol H (kharlh)]

Hi Paride - I think your idea to put this fix in -backports pockets sounds good. Do you have any idea when the backport for bionic will be ready?

[Paride Legovini (paride)]

Hi Karol, my plan is to work at this in January, following the new backporting process [1].

https://wiki.ubuntu.com/UbuntuBackports

[Lucas Kanashiro (lucaskanashiro)]

Hi Paride,

Any update on the backport of this feature to Focal (and maybe Bionic?)? There is this other bug report requesting the same thing:

https://bugs.launchpad.net/ubuntu/+source/cloud-utils/+bug/2008713

FWIW I am marking the above as a dup.