ec2-hibinit-agent: Add support for IMDSv2

[Tim Gardner]

[Impact]
If an Amazon Instance is set to require IMDSv2 and enable hibernation, the instance will get stuck in a "stopping" state when hibernation is triggered, and no VM state is preserved. Changes to support IMDSv2 in ec2-hibinit-agent are needed to prevent this behavior when only using IMDSv2.

[Fix]
Backport a portion of https://github.com/aws/amazon-ec2-hibinit-agent.git commit 9d9bca5c61fa9256289e68c88bd3747af2f62e28 ("Add IMDSv2 support")

[Test Case]

# To test the failure
# This test case is written for Mantic but will apply to all distributions
1. Create an amazon EC2 instance with the following properties
   - AMI - ami-0c2d04617e311f0d0 (For Mantic)
   - t3.micro
   - encrypted 8GB EBS volume with default key
   - Stop - Hibernate behavior: Enable
   - Metadata accessible : Enable
   - Metadata version Info : V2 only (token required)
2. Wait for instance to say "Running"
3. Initiate Hibernation
# At this point the machine will stay stuck in the "Stopping" State and you will have to force stop the machine.

# To test the fix
1. Create an amazon EC2 instance with the following properties
   - AMI - ami-0c2d04617e311f0d0 (For Mantic)
   - t3.micro
   - encrypted 8GB EBS volume with default key
   - Stop - Hibernate behavior: Enable
   - Metadata accessible : Enable
   - Metadata version Info : V2 only (token required)
2. Wait for instance to say "Running"
3. ssh into your instance
4. (Before Change is Merged) add PPA for updated package; apt update && apt upgrade
5. Create a test program
```
/bin/cat <<EOM >~/allocate_mem.py
#!/usr/bin/python3
import time

# Allocate 200MB chunk of memory
size = 200 * 1024 * 1024 # 200MB
memory_chunk = bytearray(size)

print("Allocated 200MB of memory.")

# Enter indefinite loop
while True:
    time.sleep(1) # Wait for 1 second

# The script will never reach this point
EOM
```
6. Run a background process
```
python3 ~/allocate_mem.py &
```
7. Hibernate the instance
8. Wait for Instance to be in "Stopped" State
9. Start the intsance
10. ssh into instance
11. Check that the process is running
$ ps aux | grep allocate_mem
ubuntu 2532 1.1 23.0 221948 213248 pts/0 S 13:43 0:00 python3 /home/ubuntu/allocate_mem.py

[Where problems could occur]
Hibernation may not properly occur which would result in a loss of VM state and the end user needing to force stop the instance.

[Brian Murray (brian-murray)]

Ubuntu 21.10 (Impish Indri) has reached end of life, so this bug will not be fixed for that specific release.

[Mitchell Dzurick (mitchdz)]

I have tested the change on Mantic and created the MP for it already. I will add the change through Focal and make the MP as those are created and tested.

[Mitchell Dzurick (mitchdz)]

All MPs made. Each MP is the same, which backports just features of enabling IMDSv2 on EC2 instances on top of the latest package for each release.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ec2-hibinit-agent - 1.0.0-0ubuntu14

---------------
ec2-hibinit-agent (1.0.0-0ubuntu14) mantic; urgency=medium

   * d/p/lp1941785-Add-support-for-IMDSv2.patch: add IMDSv2 support
     (LP: #1941785)

 -- Mitchell Dzurick <email address hidden> Tue, 16 May 2023 06:58:08 -0700

[Robie Basak (racb)]

Hello Tim, or anyone else affected,

Accepted ec2-hibinit-agent into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2-hibinit-agent/1.0.0-0ubuntu13.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello Tim, or anyone else affected,

Accepted ec2-hibinit-agent into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2-hibinit-agent/1.0.0-0ubuntu12.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello Tim, or anyone else affected,

Accepted ec2-hibinit-agent into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2-hibinit-agent/1.0.0-0ubuntu11.22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robie Basak (racb)]

Hello Tim, or anyone else affected,

Accepted ec2-hibinit-agent into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2-hibinit-agent/1.0.0-0ubuntu9.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mitchell Dzurick (mitchdz)]

I have tested the change in proposed for each release. The steps are almost identical (only the AMI changes) but lengthy, so I'll provide a single comment with the steps performed for each release.

1. Create an amazon EC2 instance with the following properties
   - AMI - lunar - ami-05479a7ac80766d9f
           kinetic - ami-03e20864bf1b107ee
           jammy - ami-0c7b1f327a97d8cec
           focal - ami-0bb56dbe79558e0f0
   - t2.micro
   - encrypted 8GB EBS volume with default key
   - Stop - Hibernate behavior: Enable
   - Metadata accessible : Enable
   - Metadata version Info : V2 only (token required)
2. Wait for instance to say "Running"
3. ssh into your instance
4. add proposed; apt update && apt upgrade
```
echo "deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe" | sudo tee /etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
sudo apt update -y && sudo apt upgrade -y
```
5. Create a test program
```
/bin/cat <<EOM >~/allocate_mem.py
#!/usr/bin/python3
import time

# Allocate 200MB chunk of memory
size = 200 * 1024 * 1024 # 200MB
memory_chunk = bytearray(size)

print("Allocated 200MB of memory.")

# Enter indefinite loop
while True:
    time.sleep(1) # Wait for 1 second

# The script will never reach this point
EOM
```
6. Run a background process
```
python3 ~/allocate_mem.py &
```
7. Hibernate the instance
8. Wait for Instance to be in "Stopped" State
9. Start the instance
10. ssh into instance
11. Check that the process is running
$ ps aux | grep allocate_mem | grep -v grep
ubuntu 2532 1.1 23.0 221948 213248 pts/0 S 13:43 0:00 python3 /home/ubuntu/allocate_mem.py

[Launchpad Janitor (janitor)]

This bug was fixed in the package ec2-hibinit-agent - 1.0.0-0ubuntu13.1

---------------
ec2-hibinit-agent (1.0.0-0ubuntu13.1) lunar; urgency=medium

  * d/p/lp1941785-Add-support-for-IMDSv2.patch: allow hibernation of
    AWS EC2 instances with IMDSv2 (LP: #1941785)

 -- Mitchell Dzurick <email address hidden> Tue, 16 May 2023 15:43:29 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ec2-hibinit-agent has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ec2-hibinit-agent - 1.0.0-0ubuntu12.1

---------------
ec2-hibinit-agent (1.0.0-0ubuntu12.1) kinetic; urgency=medium

  * d/p/lp1941785-Add-support-for-IMDSv2.patch: allow hibernation of
    AWS EC2 instances with IMDSv2 (LP: #1941785)

 -- Mitchell Dzurick <email address hidden> Tue, 16 May 2023 15:43:29 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package ec2-hibinit-agent - 1.0.0-0ubuntu11.22.04.2

---------------
ec2-hibinit-agent (1.0.0-0ubuntu11.22.04.2) jammy; urgency=medium

  * d/p/lp1941785-Add-support-for-IMDSv2.patch: allow hibernation of
    AWS EC2 instances with IMDSv2 (LP: #1941785)

 -- Mitchell Dzurick <email address hidden> Tue, 16 May 2023 15:43:29 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package ec2-hibinit-agent - 1.0.0-0ubuntu9.3

---------------
ec2-hibinit-agent (1.0.0-0ubuntu9.3) focal; urgency=medium

  * d/p/lp1941785-Add-support-for-IMDSv2.patch: allow hibernation of
    AWS EC2 instances with IMDSv2 (LP: #1941785)

 -- Mitchell Dzurick <email address hidden> Tue, 16 May 2023 16:39:43 -0700

[Mitchell Dzurick (mitchdz)]

Bionic/Xenial update is released in esm. Pro users will get it with a regular apt update && apt upgrade.