Write redacted metadata to /run/cloud-init/instance-data.json

[Chad Smith]

Cloud-init persists world-readable instance metadata in /run/cloud-init/instance-data.json and a read-only root /run/cloud-init/instance-data-sensitive.json.

Cloud-init has a facility whereby clouds could defined a via sensitive_metadata_keys list as a class attribute in the platform's supported DataSource subclass.

No clouds are redacting metadata using this mechanism currently.

When cloud-init persists instannce-data.json it should write the redacted content to the world-readable /run/cloud-init/instance-data.json and unredacted content to root read-only /run/cloud-init/instance-data-sensitive.json.

It currently writes the wrong content to each file. No clouds currently are exposed to this bug.

[Chad Smith (chad.smith)]

Validated current broken state if I follow this procedure:

1. Create and launch a VM using an IAM role (which exposes the 'security-credentials' metadata keys to the instance):
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

2. And then disable the cloud-init's logic which skips 'security-credentials' when crawling IMDS

cat > enable-security-creds.patch <<EOF
ubuntu@ip-172-31-80-198:~$ diff -urN /usr/lib/python3/dist-packages/cloudinit/ec2_utils.py.orig /usr/lib/python3/dist-packages/cloudinit/ec2_utils.py
--- /usr/lib/python3/dist-packages/cloudinit/ec2_utils.py.orig 2020-03-03 23:13:02.791518559 +0000
+++ /usr/lib/python3/dist-packages/cloudinit/ec2_utils.py 2020-03-03 23:12:46.679999055 +0000
@@ -85,8 +85,8 @@
             if not field or not field_name:
                 continue
             # Don't materialize credentials
- if field_name == 'security-credentials':
- continue
+ #if field_name == 'security-credentials':
+ # continue
             if has_children(field):
                 if field_name not in children:
                     children.append(field_name)
EOF

scp enable-security-creds.patch ubuntu@<MY_EC2_IAM_VM>:.
ssh ubuntu@<MY_EC2_IAM_VM>
cd /
sudo patch -p1 < /home/ubuntu/enable-security-creds.path

3. Reboot/rerun cloudinit
cloud-init clean --logs --reboot

4. sudo grep redacted /run/cloud-init/instance-data*
# Note redacted content should *not* be in instance-data-sensitive.json
/run/cloud-init/instance-data-sensitive.json: "security-credentials": "redacted for non-root user"

[Chad Smith (chad.smith)]

An upstream fix landed for this bug at
https://github.com/canonical/cloud-init/commit/fa639704f67539d9c1d8668383f755cb0213fd4a

We expect an upload into Ubuntu Focal this week

[Chad Smith (chad.smith)]

A version of cloud-init containing this fix was published to Ubuntu Focal (20.04) cloud-init 20.1-9-g1f860e5a-0ubuntu1.

If this is still a problem for you, please re-open this bug or submit a new bug with related context.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3616