additional key added to ssh authorized_keys on azure

[Doug Byrne]

Hi, running a standard Ubuntu 18.04 image on Azure.

If the System-Assigned Managed Identity is enabled at the launch of the VM, the public key of this identity will be added as an ssh authorized_key for both the admin user and root, in addition to the key provided in the launch details.

I launched a test VM with the managed identity enabled. Immediately after launch, ~/.ssh/authorized_keys contained two keys: one that I specified, and one that I did not. It looks like waagent downloads certificates from the metadata service to /var/lib/waagent/Certificates.p7m. This can was decrypted and stored in Certificates.pem. On my test instance it contains 2 certificates, and 1 private key in pem-encoded format. One of these certificates is the public key that I provided when I provisioned the instance. The other is the extra key that I did not specify.

It appears that cloud init reads this Certificates.pem, and converts them to the format needed by authorized_keys. So it appears that both public keys are coming from here.

This second Certificate has issuer=/CN=Microsoft.ManagedIdentity. This VM does have a System Managed Identity attached. If I look at another VM that does not have the managed identity enabled, the Certificates.pem includes only the public key I provided.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: cloud-init 18.4-0ubuntu1~18.04.1
ProcVersionSignature: Ubuntu 4.15.0-1035.36-azure 4.15.18
Uname: Linux 4.15.0-1035-azure x86_64
ApportVersion: 2.20.9-0ubuntu7.5
Architecture: amd64
CloudName: Azure
Date: Thu Jan 10 17:52:56 2019
PackageArchitecture: all
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)
user_data.txt:

[Chad Smith (chad.smith)]

Thanks for filing the bug, and making Ubuntu better.

Quick note: cloud-init.log reports all files read and written to /var/lib/waagent with 'Writing to' && 'Reading from' prefixes. It doesn't directly parse those pem files (that's all walinuxagent effort)

The only thing cloud-init writes is /var/lib/waagent/ovf-env.xml whose content it receives from Azure's IMDS service or reprovision API route.

I have a couple of questions to help me understand:

1. How did you provide an additional ssh-key (Azure cli, UI or #cloud-config userdata)?

2.Could you explain what the bug is here? Your instance was configured to use a System-Assigned Managed Identity key by default and you provided at launch time a second key.
    It feels a bit like having the 2 keys supplement may be intended behavior as far as Azure is concerned right?

[Chad Smith (chad.smith)]

Please make this bug back to 'New' from Incomplete and we'll look over it again

[Doug Byrne (shareableedoug)]

1. Yes, I provided one key in the launch request. It's one of the two keys that appear in authorized_keys.
2. It is my understanding that the managed identities are only useful for the VM to authenticate to Azure services, not the other way around. It's an identity, but not used for SSH login.

[Chad Smith (chad.smith)]

Thank you again Doug for the followup here.

In looking at the Managed Identities doc[1] it seems like this is intended behavior is the general intent of the 'managed identities' Azure feature. Because this is an Azure cloud feature that the user has opted into when they launched the vm, cloud-init user-data won't override that authentication data and ignore the auth keys as provided by Azure metadata (in /var/lib/waagent/Certificates.p7m). But, because that same user did also sent #cloud-config which defines additional auth keys, we supplement them in this case.

There are cases in other clouds where cloud-init has this same behavior, allow the cloud's metadata to provide some level of base common credentials and cloud-init provided user-data supplements.

It is an option for cloud-init add a logged warning that it is obtaining auth keys from two sources to help shed light to the user that they may not be adhering to instances cloud auth policy if we see the additional keys present, but I'm not certain what more should be done here.

[Chad Smith (chad.smith)]

If there is something else I'm missing that would be helpful in this regard please feel free to comment and mark back to New status.

[Doug Byrne (shareableedoug)]

Are you able to share the Managed Identities doc[1] you are referencing? I haven't been able to find anything that would suggest that the identity is a base common credential.

[Doug Byrne (shareableedoug)]

Hi,

My support contact at MS tells me that this issue has been resolved, and patches are being readied. Related MS disclosure: https://support.microsoft.com/en-us/help/4491476/extraneous-ssh-public-keys-added-to-authorized-keys-file-on-linux-vm

This ticket can be closed as a duplicate of the ticket for that work, whatever ticket that may be.

Thanks.

[Daniel Sol (danis)]

Hi Joshua, Francis,
We have tested '18.5-44-g7c07af28' successfully.

When this is available in a daily image, can you let us know, so we can retest.

Thanks,

[Joshua Powers (powersj)]

Following up in the bug as well, cloud-init (18.5-45-g3554ffe8-0ubuntu1) was released yesterday with fixes for this bug. Marking fix released.