ec2: ssh public key fingerprint in console output does not match EC2 standards

Bug #458576 reported by Eric Hammond
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ec2-init (Ubuntu)
Fix Released
Low
Scott Moser
Karmic
Fix Released
Low
Scott Moser

Bug Description

Binary package hint: ec2-init

ami-9733d0fe
ubuntu-images-testing-us/ubuntu-karmic-daily-i386-server-20091022.manifest.xml

The Canonical images look like they are trying to match the ssh host key output in the console log using the format as set by Amazon which has become the defacto standard.

Here is the current format in Amazon's Fedora 8 AMI when console output is requested:

ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 2048 a6:7f:6a:a5:8a:7c:26:45:46:ca:d9:d9:8c:f2:64:27 /etc/ssh/ssh_host_key.pub
ec2: 2048 b6:57:b7:52:4e:36:94:ab:9c:ec:a1:b3:56:71:80:e0 /etc/ssh/ssh_host_rsa_key.pub
ec2: 1024 62:47:49:82:83:9a:d8:1d:b8:c6:8f:dd:4d:d8:9a:2e /etc/ssh/ssh_host_dsa_key.pub
ec2: -----END SSH HOST KEY FINGERPRINTS-----

Here is the current format in the above Ubuntu AMI when console output is requested:

-----BEGIN SSH HOST KEY FINGERPRINTS-----
2048 2e:68:49:26:49:07:67:31:f1:33:92:18:09:c3:6a:ae /etc/ssh/ssh_host_rsa_key.pub (RSA)
1024 4b:99:0e:4a:a4:3e:b4:e5:ef:42:5e:43:07:93:91:a0 /etc/ssh/ssh_host_dsa_key.pub (DSA)
-----END SSH HOST KEY FINGERPRINTS-----

Note the lack of "ec2: " in front of each line as well as the (RSA) and (DSA) appended.

Consider matching the Amazon standard format exactly so that tools which have been written to check the ssh host key for security would not need to be modified to also support a different format in the Canonical images.

The older (April) images from Canonical used to have the "ec2: " in the console output, so it looks like this was a more recent change.

ProblemType: Bug
Architecture: i386
Date: Thu Oct 22 21:30:29 2009
DistroRelease: Ubuntu 9.10
Ec2AMI: ami-9733d0fe
Ec2AMIManifest: ubuntu-images-testing-us/ubuntu-karmic-daily-i386-server-20091022.manifest.xml
Ec2AvailabilityZone: us-east-1a
Ec2InstanceType: m1.small
Ec2Kernel: aki-f9c52690
Ec2Ramdisk: ari-9b33d0f2
Package: ec2-init 0.4.999-0ubuntu5
PackageArchitecture: all
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: User Name 2.6.31-302.7-ec2
SourcePackage: ec2-init
Tags: ec2-images
Uname: Linux 2.6.31-302-ec2 i686

Related branches

Revision history for this message
Eric Hammond (esh) wrote :
Revision history for this message
Scott Moser (smoser) wrote : Re: [Bug 458576] [NEW] ec2: ssh public key fingerprint in console output does not match EC2 standards

On Thu, 22 Oct 2009, Eric Hammond wrote:

> Here is the current format in Amazon's Fedora 8 AMI when console output
> is requested:
>
> ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
> ec2: 2048 a6:7f:6a:a5:8a:7c:26:45:46:ca:d9:d9:8c:f2:64:27 /etc/ssh/ssh_host_key.pub
> ec2: 2048 b6:57:b7:52:4e:36:94:ab:9c:ec:a1:b3:56:71:80:e0 /etc/ssh/ssh_host_rsa_key.pub
> ec2: 1024 62:47:49:82:83:9a:d8:1d:b8:c6:8f:dd:4d:d8:9a:2e /etc/ssh/ssh_host_dsa_key.pub
> ec2: -----END SSH HOST KEY FINGERPRINTS-----
>
> Here is the current format in the above Ubuntu AMI when console output
> is requested:
>
> -----BEGIN SSH HOST KEY FINGERPRINTS-----
> 2048 2e:68:49:26:49:07:67:31:f1:33:92:18:09:c3:6a:ae /etc/ssh/ssh_host_rsa_key.pub (RSA)
> 1024 4b:99:0e:4a:a4:3e:b4:e5:ef:42:5e:43:07:93:91:a0 /etc/ssh/ssh_host_dsa_key.pub (DSA)
> -----END SSH HOST KEY FINGERPRINTS-----
>
> Note the lack of "ec2: " in front of each line as well as the (RSA) and
> (DSA) appended.

For "(RSA)" or "(DSA)", I'm guessing this is a change in ssh-keygen
versions that did this. The output you see is simply from
   ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

For "ec2:", that used to get to the console because we sent the output of
the above ssh-keygen through 'logger -s -t ec2'. In bug 451881 I changed
that to go directly to /dev/console.

ssh-keygen -> logger -s ec2 produced "ec2:" prefixed text to stderr. In
the beta, stderr and stdout of the sysvinit scripts went to /dev/console
(and thus to logs).

Upstart changed the behavior, putting stdout and stderr to /dev/null (or
somewhere not the console), to reach the goal of "no console output in
boot".

I saw at least the following options for fixing this:
a.) configure syslog to to /dev/console , and send output to it via
'logger'
b.) write directly to /dev/console (via tee) keeping info going to
syslog throught tee. Ie:
   regenerate_ssh_host_keys 2>&1 | tee /dev/console |
       logger -p user.info -s -t "ec2"

I decided against 'a' because that isn't how we had things configured
before. It would absolutely result in *more* text going to console, and
at this point I wanted to have smallest set of changes. Also, we have
no guarantee that rsyslog will even be up at the point in which ec2-init
runs. I decided to write to /dev/console to be sure to get the text
there.

I now see that we can get the 'ec2:' prefix back by switching the order:
   regenerate_ssh_host_keys 2>&1 |
      logger -p user.info -s -t "ec2" 2>&1 |
      tee /dev/console

I've verified that logger will copy input to output even if rsyslog is
not up. So, I think the above change would be good to get us 'ec2:'
back.

Regarding '(RSA)' or '(DSA)', I think thats something we live with.

Revision history for this message
Scott Moser (smoser) wrote :

This patch just swaps the order of logger and tee, so that tee gets the 'ec2:' prefixed output of logger.
I would consider this very safe.

Revision history for this message
Scott Moser (smoser) wrote :

I applied the above change, did 'sudo rm /var/lib/ec2/*.ami-*' and restarted
on reboot, i get the following:
ec2: #############################################################
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 2048 df:6f:cf:db:d3:60:1d:cf:b6:3f:da:80:d8:7a:8d:98 /etc/ssh/ssh_host_rsa_key.pub (RSA)
ec2: 1024 5b:9a:a1:cb:de:2d:99:16:c4:f1:49:c6:e9:48:b3:34 /etc/ssh/ssh_host_dsa_key.pub (DSA)
ec2: -----END SSH HOST KEY FINGERPRINTS-----
ec2: #############################################################

Revision history for this message
Eric Hammond (esh) wrote :

Agree on all points and the patch looks good.

Setting the priority to Low because I think the number of people this affects is probably low based on
  http://alestic.com/2009/08/ec2-poll-fingerprint

However, it is a simple change and being compatible is a good thing, so it would be great to include in the release if possible.

Changed in ec2-init (Ubuntu):
importance: Undecided → Low
Revision history for this message
Thierry Carrez (ttx) wrote :

Nominating for karmic since it could be part of the reroll needed for the ephemeral disk automount bug.

Changed in ec2-init (Ubuntu Karmic):
status: New → Triaged
Thierry Carrez (ttx)
Changed in ec2-init (Ubuntu Karmic):
status: Triaged → In Progress
assignee: nobody → Scott Moser (smoser)
Scott Moser (smoser)
Changed in ec2-init (Ubuntu Karmic):
milestone: none → ubuntu-9.10
Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Scott Moser (smoser) wrote :

The above can/should be tested by starting an instance, then:
sudo cp /etc/fstab /etc/fstab.dist
sudo dpkg -i ec2-init_0.4.999-0ubuntu7_all.deb
diff -u /etc/fstab.dist /etc/fstab
# you should see that (bug 458850)
# i686:
# device /dev/sdb changed to /dev/sda2
# both i686 x86_64:
# ext2 -> ext3
sudo cp /etc/fstab.dist /etc/fstab
sudo rm /var/lib/ec2/*
echo "============ MARK ==============" | sudo tee /dev/console
sudo reboot

# now, on reboot, you should be able to see that
# bug 458850
# /dev/sda2 is mounted at /mnt
# /etc/fstab.dist differs from /etc/fstab as described above
# bug 458576
# euca-get-console-output has:
# ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
# rather than
# -----BEGIN SSH HOST KEY FINGERPRINTS-----

Revision history for this message
Scott Moser (smoser) wrote :

first 'suggested fix' did not get ssh keys written to console because stderr wsan't redirected through the 'tee /dev/console'. This does.

Revision history for this message
Scott Moser (smoser) wrote :

clean version of 2. (previous patch had 'fix.debdiff' recursively included')

Revision history for this message
Scott Moser (smoser) wrote :

in comment 8, i should have mentioned, you may still see 'mountall' errors from boot, complaining about lack of /dev/sdb. thats to be expected as /etc/fstab had that, and we cant overwrite it before mountall runs.

also, you can verify in both UEC and ec2 /mnt should be mounted.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

I verified Scott's patch, and uploaded the fix.

It is now up to the release team to accept this upload.

:-Dustin

Changed in ec2-init (Ubuntu Karmic):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ec2-init - 0.4.999-0ubuntu7

---------------
ec2-init (0.4.999-0ubuntu7) karmic; urgency=low

  * work around differences in eucalyptus ephemeral mounts (LP: #458850)
  * get 'ec2:' prefix on ssh public key fingerprint (LP: #458576)

 -- Scott Moser <email address hidden> Mon, 26 Oct 2009 16:18:06 -0400

Changed in ec2-init (Ubuntu Karmic):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

Fixed in 20091027 daily, verified as below:

$ euca-describe-instances i-62de5f0a
RESERVATION r-fe768596 950047163771 default
INSTANCE i-62de5f0a ami-a50ae9cc ec2-67-202-43-236.compute-1.amazonaws.com domU-12-31-39-0A-0A-31.compute-1.internal running ec2-keypair 0 m1.large 2009-10-27T02:43:50.000Z us-east-1b aki-d5c526bc ari-d10ae9b8
$ euca-describe-images -a | grep ami-a50ae9cc
IMAGE ami-a50ae9cc ubuntu-images-testing-us/ubuntu-karmic-daily-amd64-server-20091027.manifest.xml 099720109477 available public x86_64 machine
$ euca-get-console-output i-62de5f0a | sed -n '/ec2: #/,/ec2: #/p'
ec2: #############################################################
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 2048 e0:3f:0a:bd:fa:6c:43:18:15:84:2e:2c:a4:92:64:5b /etc/ssh/ssh_host_rsa_key.pub (RSA)
ec2: 1024 3a:7f:79:15:35:28:50:ed:6a:6f:16:d0:87:61:2b:76 /etc/ssh/ssh_host_dsa_key.pub (DSA)
ec2: -----END SSH HOST KEY FINGERPRINTS-----
ec2: #############################################################

Revision history for this message
Eric Hammond (esh) wrote :

Thank you!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.