cloud-init ca-certs leaves a blank line in /etc/ca-certificates.conf

Bug #1077020 reported by Luis Arias on 2012-11-09
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
High
Scott Moser
cloud-init (Ubuntu)
High
Scott Moser
Precise
High
Unassigned
Quantal
High
Scott Moser
Raring
High
Scott Moser

Bug Description

== Begin SRU Information ==
[Impact]
 * a documented feature of cloud-init, for adding ca-certificates does not function as it should. Instead, certificates added in this manner simply are ignored. This is because apparently, a line directly following a blank line in /etc/ca-certificates.conf is ignored.

[Test Case]
  - start a cloud instance with no user-data
  - add content below to /etc/cloud/cloud.cfg.d/99-local-certs.cfg
  - run the ca-certs code through cloud-init single
    you will see output from update-ca-certificates indicating no
    new certificates were added
    $ sudo cloud-init single --name=ca_certs --frequency=always
    Cloud-init v. 0.7 running 'single' at Sun, 02 Dec 2012 02:23:21 +0000. Up 2429.68 seconds.
    Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done.
    Running hooks in /etc/ca-certificates/update.d....done.

    # this can be fixed by removing the blank line and re-running
    # update-ca-certificates
    Also, note that the following has no output:
    $ ls -l /usr/lib/ssl/certs/ | grep -i cloud
  - edit /etc/ca-certificates.conf, remove cloud-init added entry
    $ sed -i '/cloud-init-ca-certs.crt/d'
  - upgrade cloud-init, re-run the ca_certs
    $ sudo dpkg -i cloud-init_all.deb
    $ sudo cloud-init single --name=ca_certs --frequency=always

    This time, you will see output containing: "1 added, 0 removed; done."
    Also, (trimmed output), you will see:
    $ ls -l /usr/lib/ssl/certs/ | grep -i cloud
    lrwxrwxrwx b1d2b355.0 -> cloud-init-ca-certs.pem
    lrwxrwxrwx cbbf81bb.0 -> cloud-init-ca-certs.pem
    lrwxrwxrwx cloud-init-ca-certs.pem -> /usr/share/ca-certificates/cloud-init-ca-certs.crt

[Regression Potential]
 * regression potential is low. It could break the ca_certs module further, but the module is not functional as it is. Tracebacks are caught when modules are executed, so there is really no potential for further harm.

== End SRU Information ==

Using a cloud-init yaml file adding a certificate like this:

# BEGIN /etc/cloud/cloud.cfg.d/99-local-certs.cfg
ca-certs:
  # If present, the 'trusted' parameter should contain a certificate (or list
  # of certificates) to add to the system as trusted CA certificates.
  # Pay close attention to the YAML multiline list syntax. The example shown
  # here is for a list of multiline certificates.
  # - Amazon RDS SSL Certificate (http://s3.amazonaws.com/rds-downloads/mysql-ssl-ca-cert.pem)
  trusted:
    - |
      -----BEGIN CERTIFICATE-----
      MIIDQzCCAqygAwIBAgIJAOd1tlfiGoEoMA0GCSqGSIb3DQEBBQUAMHUxCzAJBgNV
      BAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdTZWF0dGxlMRMw
      EQYDVQQKEwpBbWF6b24uY29tMQwwCgYDVQQLEwNSRFMxHDAaBgNVBAMTE2F3cy5h
      bWF6b24uY29tL3Jkcy8wHhcNMTAwNDA1MjI0NDMxWhcNMTUwNDA0MjI0NDMxWjB1
      MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHU2Vh
      dHRsZTETMBEGA1UEChMKQW1hem9uLmNvbTEMMAoGA1UECxMDUkRTMRwwGgYDVQQD
      ExNhd3MuYW1hem9uLmNvbS9yZHMvMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
      gQDKhXGU7tizxUR5WaFoMTFcxNxa05PEjZaIOEN5ctkWrqYSRov0/nOMoZjqk8bC
      med9vPFoQGD0OTakPs0jVe3wwmR735hyVwmKIPPsGlaBYj1O6llIpZeQVyupNx56
      UzqtiLaDzh1KcmfqP3qP2dInzBfJQKjiRudo1FWnpPt33QIDAQABo4HaMIHXMB0G
      A1UdDgQWBBT/H3x+cqSkR/ePSIinPtc4yWKe3DCBpwYDVR0jBIGfMIGcgBT/H3x+
      cqSkR/ePSIinPtc4yWKe3KF5pHcwdTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldh
      c2hpbmd0b24xEDAOBgNVBAcTB1NlYXR0bGUxEzARBgNVBAoTCkFtYXpvbi5jb20x
      DDAKBgNVBAsTA1JEUzEcMBoGA1UEAxMTYXdzLmFtYXpvbi5jb20vcmRzL4IJAOd1
      tlfiGoEoMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADgYEAvguZy/BDT66x
      GfgnJlyQwnFSeVLQm9u/FIvz4huGjbq9dqnD6h/Gm56QPFdyMEyDiZWaqY6V08lY
      LTBNb4kcIc9/6pc0/ojKciP5QJRm6OiZ4vgG05nF4fYjhU7WClUx7cxq1fKjNc2J
      UCmmYqgiVkAGWRETVo+byOSDZ4swb10=
      -----END CERTIFICATE-----
# END /etc/cloud/cloud.cfg.d/99-local-certs.cfg

The certificate is added to the /etc/ca-certificates.conf file but there is a blank line between the previous content and the line added by cloud-init. In this situation running update-ca-certificates doesn't take the cloud-init certificates into account. Removing the blank line and running update-ca-certificates again fixes the issue.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: cloud-init 0.6.3-0ubuntu1.1
ProcVersionSignature: User Name 3.2.0-31.50-virtual 3.2.28
Uname: Linux 3.2.0-31-virtual x86_64
ApportVersion: 2.0.1-0ubuntu14
Architecture: amd64
Date: Fri Nov 9 15:01:03 2012
Ec2AMI: ami-3d4ff254
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1d
Ec2InstanceType: m1.medium
Ec2Kernel: aki-825ea7eb
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 TERM=screen
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Luis Arias (kaaloo) wrote :
James Page (james-page) wrote :

Using the cloud-config provided by the bug reported I was able to confirm this issue on a recent 12.04 image.

Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → High
James Page (james-page) wrote :

Also confirmed on quantal release images.

Scott Moser (smoser) on 2012-11-16
Changed in cloud-init:
status: New → Triaged
importance: Undecided → High
milestone: none → 0.7.2
Changed in cloud-init (Ubuntu Quantal):
status: New → Triaged
Changed in cloud-init (Ubuntu Precise):
status: New → Triaged
importance: Undecided → High
Changed in cloud-init (Ubuntu Quantal):
importance: Undecided → High
Scott Moser (smoser) on 2012-12-02
description: updated
Scott Moser (smoser) wrote :

fixed in revno 744

Changed in cloud-init:
assignee: nobody → Scott Moser (smoser)
status: Triaged → Fix Committed
Scott Moser (smoser) on 2012-12-02
Changed in cloud-init (Ubuntu Quantal):
status: Triaged → In Progress
assignee: nobody → Scott Moser (smoser)
todaioan (alan-ar06) on 2012-12-02
Changed in cloud-init (Ubuntu Precise):
status: Triaged → Fix Committed
Changed in cloud-init (Ubuntu Quantal):
status: In Progress → Opinion
Scott Moser (smoser) on 2012-12-03
Changed in cloud-init (Ubuntu Quantal):
status: Opinion → In Progress
Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Triaged
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.1-0ubuntu3

---------------
cloud-init (0.7.1-0ubuntu3) raring; urgency=low

  * cherry pick relevant patches from trunk up to revision 745
  * fix writing of sudoers when suders rule is a string rather than
    an array (LP: #1079002)
  * add trailing slash to sudoers files that are written
  * fix resizefs module when 'noblock' was provided (LP: #1080985)
  * make sure there is no blank line before cloud-init entry in
    there are no blank lines in /etc/ca-certificates.conf (LP: #1077020)
 -- Scott Moser <email address hidden> Mon, 03 Dec 2012 21:45:48 -0500

Changed in cloud-init (Ubuntu Raring):
status: Confirmed → Fix Released
Scott Moser (smoser) on 2012-12-04
Changed in cloud-init (Ubuntu Raring):
assignee: nobody → Scott Moser (smoser)
todaioan (alan-ar06) wrote :

ok

Changed in cloud-init (Ubuntu Precise):
assignee: nobody → todaioan (alan-ar06)
status: Triaged → Fix Released
Scott Moser (smoser) wrote :

Why did you mark this fix-released in precise? The upload went to raring to fix it.

Changed in cloud-init (Ubuntu Precise):
status: Fix Released → Triaged

Hello Luis, or anyone else affected,

Accepted cloud-init into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.7.0-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Quantal):
status: In Progress → Fix Committed
tags: added: verification-needed
Adolfo Jayme (fitojb) wrote :

Hi Luis, does the update in quantal-proposed fix the bug for you?

Changed in cloud-init (Ubuntu Precise):
assignee: todaioan (alan-ar06) → nobody
Clint Byrum (clint-fewbar) wrote :

Hello Luis, or anyone else affected,

Accepted cloud-init into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.7.0-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser) on 2013-01-09
tags: added: verification-done
removed: verification-needed
Luis Arias (kaaloo) wrote :

Hi,

I have been trying to run a test against the package in quantal-proposed this morning but it is not clear to me how to get cloud-init to use the package in proposed. Adding quantal-proposed as a package source is not working out because it comes too late in the process. The package in proposed would have to be installed before cloud-init kicks in somehow. If you have some tips in this area I can give it another try. Thanks!

Luis

Scott Moser (smoser) wrote :

Luis, for testing with -proposed, I used the attached following cloud-config file passed in as user-data.
It adds -proposed and moves the ca-cert config module to run later in the process so the update has already occurred.

Just for reference, I launched instance like this:
$ EC2PRE=euca- ubuntu-ec2-run quantal daily --dry-run --user-data-file=/tmp/my.ud --key=brickies
# us-east-1/ebs/ubuntu-quantal-daily-amd64-server-20130109
euca-run-instances --user-data-file=/tmp/my.ud --key=brickies --instance-type=t1.micro ami-ed21a884

Then, in console output and /var/log/cloud-init-output.log I see:

Cloud-init v. 0.7 running 'modules:final' at Wed, 09 Jan 2013 14:26:23 +0000. Up 58.53 seconds.
Updating certificates in /etc/ssl/certs... Error opening Certificate cert-ec2.pem
140485162526368:error:02001002:system library:fopen:No such file or directory:bss_file.c:398:fopen('cert-ec2.pem','r')
140485162526368:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:
unable to load certificate
WARNING: cert-ec2.pem does not contain a certificate or CRL: skipping
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.

The noise about invalid certificicate is due to bug 1085537.

Then, inside the instance:
$ ls -l /usr/lib/ssl/certs/ | grep -i cloud
lrwxrwxrwx 1 root root 23 Jan 9 14:26 b1d2b355.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 23 Jan 9 14:26 cbbf81bb.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 50 Jan 9 14:26 cloud-init-ca-certs.pem -> /usr/share/ca-certificates/cloud-init-ca-certs.crt

Luis Arias (kaaloo) wrote :

Nice ! Didn't know you could control cloud-init with such precision. I'll give it a try on my setup and post back.

Luis Arias (kaaloo) wrote :

Worked great. I can confirm the package in quantal proposed fixed this issue. Thanks for the fix guys!

Scott Moser (smoser) wrote :

I've committed changes for this in a precise branch at lp:~smoser/ubuntu/precise/cloud-init/sru . I have a ppa build of that at https://launchpad.net/~smoser/+archive/cloud-init-test/ . Any testing on that would be appreciated.

The plan is to move SRU this as soon as the current SRU moves to -updates.

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.0-0ubuntu2.2

---------------
cloud-init (0.7.0-0ubuntu2.2) quantal-proposed; urgency=low

  * debian/patches/lp-1090482-fix-cloud-config-mirrors.patch:
    fix issue with cloud-config data in user-data providing mirror
    info (LP: #1073077)

cloud-init (0.7.0-0ubuntu2.1) quantal-proposed; urgency=low

  * debian/patches/lp-1073077-zsh-workaround-for-locale_warn.patch: avoid
    warning when user's shell is zsh (LP: #1073077)
  * debian/patches/lp-1077700-config-drive-fix-ssh-authorized-keys.patch:
    fix bug in config-drive-v2 usage of authoried keys (LP: #1077700)
  * debian/patches/lp-1080985-fix-resize-root-noblock.patch:
    fix 'resize_root: noblock' (LP: #1080985)
  * debian/patches/lp-1076811-fix-userdata-update-to-distro-config.patch:
    fix updates to distro config via user-data. (LP: #1076811)
  * debian/patches/lp-1066115-install-landscape-if-needed.patch:
    fix permissions on landscape config, and ensure landscape client
    is installed if landscape config is given. (LP: #1066115)
  * debian/patches/lp-1070345-restart-landscape-if-needed.patch:
    restart the landscape-client if changes to config were made. (LP: #1070345)
  * debian/patches/lp-1077020-fix-ca-certificates-blanklines.patch: fix
    adding of empty lines in ca-certificates file (LP: #1077020)
 -- Scott Moser <email address hidden> Mon, 17 Dec 2012 10:15:03 -0500

Changed in cloud-init (Ubuntu Quantal):
status: Fix Committed → Fix Released

Hello Luis, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Precise):
status: Triaged → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Scott Moser (smoser) wrote :

Verification done using latest released 12.04 cloud-image booting it under kvm in raring.

$ sudo apt-get install genisoimage -y
$ bzr branch lp:~smoser/+junk/backdoor-image ./bi

$ imgurl="http://cloud-images.ubuntu.com/releases/precise/release-20130204/ubuntu-12.04-server-cloudimg-amd64-disk1.img"
$ deburl="https://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5/+build/4312778/+files/cloud-init_0.6.3-0ubuntu1.5_all.deb"

$ wget $imgurl -O precise-amd64.img.dist
$ wget $deburl -O cloud-init.deb

$ qemu-img convert -O qcow2 precise-amd64.img.dist disk1.img.dist
$ qemu-img create -f qcow2 -b disk1.img.dist patched.img.dist

## patch the patched.img.dist with new cloud-init
$ sudo ./bi/mount-callback-umount patched.img.dist -- \
    sh -ec 'mp=$MOUNTPOINT; cp cloud-init.deb $mp/tmp &&
            LANG=C chroot $mp dpkg -i /tmp/cloud-init.deb ;
            rm $mp/tmp/cloud-init.deb' --

$ qemu-img create -f qcow2 -b patched.img.dist patched.img

$ cat my-user-data
#cloud-config
password: passw0rd
chpasswd: { expire: False }
ssh_pwauth: True
output: {all: '| tee -a /var/log/cloud-init-output.log'}
ca-certs:
  ... ### content from above example 99-local-certs.cfg ###

$ echo "instance-id: $(uuidgen || echo i-abcdefg)" > my-meta-data
$ cloud-localds my-seed.img my-user-data my-meta-data

$ kvm -m 512 -drive file=patched.img,if=virtio -cdrom my-seed.img -curses

## now, logged in as ubuntu:passw0rd
% grep cloud-init.*found.data.source /var/log/cloud-init.log
found data source: DataSourceNoCloud [seed=/dev/sr0]
% grep -i cert /var/log/cloud-init-output.log
Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
% ls -l /usr/lib/ssl/certs/ | grep -i cloud
lrwxrwxrwx 1 root root 23 Feb 19 22:04 b1d2b355.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 23 Feb 19 22:04 cbbf81bb.0 -> cloud-init-ca-certs.pem
lrwxrwxrwx 1 root root 50 Feb 19 22:04 cloud-init-ca-certs.pem ->
    /usr/share/ca-certificates/cloud-init-ca-certs.crt

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.5

---------------
cloud-init (0.6.3-0ubuntu1.5) precise-proposed; urgency=low

  * debian/update-grub-legacy-ec2: consider kernels bootable on ec2
    that are named -generic, in addition to -virtual. This fixes a problem
    where the kernels installed by linux-lts-quantal were not added to
    /boot/grub/menu.lst (LP: #1005551)
  * debian/patches/lp-1077020-fix-ca-certificates-blanklines.patch: fix
    adding of empty lines in ca-certificates file (LP: #1077020)
  * debian/patches/lp-1031065-nonet-not-start-networking.patch: do not 'start
    networking' in cloud-init-nonet upstart job. Doing so can cause networking
    to be started earlier than it should be. Instead, add a
    cloud-init-container job that runs only in a container and emits
    net-device-added (LP: #1031065).
  * debian/patches/lp-1037567-add-config-drive-v2-support.conf:
    backport support for config-drive-v2 which is part of Openstack Nova in
    Folsom and later. (LP: #1037567) (LP: #1100545)
 -- Scott Moser <email address hidden> Wed, 16 Jan 2013 19:37:57 -0500

Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Fix Released
Scott Moser (smoser) wrote :

fixed in 0.7.2

Changed in cloud-init:
milestone: 0.7.2 → none
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers