[SRU] gce metadata api doesn't properly stream binary data

Bug #1404311 reported by Wayne Witzel III
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Undecided
Wayne Witzel III
cloud-init (Ubuntu)
Fix Released
High
Unassigned
Precise
Fix Released
Undecided
Unassigned
Trusty
Fix Released
High
Unassigned
Utopic
Fix Released
Medium
Unassigned
Vivid
Fix Released
High
Unassigned

Bug Description

[IMPACT] Due to limitations in GCE, binary user-data is mangled when sent as user-data.

[FIX] Allow user to declare binary encoding on user-data.

[VERIFICATION]
1. Create pristine image from -proposed
2. For step 6
3. Boot GCE instance w/ normal user-data, i.e.:
   $ cat user-data
   #cloud-config
   ssh_import_id: [utlemming]
   $ gcloud compute instances create <image from step 1> \
            --metadata-from-file user-data=user-data
4. Confirm that user-data was parsed properly
5. GZIP user-data, and encode using base64:
   gzip -c user-data.txt | base64 > user-data.b64
6. Boot GCE instance w/ user-data.b64 and character encoding meta-data
   set:
   $ gcloud compute instances create <image from step 1> \
            --metadata-from-file user-data=user-data.b64 \
            --metadata user-data-encoding=base64
7. Confirm that user-data was consumed; attach /var/log/cloud-init.log
   to report.

[RISK] If a user sets the user-data-encoding to base64, but does not provide base64 data the instance will fail to provision. However, since the user has to explicitly setup the circumstances, it is low risk.

[ORIGINAL REPORT]
The GCE datasource uses the long hostname. Hostnames longer than 64 characters can break several tools.
While implementing the GCE provider for Juju we found that the metadata API breaks when trying to retrieve certain binary formats. In our case the gz of user-data. The API only streams out the first 5 bytes, encounters what it preceives as a EOF/nil character and truncates the rest of the request.

We've opened an issue with Google directly, but in the meantime a work around is to allow an explicit encoding to be set for the user-data field of the GCE metadata. This will allow use to base64 encode the binary blob, which the API returns the entire contents of without issue.

Related branches

Changed in cloud-init:
status: New → In Progress
assignee: nobody → Wayne Witzel III (wwitzel3)
Changed in cloud-init:
status: In Progress → Fix Committed
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Changed in cloud-init (Ubuntu Utopic):
status: New → Confirmed
Changed in cloud-init (Ubuntu Trusty):
status: New → Confirmed
importance: Undecided → High
Changed in cloud-init (Ubuntu Utopic):
importance: Undecided → Medium
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1055-0ubuntu1

---------------
cloud-init (0.7.7~bzr1055-0ubuntu1) vivid; urgency=medium

  * New upstream snapshot.
    * move to python3 (LP: #1247132)
    * systemd: run cloud-init before systemd-user-sessions.service
    * Use the GCE short hostname. (LP: #1383794)
    * Enable user-data encoding support for GCE. (LP: #1404311)
    * Update to use a newer and better OMNIBUS_URL
    * Be more tolerant of 'ssh_authorized_keys' types
    * Fix parse_ssh_config failing in ssh_util.py
    * Increase the robustness/configurability of the chef module.
    * retain trailing newline from template files when using
      jinja2 (LP: #1355343)
    * fix broken output handling (LP: #1387340)
    * digital ocean datasource
    * update url in config drive documentation
    * freebsd: enable correct behavior on Ec2.
    * freebsd: Use the proper virtio FreeBSD network interface name.
 -- Scott Moser <email address hidden> Wed, 11 Feb 2015 15:55:58 -0500

Changed in cloud-init (Ubuntu Vivid):
status: Confirmed → Fix Released
description: updated
summary: - gce metadata api doesn't properly stream binary data
+ [SRU] gce metadata api doesn't properly stream binary data
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Wayne, or anyone else affected,

Accepted cloud-init into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.6~bzr1022-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Utopic):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Wayne, or anyone else affected,

Accepted cloud-init into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.5-0ubuntu1.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Trusty):
status: Confirmed → Fix Committed
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Wayne, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Precise):
status: New → Fix Committed
Changed in cloud-init (Ubuntu Precise):
assignee: nobody → Ben Howard (utlemming)
Changed in cloud-init (Ubuntu Utopic):
assignee: nobody → Ben Howard (utlemming)
Changed in cloud-init (Ubuntu Trusty):
assignee: nobody → Ben Howard (utlemming)
Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Confirmed Ubuntu 14.04.

Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Confirmed Ubuntu 12.04

Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Confirmed 14.10

Calling this "verification done".

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.6~bzr1022-0ubuntu2

---------------
cloud-init (0.7.6~bzr1022-0ubuntu2) utopic; urgency=medium

  [ Ben Howard ]
  * d/patches/lp-1383794-gce-short_name.patch: Use short hostname for GCE
    (LP: #1383794).

  [ Wayne Witzel III ]
  * d/patches/lp-1404311-gce-data_encoding.patch: Enable user-data encoding
    support for GCE (LP: #1404311).

  [ Daniel Watkins ]
  * d/patches/lp-1422919-azure-g5_ephemeral.patch: Properly format G5 series
    cloud instances (LP: #1422919).
 -- Ben Howard <email address hidden> Tue, 17 Feb 2015 15:15:33 -0700

Changed in cloud-init (Ubuntu Utopic):
status: Fix Committed → Fix Released
Revision history for this message
Scott Kitterman (kitterman) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.5-0ubuntu1.4

---------------
cloud-init (0.7.5-0ubuntu1.4) trusty; urgency=medium

  [ Ben Howard ]
  * d/patches/lp-1383794-gce-short_name.patch: Use short hostname for GCE
    (LP: #1383794).

  [ Wayne Witzel III ]
  * d/patches/lp-1404311-gce-data_encoding.patch: Enable user-data encoding
    support for GCE (LP: #1404311).

  [ Daniel Watkins ]
  * d/patches/lp-1422919-azure-g5_ephemeral.patch: Properly format G5 series
    cloud instances (LP: #1422919).
 -- Ben Howard <email address hidden> Tue, 17 Feb 2015 14:56:16 -0700

Changed in cloud-init (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.16

---------------
cloud-init (0.6.3-0ubuntu1.16) precise-proposed; urgency=medium

   [ Ben Howard ]
   * debian/patches/lp-1383794-gce-short_name.patch: Use short hostname for GCE
     (LP: #1383794).

   [ Wayne Witzel III ]
   * debian/patches/lp-1404311-gce-data_encoding.patch: Enable user-data
     encoding support for GCE (LP: #1404311).
 -- Ben Howard <email address hidden> Tue, 21 Oct 2014 14:56:23 -0600

Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.