Cloud-init on Xenial won't deploy compressed content in "write_files" - says DecompressionError: Not a gzipped file

Bug #1565638 reported by Oded Arbel on 2016-04-04
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned
cloud-init (Ubuntu)
Medium
Unassigned

Bug Description

I'm trying to deploy an Amazon EC2 instance using cloud-init.

When I'm compressing the files in the write_files section, the cloud-init process doesn't write the files, and I get the following error in the cloud-init.log :

Apr 3 16:27:16 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: init-network/config-write-files: FAIL: running config-write-files with frequency once-per-instance
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[WARNING]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>)
 failed
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[DEBUG]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 393, in decomp_gzip
    return decode_binary(gh.read())
  File "/usr/lib/python3.5/gzip.py", line 274, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.5/gzip.py", line 461, in read
    if not self._read_gzip_header():
  File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b"b'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 735, in _run_modules
    freq=freq)
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 39, in handle
    write_files(name, files, log)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 74, in write_files
    contents = extract_contents(f_info.get('content', ''), extractions)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 98, in extract_contents
    result = util.decomp_gzip(result, quiet=False)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 400, in decomp_gzip
    raise DecompressionError(six.text_type(e))
cloudinit.util.DecompressionError: Not a gzipped file (b"b'")

I've verified that the cloud init I'm submitting does encode the files correctly by running this ruby code on the cloud-init YAML file:

> File.write("test.gz", YAML.load(File.read("test.init"))["write_files"].first["content"])

Then:

$ file test.gz
test.gz: gzip compressed data, last modified: Mon Apr 4 06:55:53 2016, max compression, from Unix
$ python
Python 2.7.11+ (default, Mar 30 2016, 21:00:42)
[GCC 5.3.1 20160330] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gzip
>>> with gzip.open('test.gz', 'rb') as f:
... file_content = f.read()
... print file_content
...
<TEXT CONTENT THAT I EXPECTED>

I've never implemented this with trusty, so I'm not sure how cloud-init on trusty handles that.

Related branches

Oded Arbel (oded-geek) wrote :

After disabling compression of write_files it works fine - just compressed file don't work.

Scott Moser (smoser) wrote :

fixed in trunk at 1199.

Changed in cloud-init:
status: New → Confirmed
Changed in cloud-init (Ubuntu):
status: New → Confirmed
Changed in cloud-init:
importance: Undecided → Medium
Changed in cloud-init (Ubuntu):
importance: Undecided → Medium
Changed in cloud-init:
status: Confirmed → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1200-0ubuntu1

---------------
cloud-init (0.7.7~bzr1200-0ubuntu1) xenial; urgency=medium

  * New upstream snapshot.
    - fix adding of users when no group is specified (LP: #1562918)
    - fix write_files with compressed content (LP: #1565638)
    - systemd: do not specify After of obsolete syslog.target (LP: #1536964)
    - chef: fix chef installation from gems (LP: #1553345)
    - disk_setup: correctly send force to mkfs on block devices (LP: #1548772)
    - locale: list unsupported environment settings in warning (LP: #1558069)
    - fix NoCloud and other datasources if no datasource config (LP: #1514407)

 -- Scott Moser <email address hidden> Wed, 06 Apr 2016 13:07:27 -0400

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers