Cloud-init on Xenial won't deploy compressed content in "write_files" - says DecompressionError: Not a gzipped file

Bug #1565638 reported by Oded Arbel
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Unassigned
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

I'm trying to deploy an Amazon EC2 instance using cloud-init.

When I'm compressing the files in the write_files section, the cloud-init process doesn't write the files, and I get the following error in the cloud-init.log :

Apr 3 16:27:16 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: init-network/config-write-files: FAIL: running config-write-files with frequency once-per-instance
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[WARNING]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>)
 failed
Apr 3 16:27:16 ubuntu [CLOUDINIT] util.py[DEBUG]: Running module write-files (<module 'cloudinit.config.cc_write_files' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 393, in decomp_gzip
    return decode_binary(gh.read())
  File "/usr/lib/python3.5/gzip.py", line 274, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.5/gzip.py", line 461, in read
    if not self._read_gzip_header():
  File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b"b'")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 735, in _run_modules
    freq=freq)
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 39, in handle
    write_files(name, files, log)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 74, in write_files
    contents = extract_contents(f_info.get('content', ''), extractions)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files.py", line 98, in extract_contents
    result = util.decomp_gzip(result, quiet=False)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 400, in decomp_gzip
    raise DecompressionError(six.text_type(e))
cloudinit.util.DecompressionError: Not a gzipped file (b"b'")

I've verified that the cloud init I'm submitting does encode the files correctly by running this ruby code on the cloud-init YAML file:

> File.write("test.gz", YAML.load(File.read("test.init"))["write_files"].first["content"])

Then:

$ file test.gz
test.gz: gzip compressed data, last modified: Mon Apr 4 06:55:53 2016, max compression, from Unix
$ python
Python 2.7.11+ (default, Mar 30 2016, 21:00:42)
[GCC 5.3.1 20160330] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gzip
>>> with gzip.open('test.gz', 'rb') as f:
... file_content = f.read()
... print file_content
...
<TEXT CONTENT THAT I EXPECTED>

I've never implemented this with trusty, so I'm not sure how cloud-init on trusty handles that.

Related branches

Revision history for this message
Oded Arbel (oded-geek) wrote :

After disabling compression of write_files it works fine - just compressed file don't work.

Revision history for this message
Scott Moser (smoser) wrote :

fixed in trunk at 1199.

Changed in cloud-init:
status: New → Confirmed
Changed in cloud-init (Ubuntu):
status: New → Confirmed
Changed in cloud-init:
importance: Undecided → Medium
Changed in cloud-init (Ubuntu):
importance: Undecided → Medium
Changed in cloud-init:
status: Confirmed → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1200-0ubuntu1

---------------
cloud-init (0.7.7~bzr1200-0ubuntu1) xenial; urgency=medium

  * New upstream snapshot.
    - fix adding of users when no group is specified (LP: #1562918)
    - fix write_files with compressed content (LP: #1565638)
    - systemd: do not specify After of obsolete syslog.target (LP: #1536964)
    - chef: fix chef installation from gems (LP: #1553345)
    - disk_setup: correctly send force to mkfs on block devices (LP: #1548772)
    - locale: list unsupported environment settings in warning (LP: #1558069)
    - fix NoCloud and other datasources if no datasource config (LP: #1514407)

 -- Scott Moser <email address hidden> Wed, 06 Apr 2016 13:07:27 -0400

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.