Unable to use a gzip for user_data on nova boot

[Samuel Matzek]

Attempting to use a gzip for user_data on Nova boot fails:
nova --debug boot --flavor m1.small --image myImage --nic net-id=025e776e-9e25-461c-a312-e88a56fa7832,v4-fixed-ip=10.10.10.5 --user-data /mygzip.gz

DEBUG (shell:803) 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 800, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.6/site-packages/novaclient/shell.py", line 730, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.6/site-packages/novaclient/v1_1/shell.py", line 470, in do_boot
    server = cs.servers.create(*boot_args, **boot_kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/v1_1/servers.py", line 902, in create
    **boot_kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/v1_1/servers.py", line 465, in _boot
    userdata = encodeutils.safe_encode(userdata)
  File "/usr/lib/python2.6/site-packages/oslo/utils/encodeutils.py", line 86, in safe_encode
    text = safe_decode(text, incoming, errors)
  File "/usr/lib/python2.6/site-packages/oslo/utils/encodeutils.py", line 57, in safe_decode
    return text.decode('utf-8', errors)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte
ERROR (UnicodeDecodeError): 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte

If I base64 encode the gzip and pass it on a REST API call using curl the deploy/create server runs properly. The value is correctly written to the instance's table, and the gzip is correctly placed in the config drive without data loss due to improper encode/decoding. So it appears that this should work once the encoding issue with novaclient is fixed.

Note that cloud-init supports gzipped userdata so that users can gzip their MIME or script files to keep user_data under the 65K DB column limit.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/170399

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/170399
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=19d4d35a4e87598f2ec0115753b147552870ceb7
Submitter: Jenkins
Branch: master

commit 19d4d35a4e87598f2ec0115753b147552870ceb7
Author: melanie witt <email address hidden>
Date: Fri Apr 3 08:52:42 2015 +0000

    Handle binary userdata files such as gzip

    The current code in the servers api assumes only text files will be
    provided as userdata. That is, the data is utf-8 encoded before it
    is base64 encoded and sent. This breaks for binary userdata files,
    (example, gzip files) as they can't be utf-8 encoded.

    This change ignores specifically the exceptions that are raised
    when utf-8 encoding is attempted on non-text data: AttributeError
    and UnicodeDecodeError. These exceptions will be ignored to let
    binary files proceed to the base64 encoding step.

    Closes-Bug: #1419859

    Change-Id: Ie96283f6ff892ae30485722cf7c3ec40f3f1b5df