UTF-8 support in User Data (text/x-shellscript) is broken
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cloud-init |
Fix Released
|
Medium
|
Unassigned |
Bug Description
Hi there,
(1),
Q. Tell us your cloud provider
A. AWS
(2),
Q. Any appropriate cloud-init configuration you can provide us
A. Default,
$ cat /var/run/
di_report:
(3),
Q. Perform the following on the system and attach it to this bug:
A. Output of `cloud-init collect-logs` (filename: cloud-init.tar.gz) in attach.
(4),
Example of User Data used (can be reproduced with any UTF-8 character),
#!/bin/bash
# Basic comment with UTF-8 character - Ä
echo "Repro"
exit 0
(5),
Version of python3 installed (only python3 is installed (i.e. no python2 packages installed)),
$ python3 -V
Python 3.5.2
(6),
Version of cloud-init installed,
$ cloud-init --version
Could you please have a look?
Thanks a lot in advance,
Alexey
Related branches
- Server Team CI bot: Approve (continuous-integration)
- Scott Moser: Pending requested
-
Diff: 1562 lines (+711/-94)26 files modifiedcloudinit/config/cc_lxd.py (+56/-8)
cloudinit/net/eni.py (+17/-3)
cloudinit/net/netplan.py (+14/-8)
cloudinit/net/sysconfig.py (+7/-0)
cloudinit/sources/DataSourceOpenStack.py (+23/-0)
cloudinit/stages.py (+3/-1)
cloudinit/user_data.py (+13/-9)
cloudinit/util.py (+90/-21)
debian/changelog (+25/-0)
doc/rtd/topics/datasources/openstack.rst (+15/-0)
doc/rtd/topics/network-config-format-v1.rst (+27/-0)
doc/rtd/topics/network-config-format-v2.rst (+6/-0)
doc/rtd/topics/tests.rst (+6/-1)
integration-requirements.txt (+1/-1)
tests/cloud_tests/args.py (+3/-0)
tests/cloud_tests/collect.py (+2/-1)
tests/cloud_tests/stage.py (+12/-3)
tests/cloud_tests/testcases/modules/ntp_chrony.py (+12/-1)
tests/cloud_tests/testcases/modules/salt_minion.yaml (+12/-2)
tests/cloud_tests/verify.py (+46/-1)
tests/unittests/test_data.py (+10/-1)
tests/unittests/test_datasource/test_openstack.py (+110/-14)
tests/unittests/test_handler/test_handler_lxd.py (+64/-16)
tests/unittests/test_net.py (+20/-1)
tests/unittests/test_runs/test_simple_run.py (+30/-2)
tests/unittests/test_util.py (+87/-0)
- Scott Moser: Approve
- Server Team CI bot: Approve (continuous-integration)
- Ryan Harper: Approve
-
Diff: 72 lines (+23/-10)2 files modifiedcloudinit/user_data.py (+13/-9)
tests/unittests/test_data.py (+10/-1)
- Server Team CI bot: Approve (continuous-integration)
- cloud-init Commiters: Pending requested
-
Diff: 42 lines (+13/-9)1 file modifiedcloudinit/user_data.py (+13/-9)
Also reproducible with boot commands as well,
`/var/lib/ cloud/instances /i-0123456789ab c/user- data.txt`
#cloud- config
# boot commands
# default: none
# this is very similar to runcmd, but commands run very early
# in the boot process, only slightly after a 'boothook' would run.
# bootcmd should really only be used for things that could not be
# done later in the boot process. bootcmd is very much like
# boothook, but possibly with more friendly.
# - bootcmd will run on every boot
# - the INSTANCE_ID variable will be set to the current instance id.
# - you can use 'cloud-init-per' command to help only run once Ä
bootcmd:
- echo Reprod | tee -a /home/ubuntu/tee
- echo Ä
$ sudo grep -iE 'err|warn|shells|v. 1' /var/log/ cloud-init. log py[DEBUG] : strict_mode: warn, cloud_platform=AWS py[DEBUG] : Found preferred metadata version 2016-09-02 x-shellscript' } from ShellScriptPart Handler: [['text/ x-shellscript' ]] Handler: [['text/ x-shellscript' ]] (__begin__, None, 2) with frequency once-per-instance
yaml.reader. ReaderError: unacceptable character #xdcc4: special characters are not allowed
AttributeError : 'NoneType' object has no attribute 'pop' Handler: [['text/ x-shellscript' ]] (__end__, None, 2) with frequency once-per-instance
2018-05-03 12:00:55,484 - util.py[DEBUG]: Cloud-init v. 17.2 running 'init-local' at Thu, 03 May 2018 12:00:55 +0000. Up 6.24 seconds.
2018-05-03 12:00:56,435 - DataSourceEc2.
2018-05-03 12:00:56,526 - DataSourceEc2.
2018-05-03 12:00:58,401 - util.py[DEBUG]: Cloud-init v. 17.2 running 'init' at Thu, 03 May 2018 12:00:58 +0000. Up 9.23 seconds.
2018-05-03 12:00:58,473 - stages.py[DEBUG]: Added default handler for {'text/
2018-05-03 12:00:58,473 - __init__.py[DEBUG]: Calling handler ShellScriptPart
2018-05-03 12:00:58,474 - util.py[WARNING]: Failed loading yaml blob
2018-05-03 12:00:58,483 - util.py[WARNING]: Failed at merging in cloud config part from part-001
2018-05-03 12:00:58,488 - __init__.py[DEBUG]: Calling handler ShellScriptPart
2018-05-03 12:01:00,631 - util.py[DEBUG]: Cloud-init v. 17.2 running 'modules:config' at Thu, 03 May 2018 12:01:00 +0000. Up 11.43 seconds.
2018-05-03 12:01:02,193 - util.py[DEBUG]: Cloud-init v. 17.2 running 'modules:final' at Thu, 03 May 2018 12:01:02 +0000. Up 12.99 seconds.
2018-05-03 12:01:02,371 - util.py[DEBUG]: Cloud-init v. 17.2 finished at Thu, 03 May 2018 12:01:02 +0000. Datasource DataSourceEc2Local. Up 13.22 seconds
Logs are attached as well (filename: 2018-05- 03T13-05_ 0100_boot_ commands_ cloud-init. tar.gz) .