util.subp regression: no longer accept commands as string

[Chad Smith]

Revision 46cb6716c27d4496ce3d2bea7684803f522f277d regressed behavior of util.subp which now no longer accepts a string as the command to run.

To test:
# failure path
$ python -c 'from cloudinit.util import subp; print(subp("echo hi mom", shell=True))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cloudinit/util.py", line 1905, in subp
    cmd=args)
cloudinit.util.ProcessExecutionError: Unexpected error while running command.
Command: echo hi mom
Exit code: 127
Reason: -
Stdout:
Stderr: c: 1: c: e: not found

# revert commit causing regression
$ git diff 46cb6716c27d4496ce3d2bea7684803f522f277d 46cb6716c27d4496ce3d2bea7684803f522f277d^ > patch1
$ patch -p1 < patch1
patching file cloudinit/util.py
patching file tests/unittests/test_util.py
$ python -c 'from cloudinit.util import subp; print(subp("echo hi mom", shell=True))'
(u'hi mom\n', u'')

This also caused symptoms like the following Azure failure described in bug 1755565.

Related bugs:
 * bug 1755565: Unable to deploy azure template with Ubuntu18.04 Daily
 * bug 1754495: get_hostname on DataSourceAzure fails

[Scott Moser (smoser)]

This is fixed in 18.1-17-g97012fbb-0ubuntu1

[Dan Watkins (oddbloke)]

I'm keeping an eye on the build system to ensure we get a good daily out.

[Mitchell Thompson (carcomp)]

The latest image:

  {
    "offer": "UbuntuServer",
    "publisher": "Canonical",
    "sku": "18.04-DAILY-LTS",
    "urn": "Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201803180",
    "version": "18.04.201803180"
  }

on Microsoft Azure does not exhibit this problem.

[Chad Smith (chad.smith)]

This bug is believed to be fixed in cloud-init in 18.2. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[Suraj Muthreja (surajm)]

I am facing the same issue when using Ubuntu 14. Can you please confirm if this has been fixed in Ubuntu 14 as well.

Below is Ubuntu version I am using:

Linux myVMTerraform 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Cloud-Init version:

cloud-init 0.7.5

Logs:

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
[ 1.674063] Floppy drive(s): fd0 is 1.44M
[ 1.676471] hv_vmbus: Hyper-V Host Build:14393-10.0-0-0.230; Vmbus version:3.0
[ 1.676471] hv_vmbus: CPU offlining is not supported by hypervisor
[ 1.699151] hv_vmbus: registering driver hv_storvsc
[ 1.702572] hv_utils: Registering HyperV Utility Driver
[ 1.706509] hv_vmbus: registering driver hv_util
[ 1.710911] hv_vmbus: registering driver hv_netvsc
[ 1.712042] FDC 0 is an 82078.
[ 1.716379] scsi2 : storvsc_host_t
[ 1.718868] scsi 2:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5
[ 1.723838] sd 2:0:0:0: Attached scsi generic sg1 type 0
[ 1.727265] sd 2:0:0:0: [sda] 169869312 512-byte logical blocks: (86.9 GB/81.0 GiB)
[ 1.728786] scsi3 : storvsc_host_t
[ 1.730024] scsi 3:0:1:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5
[ 1.730367] sd 3:0:1:0: Attached scsi generic sg2 type 0
[ 1.730419] sd 3:0:1:0: [sdb] 134217728 512-byte logical blocks: (68.7 GB/64.0 GiB)
[ 1.730420] sd 3:0:1:0: [sdb] 4096-byte physical blocks
[ 1.730773] sd 3:0:1:0: [sdb] Write Protect is off
[ 1.730868] sd 3:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[ 1.731887] sdb: sdb1
[ 1.732260] hv_netvsc: hv_netvsc channel opened successfully
[ 1.732769] sd 3:0:1:0: [sdb] Attached SCSI disk
[ 1.739271] hv_netvsc vmbus_0_15: Send section size: 6144, Section count:170
[ 1.739518] hv_netvsc vmbus_0_15: Device MAC 00:0d:3a:36:17:39 link state up
[ 1.780402] sd 2:0:0:0: [sda] 4096-byte physical blocks
[ 1.784724] sd 2:0:0:0: [sda] Write Protect is off
[ 1.788162] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 1.793728] sda: sda1 sda2 < sda5 >
[ 1.796757] sd 2:0:0:0: [sda] Attached SCSI disk
[ 1.899065] psmouse serio1: alps: Unknown ALPS touchpad: E7=12 00 64, EC=12 00 64
[ 2.104106] psmouse serio1: trackpoint: failed to get extended button data
[ 3.893299] hv_netvsc: hv_netvsc channel opened successfully
[ 3.904379] hv_netvsc vmbus_0_16: Send section size: 6144, Section count:170
[ 3.909289] hv_netvsc vmbus_0_16: Device MAC 00:0d:3a:36:13:5b link state up
[ 6.708078] psmouse serio1: trackpoint: IBM TrackPoint firmware: 0x01, buttons: 0/0
[ 6.714160] input: TPPS/2 IBM TrackPoint as /devices/platform/i8042/serio1/input/input3
[[ 33.989462] random: lvm urandom read with 99 bits of entropy available
[ 34.677278] random: nonblocking pool is initialized
Begin: Running /scripts/local-premount ... done.
[ 35.034629] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
[ 38.981307] Adding 8384508k swap on /dev/ma...

[Scott Moser (smoser)]

@Suraj,

Hi, I don't think the problem you're seeing is at all related to this bug.
If you are still having the problem file a new bug
 https://bugs.launchpad.net/ubuntu/+source/cloud-init/+filebug/

Please attach to it
a.) /var/log/cloud-init.log
b.) /var/log/cloud-init-output.log
c.) the cloud platform that you are running on (it seems like maybe MS Azure?)
d.) the full console log if available.

Thanks.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3137