util.subp regression: no longer accept commands as string

Bug #1755965 reported by Chad Smith
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fix Released
Dan Watkins
Fix Released
Chad Smith
cloud-init (Ubuntu)
Fix Released

Bug Description

Revision 46cb6716c27d4496ce3d2bea7684803f522f277d regressed behavior of util.subp which now no longer accepts a string as the command to run.

To test:
# failure path
$ python -c 'from cloudinit.util import subp; print(subp("echo hi mom", shell=True))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cloudinit/util.py", line 1905, in subp
cloudinit.util.ProcessExecutionError: Unexpected error while running command.
Command: echo hi mom
Exit code: 127
Reason: -
Stderr: c: 1: c: e: not found

# revert commit causing regression
$ git diff 46cb6716c27d4496ce3d2bea7684803f522f277d 46cb6716c27d4496ce3d2bea7684803f522f277d^ > patch1
$ patch -p1 < patch1
patching file cloudinit/util.py
patching file tests/unittests/test_util.py
$ python -c 'from cloudinit.util import subp; print(subp("echo hi mom", shell=True))'
(u'hi mom\n', u'')

This also caused symptoms like the following Azure failure described in bug 1755565.

Related bugs:
 * bug 1755565: Unable to deploy azure template with Ubuntu18.04 Daily
 * bug 1754495: get_hostname on DataSourceAzure fails

Related branches

Chad Smith (chad.smith)
description: updated
Changed in cloud-init:
importance: Undecided → High
assignee: nobody → Chad Smith (chad.smith)
status: New → In Progress
Chad Smith (chad.smith)
Changed in cloud-init:
status: In Progress → Fix Committed
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in 18.1-17-g97012fbb-0ubuntu1

Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → Critical
Scott Moser (smoser)
description: updated
Revision history for this message
Dan Watkins (oddbloke) wrote :

I'm keeping an eye on the build system to ensure we get a good daily out.

Changed in cloud-images:
status: New → In Progress
assignee: nobody → Dan Watkins (daniel-thewatkins)
importance: Undecided → Medium
Revision history for this message
Mitchell Thompson (carcomp) wrote :

The latest image:

    "offer": "UbuntuServer",
    "publisher": "Canonical",
    "sku": "18.04-DAILY-LTS",
    "urn": "Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201803180",
    "version": "18.04.201803180"

on Microsoft Azure does not exhibit this problem.

Dan Watkins (oddbloke)
Changed in cloud-images:
status: In Progress → Fix Released
Revision history for this message
Chad Smith (chad.smith) wrote : Fixed in Cloud-init 18.2

This bug is believed to be fixed in cloud-init in 18.2. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
Suraj Muthreja (surajm) wrote :
Download full text (13.9 KiB)

I am facing the same issue when using Ubuntu 14. Can you please confirm if this has been fixed in Ubuntu 14 as well.

Below is Ubuntu version I am using:

Linux myVMTerraform 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Cloud-Init version:

cloud-init 0.7.5


Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
[ 1.674063] Floppy drive(s): fd0 is 1.44M
[ 1.676471] hv_vmbus: Hyper-V Host Build:14393-10.0-0-0.230; Vmbus version:3.0
[ 1.676471] hv_vmbus: CPU offlining is not supported by hypervisor
[ 1.699151] hv_vmbus: registering driver hv_storvsc
[ 1.702572] hv_utils: Registering HyperV Utility Driver
[ 1.706509] hv_vmbus: registering driver hv_util
[ 1.710911] hv_vmbus: registering driver hv_netvsc
[ 1.712042] FDC 0 is an 82078.
[ 1.716379] scsi2 : storvsc_host_t
[ 1.718868] scsi 2:0:0:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5
[ 1.723838] sd 2:0:0:0: Attached scsi generic sg1 type 0
[ 1.727265] sd 2:0:0:0: [sda] 169869312 512-byte logical blocks: (86.9 GB/81.0 GiB)
[ 1.728786] scsi3 : storvsc_host_t
[ 1.730024] scsi 3:0:1:0: Direct-Access Msft Virtual Disk 1.0 PQ: 0 ANSI: 5
[ 1.730367] sd 3:0:1:0: Attached scsi generic sg2 type 0
[ 1.730419] sd 3:0:1:0: [sdb] 134217728 512-byte logical blocks: (68.7 GB/64.0 GiB)
[ 1.730420] sd 3:0:1:0: [sdb] 4096-byte physical blocks
[ 1.730773] sd 3:0:1:0: [sdb] Write Protect is off
[ 1.730868] sd 3:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[ 1.731887] sdb: sdb1
[ 1.732260] hv_netvsc: hv_netvsc channel opened successfully
[ 1.732769] sd 3:0:1:0: [sdb] Attached SCSI disk
[ 1.739271] hv_netvsc vmbus_0_15: Send section size: 6144, Section count:170
[ 1.739518] hv_netvsc vmbus_0_15: Device MAC 00:0d:3a:36:17:39 link state up
[ 1.780402] sd 2:0:0:0: [sda] 4096-byte physical blocks
[ 1.784724] sd 2:0:0:0: [sda] Write Protect is off
[ 1.788162] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 1.793728] sda: sda1 sda2 < sda5 >
[ 1.796757] sd 2:0:0:0: [sda] Attached SCSI disk
[ 1.899065] psmouse serio1: alps: Unknown ALPS touchpad: E7=12 00 64, EC=12 00 64
[ 2.104106] psmouse serio1: trackpoint: failed to get extended button data
[ 3.893299] hv_netvsc: hv_netvsc channel opened successfully
[ 3.904379] hv_netvsc vmbus_0_16: Send section size: 6144, Section count:170
[ 3.909289] hv_netvsc vmbus_0_16: Device MAC 00:0d:3a:36:13:5b link state up
[ 6.708078] psmouse serio1: trackpoint: IBM TrackPoint firmware: 0x01, buttons: 0/0
[ 6.714160] input: TPPS/2 IBM TrackPoint as /devices/platform/i8042/serio1/input/input3
[[ 33.989462] random: lvm urandom read with 99 bits of entropy available
[ 34.677278] random: nonblocking pool is initialized
Begin: Running /scripts/local-premount ... done.
[ 35.034629] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[ 38.981307] Adding 8384508k swap on /dev/ma...

Revision history for this message
Scott Moser (smoser) wrote :


Hi, I don't think the problem you're seeing is at all related to this bug.
If you are still having the problem file a new bug

Please attach to it
a.) /var/log/cloud-init.log
b.) /var/log/cloud-init-output.log
c.) the cloud platform that you are running on (it seems like maybe MS Azure?)
d.) the full console log if available.


Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.