Crash running custom command on fs_setup

[José Riguera]

Given this cloud-init config:

    fs_setup:
    - cmd:['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data']
      label: volumedata
      filesystem: 'btrfs'
      device: '/dev/sda'

Error log:

    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Writing to /var/lib/cloud/instances/raspberrypi/sem/config_disk_setup - wb: [420] 24 bytes
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] helpers.py[DEBUG]: Running config-disk_setup using lock (<FileLock using file '/var/lib/cloud/instances/raspberrypi/sem/config_disk_setup'>)
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: setting up filesystems: [{'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sda', '--subvol', 'data:/data'], 'device': '/dev/sda'}, {'label': 'volumedata', 'filesystem': 'btrfs', 'cmd': ['/bin/btrfs-manage-raid', 'volumedata', '/media/root-data', '/dev/sdb'], 'device': '/dev/sdb'}]
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sda against default devices
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Using the raw device to place filesystem volumedata on
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: File system type 'btrfs' with label 'volumedata' will be created on /dev/sda
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Creating fs for /dev/sda took 0.003 seconds
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[WARNING]: Failed during filesystem operation
                                           unsupported operand type(s) for %: 'list' and 'dict'
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] util.py[DEBUG]: Failed during filesystem operation
                                           unsupported operand type(s) for %: 'list' and 'dict'
                                           Traceback (most recent call last):
                                             File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 155, in handle
                                               func=mkfs, args=(definition,))
                                             File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2181, in log_time
                                               ret = func(*args, **kwargs)
                                             File "/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py", line 917, in mkfs
                                               'device': device,
                                           TypeError: unsupported operand type(s) for %: 'list' and 'dict'
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Creating new filesystem.
    Nov 05 21:12:33 raspi cloud-init[374]: [CLOUDINIT] cc_disk_setup.py[DEBUG]: Checking /dev/sdb against default devices

Link to the bug in the code: https://git.launchpad.net/cloud-init/tree/cloudinit/config/cc_disk_setup.py#n952

The problem is because `fs_cmd` is a list, not a string, which is related to: https://bugs.launchpad.net/cloud-init/+bug/1755965

Also the documentation is not accurate, because the example shows a string, and actually the type of `cmd` in the code is a list: See here: https://cloudinit.readthedocs.io/en/latest/topics/examples.html#disk-setup

[José Riguera (jriguera)]

Quick fix on https://git.launchpad.net/cloud-init/tree/cloudinit/config/cc_disk_setup.py#n952

        fs_cmd = [ i % {
                'label': label,
                'filesystem': fs_type,
                'device': device,
            } for i in fs_cfg['cmd']
        ]

This snipped fixes the issue.

[Ryan Harper (raharper)]

Hi,

Cloud-init does support cmd as string with formatters; but not cmd as a list.

This should get the command you want without changes for cloud-init.

#cloud-config
fs_setup:
  - cmd: '/bin/btrfs-manage-raid %(label)s /media/root-data %(device)s --subvol data:/data'
    label: volumedata
    device: /dev/sda

That said, the documentation on fs_setup is missing a section on how to use the cmd string with formatters; even though the example shows one. We should update it with some additional ones.

[Launchpad Janitor (janitor)]

[Expired for cloud-init because there has been no activity for 60 days.]

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3272