Failed mount of '/dev/sdb1' with Swap File cloud-init config

[Daniel Sol]

Environment : RHEL 7.5 on Azure, using cloud-init 18.2 rebase with these patches: https://git.launchpad.net/cloud-init/commit/?id=aa4eeb80

Scenario:
I provision an image with the above on Azure, in addition, i apply this cloud-init config:
#cloud-config
disk_setup:
    ephemeral0:
        table_type: gpt
        layout: [66, [33, 82]]
        overwrite: True
fs_setup:
    - device: ephemeral0.1
      filesystem: ext4
    - device: ephemeral0.2
      filesystem: swap
mounts:
    - ["ephemeral0.1", "/mnt"]
    - ["ephemeral0.2", "none", "swap", "sw", "0", "0"]

The VM provisions successfully, you see /sdb1 gets mounted and the swap file config succeeds.

If i deallocate the VM and start it, /sdb1 does not get mounted, you see the errors below.

2018-06-20 19:50:38,138 - util.py[WARNING]: Failed reading the partition table Unexpected error while running command.
Command: ['/usr/sbin/blockdev', '--rereadpt', '/dev/sdb']
Exit code: 1
Reason: -
Stdout:
Stderr: blockdev: ioctl error on BLKRRPART: Device or resource busy
2018-06-20 19:50:38,138 - util.py[DEBUG]: Failed reading the partition table Unexpected error while running command.
Command: ['/usr/sbin/blockdev', '--rereadpt', '/dev/sdb']
Exit code: 1
Reason: -
Stdout:
Stderr: blockdev: ioctl error on BLKRRPART: Device or resource busy
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cloudinit/config/cc_disk_setup.py", line 685, in read_parttbl
    util.subp(blkdev_cmd)
  File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 1958, in subp
    cmd=args)
ProcessExecutionError: Unexpected error while running command.

Running multiple deallocates and starts does not see the system resolve itself.

I check fstab:
UUID=6e4681f0-1ec3-4c27-9c16-afb90a4c4513 / xfs defaults 0 0
UUID=f2116236-4be6-4b72-b107-c2d7a68688c4 /boot xfs defaults 0 0
/dev/disk/cloud/azure_resource-part1 /mnt auto defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 2
/dev/disk/cloud/azure_resource-part2 none swap sw,comment=cloudconfig 0 0

As a test, i changed the FSTAB entry for the swap to add a dependency on cloud-init:
/dev/disk/cloud/azure_resource-part2 none swap sw,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 0

I deallocated and started the VM, sdb mounts correctly, but the FSTAB is overwritten for swap with the previous config. I modified the FSTAB again, with the addition, then rebooted, again, everything works.

Does azure_resource-part2 need a dependency like azure_resource-part1?

You can also test this using the existing Azure cloud-init preview images (cloud-init 0.7.9.x)
az vm create
--resource-group rgName
--name vmName
--image RedHat:RHEL:7-RAW-CI:latest
--admin-username anotherusr
--custom-data /../swapconfig.txt
--ssh-key-value /.../my.pub

az vm deallocate --resource-group rgName --name vmName
az vm start --resource-group rgName --name vmName

Thanks,

Related bugs:
 * bug 1785354: /etc/fstab: fs_passno is 0 for all file systems

[Daniel Sol (danis)]

ran cloud-init collect-logs...attaching

[Scott Moser (smoser)]

Daniel,
I think this is likely a duplicate of bug 1691489.

Its a tricky bug that we tried a fix for that caused other issues. There is more info there.

[Daniel Sol (danis)]

Hi Scott,
As you called out in 1691489, I can only repro this behavior when using a swap file cloud-init config.

What are you plans to revisit this?

Correct me, if i am wrong, but i should advise customers not to use that method in cloud-init config to set up the swap file?

Thanks,

[Daniel Sol (danis)]

Hi Scott, as discussed at the summit, i changed the fsck property in fstab to switch off fsck for the ephemeral disk :

fstab
/dev/disk/cloud/azure_resource-part1 /mnt auto defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 2

to :
fstab
/dev/disk/cloud/azure_resource-part1 /mnt auto defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 0

After a VM deallocate and start, SDB mounted with no errors.

As we are turning off this check, you want to follow up with another team for their thoughts on changing this option.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3201