cloud-init's growfs/resize fails with gpart dependency on FreeBSD

[Rick]

I've generated a FreeBSD qcow2 including base and kernel with cloud-init and dependencies. cloud-init runs when the instance boots, but the growpart module fails due to, what appears to be, two separate problems.

One of the dependencies is the gpart port/pkg which, incidentally, is a reason it is failing[1]. The growpart module, for a yet unknown reason, is calling /usr/local/sbin/gpart with arguments that are valid only with FreeBSD's /sbin/gpart. cloud-init executes /sbin/gpart when the dependent /usr/local/sbin/gpart is removed that subsequently results in the next failure which I think is caused by cloud-init itself...

cloud-init executes growfs, via resizefs, after successful execution of gpart recover/resize. The logs[2] illustrate the growfs command as growfs /dev/vtbd0p2. By default, FreeBSD's growfs runs interactively asking a question which can be mitigated using the '-y' command line option. The logs indicate a successful growfs operation, but df doesn't reflect it. I suspect the this is due to the default interactive nature of growfs.

[1] http://hostileadmin.com/images/cloud_init_gpart_fail.jpg
[2] http://hostileadmin.com/images/cloud_init_growfs_fail.jpg

[Harm Weites (harmw)]

Interested to see this from gpart:

2015-01-11 18:49:23,742 - util.py[DEBUG]: Failed: gpart recover /dev/vtbd0
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/cloudinit/config/cc_growpart.py", line 132, in resize
    util.subp(["gpart", "recover", diskdev])
  File "/usr/local/lib/python2.7/site-packages/cloudinit/util.py", line 1570, in subp
    cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['gpart', 'recover', '/dev/vtbd0']
Exit code: 1
Reason: -
Stdout: ''
Stderr: 'Usage: gpart [options] device\nOptions: [-b <backup MBR>][-C c,h,s][-c][-d][-E][-e][-f][-g][-h][-i]\n [-K <last sector>][-k <# of sectors>][-L][-l <log file>]\n [-n <increment>][-q][-s <sector-size>][-t <module-name>]\n [-V][-v][-W <device>][-w <module-name,weight>]\ngpart v0.1h (c) 1999-2001 Michail Brzitwa <email address hidden>.\nGuess PC-type hard disk partitions.\n\n'

This was stderr:

Usage: gpart [options] device
Options: [-b <backup MBR>][-C c,h,s][-c][-d][-E][-e][-f][-g][-h][-i]
         [-K <last sector>][-k <# of sectors>][-L][-l <log file>]
         [-n <increment>][-q][-s <sector-size>][-t <module-name>]
         [-V][-v][-W <device>][-w <module-name,weight>]
gpart v0.1h (c) 1999-2001 Michail Brzitwa <email address hidden>.
Guess PC-type hard disk partitions.

A little rude, but lets see what happens, since the above is most certainly nothing compared with man gpart...

# find / -name gpart
/rescue/gpart
/usr/local/sbin/gpart
/sbin/gpart

Bingo: we have gpart in base -and- from ports, and according the initscript cloudinit will use /usr/local/sbin first.

Workaround:
After installing the cloudinit port remove gpart: pkg delete -f gpart

Fix:
Lets remove that port from the required dependencies. Ill see what's required to get that going, I think this only involves pushing out a new port release.

The other issue seems to be with growfs, which doesn't grow the filesystem. The cc_resizefs module should be looking like this:

def _resize_ufs(mount_point, devpth):
    return ('growfs', '-y', devpth)

After this it'll resize as planned. (I can't believe I missed that -y though...)

[Harm Weites (harmw)]

Added my branch with the cloud-init code change.

[Rick (vmiller)]

It has been confirmed that the fixes proposed here do correct the behavior.

[Scott Moser (smoser)]

Hi, there were some recent changes in trunk at
 https://git.launchpad.net/cloud-init/commit/?id=0a71d5a870b416f2c86c8bc196004bb3fc0768a0
I think this is now probably fixed.

Could you please test with a new version of cloud-init and report back?
If the issue is still present, please set teh state to 'confirmed'.
If it is not, then set to fix-committed.

Thanks,
Scott

[Rick (vmiller)]

I can test this in a few months when at the point of generating new FreeBSD qcow2 images. There are competing priorities at this time.

[Sergei Butov (myser90)]

The problem is still relevant.

https://www.freebsd.org/cgi/man.cgi?growfs(8)
  -y ``Expert mode''. Usually growfs will ask you if you took a
      backup of your data before and will do some tests whether special
      is currently mounted or whether there are any active snapshots on
      the file system specified. This will be suppressed. So use this
      option with great care!

Without this option FS doesn't resize.

[Scott Moser (smoser)]

Sergei,
can you verify that the fix suggested
 https://code.launchpad.net/~harmw/cloud-init/growfs-gpart-fixes_for_fbsd/+merge/246075
will actually work ?

that is to say all we need to do is call growfs with '-y' ?

[Sergei Butov (myser90)]

> Sergei,
> can you verify that the fix suggested
> https://code.launchpad.net/~harmw/cloud-init/growfs-gpart-fixes_for_fbsd/+merge/246075
> will actually work ?
Yes, it's work.

> that is to say all we need to do is call growfs with '-y' ?
Yes. Package 'gpart' is already removed from dependencies.

[Chad Smith (chad.smith)]

An upstream commit landed for this bug.

To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=fed07fc6

[Scott Moser (smoser)]

This bug is believed to be fixed in cloud-init in version 18.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[Scott Moser (smoser)]

Just an FYI, This change is being released to Ubuntu 16.04, 17.10 and 18.04 under bug 1777912.