Not obvious how to add more storage when you run out

[James Troup]

For whatever reason I chose overly conservatively when asked how much
space I wanted to allocate during 'lxd init'. When I then ran out, it
wasn't super obvious (at least to me) how to fix the situation.

(Even when googling for help, most of the ZFS documentation
 (reasonably) assumes you're dealing with drives not a file backed
 disk image.)

I ended up doing this:

 * dd if=/dev/zero of=/var/lib/lxd/zfs1.img bs=1k count=1 seek=20M
 * zpool add lxd /var/lib/lxd/zfs1.img

Which ... appears to work? It'd be nice if lxd had some documentation
around how to manage storage.

And/or perhaps lxd itself could help you manage your storage?

[Stéphane Graber (stgraber)]

Hey there,

Yes, your approach to grow ZFS is certainly the easiest. I believe you can also grow the existing file and then tell ZFS about it having grown, but that's obviously much more error prone.

We have a generic "production setup" document with useful tips when operating LXD, I'll see that we add something about growing an existing file backed ZFS pool in there. As well as the restriction that you can never shrink a ZFS pool and that overcommitting on disk space will lead to corruption, so some care as to be taken when doing that.

As for managing storage, we're working on this right now. The new LXD storage API will let you define multiple storage pools, choose which you want to use on a per-container basis, as well as allocate additional volumes from any of the pools to be attached to a particular path in the container.

It shouldn't be too difficult to allow for growing a loop backed bool through the API. We already have a "size" property in our design for the initial pool size when creating it, allowing it to be increased should be pretty straightforward.

[Colin Watson (cjwatson)]

FWIW, here's my approach for resizing an existing file-backed device, which isn't too much harder than James's original sequence:

  sudo truncate -s +5G /var/lib/lxd/zfs.img
  sudo zpool set autoexpand=on lxd
  sudo zpool online -e lxd /var/lib/lxd/zfs.img
  sudo zpool set autoexpand=off lxd

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxd - 2.11-0ubuntu1

---------------
lxd (2.11-0ubuntu1) zesty; urgency=medium

  * New upstream release (2.11):
    - New "aliases" field in POST /1.0/images
    - Reworked help messages and "lxc manpage" command (LP: #1544984)
    - New "vlan" nic property for "macvlan" devices
    - doc: Add instructions to grow ZFS loop (LP: #1648995)
    - doc: Improve storage doc
    - global: Use RunCommand everywhere
    - i18n: Refresh templates
    - i18n: Update translations from weblate
    - i18n: Update translation templates
    - lxc: Allow --version to be passed with any command
    - lxc: Make help/usage a bit more consistent
    - lxc: Rework for better manpages
    - lxc/image: Show the alias description
    - lxc/profile: Properly implement "profile unset" (LP: #1669772)
    - lxd/containers: Don't use FindProcess, just pass exec.Cmd
    - lxd/containers: Properly revert memory limits on failure
    - lxd/images: Properly return the alias description
    - lxd/images: Refactor code a bit
    - lxd/migration: Actually unset the storage pool if unavailable
    - lxd/migration: Better handle rsync errors (subprocesses)
    - lxd/migration: Set correct pool property for btrfs
    - lxd/migration: Set correct pool property for zfs
    - lxd/migration: Tweak rsync logging a bit
    - lxd/patches: Call tryMount() if not already mounted
    - lxd/patches: Conditionalize lvrename
    - lxd/patches: Delete image db entry if LV is missing
    - lxd/patches: Detect the logical volume size
    - lxd/patches: Fix incorrect btrfs source properties
    - lxd/patches: Handle mixed-storage upgrade
    - lxd/patches: Use MNT_DETACH for lvm
    - lxd/patches: Use RemoveAll() for lvm snapshots dir
    - lxd/storage/btrfs: Correctly handle loop-backed pools
    - lxd/storage/btrfs: Handle custom subvolume paths
    - lxd/storage/dir: Limit valid pool source paths
    - lxd/storage/lvm: Call {pv,vg}scan
    - lxd/storage/lvm: Dumb down functions from methods to functions
    - lxd/storage: Deal with source not being btrfs
    - lxd/storage: Ensure correct pool for snapshots
    - lxd/storage: Harden the btrfs migration code
    - lxd/storage: Report prepareLoopDev() error directly
    - shared/idmap: Fix various issues
    - tests: Add more dir and btrfs tests
    - tests: Improve lvm part of storage tests
  * Use "lxc manpage" to generate the manpages.
  * Update debian/tests/control to pull in lvm2 and thin-provisioning-tools.
  * Include lxd-benchmark in lxd-tools.

 -- Stéphane Graber <email address hidden> Wed, 08 Mar 2017 16:24:35 -0500