openstack server rescue is broken with ZFS storage pool for rootfs

[Alex Kavanagh]

If a ZFS storage pool is used for the rootfs of a container, then the rescue functionality doesn't work, and breaks the container (moves it to an error state).

Essentially, when LXD stops a container which has its rootfs on a ZFS storage-pool, it unmounts the ZFS mount for that rootfs. This means it's not "there" for the rescued container to mount as an additional drive in rescue mode. This only happens with ZFS.

Work around is to NOT use ZFS; i.e. a dir (slow) or BTRFS (not stable).

Fix is to, in the rescue function, mount the rootfs somewhere else (using zfs directly), and then pass that into the container via the profile. Obviously, it will require unmounting when either the instance is un-rescued, or deleted, etc.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648143
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=aee3ef71d91e89096fc8e6c45708c994b243d8fc
Submitter: Zuul
Branch: master

commit aee3ef71d91e89096fc8e6c45708c994b243d8fc
Author: Alex Kavanagh <email address hidden>
Date: Wed Mar 27 14:50:07 2019 +0000

    On Bionic ensure that a default storage backend is configured

    Sets up the default dir backend and configures it as the default pool
    for lxd. This is required on bionic (LXD 3.0.x) as there is no default
    storage pool by default.

    Also installs apparmor which LXD requires to operate, but that is
    missing from ubuntu image that is built for devstack-gate.

    Related-Bug: 1822182
    Change-Id: I8cc83486d2e040e02dfbeec570d8c8b59b2c2a97

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/649543

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649543
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=1060782cbfd365f1ffeb3d419e27d43fd6ca359e
Submitter: Zuul
Branch: stable/rocky

commit 1060782cbfd365f1ffeb3d419e27d43fd6ca359e
Author: Alex Kavanagh <email address hidden>
Date: Wed Mar 27 14:50:07 2019 +0000

    On Bionic ensure that a default storage backend is configured

    Sets up the default dir backend and configures it as the default pool
    for lxd. This is required on bionic (LXD 3.0.x) as there is no default
    storage pool by default.

    Also installs apparmor which LXD requires to operate, but that is
    missing from ubuntu image that is built for devstack-gate.

    Related-Bug: 1822182
    (Cherry-picked from aee3ef71d91e89096fc8e6c45708c994b243d8fc)

    Change-Id: I910d6f4b64990ff3068badc693321165317c10e3