nova-lxd breaks when lxd is deployed on a zfs filesystem containing the '/' character

[Francois Deppierraz]

OpenStack queens deployed on Ubuntu 16.04 with openstack-ansible.

root@alpha:~# lxc config show
config:
  core.https_address: '[::]:8443'
  core.trust_password: true
  storage.zfs_pool_name: tank/lxd
root@alpha:~#

Here's nova-compute log:

2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager Traceback (most recent call last):
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/compute/manager.py", line 7227, in update_available_resource_for_node
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager rt.update_available_resource(context, nodename)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 664, in update_available_resource
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager resources = self.driver.get_available_resource(nodename)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/virt/lxd/driver.py", line 1016, in get_available_resource
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager lxd_config['config']['storage.zfs_pool_name']
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/virt/lxd/driver.py", line 209, in _get_zpool_info
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager total = _get_zpool_attribute('size')
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/virt/lxd/driver.py", line 201, in _get_zpool_attribute
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager run_as_root=True)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/utils.py", line 230, in execute
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager return RootwrapProcessHelper().execute(*cmd, **kwargs)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/nova/utils.py", line 113, in execute
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager return processutils.execute(*cmd, **kwargs)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager File "/openstack/venvs/nova-17.0.0/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execute
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager cmd=sanitized_cmd)
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager ProcessExecutionError: Unexpected error while running command.
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager Command: sudo nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H tank/lxd
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager Exit code: 1
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager Stdout: u''
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager Stderr: u"cannot open 'tank/lxd': invalid character '/' in pool name\n"
2018-03-19 12:52:41.354 33414 ERROR nova.compute.manager
2018-03-19 12:53:40.910 33414 ERROR nova.compute.manager [req-a858d51b-421b-48af-933c-a37b4ba72f96 - - - - -] No compute node record for host alpha: ComputeHostNotFound_Remote: Compute host alpha could not be found.
2018-03-19 12:53:41.396 33414 ERROR nova.compute.manager [req-a858d51b-421b-48af-933c-a37b4ba72f96 - - - - -] Error updating resources for node alpha.: ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf zpool list -o size -H tank/lxd
Exit code: 1
Stdout: u''
Stderr: u"cannot open 'tank/lxd': invalid character '/' in pool name\n"

[Alex Kavanagh (ajkavanagh)]

I think that this might be a current limitation of nova-lxd. Obviously, a ZFS pool name can't have a '/' in it. This is a limitation of ZFS itself. e.g. try to create a pool name with:

# zpool create tank/lxd /mnt/zpool100M
cannot create 'tank/lxd': invalid character '/' in pool name
use 'zfs create' to create a dataset

So this will be dataset called 'lxd' in the 'tank' pool.

Now, lxd also (obviously) allows the use of zfs datasets to be use in place of a pool. So this is a feature mismatch between nova-lxd and lxd.

So, this is an enhancement. For a dataset, the relevant command will most likely be:

zfs list -o available -H tank/lxd

Which will provide the amount available in dataset. We probably ought to use '-p' (print it in parsable to also solve another bug (https://bugs.launchpad.net/nova-lxd/+bug/1757371) re locales.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/571226
Committed: https://git.openstack.org/cgit/openstack/nova-lxd/commit/?id=97cb7ea742d4dbcc83079da33ebb4d7e66d0ee9c
Submitter: Zuul
Branch: master

commit 97cb7ea742d4dbcc83079da33ebb4d7e66d0ee9c
Author: Alex Kavanagh <email address hidden>
Date: Wed May 30 16:19:00 2018 +0100

    Enable ZFS to work with datasets and all locales

    This fixes two bugs (independently) to get nova-lxd to work with
    datasets (pool/set) as well as pools (just 'pool'). It also, switches
    the attributes to use the '-p' option with 'zfs list' and 'zpool list'
    which returns the values in bytes rather than human readable format,
    which is more useful in nova-lxd.

    Change-Id: I2eb1b3ba7bd482e680933808414a3992a9a1feba
    Closes-Bug: #1756900
    Closes-Bug: #1757371