Comment 9 for bug 1881747

I think the reason that ZFS behaves differently is because of this...

/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/activate.py

from ceph_volume.util import system

    # mount on tmpfs the osd directory
    osd_path = '/var/lib/ceph/osd/%s-%s' % (conf.cluster, osd_id)
    if not system.path_is_mounted(osd_path):
        # mkdir -p and mount as tmpfs
        prepare_utils.create_osd_path(osd_id, tmpfs=tmpfs)

This "path_is_mounted" test that it does appears to misbehave on a ZFS root, causing it to then resort to using the tmpfs

The test is ultimately traced to "get_mounts" in
/usr/lib/python3.6/site-packages/ceph_volume/util/system.py

On Linux, this reads through /proc/mounts

On ZFS root, the line is should be finding is resembles this...

rpool/ROOT/ubuntu_4trzhh/var/lib /var/lib/ceph/osd/ceph-0 zfs rw,relatime,xattr,posixacl 0 0

...where as on a normal EXT4 root, it looks like this...

/dev/nvme0n1p2 /var/lib/ceph/osd/ceph-0 ext4 rw,relatime,errors=remount-ro 0 0

There's some logic in there about the device needing to start with leading "/", and I think that is what confuses the test when ZFS root has "rpool" with no leading slash.