zpool export mpu1pool cannot export 'mpu1pool': pool is busy

[Andrey Arapov]

$ lsb_release -rd
Description: Ubuntu 16.04.2 LTS
Release: 16.04

Linux 4.4.0-64-generic

$ apt-cache policy zfsutils-linux
zfsutils-linux:
  Installed: 0.6.5.6-0ubuntu15

# modinfo zfs |head
filename: /lib/modules/4.4.0-64-generic/kernel/zfs/zfs/zfs.ko
version: 0.6.5.6-0ubuntu15

I was not able to unmount a ZFS pool:

# zpool export mpu1pool
cannot export 'mpu1pool': pool is busy

The filesystems were unmounted and there was nothing that would be blocking them FS.
Neither sync, echo 3 > /proc/sys/vm/drop_caches helped.

The only way worked to export a ZFS pool was to turn off the SWAP:

# lsof -Pn |grep -i mpu1
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
# free -mh
              total used free shared buff/cache available
Mem: 7.7G 5.1G 502M 837M 2.1G 1.0G
Swap: 8.0G 428M 7.6G

# swapoff -a
# free -mh
              total used free shared buff/cache available
Mem: 7.7G 1.5G 4.8G 366M 1.4G 5.2G
Swap: 0B 0B 0B

# zpool export mpu1pool
# echo $?
0
# swapon -a

It is not obvious that the SWAP has to be freed before one can export ZFS pool.
Would it be possible to improve this situation?

Kind regards,
Andrey Arapov

[Colin Ian King (colin-king)]

Internally zpool export will call unshare and umount, so a "pool is busy" indicates that process(es) has/have one or more files open and so this operation fails.

One can check for these processes using either:

lsof -Pn

or

fuser -vm /poolname

(where poolname is the name of your pool)

and it's worth checking the status of the pool using:

zpool status -v

[Colin Ian King (colin-king)]

There has been no reply to this information request in comment #1, so I am closing this bug. Please re-open it if it still an issue.

[Andrey Arapov (andrey-arapov)]

Hi,

sorry, I have missed your reply of 2018-07-17.

I wanted to highlight that ZFS's export <pool> was blocked until "swapoff -a" command was ran.
Then, I have mentioned in the OP that the filesystem was unmounted, hence there was no possibility for any file to be opened.

And what information was requested in the comment #1?

[Andrey Arapov (andrey-arapov)]

Hum, just came across the same issue again:

# modinfo zfs | head -2
filename: /lib/modules/4.15.0-43-generic/kernel/zfs/zfs/zfs.ko
version: 0.7.5-1ubuntu16.4

# sync
# echo 3 > /proc/sys/vm/drop_caches
# zfs unmount -a
# swapoff -a
# zpool export WD4Gpool
cannot export 'WD4Gpool': pool is busy

# lsof nor fuser do not show anything related to the WD4Gpool mountpoints.

# dmsetup table
does not show anything related to ZFS as expected (I do not have the volume datasets)

Please re-open this bug.

[Andrey Arapov (andrey-arapov)]

Actually I've just figured why I could not export the pool.
One of my docker containers was using the FS, so I could not see it with the "lsof" :/ (mount namespace). I figured when I saw multiple zfs kernel calls with bpftrace :-)

I am wondering what would be required for the "zfs unmount -a" to actually fail when its FS is actually used in some other namespace?

[Andrey Arapov (andrey-arapov)]

Ok, this is not limited to ZFS. So that's solely due to namespace isolation.

"It's not a bug, it's a feature" (c) :-)

[Colin Ian King (colin-king)]

OK, thanks for the update.