Snapshots of encrypted zfs 2.1.5 filesystems sometimes broken

[Walter]

As reported in https://github.com/openzfs/zfs/issues/15474 -- I have had occasional broken snapshots with ZFS 2.1.5 on Jammy on encrypted filesystems.

This is _not_ the same problem as https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1987190

**Manifestation**

  # ls -l /var/lib/mysql/redacted/.zfs/snapshot/planb-20231101T0500Z
  ls: cannot access '/var/lib/mysql/redacted/.zfs/snapshot/planb-20231101T0500Z/aria_log.00000001': Input/output error
  ls: cannot access '/var/lib/mysql/redacted/.zfs/snapshot/planb-20231101T0500Z/aria_log_control':
  Input/output error
  total 10129
  -????????? ? ? ? ? ? aria_log.00000001
  -????????? ? ? ? ? ? aria_log_control
  -rw-rw---- 1 mysql mysql 0 nov 1 02:55 ddl.log
  -rw-rw---- 1 mysql mysql 16384 okt 17 10:26 ddl_recovery.log
  -rw-rw---- 1 mysql mysql 10737419544 nov 1 02:55 galera.cache
  -rw-rw---- 1 mysql mysql 8 okt 17 10:26 galera.pid
  ...

Creating newer snapshots succeeded just fine, but this snapshot was unreadable: each and every file in in it was not readable (except for 0-sized files).

Trying to 'zfs send' this snapshot failed as well.

**Workaround**

Deleting the snapshot worked. But at some point a new snapshot would be broken again.

**Fix**

Upgrading to 2.1.13 -- as described in the Github ticket -- fixed the problem.

Cheers,
Walter Doekes
OSSO B.V.

[Seth Arnold (seth-arnold)]

It looks like you're still working with rincebrain to figure out the root cause; please do update this if you come to a conclusion. (On the one hand, I'm optimistic, he's really good at this stuff. On the other hand, the encryption stuff feels more brittle than than usual.)

Thanks

[Launchpad Janitor (janitor)]

[Expired for zfs-linux (Ubuntu) because there has been no activity for 60 days.]