Activity log for bug #1875577

Date Who What changed Old value New value Message
2020-04-28 07:51:44 John Gray bug added bug
2020-04-28 18:31:25 Steve Langasek cryptsetup (Ubuntu): status New Incomplete
2020-04-30 08:19:29 John Gray affects cryptsetup (Ubuntu) systemd (Ubuntu)
2020-05-01 22:04:06 Steve Langasek systemd (Ubuntu): status Incomplete New
2020-05-03 21:56:47 Launchpad Janitor systemd (Ubuntu): status New Confirmed
2020-05-05 00:20:45 Dan Streetman affects systemd (Ubuntu) zfs-linux (Ubuntu)
2020-05-05 00:21:01 Dan Streetman bug added subscriber Didier Roche
2020-05-05 02:26:06 Richard Laager attachment added 2150-fix-systemd-dependency-loops.patch https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1875577/+attachment/5366544/+files/2150-fix-systemd-dependency-loops.patch
2020-05-05 04:27:30 Ubuntu Foundations Team Bug Bot tags patch
2020-05-05 04:27:36 Ubuntu Foundations Team Bug Bot bug added subscriber Ubuntu Review Team
2020-05-05 21:48:40 Rob David bug added subscriber Rob David
2020-05-15 17:25:13 Guido Berhoerster bug added subscriber Guido Berhoerster
2020-06-04 11:21:09 Colin Ian King zfs-linux (Ubuntu): importance Undecided Low
2020-06-04 11:21:12 Colin Ian King zfs-linux (Ubuntu): assignee Colin Ian King (colin-king)
2020-06-10 16:06:28 Launchpad Janitor zfs-linux (Ubuntu): status Confirmed Fix Released
2020-07-14 13:45:45 Michael-250 bug added subscriber Michael-250
2020-08-12 06:05:49 Andy Sayler bug added subscriber Andy Sayler
2020-08-16 22:10:11 Randall Leeds bug added subscriber Randall Leeds
2020-12-01 22:57:58 Simon May bug added subscriber Simon May
2021-05-20 21:14:25 Dan Streetman bug added subscriber Heitor Alves de Siqueira
2021-05-20 21:14:58 Dan Streetman bug added subscriber Dan Streetman
2021-05-20 21:15:23 Dan Streetman nominated for series Ubuntu Focal
2021-05-20 21:15:23 Dan Streetman bug task added zfs-linux (Ubuntu Focal)
2021-07-12 09:44:24 Launchpad Janitor zfs-linux (Ubuntu Focal): status New Confirmed
2021-07-12 13:52:02 Heitor Alves de Siqueira zfs-linux (Ubuntu Focal): importance Undecided Medium
2021-07-12 13:52:03 Heitor Alves de Siqueira zfs-linux (Ubuntu Focal): assignee Heitor Alves de Siqueira (halves)
2021-07-12 15:06:38 Heitor Alves de Siqueira description root@eu1:/var/log# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal root@eu1:/var/log# apt-cache policy cryptsetup cryptsetup: Installed: (none) Candidate: 2:2.2.2-3ubuntu2 Version table: 2:2.2.2-3ubuntu2 500 500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages OTHER BACKGROUND INFO: ====================== 1. machine has 2 drives. each drive is partitioned into 2 partitions, zfs and swap 2. Ubuntu 20.04 installed on ZFS root using debootstrap (debootstrap_1.0.118ubuntu1_all) 3. The ZFS root pool is a 2 partition mirror (the first partition of each disk) 4. /etc/crypttab is set up as follows: swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802933-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 WHAT I EXPECTED =============== I expected machine would reboot and have encrypted swap that used two devices under /dev/mapper WHAT HAPPENED INSTEAD ===================== On reboot, swap setup fails with the following messages in /var/log/syslog: Apr 28 17:13:01 eu1 kernel: [ 5.360793] systemd[1]: cryptsetup.target: Found ordering cycle on systemd-cryptsetup@swap.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360795] systemd[1]: cryptsetup.target: Found dependency on systemd-random-seed.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360796] systemd[1]: cryptsetup.target: Found dependency on zfs-mount.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360797] systemd[1]: cryptsetup.target: Found dependency on zfs-load-module.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360798] systemd[1]: cryptsetup.target: Found dependency on cryptsetup.target/start Apr 28 17:13:01 eu1 kernel: [ 5.360799] systemd[1]: cryptsetup.target: Job systemd-cryptsetup@swap.service/start deleted to break ordering cycle starting with cryptsetup.target/start . . . . . . Apr 28 17:13:01 eu1 kernel: [ 5.361082] systemd[1]: Unnecessary job for /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 was removed Also, /dev/mapper does not contain any swap devices: root@eu1:/var/log# ls -l /dev/mapper total 0 crw------- 1 root root 10, 236 Apr 28 17:13 control root@eu1:/var/log# And top shows no swap: MiB Swap: 0.0 total, 0.0 free, 0.0 used. 63153.6 avail Mem [Impact] Encrypted swap partitions may not load correctly with ZFS root, due to ordering cycle on zfs-mount.service. [Test Plan] 1. Install Ubuntu 20.04 using ZFS-on-root 2. Add encrypted partition to /etc/crypttab: swap /dev/nvme0n1p1 /dev/urandom swap,cipher=aes-xts-plain64,size=256 3. Add swap partition to /etc/fstab: /dev/mapper/swap none swap sw 0 0 4. Reboot and check whether swap has loaded correctly, and whether boot logs show ordering cycle: [ 6.638228] systemd[1]: systemd-random-seed.service: Found ordering cycle on zfs-mount.service/start [ 6.639418] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import.target/start [ 6.640474] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import-cache.service/start [ 6.641637] systemd[1]: systemd-random-seed.service: Found dependency on cryptsetup.target/start [ 6.642734] systemd[1]: systemd-random-seed.service: Found dependency on systemd-cryptsetup@swap.service/start [ 6.643951] systemd[1]: systemd-random-seed.service: Found dependency on systemd-random-seed.service/start [ 6.645098] systemd[1]: systemd-random-seed.service: Job zfs-mount.service/start deleted to break ordering cycle starting with systemd-random-seed.service/start [ SKIP ] Ordering cycle found, skipping Mount ZFS filesystems [Where problems could occur] Since we're changing the zfs-mount-generator service, regressions could show up during mounting of ZFS partitions. We should thoroughly test different scenarios of ZFS such as ZFS-on-root, separate ZFS partitions and the presence of swap, to make sure all partitions are mounted correctly and no ordering cycles are present. [Other Info] This has been fixed upstream by the following commits: * ec41cafee1da Fix a dependency loop [0] * 62663fb7ec19 Fix another dependency loop [1] The patches above have been introduced in version 2.1.0, with upstream backports to zfs-2.0. In Ubuntu, it's present in Groovy and later releases, so it's still needed in Focal. $ rmadison -a source zfs-linux zfs-linux | 0.8.3-1ubuntu12 | focal | source zfs-linux | 0.8.3-1ubuntu12.9 | focal-security | source zfs-linux | 0.8.3-1ubuntu12.10 | focal-updates | source zfs-linux | 0.8.4-1ubuntu11 | groovy | source zfs-linux | 0.8.4-1ubuntu11.2 | groovy-updates | source zfs-linux | 2.0.2-1ubuntu5 | hirsute | source zfs-linux | 2.0.3-8ubuntu5 | impish | source [0] https://github.com/openzfs/zfs/commit/ec41cafee1da [1] https://github.com/openzfs/zfs/commit/62663fb7ec19 ORIGINAL DESCRIPTION ==================== root@eu1:/var/log# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal root@eu1:/var/log# apt-cache policy cryptsetup cryptsetup:   Installed: (none)   Candidate: 2:2.2.2-3ubuntu2   Version table:      2:2.2.2-3ubuntu2 500         500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages OTHER BACKGROUND INFO: ====================== 1. machine has 2 drives. each drive is partitioned into 2 partitions, zfs and swap 2. Ubuntu 20.04 installed on ZFS root using debootstrap (debootstrap_1.0.118ubuntu1_all) 3. The ZFS root pool is a 2 partition mirror (the first partition of each disk) 4. /etc/crypttab is set up as follows: swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802933-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 WHAT I EXPECTED =============== I expected machine would reboot and have encrypted swap that used two devices under /dev/mapper WHAT HAPPENED INSTEAD ===================== On reboot, swap setup fails with the following messages in /var/log/syslog: Apr 28 17:13:01 eu1 kernel: [ 5.360793] systemd[1]: cryptsetup.target: Found ordering cycle on systemd-cryptsetup@swap.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360795] systemd[1]: cryptsetup.target: Found dependency on systemd-random-seed.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360796] systemd[1]: cryptsetup.target: Found dependency on zfs-mount.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360797] systemd[1]: cryptsetup.target: Found dependency on zfs-load-module.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360798] systemd[1]: cryptsetup.target: Found dependency on cryptsetup.target/start Apr 28 17:13:01 eu1 kernel: [ 5.360799] systemd[1]: cryptsetup.target: Job systemd-cryptsetup@swap.service/start deleted to break ordering cycle starting with cryptsetup.target/start . . . . . . Apr 28 17:13:01 eu1 kernel: [ 5.361082] systemd[1]: Unnecessary job for /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 was removed Also, /dev/mapper does not contain any swap devices: root@eu1:/var/log# ls -l /dev/mapper total 0 crw------- 1 root root 10, 236 Apr 28 17:13 control root@eu1:/var/log# And top shows no swap: MiB Swap: 0.0 total, 0.0 free, 0.0 used. 63153.6 avail Mem
2021-07-12 15:07:59 Heitor Alves de Siqueira attachment added lp1875577-focal.debdiff https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1875577/+attachment/5510557/+files/lp1875577-focal.debdiff
2021-07-12 15:10:48 Heitor Alves de Siqueira zfs-linux (Ubuntu Focal): status Confirmed In Progress
2021-07-14 12:52:47 Heitor Alves de Siqueira description [Impact] Encrypted swap partitions may not load correctly with ZFS root, due to ordering cycle on zfs-mount.service. [Test Plan] 1. Install Ubuntu 20.04 using ZFS-on-root 2. Add encrypted partition to /etc/crypttab: swap /dev/nvme0n1p1 /dev/urandom swap,cipher=aes-xts-plain64,size=256 3. Add swap partition to /etc/fstab: /dev/mapper/swap none swap sw 0 0 4. Reboot and check whether swap has loaded correctly, and whether boot logs show ordering cycle: [ 6.638228] systemd[1]: systemd-random-seed.service: Found ordering cycle on zfs-mount.service/start [ 6.639418] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import.target/start [ 6.640474] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import-cache.service/start [ 6.641637] systemd[1]: systemd-random-seed.service: Found dependency on cryptsetup.target/start [ 6.642734] systemd[1]: systemd-random-seed.service: Found dependency on systemd-cryptsetup@swap.service/start [ 6.643951] systemd[1]: systemd-random-seed.service: Found dependency on systemd-random-seed.service/start [ 6.645098] systemd[1]: systemd-random-seed.service: Job zfs-mount.service/start deleted to break ordering cycle starting with systemd-random-seed.service/start [ SKIP ] Ordering cycle found, skipping Mount ZFS filesystems [Where problems could occur] Since we're changing the zfs-mount-generator service, regressions could show up during mounting of ZFS partitions. We should thoroughly test different scenarios of ZFS such as ZFS-on-root, separate ZFS partitions and the presence of swap, to make sure all partitions are mounted correctly and no ordering cycles are present. [Other Info] This has been fixed upstream by the following commits: * ec41cafee1da Fix a dependency loop [0] * 62663fb7ec19 Fix another dependency loop [1] The patches above have been introduced in version 2.1.0, with upstream backports to zfs-2.0. In Ubuntu, it's present in Groovy and later releases, so it's still needed in Focal. $ rmadison -a source zfs-linux zfs-linux | 0.8.3-1ubuntu12 | focal | source zfs-linux | 0.8.3-1ubuntu12.9 | focal-security | source zfs-linux | 0.8.3-1ubuntu12.10 | focal-updates | source zfs-linux | 0.8.4-1ubuntu11 | groovy | source zfs-linux | 0.8.4-1ubuntu11.2 | groovy-updates | source zfs-linux | 2.0.2-1ubuntu5 | hirsute | source zfs-linux | 2.0.3-8ubuntu5 | impish | source [0] https://github.com/openzfs/zfs/commit/ec41cafee1da [1] https://github.com/openzfs/zfs/commit/62663fb7ec19 ORIGINAL DESCRIPTION ==================== root@eu1:/var/log# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal root@eu1:/var/log# apt-cache policy cryptsetup cryptsetup:   Installed: (none)   Candidate: 2:2.2.2-3ubuntu2   Version table:      2:2.2.2-3ubuntu2 500         500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages OTHER BACKGROUND INFO: ====================== 1. machine has 2 drives. each drive is partitioned into 2 partitions, zfs and swap 2. Ubuntu 20.04 installed on ZFS root using debootstrap (debootstrap_1.0.118ubuntu1_all) 3. The ZFS root pool is a 2 partition mirror (the first partition of each disk) 4. /etc/crypttab is set up as follows: swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802933-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 WHAT I EXPECTED =============== I expected machine would reboot and have encrypted swap that used two devices under /dev/mapper WHAT HAPPENED INSTEAD ===================== On reboot, swap setup fails with the following messages in /var/log/syslog: Apr 28 17:13:01 eu1 kernel: [ 5.360793] systemd[1]: cryptsetup.target: Found ordering cycle on systemd-cryptsetup@swap.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360795] systemd[1]: cryptsetup.target: Found dependency on systemd-random-seed.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360796] systemd[1]: cryptsetup.target: Found dependency on zfs-mount.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360797] systemd[1]: cryptsetup.target: Found dependency on zfs-load-module.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360798] systemd[1]: cryptsetup.target: Found dependency on cryptsetup.target/start Apr 28 17:13:01 eu1 kernel: [ 5.360799] systemd[1]: cryptsetup.target: Job systemd-cryptsetup@swap.service/start deleted to break ordering cycle starting with cryptsetup.target/start . . . . . . Apr 28 17:13:01 eu1 kernel: [ 5.361082] systemd[1]: Unnecessary job for /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 was removed Also, /dev/mapper does not contain any swap devices: root@eu1:/var/log# ls -l /dev/mapper total 0 crw------- 1 root root 10, 236 Apr 28 17:13 control root@eu1:/var/log# And top shows no swap: MiB Swap: 0.0 total, 0.0 free, 0.0 used. 63153.6 avail Mem [Impact] Encrypted swap partitions may not load correctly with ZFS root, due to ordering cycle on zfs-mount.service. [Test Plan] 1. Install Ubuntu 20.04 using ZFS-on-root 2. Add encrypted partition to /etc/crypttab:    swap /dev/nvme0n1p1 /dev/urandom swap,cipher=aes-xts-plain64,size=256 3. Add swap partition to /etc/fstab:    /dev/mapper/swap none swap sw 0 0 4. Reboot and check whether swap has loaded correctly, and whether boot logs show ordering cycle: [ 6.638228] systemd[1]: systemd-random-seed.service: Found ordering cycle on zfs-mount.service/start [ 6.639418] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import.target/start [ 6.640474] systemd[1]: systemd-random-seed.service: Found dependency on zfs-import-cache.service/start [ 6.641637] systemd[1]: systemd-random-seed.service: Found dependency on cryptsetup.target/start [ 6.642734] systemd[1]: systemd-random-seed.service: Found dependency on systemd-cryptsetup@swap.service/start [ 6.643951] systemd[1]: systemd-random-seed.service: Found dependency on systemd-random-seed.service/start [ 6.645098] systemd[1]: systemd-random-seed.service: Job zfs-mount.service/start deleted to break ordering cycle starting with systemd-random-seed.service/start [ SKIP ] Ordering cycle found, skipping Mount ZFS filesystems [Where problems could occur] Since we're changing the zfs-mount-generator service, regressions could show up during mounting of ZFS partitions. We should thoroughly test different scenarios of ZFS such as ZFS-on-root, separate ZFS partitions and the presence of swap, to make sure all partitions are mounted correctly and no ordering cycles are present. Below is a list of suggested test scenarios that we should check for regressions: 1. ZFS-on-root + encrypted swap (see "Test Plan" section above) 2. Encrypted root + separate ZFS partitions 3. ZFS on LUKS 4. ZFS on dm-raid Although scenario 4 is usually advised against (ZFS itself should handle RAID), it's a good smoke test to validate that mount order is being handled correctly. [Other Info] This has been fixed upstream by the following commits: * ec41cafee1da Fix a dependency loop [0] * 62663fb7ec19 Fix another dependency loop [1] The patches above have been introduced in version 2.1.0, with upstream backports to zfs-2.0. In Ubuntu, it's present in Groovy and later releases, so it's still needed in Focal. $ rmadison -a source zfs-linux  zfs-linux | 0.8.3-1ubuntu12 | focal | source  zfs-linux | 0.8.3-1ubuntu12.9 | focal-security | source  zfs-linux | 0.8.3-1ubuntu12.10 | focal-updates | source  zfs-linux | 0.8.4-1ubuntu11 | groovy | source  zfs-linux | 0.8.4-1ubuntu11.2 | groovy-updates | source  zfs-linux | 2.0.2-1ubuntu5 | hirsute | source  zfs-linux | 2.0.3-8ubuntu5 | impish | source [0] https://github.com/openzfs/zfs/commit/ec41cafee1da [1] https://github.com/openzfs/zfs/commit/62663fb7ec19 ORIGINAL DESCRIPTION ==================== root@eu1:/var/log# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal root@eu1:/var/log# apt-cache policy cryptsetup cryptsetup:   Installed: (none)   Candidate: 2:2.2.2-3ubuntu2   Version table:      2:2.2.2-3ubuntu2 500         500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages OTHER BACKGROUND INFO: ====================== 1. machine has 2 drives. each drive is partitioned into 2 partitions, zfs and swap 2. Ubuntu 20.04 installed on ZFS root using debootstrap (debootstrap_1.0.118ubuntu1_all) 3. The ZFS root pool is a 2 partition mirror (the first partition of each disk) 4. /etc/crypttab is set up as follows: swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 swap /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802933-part2 /dev/urandom swap,cipher=aes-xts-plain64,size=256 WHAT I EXPECTED =============== I expected machine would reboot and have encrypted swap that used two devices under /dev/mapper WHAT HAPPENED INSTEAD ===================== On reboot, swap setup fails with the following messages in /var/log/syslog: Apr 28 17:13:01 eu1 kernel: [ 5.360793] systemd[1]: cryptsetup.target: Found ordering cycle on systemd-cryptsetup@swap.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360795] systemd[1]: cryptsetup.target: Found dependency on systemd-random-seed.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360796] systemd[1]: cryptsetup.target: Found dependency on zfs-mount.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360797] systemd[1]: cryptsetup.target: Found dependency on zfs-load-module.service/start Apr 28 17:13:01 eu1 kernel: [ 5.360798] systemd[1]: cryptsetup.target: Found dependency on cryptsetup.target/start Apr 28 17:13:01 eu1 kernel: [ 5.360799] systemd[1]: cryptsetup.target: Job systemd-cryptsetup@swap.service/start deleted to break ordering cycle starting with cryptsetup.target/start . . . . . . Apr 28 17:13:01 eu1 kernel: [ 5.361082] systemd[1]: Unnecessary job for /dev/disk/by-id/nvme-SAMSUNG_MZVLB1T0HALR-00000_S3W6NX0M802914-part2 was removed Also, /dev/mapper does not contain any swap devices: root@eu1:/var/log# ls -l /dev/mapper total 0 crw------- 1 root root 10, 236 Apr 28 17:13 control root@eu1:/var/log# And top shows no swap: MiB Swap: 0.0 total, 0.0 free, 0.0 used. 63153.6 avail Mem
2021-07-27 12:04:17 Łukasz Zemczak zfs-linux (Ubuntu Focal): status In Progress Fix Committed
2021-07-27 12:04:18 Łukasz Zemczak bug added subscriber Ubuntu Stable Release Updates Team
2021-07-27 12:04:20 Łukasz Zemczak bug added subscriber SRU Verification
2021-08-02 18:05:04 Heitor Alves de Siqueira tags patch patch verification-done verification-done-focal
2021-08-04 04:07:05 Chris Halse Rogers removed subscriber Ubuntu Stable Release Updates Team
2021-08-04 04:21:57 Launchpad Janitor zfs-linux (Ubuntu Focal): status Fix Committed Fix Released