| 2019-12-03 18:06:31 |
Eric Desrochers |
bug |
|
|
added bug |
| 2019-12-03 18:07:09 |
Eric Desrochers |
bug task added |
|
initramfs-tools (Ubuntu) |
|
| 2019-12-03 18:07:19 |
Eric Desrochers |
tags |
|
sts |
|
| 2019-12-03 18:51:16 |
Eric Desrochers |
attachment added |
|
initramfs.debug_DISCO https://bugs.launchpad.net/ubuntu/+source/lvm2/+bug/1854981/+attachment/5309607/+files/initramfs.debug |
|
| 2019-12-03 19:50:30 |
Eric Desrochers |
description |
Only the lv for root volume get activated, most likely because of the grub parameter that specifies it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are, but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing. |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are, but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
NOTE:
That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context. |
|
| 2019-12-03 19:56:21 |
Eric Desrochers |
lvm2 (Ubuntu): importance |
Undecided |
High |
|
| 2019-12-03 19:56:22 |
Eric Desrochers |
initramfs-tools (Ubuntu): importance |
Undecided |
High |
|
| 2019-12-03 19:58:41 |
Eric Desrochers |
description |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are, but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
NOTE:
That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context. |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are, but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing. |
|
| 2019-12-03 21:00:14 |
Eric Desrochers |
description |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are, but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing. |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing. |
|
| 2019-12-03 21:17:57 |
Eric Desrochers |
description |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing. |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the user merge approach, one can install package 'usrmerge' and make their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-03 21:19:46 |
Eric Desrochers |
description |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the user merge approach, one can install package 'usrmerge' and make their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-03 21:21:30 |
Dan Streetman |
bug |
|
|
added subscriber Dan Streetman |
| 2019-12-05 14:04:21 |
Eric Desrochers |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
lvm2 (Ubuntu Bionic) |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
initramfs-tools (Ubuntu Bionic) |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
nominated for series |
|
Ubuntu Disco |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
lvm2 (Ubuntu Disco) |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
initramfs-tools (Ubuntu Disco) |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
nominated for series |
|
Ubuntu Xenial |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
lvm2 (Ubuntu Xenial) |
|
| 2019-12-05 14:04:21 |
Eric Desrochers |
bug task added |
|
initramfs-tools (Ubuntu Xenial) |
|
| 2019-12-05 14:06:38 |
Eric Desrochers |
initramfs-tools (Ubuntu Disco): status |
New |
Won't Fix |
|
| 2019-12-05 14:06:42 |
Eric Desrochers |
initramfs-tools (Ubuntu Bionic): status |
New |
Won't Fix |
|
| 2019-12-05 14:06:44 |
Eric Desrochers |
initramfs-tools (Ubuntu Xenial): status |
New |
Won't Fix |
|
| 2019-12-05 14:06:47 |
Eric Desrochers |
initramfs-tools (Ubuntu): status |
New |
Won't Fix |
|
| 2019-12-05 14:06:55 |
Eric Desrochers |
lvm2 (Ubuntu): status |
New |
Fix Released |
|
| 2019-12-05 14:07:04 |
Eric Desrochers |
lvm2 (Ubuntu Disco): status |
New |
Confirmed |
|
| 2019-12-05 14:07:07 |
Eric Desrochers |
lvm2 (Ubuntu Bionic): status |
New |
Confirmed |
|
| 2019-12-05 14:07:11 |
Eric Desrochers |
lvm2 (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2019-12-05 14:07:13 |
Eric Desrochers |
lvm2 (Ubuntu Disco): importance |
Undecided |
Medium |
|
| 2019-12-05 14:07:18 |
Eric Desrochers |
lvm2 (Ubuntu): importance |
High |
Undecided |
|
| 2019-12-05 14:18:52 |
Eric Desrochers |
lvm2 (Ubuntu Xenial): status |
New |
Won't Fix |
|
| 2019-12-05 14:31:58 |
Eric Desrochers |
description |
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
## DRAFT in progress ....##
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 14:32:44 |
Eric Desrochers |
description |
## DRAFT in progress ....##
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
## DRAFT in progress ....##
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 14:33:29 |
Eric Desrochers |
description |
## DRAFT in progress ....##
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 14:34:27 |
Eric Desrochers |
description |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 16:03:03 |
Eric Desrochers |
lvm2 (Ubuntu Bionic): status |
Confirmed |
In Progress |
|
| 2019-12-05 16:03:04 |
Eric Desrochers |
lvm2 (Ubuntu Disco): status |
Confirmed |
In Progress |
|
| 2019-12-05 16:03:07 |
Eric Desrochers |
lvm2 (Ubuntu Disco): assignee |
|
Eric Desrochers (slashd) |
|
| 2019-12-05 16:03:09 |
Eric Desrochers |
lvm2 (Ubuntu Bionic): assignee |
|
Eric Desrochers (slashd) |
|
| 2019-12-05 19:08:20 |
Eric Desrochers |
description |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
Xenial and Eoan and late didn't exhibit the situation.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 20:14:06 |
Eric Desrochers |
lvm2 (Ubuntu Xenial): status |
Won't Fix |
Fix Released |
|
| 2019-12-05 20:59:31 |
Łukasz Zemczak |
lvm2 (Ubuntu Disco): status |
In Progress |
Fix Committed |
|
| 2019-12-05 20:59:32 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2019-12-05 20:59:34 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2019-12-05 20:59:37 |
Łukasz Zemczak |
tags |
sts |
sts verification-needed verification-needed-disco |
|
| 2019-12-05 21:02:02 |
Łukasz Zemczak |
lvm2 (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2019-12-05 21:02:08 |
Łukasz Zemczak |
tags |
sts verification-needed verification-needed-disco |
sts verification-needed verification-needed-bionic verification-needed-disco |
|
| 2019-12-05 21:05:22 |
Eric Desrochers |
description |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, user will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Bionic and Disco.
So one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
Xenial and Eoan and late didn't exhibit the situation.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, for users who will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Disco. (it is not recommended to do new disco installs anyway since it's going EOL shortly) One would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
IIRC Bionic will have a new point release somewhere in Feb 2020 but until then one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive as well.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
Xenial and Eoan and late didn't exhibit the situation.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-05 21:08:58 |
Eric Desrochers |
description |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, for users who will see this fixed, will still experience the situation in standard ISO, since I'm afraid there is no more point release for Disco. (it is not recommended to do new disco installs anyway since it's going EOL shortly) One would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
IIRC Bionic will have a new point release somewhere in Feb 2020 but until then one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive as well.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
Xenial and Eoan and late didn't exhibit the situation.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
[Impact]
In summary, the problem is a LVM 'auto-activation' issue. Basically the LVM activation doesn't occurs during the boot process (except for LVM root partition) and since /usr is an important component before the 'pivot'[0], the init inside the initramfs space tries to mount /usr. If /usr is a LVM volume then it will try to mount it while its backend device is not activated yet (due to the actual auto-activation issue), thus the mounting will fails and bring the user to the initramfs prompt for debugging.
Workaround:
At the initramfs prompt: lvm vgchange -ay and then 'ctrl-d' to resume the boot process.
[Test Case]
- Install Bionic or Disco
- Create a VG with /usr on its on LV
- Complete the installation
- At reboot the system will be stuck at mounting /usr file system
Extra notes:
Since this problem is ONLY related to LVM activation. If one create a non-LVM /usr separate partition, everything will work as expected.
Other separate LVM (e.g. /home, /var/, /srv) if on its own separate LVM volume will not be activate either, but since they don't need to be mounted in the initramfs space, it's not a problem, they can be activated later on (after the pivot) by systemd.
So the only combination possible is when /usr is on its separate LV only.
[Regression potential]
I don't foresee any regression, the fix will instruct udev rule to pvscan activate volume based on their major/minor number if LVM is scanned.
+RUN+="(LVM_EXEC)/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"
The current situation doesn't auto-activate at all.
One downside, for users who will see this fixed, will still experience the situation in standard ISO, since I'm afraid Disco won't produces new ISO. (it is not recommended to do new disco installs anyway since it's going EOL shortly) One would need to rely on the mini.iso for fetching the latest lvm2 package in the archive.
IIRC Bionic will have a new point release somewhere in Feb 2020 but until then one would need to rely on the mini.iso for fetching the latest lvm2 package in the archive as well.
or use the workaround in the [Impact] section and then apt-get dist-upgrade in order to get the latest lvm2.
[Other information]
This problem only exhibit in Bionic and Disco.
Xenial and Eoan and late didn't exhibit the situation.
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
https://wiki.debian.org/UsrMerge
[Original Description]
Only the lv for root volume get activated, because of the grub parameter that specifies/enforce it "root=/dev/mapper/ubuntu-vg-ubuntu--lv"
http://man7.org/linux/man-pages/man7/bootparam.7.html
'root=...'
This argument tells the kernel what device is to be used as
the root filesystem while booting.
If one add a separate LV for /usr, the system will go straight to initramfs prompt failling to mount /usr.
At initramfs prompt, we notice that 'lv-usr' status is 'NOT available'.
Performing 'lvm vgchange -ay' at initramfs prompt workaround the problem and allow one to successfully boot.
Adding 'debug' parameter, we clearly we see /root being detected and mounted:
initramfs.debug:
=> + mount -r -t ext4 /dev/mapper/ubuntu--vg-ubuntu--lv /root
=> + mountroot_status=0
+ [ ]
+ log_end_msg
+ _log_msg done.\n
+ [ n = y ]
+ printf done.\n
done.
+ read_fstab_entry /usr
+ found=1
+ [ -f /root/etc/fstab ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ / = /usr ]
+ read MNT_FSNAME MNT_DIR MNT_TYPE MNT_OPTS MNT_FREQ MNT_PASS MNT_JUNK
+ [ /usr = /usr ]
+ [ -n ]
+ found=0
+ break 2
+ return 0
+ log_begin_msg Mounting /usr file system
+ _log_msg Begin: Mounting /usr file system ...
then the code read /etc/fstab and specifically search for /usr (most likely because of the /usr binary merged) and try to mount if if found.
initramfs-tools:init
271 maybe_break mount
272 log_begin_msg "Mounting root file system"
273 # Always load local and nfs (since these might be needed for /etc or
274 # /usr, irrespective of the boot script used to mount the rootfs).
275 . /scripts/local
276 . /scripts/nfs
277 . /scripts/${BOOT}
278 parse_numeric "${ROOT}"
279 maybe_break mountroot
280 mount_top
281 mount_premount
282 mountroot
283 log_end_msg
284
=> 285 if read_fstab_entry /usr; then
=> 286 log_begin_msg "Mounting /usr file system"
=> 287 mountfs /usr
=> 288 log_end_msg
=> 289 fi
In this case, /usr is present in /etc/fstab but the logical volume is not available, so it is mounting a filesystem without his backend device, thus goes straight to the initramfs prompt because /usr couldn't be mounted.
It clearly seems to be an 'auto-activation' issue at boot.
For other such as /home, /var, it's not a big deal cause they can be activated later on in the process and they are (I haven't check but I guess systemd or systemd unit/generator is taking care of it at some point), but /usr is a important piece to have mounted before the pivot since it contains most of the crucial binary, especially that nowadays /sbin, /bin, and /lib are pointing to /usr:
lrwxrwxrwx 1 root root 10 Aug 26 13:51 libx32 -> usr/libx32
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 9 Aug 26 13:51 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 7 Aug 26 13:51 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Aug 26 13:51 bin -> usr/bin
lrwxrwxrwx 1 root root 8 Aug 26 13:51 sbin -> usr/sbin
NOTE:
* That doesn't affect /usr found in /etc/fstab on its separate partition when no LVM involve (e.g. /dev/vdb /usr ext4 ....). It only cause issue when /usr is in a LVM context.
* I was able to reproduce on Bionic and Disco so far.
Eoan doesn't seem to exhibit the situation so far in my testing.
* While certain release such as Bionic, Xenial doesn't come implicitly with the /usr merge approach. One can install package 'usrmerge' and convert their system /usr merged. I don't think removing the read_fstable_entry for /usr is an option here, as some user could potentially decide to convert their system with 'usrmerge' pkg. |
|
| 2019-12-06 14:04:02 |
Eric Desrochers |
tags |
sts verification-needed verification-needed-bionic verification-needed-disco |
sts verification-done-bionic verification-needed verification-needed-disco |
|
| 2019-12-06 15:21:01 |
Eric Desrochers |
tags |
sts verification-done-bionic verification-needed verification-needed-disco |
sts verification-done-bionic verification-done-disco verification-needed |
|
| 2019-12-10 09:00:23 |
Joachim Bremer |
bug |
|
|
added subscriber Joachim Bremer |
| 2019-12-12 14:28:24 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2019-12-12 14:34:50 |
Launchpad Janitor |
lvm2 (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2019-12-12 14:38:37 |
Launchpad Janitor |
lvm2 (Ubuntu Disco): status |
Fix Committed |
Fix Released |
|
| 2019-12-13 03:12:10 |
Brett Frankenberger |
bug |
|
|
added subscriber Brett Frankenberger |
