Udev slows down the bootup by a lot

[i2000s]

I think this is a bug as it started from recent updates (I don't remember when was the start). Basically, systemd-udev-settle takes 1min 30sec in the booting process. Since I use LVM and btrfs systems, I cannot mask the udev service from booting. Any insight would be appreciated. Below are some of my log. Computer: Lenovo Thinkpad P50, Ubuntu 16.04.3.

$ uname -a
Linux QC5-ubuntu 4.10.0-38-generic #42~16.04.1-Ubuntu SMP Tue Oct 10 16:32:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ sudo lspci -vvnn
see https://paste.ubuntu.com/25900851/

$ journalctl -xb
see https://paste.ubuntu.com/25900867/

$ systemd-analyze blame
1min 29.765s systemd-udev-settle.service
      9.672s NetworkManager-wait-online.service
      2.955s plymouth-quit-wait.service
      2.449s upower.service
       738ms lvm2-activation-early.service
       645ms dev-mapper-ubuntuvg\x2droot.device
       519ms nmbd.service
       436ms grub-common.service
       431ms sysfsutils.service
       430ms dictd.service
       430ms apport.service
       423ms irqbalance.service
       417ms plymouth-start.service
       416ms speech-dispatcher.service
       381ms ondemand.service
       325ms vboxdrv.service
       312ms media-I.mount
       291ms lvm2-activation.service
       237ms media-F.mount
       213ms lvm2-monitor.service
       170ms lvm2-activation-net.service
       163ms systemd-hwdb-update.service
       155ms media-H.mount
...

$ system-analyze critical-chain systemd-udev-settle.service
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.

systemd-udev-settle.service +1min 29.765s
└─systemd-udev-trigger.service @427ms +60ms
  └─systemd-hwdb-update.service @263ms +163ms
    └─systemd-remount-fs.service @254ms +5ms
      └─systemd-journald.socket @175ms
        └─-.slice @167ms

$ journalctl -b -u systemd-udev-settle.service
-- Logs begin at Sun 2017-11-05 16:49:28 MST, end at Sun 2017-11-05 17:00:17 MST. --
Nov 05 16:49:28 QC5-ubuntu systemd[1]: Starting udev Wait for Complete Device Initialization...
Nov 05 16:50:58 QC5-ubuntu systemd[1]: Started udev Wait for Complete Device Initialization.

$ systemctl list-dependencies --reverse systemd-udev-settle.service
systemd-udev-settle.service
● ├─lvm2-activation-early.service
● └─lvm2-activation.service

$ sudo dmidecode
see https://paste.ubuntu.com/25900903/

I have also tried the network service. The network manager needs a lot of time to boot up, but really? It requires internet to boot up? So, I check which services use it:

$ systemctl show network-online.target | grep -E 'WantedBy|Before'
WantedBy=whoopsie.service samba-ad-dc.service nmbd.service apt-daily.timer smbd.service
Before=whoopsie.service nmbd.service smbd.service rc-local.service apt-daily.timer samba-ad-dc.service snapd.refresh.service

Thanks,
Qi

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1730278

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[i2000s (i2000s)]

The bot was searching for "linux" using dpkg-query, but the problem may be related to the Linux kernel. So, I have to stop the bot as it goes to nowhere.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.14 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.14-rc8

[i2000s (i2000s)]

So, I got basically the same error as before with the kernel 4.14-rc8. There is an error on the USB port from my DisplayLink dock station, but the major slowdown part is to find and mount my LVM /home volume (see the full log of journalctl -abl from the attachment):

Nov 06 17:08:21 QC5-ubuntu systemd[1]: Stopped NVIDIA Persistence Daemon.
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Starting Show Plymouth Boot Screen...
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Started Braille Device Support.
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Started Show Plymouth Boot Screen.
Nov 06 17:09:21 QC5-ubuntu systemd-udevd[454]: seq 2806 '/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1.4/2-1.4.1/2-1.4.1:1.0'
Nov 06 17:09:50 QC5-ubuntu systemd[1]: dev-mapper-ubuntuvg\x2dhome.device: Job dev-mapper-ubuntuvg\x2dhome.device/start timed
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Timed out waiting for device dev-mapper-ubuntuvg\x2dhome.device.
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Dependency failed for /home.
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Dependency failed for Local File Systems.
Nov 06 17:08:21 QC5-ubuntu kernel: (NULL device *): hwmon_device_register() is deprecated. Please convert the driver to use hw
Nov 06 17:08:21 QC5-ubuntu kernel: thermal thermal_zone3: failed to read out thermal zone (-61)
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Found device ST500LM021-1KJ152 7.
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Started Show Plymouth Boot Screen.
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Reached target Sound Card.
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Starting Show Plymouth Boot Screen...
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Started Braille Device Support.
Nov 06 17:08:21 QC5-ubuntu kernel: cdc_ncm 2-1.4.1:1.5 enx0050b660d5b6: renamed from usb0
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Started Show Plymouth Boot Screen.
Nov 06 17:08:21 QC5-ubuntu kernel: iwlwifi 0000:04:00.0 wlp4s0: renamed from wlan0
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Started Load/Save RF Kill Switch Status.
Nov 06 17:08:21 QC5-ubuntu kernel: input: ThinkPad Extra Buttons as /devices/platform/thinkpad_acpi/input/input17
Nov 06 17:08:21 QC5-ubuntu kernel: input: HDA Intel PCH Mic as /devices/pci0000:00/0000:00:1f.3/sound/card0/input20
Nov 06 17:08:21 QC5-ubuntu kernel: input: HDA Intel PCH Headphone as /devices/pci0000:00/0000:00:1f.3/sound/card0/input21
Nov 06 17:08:21 QC5-ubuntu systemd[1]: Stopped NVIDIA Persistence Daemon.
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Starting Show Plymouth Boot Screen...
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Started Braille Device Support.
Nov 06 17:08:22 QC5-ubuntu systemd[1]: Started Show Plymouth Boot Screen.
Nov 06 17:09:21 QC5-ubuntu systemd-udevd[454]: seq 2806 '/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1.4/2-1.4.1/2-1.4.1:1.0'
Nov 06 17:09:50 QC5-ubuntu systemd[1]: dev-mapper-ubuntuvg\x2dhome.device: Job dev-mapper-ubuntuvg\x2dhome.device/start timed
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Timed out waiting for device dev-mapper-ubuntuvg\x2dhome.device.
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Dependency failed for /home.
Nov 06 17:09:50 QC5-ubuntu systemd[1]: Dependency failed for Local File Systems.

$ systemctl cat systemd-udev-settle.service
...
[Service]
Type...