| 2009-02-20 21:23:08 |
Michael Flaig |
bug |
|
|
added bug |
| 2009-02-20 21:39:15 |
Martin Kossick |
None: bugtargetdisplayname |
Ubuntu |
udev (Ubuntu) |
|
| 2009-02-20 21:39:15 |
Martin Kossick |
None: bugtargetname |
ubuntu |
udev (Ubuntu) |
|
| 2009-02-20 21:39:15 |
Martin Kossick |
None: statusexplanation |
|
reassigning this bug to package udev |
|
| 2009-02-20 21:39:15 |
Martin Kossick |
None: title |
Bug #332270 in Ubuntu: "[jaunty] doesn't boot anymore after udev upgrade" |
Bug #332270 in udev (Ubuntu): "[jaunty] doesn't boot anymore after udev upgrade" |
|
| 2009-02-20 23:09:35 |
Martin Kossick |
udev: status |
New |
Confirmed |
|
| 2009-02-20 23:09:35 |
Martin Kossick |
udev: statusexplanation |
reassigning this bug to package udev |
Okay, thank you.
Setting status to confirmed.
For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status .
|
|
| 2009-02-21 13:46:20 |
Albert Damen |
bug |
|
|
added attachment 'syslog.gz' (syslog.gz) |
| 2009-02-21 20:17:54 |
Michael Evans |
udev: status |
Confirmed |
In Progress |
|
| 2009-02-21 20:17:54 |
Michael Evans |
udev: assignee |
|
intuitivenipple |
|
| 2009-02-21 20:17:54 |
Michael Evans |
udev: statusexplanation |
Okay, thank you.
Setting status to confirmed.
For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status .
|
20090221-12:13:33 < IntuitiveNipple> Sure... you do that, I'll do the new package and test it in the VM
|
|
| 2009-02-22 02:08:29 |
Daniel T Chen |
udev: importance |
Undecided |
Critical |
|
| 2009-02-22 02:08:29 |
Daniel T Chen |
udev: statusexplanation |
20090221-12:13:33 < IntuitiveNipple> Sure... you do that, I'll do the new package and test it in the VM
|
|
|
| 2009-02-22 03:25:27 |
Scott James Remnant (Canonical) |
udev: status |
In Progress |
Incomplete |
|
| 2009-02-22 04:41:35 |
Anders Kaseorg |
udev: status |
Incomplete |
Confirmed |
|
| 2009-02-22 04:41:35 |
Anders Kaseorg |
udev: assignee |
intuitivenipple |
|
|
| 2009-02-22 04:41:35 |
Anders Kaseorg |
udev: statusexplanation |
|
For me:
http://web.mit.edu/andersk/Public/lp-332270/dpkg_-l.txt
http://web.mit.edu/andersk/Public/lp-332270/ls_sys_block.txt
http://web.mit.edu/andersk/Public/lp-332270/lvm_pvs.txt
http://web.mit.edu/andersk/Public/lp-332270/lvm_vgs.txt
|
|
| 2009-02-22 06:03:53 |
Marc Deslauriers |
bug |
|
|
added attachment 'dpkg_l.txt' (mdeslaur dpkg -l) |
| 2009-02-22 06:04:16 |
Marc Deslauriers |
bug |
|
|
added attachment 'ls_sys_block.txt' (mdeslaur ls /sys/block) |
| 2009-02-22 06:04:37 |
Marc Deslauriers |
bug |
|
|
added attachment 'lvm_pvs.txt' (mdeslaur lvm pvs) |
| 2009-02-22 06:05:09 |
Marc Deslauriers |
bug |
|
|
added attachment 'lvm_vgs.txt' (mdeslaur lvm vgs) |
| 2009-02-22 10:27:42 |
Luka Renko |
bug |
|
|
added attachment 'nw8440-lvm-udev-bug.tar.gz' (collected logs from LVM laptop) |
| 2009-02-22 10:31:31 |
Peter Sabaini |
bug |
|
|
added attachment 'dpkg-l.txt' (dpkg-l.txt) |
| 2009-02-22 10:31:51 |
Peter Sabaini |
bug |
|
|
added attachment 'ls-sys-block.txt' (ls-sys-block.txt) |
| 2009-02-22 10:32:10 |
Peter Sabaini |
bug |
|
|
added attachment 'lvm-pvs.txt' (lvm-pvs.txt) |
| 2009-02-22 10:32:25 |
Peter Sabaini |
bug |
|
|
added attachment 'lvm-vgs.txt' (lvm-vgs.txt) |
| 2009-02-22 10:54:01 |
hunger |
bug |
|
|
added attachment '332270_info.tar.gz' (requested system information) |
| 2009-02-22 13:54:19 |
TJ |
bug |
|
|
added attachment 'boot-log-udev-debug.log.gz' (Gzipped boot log with udev_log=debug) |
| 2009-02-22 14:01:46 |
TJ |
bug |
|
|
added attachment 'boot-log-udev-debug.log.gz' (Gzipped boot log with udev_log=debug) |
| 2009-02-22 14:04:19 |
Daniel Hahler |
bug |
|
|
added attachment 'bug332270.tar.gz' (bug332270.tar.gz) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'dpkgl.txt' (dpkgl.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'lvm-pvs.txt' (lvm-pvs.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'lvm-vgs.txt' (lvm-vgs.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'lvm-lvs.txt' (lvm-lvs.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'sys-block.txt' (sys-block.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment '60-persistent-storage-modified-rules.txt' (60-persistent-storage-modified-rules.txt) |
| 2009-02-22 14:09:12 |
Albert Damen |
bug |
|
|
added attachment 'dev-udev-watch.txt' (dev-udev-watch.txt) |
| 2009-02-22 14:32:14 |
Daniel Hahler |
udev: status |
Confirmed |
Triaged |
|
| 2009-02-22 14:32:14 |
Daniel Hahler |
udev: statusexplanation |
For me:
http://web.mit.edu/andersk/Public/lp-332270/dpkg_-l.txt
http://web.mit.edu/andersk/Public/lp-332270/ls_sys_block.txt
http://web.mit.edu/andersk/Public/lp-332270/lvm_pvs.txt
http://web.mit.edu/andersk/Public/lp-332270/lvm_vgs.txt
|
Scott, I think it makes sense to provide an udev update, which includes a/the workaround - unless a proper fix is not found already.
There should be enough info for debugging provided already and given the impact of this bug, it makes not much sense to upset more users using the development branch. |
|
| 2009-02-22 17:03:28 |
TJ |
bug |
|
|
added attachment 'boot-udev_log=debug.log.gz' (udev_log=debug via serial console) |
| 2009-02-22 17:25:56 |
Scott James Remnant (Canonical) |
title |
[jaunty] doesn't boot anymore after udev upgrade |
udev repeatedly generates "change" events for the same block device(s) |
|
| 2009-02-22 19:20:43 |
Scott James Remnant (Canonical) |
udev: importance |
Critical |
High |
|
| 2009-02-22 19:20:43 |
Scott James Remnant (Canonical) |
udev: statusexplanation |
Scott, I think it makes sense to provide an udev update, which includes a/the workaround - unless a proper fix is not found already.
There should be enough info for debugging provided already and given the impact of this bug, it makes not much sense to upset more users using the development branch. |
Reducing Importance.
This bug so far seems to affect a very small number of people, and is not replicable by installing with a configuration involving a combination of MD, LVM and encrypted swap. It's therefore likely to be a problem caused by some people's existing configuration, which may not even be a configuration our installer can create.
Please note that Importance is not Priority - I will still be investigating this, but right now, we wouldn't stop the release for it.
Your help is required to determine what is unique about your systems that is causing this bug.
Please attach a tar file containing your /etc/udev/rules.d and /lib/udev/rules.d directories as well as /dev itself |
|
| 2009-02-22 19:51:28 |
Scott James Remnant (Canonical) |
udev: statusexplanation |
Reducing Importance.
This bug so far seems to affect a very small number of people, and is not replicable by installing with a configuration involving a combination of MD, LVM and encrypted swap. It's therefore likely to be a problem caused by some people's existing configuration, which may not even be a configuration our installer can create.
Please note that Importance is not Priority - I will still be investigating this, but right now, we wouldn't stop the release for it.
Your help is required to determine what is unique about your systems that is causing this bug.
Please attach a tar file containing your /etc/udev/rules.d and /lib/udev/rules.d directories as well as /dev itself |
But do track for the release |
|
| 2009-02-22 19:51:28 |
Scott James Remnant (Canonical) |
udev: milestone |
|
jaunty-alpha-5 |
|
| 2009-02-22 20:13:45 |
Jonathan Hudson |
bug |
|
|
added attachment 'jh-332270.tar.gz' (jh-332270.tar.gz) |
| 2009-02-22 20:53:44 |
TJ |
bug |
|
|
added attachment 'boot-udev-storm.log.gz' (Udev event storm during init-premount) |
| 2009-02-22 20:56:27 |
jayboi |
bug |
|
|
added attachment 'udev.tar.gz' (udev.tar.gz) |
| 2009-02-22 21:32:44 |
TJ |
bug |
|
|
added attachment 'boot-udev-storm-handle_inotify.log' (handle_inotify log messages) |
| 2009-02-22 22:48:09 |
Albert Damen |
bug |
|
|
added attachment 'udev_rules.tar.gz' (udev_rules.tar.gz) |
| 2009-02-22 22:48:09 |
Albert Damen |
bug |
|
|
added attachment 'dev.tar.gz' (dev.tar.gz) |
| 2009-02-23 02:52:20 |
Luke Yelavich |
bug |
|
|
added attachment 'dpkg-l.log' (Dpkg package list) |
| 2009-02-23 02:52:57 |
Luke Yelavich |
bug |
|
|
added attachment 'sysblock.log' (/sys/block) |
| 2009-02-23 02:53:23 |
Luke Yelavich |
bug |
|
|
added attachment 'lvm-vgs.log' (lvm vgs) |
| 2009-02-23 02:53:42 |
Luke Yelavich |
bug |
|
|
added attachment 'lvm-pvs.log' (lvm pvs) |
| 2009-02-23 11:18:10 |
Oleksij Rempel |
bug |
|
|
added attachment 'trace' (trace) |
| 2009-02-23 12:39:42 |
Christian Neugebauer |
bug |
|
|
added attachment 'munin-df-day.png' (munin-df-day.png) |
| 2009-02-23 16:23:44 |
TJ |
description |
Since todays upgrade of udev jaunty is not able to boot anymore
affects also recovery mode, of course.
I can see messages like
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda (9367)
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda/sda2 (9366)
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda/sda2 (9365)
the messages appear in blocks of quite many, always repeating after 5 minutes or so and the number is counting up.
Harddisk LED is flashing rapidly but no hearable access to the hard disk
I think it has something to do with udev as this is in the last upgrade log
2009-02-20 20:59:24 upgrade libvolume-id1 137-2 138-1
2009-02-20 20:59:24 status unpacked libvolume-id1 138-1
2009-02-20 20:59:24 status unpacked libvolume-id1 138-1
2009-02-20 20:59:25 configure libvolume-id1 138-1 138-1
2009-02-20 20:59:25 status unpacked libvolume-id1 138-1
2009-02-20 20:59:25 status half-configured libvolume-id1 138-1
2009-02-20 20:59:25 status installed libvolume-id1 138-1
2009-02-20 20:59:26 upgrade udev 137-2 138-1
2009-02-20 20:59:27 status unpacked udev 138-1
2009-02-20 20:59:27 status unpacked udev 138-1
2009-02-20 20:59:34 upgrade libudev0 137-2 138-1
2009-02-20 20:59:34 status unpacked libudev0 138-1
2009-02-20 20:59:34 status unpacked libudev0 138-1
2009-02-20 20:59:41 configure udev 138-1 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status half-configured udev 138-1
2009-02-20 20:59:41 status installed udev 138-1
2009-02-20 20:59:47 configure libudev0 138-1 138-1
2009-02-20 20:59:47 status unpacked libudev0 138-1
2009-02-20 20:59:47 status half-configured libudev0 138-1
2009-02-20 20:59:47 status installed libudev0 138-1 |
SYMPTOMS
Early in the boot process (during initial ram-disk script processing) the PC appears to stop although there may be a lot of repeated disk activity, possibly accompanied by messages to screen of the form:
UEVENT[1235217664.944992] change /devices/pci0000:00/0000:00:09.0/host0/target0:0:0/0:0:0:0/block/sda (block)
This happens after an automatic package upgrade to udev to 138-1.
RECOVERY PROCEDURE
If the PC is unable to complete start-up, effectively locking you out, this is how to regain control and install a work-around until a solution is available.
1. When the PC starts enter the GRUB menu.
2. Select the Recovery entry and edit it (press "e") - this starts in single-user mode with minimal services.
3. Select the "kernel" line and edit it (press "e")
4. Add " break=top" to the end of the line and press Enter.
5. With the "kernel" line highlighted press "b" to boot using the edited entry.
The boot process will stop when the initial ram-disk scripts begin and you'll be faced with a busybox shell.
Edit the rules that contain the problematic "watch" and remove it:
for rule in /lib/udev/rules.d/*; do sed -i -e '/watch/s/watch//' $rule; done
Modify the init script so the boot can continue:
sed -i -e '/run-init/s,${init},/bin/bash,' /init
Restart the boot process by pressing Ctrl+D
The system *should* now boot to a root prompt although there may still be a delay. Once there, edit the non-initrd system rules:
for rule in /lib/udev/rules.d/*; do sed -i -e '/watch/s/watch//' $rule; done
And rebuild the initial ram-disk image so you don't have to go through this procedure at every boot:
To update just the kernel version that is currently running:
update-initramfs -u -k `uname -r`
or to update for all installed kernels:
update-initramfs -u -k all
Now restart the PC and hopefully the regular multiuser start-up should go normally.
-- TJ
-----------------------------
Since todays upgrade of udev jaunty is not able to boot anymore
affects also recovery mode, of course.
I can see messages like
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda (9367)
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda/sda2 (9366)
/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0/block/sda/sda2 (9365)
the messages appear in blocks of quite many, always repeating after 5 minutes or so and the number is counting up.
Harddisk LED is flashing rapidly but no hearable access to the hard disk
I think it has something to do with udev as this is in the last upgrade log
2009-02-20 20:59:24 upgrade libvolume-id1 137-2 138-1
2009-02-20 20:59:24 status unpacked libvolume-id1 138-1
2009-02-20 20:59:24 status unpacked libvolume-id1 138-1
2009-02-20 20:59:25 configure libvolume-id1 138-1 138-1
2009-02-20 20:59:25 status unpacked libvolume-id1 138-1
2009-02-20 20:59:25 status half-configured libvolume-id1 138-1
2009-02-20 20:59:25 status installed libvolume-id1 138-1
2009-02-20 20:59:26 upgrade udev 137-2 138-1
2009-02-20 20:59:27 status unpacked udev 138-1
2009-02-20 20:59:27 status unpacked udev 138-1
2009-02-20 20:59:34 upgrade libudev0 137-2 138-1
2009-02-20 20:59:34 status unpacked libudev0 138-1
2009-02-20 20:59:34 status unpacked libudev0 138-1
2009-02-20 20:59:41 configure udev 138-1 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status unpacked udev 138-1
2009-02-20 20:59:41 status half-configured udev 138-1
2009-02-20 20:59:41 status installed udev 138-1
2009-02-20 20:59:47 configure libudev0 138-1 138-1
2009-02-20 20:59:47 status unpacked libudev0 138-1
2009-02-20 20:59:47 status half-configured libudev0 138-1
2009-02-20 20:59:47 status installed libudev0 138-1 |
|
| 2009-02-23 16:38:52 |
Scott James Remnant (Canonical) |
bug |
|
|
assigned to lvm2 (Ubuntu) |
| 2009-02-23 16:45:18 |
Scott James Remnant (Canonical) |
lvm2: status |
New |
Triaged |
|
| 2009-02-23 16:45:18 |
Scott James Remnant (Canonical) |
lvm2: importance |
Undecided |
High |
|
| 2009-02-23 16:45:18 |
Scott James Remnant (Canonical) |
lvm2: statusexplanation |
|
There are two parts to this bug.
The actual bug is being caused by "lvm vgscan" opening all block devices on the system writable and then closing them again, which triggers the inotify event. There seems to be no reason they cannot be opened readonly, since the only thing it does is call fstat() and read() on them.
Since the devices include any LVM PVs (which is what it's looking for), this gives you a recursive loop where LVM PVs appear to have been changed, so udev calls "lvm vgscan" again to update its tables.
Part #1 of the fix will be a patch to LVM to only open block devices with O_RDONLY
Part #2 of the fix will be a patch to udev to not add the inotify watch until all RUN rules are processed; this will prevent the initial onslaught on boot, and hopefully guard against anything it runs changing the device resulting in reprocessing (which is better handled with udev rules ordering anyway).
It will obviously not protect against a "udevadm trigger" once booted, so it's possible that we'll trip over this with another program later and will have to fix that one in the same manner as LVM.
It's not yet clear why this does not affect _everyone_, there appears to be a race involved as slowing udev down with various log levels - or using only one core - is enough to hide the bug for some people. I need to work out why that is before uploading the fixes
|
|
| 2009-02-23 16:45:18 |
Scott James Remnant (Canonical) |
lvm2: milestone |
|
jaunty-alpha-5 |
|
| 2009-02-23 16:47:16 |
Scott James Remnant (Canonical) |
bug |
|
|
added attachment 'udev.inotify.patch' (udev patch to add inotify watch after processing RUN rules) |
| 2009-02-23 20:35:14 |
Launchpad Janitor |
udev: status |
Triaged |
Fix Released |
|
| 2009-02-23 22:30:07 |
Launchpad Janitor |
lvm2: status |
Triaged |
Fix Released |
|
| 2009-02-23 22:33:58 |
Scott James Remnant (Canonical) |
bug |
|
|
added attachment 'open-readonly.patch' (Patch for LVM2 to open in readonly when only reading) |
| 2009-04-13 16:56:32 |
Francisco Borges |
removed subscriber Francisco Borges |
|
|
|
| 2009-06-28 04:08:09 |
Launchpad Janitor |
branch linked |
|
lp:ubuntu/karmic/lvm2 |
|
| 2010-08-23 07:20:39 |
Launchpad Janitor |
branch linked |
|
lp:~ubuntu-core-dev/ubuntu/maverick/udev/ubuntu |
|
