parted calls 'udevadm settle' after every change causing it to hang for 180s if no udevd with matching magic key is running.

[Al Stone]

parted can take a very long time when run within a chroot.

The issue here is an interface mismatch between udevd and udevadm.

'udevadm settle' works by sending a netlink message to the udevd process. However, the magic key (UDEV_MONITOR_MAGIC) udevd uses to communicate w/ udevadm changed between lucid and maverick (presumably due to a non-backwards-compatible interface change). So, if you are running a maverick system and try to run lucid's udevadm in a chroot, there will be no compatible udevd to respond. The long delay is due to a hardcoded timeout default in udevadm - it will wait up to 180s for udevd to signal settle completion.

Development branch: Fixed in parted 2.3-4ubuntu2 by not calling 'udevadm settle' if we're chrooted.

TEST CASE: I verified this by debootstrapping both a lucid and a maverick chroot on a maverick host. In a pristine host config:

   chroot /chroot/lucid udevadm settle -> hang
   chroot /chroot/maverick udevadm settle -> no hang

If I modify the host's udev to listen to lucid's UDEV_MONITOR_MAGIC define, I get the inverse results.

So how does live-helper trigger this? parted has an ubuntu-specific patch that calls 'udevadm settle' after manipulating partitions to avoid races with other applications that might be triggered by partition changes.

[dann frazier (dannf)]

The issue here is as actually an interface mismatch between udevd and udevadm.

'udevadm settle' works by sending a netlink message to the udevd process. However, the magic key (UDEV_MONITOR_MAGIC) udevd uses to communicate w/ udevadm changed between lucid and maverick (presumably due to a non-backwards-compatible interface change). So, if you are running a maverick system and try to run lucid's udevadm in a chroot, there will be no compatible udevd to respond. The long delay is due to a hardcoded timeout default in udevadm - it will wait up to 180s for udevd to signal settle completion.

I verified this by debootstrapping both a lucid and a maverick chroot on a maverick host. In a pristine host config:

   chroot /chroot/lucid udevadm settle -> hang
   chroot /chroot/lucid udevadm settle -> no hang

If I modify the host's udev to listen to lucid's UDEV_MONITOR_MAGIC define, I get the inverse results.

So how does live-helper trigger this? parted has an ubuntu-specific patch that calls 'udevadm settle' after manipulating partitions to avoid races with other applications that might be triggered by partition changes.

In my opinion, the right place to fix this is in live-helper, since it could use the host's parted (and therefore the host's udevadm) to tweak devices in the chroot.

[Al Stone (ahs3)]

Re-assigning to live-build

[Al Stone (ahs3)]

Attaching a patch that changes which parted is used by lh, thus avoiding the udev problems previously mentioned in this bug.

[Cody A.W. Somerville (cody-somerville)]

The bug isn't in live-build, even if we can work around this issue with a hack, and there is no way upstream will accept this patch. Reassigning to udev.

[Steve Magoun (smagoun)]

Marking oem-priority; this dramatically increases (order of magnitude) the time needed to build a Maverick or Natty OS image in our infrastructure.

[Pedro Villavicencio (pedro)]

Thanks for the report and the patch, Could someone on the foundations team have a look into it? Thanks a lot.

[Cody A.W. Somerville (cody-somerville)]

Deleting patch from bug since its already been rejected.

[Colin Watson (cjwatson)]

Sorry about the delay on this. My preference, I think, is simply to change parted to avoid calling 'udevadm settle' when in a chroot. The fact that it calls 'udevadm settle' is an Ubuntu patch for the benefit of our installer in the first place, and we never rely on calling it in a chroot. I'd be more comfortable with doing this than with going round chasing up everything that might call parted, or with trying to persuade udev upstream to maintain compatibility (which I sort of doubt they're going to care much about).

I'll get to this as soon as possible.

[Colin Watson (cjwatson)]

Incidentally, I did look at live-helper and it seems to me that it isn't going to run across the races that the 'udevadm settle' call was originally introduced to prevent, so I believe that my fix is safe from that point of view.

[Launchpad Janitor (janitor)]

This bug was fixed in the package parted - 2.3-4ubuntu2

---------------
parted (2.3-4ubuntu2) natty; urgency=low

  * Don't call 'udevadm settle' when chrooted (LP: #664115).
 -- Colin Watson <email address hidden> Mon, 06 Dec 2010 16:29:47 +0000

[Martin Pitt (pitti)]

Accepted parted into maverick-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Cody A.W. Somerville (cody-somerville)]

I tested this update and can confirm that the hanging behaviour described in this bug no longer occurs with the proposed update installed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package parted - 2.3-2ubuntu2

---------------
parted (2.3-2ubuntu2) maverick-proposed; urgency=low

  * Don't call 'udevadm settle' when chrooted (LP: #664115).
 -- Colin Watson <email address hidden> Mon, 06 Dec 2010 16:47:27 +0000

[Chris Halse Rogers (raof)]

SRU team ack. Please accept into lucid-proposed.

[Martin Pitt (pitti)]

Accepted parted into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Chris Van Hoof (vanhoof)]

Hi Al -- Mind giving this one a test on Lucid now that an updated parted is in -proposed?

Cheers,
Chris

[Al Stone (ahs3)]

I can no longer get the original behavior to re-occur. Looks like we're good with this one, Chris.

[Launchpad Janitor (janitor)]

This bug was fixed in the package parted - 2.2-5ubuntu5.2

---------------
parted (2.2-5ubuntu5.2) lucid-proposed; urgency=low

  * Don't call 'udevadm settle' when chrooted (LP: #664115).
 -- Colin Watson <email address hidden> Thu, 30 Jun 2011 09:11:45 +0100