lack of CONFIG_GETOPT_LONG in busybox-udeb completely breaks Kickstart

[Samuel Kleiner]

The 'getopt -l' feature (for parsing long options) is not built into busybox in Ubuntu 8.10. This completely breaks almost all uses of Kickstart, which relies on it. Kickstart is typically used from netboot installations, and so it is worth fixing this in an update to 8.10 even though we don't expect to do a full point release with updated CD images.

DEVELOPMENT BRANCH: Fixed in busybox 1:1.10.2-2ubuntu1 in Jaunty by making the same configuration change proposed here.

PATCH: http://launchpadlibrarian.net/21537368/busybox_1.10.2-1ubuntu7.debdiff

TEST CASE: Boot with the Kickstart file in bug 320053 (you may want to fill in proper MD5-encrypted passwords there; see https://help.ubuntu.com/8.10/installation-guide/i386/automatic-install.html for how to boot with a Kickstart file). Without this patch, partitioning and user information will not be correctly preseeded, among other things, and the installation will fail. (I have not tested this test case extensively; contact cjwatson on IRC if you have problems.)

REGRESSION POTENTIAL: Very little for the installer since we still provide the released installer build. However, the same source package builds a component of the initramfs for normal systems too. Although it shouldn't change, there's always the possibility of insanity, so after installing busybox-initramfs from this update, make sure that you can still boot.

[Launchpad Janitor (janitor)]

This bug was fixed in the package busybox - 1:1.10.2-2ubuntu1

---------------
busybox (1:1.10.2-2ubuntu1) jaunty; urgency=low

  * Resynchronise with Debian. Remaining changes:
    - Enable chvt, getopt, killall, losetup, mktemp, NFS mount, od, ping,
      stat, and remote syslog for udeb targets.
    - Enable 'mount -f' and mount helpers for all targets.
    - Add busybox-initramfs.
    - mkswap-uuid.patch: Set UUIDs on swap areas.
    - sleep-fractional.patch: Add support for fractional times.
    - procps-kernel-threads.patch: Avoid killing kernel threads in init when
      killing all processes.
    - test-bin.patch: Move test and friends to /bin.
    - mount-helper-fixes.patch: Add proper support for mount -i (don't use
      external mount helper), and trivial support for umount -i although
      busybox umount doesn't support external helpers yet. Fix handling of
      mount -f and -n.
  * Drop busybox-initramfs Breaks, since busybox prefers applets again.
  * getopt-long-only.patch: Allow 'getopt -l' to be enabled independently of
    the whole of CONFIG_GETOPT_LONG, so that it can be enabled more cheaply.
  * Enable 'getopt -l' for udeb targets (LP: #293586).
  * Tag Ubuntu patches per
    https://wiki.ubuntu.com/UbuntuDevelopment/PatchTaggingGuidelines.

busybox (1:1.10.2-2) unstable; urgency=low

  * [deb, static] Again prefer applets and fallback to real executable if it
    is not found. (closes: #498029)
  * [deb, static] Reenable long options for several applets. (closes: #490396)
  * [udeb] Don't recompile during installation.

 -- Colin Watson <email address hidden> Thu, 06 Nov 2008 17:38:28 +0000

[Marc Bevand (bevand-m)]

This is a huge bug. I hit it when trying to kickstart Intrepid Ibex yesterday, and I tracked it down exactly to that missing CONFIG_GETOPT_LONG. When will the initrd...

http://archive.ubuntu.com/ubuntu/dists/intrepid/main/installer-$ARCH/current/images/netboot/ubuntu-installer/$ARCH/initrd.gz

...be regenerated to include the new version of busybox ? For those waiting for it to be regenerated, here is a workaround that consists of patching a broken initrd, to make kickstart's "url --url http://xxx" option work again (and only this option, others will remain broken):

 $ mkdir i && cd i
 $ zcat ../initrd.gz | cpio --make-dir -i
 $ patch -Np1 <../fix-kickseed-url-handler.patch
 $ find . | cpio -o -H newc | gzip -9 >../initrd.gz

-marc

[netllama (netllama)]

There's a fixed i386 initrd available here:
http://people.ubuntu.com/~cjwatson/tmp/intrepid-busybox-fix/netboot/ubuntu-installer/i386/

Unfortunately, nothing for x86_64. Until this is fixed, Ubuntu-8.10 is 100% useless to me. I'm not about to manually install 8.10 a few hundred times.

[Marc Bevand (bevand-m)]

IMHO this bug should be mentioned in the release notes.
http://www.ubuntu.com/getubuntu/releasenotes/810

[Cody Herriges (ody-cat)]

Any update on new initrd.gz for netboot installers to address this bug?

[netllama (netllama)]

At the very least, it would be great if someone provided an estimate on when a fixed initrd would be available. I'm rapidly approaching a cutoff where support for 8.10 (in my employer's software) is going to be dropped completely if I can't perform any automated installations for testing purposes.

[lcars (andrea-inversepath)]

Confirmed on hardy too, it prevents using kickstart 'part' keyword for setting partitions and have unattended installations.

[Colin Watson (cjwatson)]

Sorry about this 8.10 update taking so long. I've started the process now.

[Martin Pitt (pitti)]

Accepted into intrepid-proposed, please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[netllama (netllama)]

I don't understand the last comment. Can someone point me to where or how I get a fixed installer initrd for 8.10 ?

[Colin Watson (cjwatson)]

Martin's comment was automatically generated from a template used for all packages accepted into stable releases. I'm uploading debian-installer now, which will cause installer initrds to be built; after that, you'll be able to get fixed installer initrds.

[Colin Watson (cjwatson)]

You can now find images that fix this bug here:

  http://archive.ubuntu.com/ubuntu/dists/intrepid-proposed/main/installer-i386/20080522ubuntu26/images/
  http://archive.ubuntu.com/ubuntu/dists/intrepid-proposed/main/installer-amd64/20080522ubuntu26/images/

[netllama (netllama)]

I just attempted to use this:
http://archive.ubuntu.com/ubuntu/dists/intrepid-proposed/main/installer-amd64/20080522ubuntu26/images/netboot/ubuntu-installer/amd64/initrd.gz

And when booting with it and the original 8.10 installer kernel, it fails almost immediately claiming that there are no NICs detected. That problem doesn't happen if I go back to using the original 8.10 netboot initrd.gz (but then I hit the bug that started this entire mess).

Either I'm confused, or the images referenced in the last comment are broken in a new & different way.

[Colin Watson (cjwatson)]

Oh, there are some special procedures you need to follow when testing installer images from -proposed: specifically, you need to boot with this kernel parameter:

  apt-setup/proposed=true

Could you retest with this?

[Steve Beattie (sbeattie)]

I've verified that the 20080522ubuntu26 intrepid-proposed netboot image correctly installs with the kickseed file described in bug 320053 (with real passwords put in place). I've also verified that, after updating to busybox-initramfs 1:1.10.2-1ubuntu7 in intrepid-proposed and forcing the initramfs to be regenerated, my test instance still boots.

[netllama (netllama)]

@Colin
I added "apt-setup/proposed=true" to the 'append' line of my kickstart config. That sort of helped, but not enough to do anything useful. Now the install fails immediately after getting a DHCP lease with the error "No kernel modules were found. This probable is due to a mismatch between the kernel used by this version of the installer and the kernel version available in the archive".

 I run a local, behind the firewall, mirror of 8.10 for kickstart installations. The location of this mirror has the contents of the 8.10 DVD. I can only assume that the new netboot kernel & initrd are a completely different version than what shipped with 8.10, and that's what is triggering this error.

Perhaps I'm misdiagnosing the current state of things, but for me, this still isn't working.

[Colin Watson (cjwatson)]

netllama: You can't use this against just the 8.10 DVD, sorry; the 8.10 DVD had a different kernel ABI than the security-updated one currently in intrepid-updates, and we always build the installer against the latest available kernel. You'll have to use a real mirror, although you can probably start this by taking what you have now and running debmirror against it (or similar).

This does not represent a test either way of this busybox change; it would have happened with any updated installer image.

[Colin Watson (cjwatson)]

By the way, I find that I very often have to help out people who have attempted to construct reduced mirrors by mounting CD or DVD images and making them available through a web server. My experience is that the time saved avoiding the initial download is not worth the time you spend debugging problems with it, and I do not recommend this setup in general.

[netllama (netllama)]

That's rather unfortunate that no one is going to fix the actual bug. I cannot use a real mirror for numerous reasons. First, all the systems that are performing the 8.10 install are behind a firewall with no internet access. Additionally, these systems are used for my employer's regression testing, and are reloaded from bare metal several hundred times/month. Even if they had access to a 'real' mirror, the increase in OS install time would be quite significant, and therefore unacceptable. Support for 8.10 had been put on hold as a result of this bug. Since no actual fix is going to be forthcoming, it looks like we're going to have to drop 8.10 altogether.

[Steve Langasek (vorlon)]

netllama,

"real mirror" doesn't imply a public mirror; it just implies a complete mirror that includes intrepid and intrepid-updates (and in this case, intrepid-proposed). As Colin says, you can use the packages from the DVD to /start/ the mirror; but you need to have a full set of updates on your mirror, not just the set of packages on the original DVD.

[Colin Watson (cjwatson)]

Yes, exactly what Steve said. I certainly did not mean to imply that you must use a public mirror; it's entirely possible and reasonable to use a private mirror for installations - it just needs to be a little more than a mounted DVD image. For example, something like the following debmirror invocation is sufficient (based on what I use myself):

  debmirror /mirror/ubuntu --host gb.archive.ubuntu.com --method http \
          --root /ubuntu/ --dist=intrepid,intrepid-security,intrepid-updates,intrepid-proposed \
          --section main,main/debian-installer,restricted,restricted/debian-installer \
          --arch=amd64,i386 --ignore=main/installer-

If you're reloading machines several times a month, it will be in your interest to have them picking up updates from a single local mirror rather than over the Internet anyway! (I'm assuming that you want to test against 8.10 plus updates as well as, or perhaps instead of, merely 8.10. Most software vendors do, since 8.10 plus updates is what real users use.)

[Launchpad Janitor (janitor)]

This bug was fixed in the package busybox - 1:1.10.2-1ubuntu7

---------------
busybox (1:1.10.2-1ubuntu7) intrepid-proposed; urgency=low

  * Enable 'getopt -l' for udeb targets, needed by kickseed (LP: #293586).

 -- Colin Watson <email address hidden> Thu, 22 Jan 2009 15:21:42 +0000

[zapster (zapster)]

Is it possible that this bug is related to Question 59032 in any way?
(https://answers.launchpad.net/ubuntu/+question/59032)

[Colin Watson (cjwatson)]

On Thu, Jan 29, 2009 at 03:08:21PM -0000, zapster wrote:
> Is it possible that this bug is related to Question 59032 in any way?
> (https://answers.launchpad.net/ubuntu/+question/59032)

No, it is entirely unrelated.

[Colin Watson (cjwatson)]

I've built an amd64 netboot image with this fix but with no other updates from intrepid, to cater for people who don't want to use intrepid-security or intrepid-updates for one reason or another. New URLs are as follows:

  http://people.ubuntu.com/~cjwatson/tmp/intrepid-busybox-fix/amd64/netboot/
  http://people.ubuntu.com/~cjwatson/tmp/intrepid-busybox-fix/i386/netboot/

I'm afraid I have been unable to test the amd64 image, so caveat emptor, but it *should* work.

[netllama (netllama)]

Thank you, this is exactly what I've needed, and I've confirmed that it works perfectly for kickstart installations.