Jockey not offer to install nvidia/fglrx/etc. drivers in LiveCD environment, else get error "package nvidia-current 260.19.21-0ubuntu1 failed to install/upgrade: subprocess installed post-installation script returned error exit status 1"

[Andreas Hasenack]

[Problem]
This is a common issue encountered by users trying to install -nvidia on systems with insufficient space to install the driver. This can happen for instance if the user is booting from a usb disk.

It is further noticed that in many cases the system may appear to have enough space, but it still fails on an out of space issue. It is suspected that either more space is used than expected, or the amount of available space is overreported.

Jockey should check for available space on installation, and should include a sizable margin (like 2.5x space asked). If the user has less than that amount of space, it should give a dialog warning them that they won't have enough space.

Jockey also needs to be aware that in a LiveCD environment, there may be ample space available in the reserved persistent space (the casper-rw file on the usb key), but it needs to have space on the underlying filesystem too, for creating initrd and storing the kernel image and so on.

[Steps to Reproduce]
1. Download an ISO
2. Generate a USB install key using usb-creator. Select the 'Stored in reserved extra space' option and set it to the maximum available space.
3. Using df note the remaining free space (496k in my case)
4. Boot the usb device on a system with NVIDIA or AMD graphics
5. Go to the applications menu. Select 'Additional Drivers'. Install fglrx or nvidia (or any other proprietary driver).
6. Installation of the driver fails. (I see "SystemError: installArchives() failed" but might error differently depending on how much space was available).
7. Shut down system and re-examine usb drive. The persistence file has not changed size, but other things on the USB drive have increased and used up the remaining space. (I see initrd.lz.new of size 496k)

[Original Report]
Booted from usb disk

Got a notice saying that unity was not available because I didn't have 3D.

Went to System->Administration->Hardware drivers, selected to install nvidia drivers.

Apport detected crash, driver installation failed.

ProblemType: Package
DistroRelease: Ubuntu 11.04
Package: nvidia-current 260.19.21-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.37-7.19-generic 2.6.37-rc3
Uname: Linux 2.6.37-7-generic i686
Architecture: i386
Date: Fri Dec 3 23:57:10 2010
DkmsStatus:

ErrorMessage: subprocess installed post-installation script returned error exit status 1
GdmLog1: Error: command ['cat', '/var/log/gdm/:0.log.1'] failed with exit code 1: cat: /var/log/gdm/:0.log.1: No such file or directory
GdmLog2: Error: command ['cat', '/var/log/gdm/:0.log.2'] failed with exit code 1: cat: /var/log/gdm/:0.log.2: No such file or directory
LiveMediaBuild: Ubuntu 11.04 "Natty Narwhal" - Alpha i386 (20101203)
MachineType: System manufacturer System Product Name
ProcKernelCmdLine: BOOT_IMAGE=/casper/vmlinuz noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/hostname.seed boot=casper initrd=/casper/initrd.lz quiet splash -- maybe-ubiquity
SourcePackage: nvidia-graphics-drivers
Title: package nvidia-current 260.19.21-0ubuntu1 failed to install/upgrade: subprocess installed post-installation script returned error exit status 1
dmi.bios.date: 11/23/2007
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1226
dmi.board.asset.tag: To Be Filled By O.E.M.
dmi.board.name: P5B-Deluxe
dmi.board.vendor: ASUSTeK Computer INC.
dmi.board.version: Rev 1.xx
dmi.chassis.asset.tag: Asset-1234567890
dmi.chassis.type: 3
dmi.chassis.vendor: Chassis Manufacture
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1226:bd11/23/2007:svnSystemmanufacturer:pnSystemProductName:pvrSystemVersion:rvnASUSTeKComputerINC.:rnP5B-Deluxe:rvrRev1.xx:cvnChassisManufacture:ct3:cvrChassisVersion:
dmi.product.name: System Product Name
dmi.product.version: System Version
dmi.sys.vendor: System manufacturer
glxinfo: Error: [Errno 2] No such file or directory
system:
 distro: Ubuntu
 codename: natty
 architecture: i686
 kernel: 2.6.37-7-generic

[Robert Hooker (sarvatt)]

How much persistent storage space do you have allocated? nvidia-current needs quite a bit to install properly, it could be a case of just not having enough available here.

[Andreas Hasenack (ahasenack)]

That may be the case. Let me try again with 512Mb (it's a 4Gb pendrive).

[Bryce Harrington (bryce)]

In discussing this with Sarvatt, it sounds like while this has been a long standing issue, the unity focus for natty makes this issue especially important now since.

I hope you don't mind that we're going to use this bug report as a master bug against jockey for adding checks for this particular workflow. I'll be revising the description of the bug to focus on the issue as we understand it.

[Andreas Hasenack (ahasenack)]

Wait, you mean the slider in startup-disk-creator where you can choose how much space you want for settings and documents, right? Or something else?

[Bryce Harrington (bryce)]

This issue has existed for some time, however in natty with Unity now being the default interface users are shown a warning that they lack 3D. This motivates them to try installing a binary driver via jockey. Especially if they're just testing to kick the tires, they may attempt to do so on media without sufficient space. Since for all NVIDIA owners the LiveCD boots with nouveau (currently defaulted to 2D-only), a lot of people will be encountering this particular bug work flow.

Because of this, I think the severity of this bug is critical.

[Bryce Harrington (bryce)]

I'm adding a Jockey task because I think this needs fixed via better installation space checking during installation by jockey.

But I'm leaving the -nvidia task open for now, for tracking/duping purposes. I don't *think* there is anything we can change in -nvidia to correct the bug, but until it's fixed we may see an increase number of reports of failures along these lines.

[Bryce Harrington (bryce)]

Btw, note that while insufficient disk space can cause the "post-installation script returned error" error, there may well be other issues that lead to this same error message.

[Andreas Hasenack (ahasenack)]

Where exactly should there be sufficient disk space? The df output below shows the free space right after I got the error:
ubuntu@ubuntu:~$ df -h
Filesystem Size Used Avail Use% Mounted on
aufs 1006M 213M 794M 22% /
none 1000M 352K 1000M 1% /dev
/dev/sde1 3.8G 1.3G 2.6G 32% /cdrom
/dev/loop0 684M 684M 0 100% /rofs
none 1006M 104K 1006M 1% /dev/shm
tmpfs 1006M 28K 1006M 1% /tmp
none 1006M 88K 1006M 1% /var/run
none 1006M 4.0K 1006M 1% /var/lock
ubuntu@ubuntu:~$

I did create this USB boot pendrive using 512Mb of space for "documents and settings", FWIW.

Here is the mount output:
ubuntu@ubuntu:~$ mount
aufs on / type aufs (rw)
none on /proc type proc (rw,noexec,nosuid,nodev)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
/dev/sde1 on /cdrom type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
/dev/loop0 on /rofs type squashfs (ro,noatime)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
gvfs-fuse-daemon on /home/ubuntu/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=ubuntu)

[Bryce Harrington (bryce)]

The way we determined it as an out of space issue, is Sarvatt reproduced your problem on a USB stick by manually installing nvidia, and got the error message that way:

<Sarvatt> Setting up initramfs-tools (0.98.1ubuntu7) ...
 update-initramfs: deferring update (trigger activated)
 lzma: Encoder error: -2147467259
 dpkg: error processing initramfs-tools (--configure):
  subprocess installed post-installation script returned error exit status 1
 after sudo apt-get install nvidia-current from a terminal

He wasn't able to find the error message in any logs. Typical places to look would be /var/log/dpkg.log or the make.log (found a couple levels deep under /var/lib/dkms). You might poke around in your /var/log/ to see if you can find an error message which indicates a problem other than disk usage.

lzma error "-2147467259" means "no space to unpack the archive". See bug #562312 as another example.

There is a possibility that you're not actually truly out of space at all, it's just that the installer *thinks* there's not enough. Some further analysis is needed to pinpoint the exact steps to reproduce the issue. If you could do some additional testing to identify the steps, that'd help move things forward faster.

[Bryce Harrington (bryce)]

Comment #7 on bug #562312 sounds quite pertinent. Please review and see if that describes the procedure you used to do your install?

[Robert Hooker (sarvatt)]

Yeah I can reproduce this by creating a liveusb with the persistent storage space slider maxed out, it requires space on the fat32 file system to manipulate the initramfs.

[Andreas Hasenack (ahasenack)]

I get this when trying to install nvidia-current with apt-get:
root@ubuntu:~# apt-get install nvidia-current
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  dkms fakeroot nvidia-settings patch screen-resolution-extra
Suggested packages:
  diffutils-doc
The following NEW packages will be installed:
  dkms fakeroot nvidia-current nvidia-settings patch screen-resolution-extra
0 upgraded, 6 newly installed, 0 to remove and 244 not upgraded.
Need to get 27.6 MB of archives.
After this operation, 81.8 MB of additional disk space will be used.
Do you want to continue [Y/n]?
Get:1 http://br.archive.ubuntu.com/ubuntu/ natty/main patch i386 2.6-3 [122 kB]
Get:2 http://br.archive.ubuntu.com/ubuntu/ natty/main dkms all 2.1.1.2-5ubuntu1 [71.9 kB]
Get:3 http://br.archive.ubuntu.com/ubuntu/ natty/main fakeroot i386 1.14.4-1ubuntu1 [118 kB]
Get:4 http://br.archive.ubuntu.com/ubuntu/ natty/restricted nvidia-current i386 260.19.21-0ubuntu1 [26.4 MB]
Get:5 http://br.archive.ubuntu.com/ubuntu/ natty/main screen-resolution-extra all 0.14 [13.9 kB]
Get:6 http://br.archive.ubuntu.com/ubuntu/ natty/main nvidia-settings i386 260.19.21-0ubuntu2 [844 kB]
Fetched 27.6 MB in 22s (1,218 kB/s)
Selecting previously deselected package patch.
(Reading database ... 129416 files and directories currently installed.)
Unpacking patch (from .../archives/patch_2.6-3_i386.deb) ...
Selecting previously deselected package dkms.
Unpacking dkms (from .../dkms_2.1.1.2-5ubuntu1_all.deb) ...
Selecting previously deselected package fakeroot.
Unpacking fakeroot (from .../fakeroot_1.14.4-1ubuntu1_i386.deb) ...
Selecting previously deselected package nvidia-current.
Unpacking nvidia-current (from .../nvidia-current_260.19.21-0ubuntu1_i386.deb) ...
Selecting previously deselected package screen-resolution-extra.
Unpacking screen-resolution-extra (from .../screen-resolution-extra_0.14_all.deb) ...
Selecting previously deselected package nvidia-settings.
Unpacking nvidia-settings (from .../nvidia-settings_260.19.21-0ubuntu2_i386.deb) ...
Processing triggers for man-db ...
Setting up patch (2.6-3) ...
Setting up dkms (2.1.1.2-5ubuntu1) ...
Setting up fakeroot (1.14.4-1ubuntu1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode.
Setting up nvidia-current (260.19.21-0ubuntu1) ...
update-alternatives: using /usr/lib/nvidia-current/ld.so.conf to provide /etc/ld.so.conf.d/GL.conf (gl_conf) in auto mode.
update-alternatives: warning: skip creation of /usr/lib32/vdpau/libvdpau_nvidia.so.1 because associated file /usr/lib32/nvidia-current/vdpau/libvdpau_nvidia.so.1 (of link group gl_conf) doesn't exist.
update-alternatives: warning: skip creation of /usr/lib32/libvdpau_nvidia.so because associated file /usr/lib32/nvidia-current/vdpau/libvdpau_nvidia.so (of link group gl_conf) doesn't exist.
update-initramfs: deferring u...

[Andreas Hasenack (ahasenack)]

/vmlinuz is a broken link:
lrwxrwxrwx 1 root root 29 2010-12-03 08:12 vmlinuz -> boot/vmlinuz-2.6.37-7-generic

root@ubuntu:~# ll /boot/
total 20569
drwxr-xr-x 4 root root 80 2010-12-08 13:16 ./
drwxr-xr-x 33 root root 320 2010-12-08 13:09 ../
-rw-r--r-- 1 root root 714835 2010-12-01 04:00 abi-2.6.37-7-generic
-rw-r--r-- 1 root root 132160 2010-12-01 04:00 config-2.6.37-7-generic
drwxr-xr-x 2 root root 60 2010-12-08 13:10 grub/
-rw-r--r-- 1 root root 17922686 2010-12-08 13:16 initrd.img-2.6.37-7-generic
-rw-r--r-- 1 root root 160988 2010-10-22 11:43 memtest86+.bin
-rw-r--r-- 1 root root 163168 2010-10-22 11:43 memtest86+_multiboot.bin
-rw-r--r-- 1 root root 1939255 2010-12-01 04:00 System.map-2.6.37-7-generic
-rw-r--r-- 1 root root 1191 2010-12-01 04:02 vmcoreinfo-2.6.37-7-generic

[Pedro Machado Santa (pedro-santa)]

Sorry for the duplicate bug #688116. It was submitted semi-automatically by the bug report application. :/

Cheers.

[Pedro Machado Santa (pedro-santa)]

By the way, if it helps I was running a live USB session, from a disk I made by following the Jono Bacon guide (http://www.jonobacon.org/2010/11/30/testing-natty-and-unity-safely-with-a-usb-stick/), with 4Gb of storage for documents and stuff - the drive has 8Gb total.

[Bryce Harrington (bryce)]

@Andreas, mount the usb disk in a different working linux installation, and look at how much free disk space there is on the usb drive itself. My guess is maybe it ran out of space while trying to create vmlinuz-2.6.37-7-generic. What % of the free space did you allocate to reserve for documents and such? If you set it to 100% then it does not leave room for system files like initrd or the kernel, I guess.

[Bryce Harrington (bryce)]

@Pedro, I believe that casper can only handle a 4G filesystem max (FAT limitation?) so even though it is 8Gb total the live environment can't address the upper 5-8G. So since your storage was set so large, there wasn't space to write the kernel.

Or something like that. Try reducing the size of your storage to 2 or 3G. Or maybe manually partition the 8G drive into smaller chunks.

[Bryce Harrington (bryce)]

I've reproduced this myself on a 4G usb key, setting the reserved space to max (3.1G), as I've outlined in the description.

Comment #7 on bug #562312 does indeed describe what is happening. I think there's a whole class of issues that can occur, which produce a range of different (and confusing) error messages, crashes, and so on.

[Bryce Harrington (bryce)]

Alright, I've sufficiently reproduced the issue and its workarounds now that I'm comfortable closing the -nvidia part of this bug. I'll update the description with steps I used to work around it.

I reckon there's 3 separate bugs at play here:

1. usb-creator allows people to specify too large a proportion of usb drive space to the persistent file, leaving an insufficient amount to do initrd updates (bug #562312).

2. jockey does not detect the limited system space issue (this bug)

3. casper is not creating the /vmlinuz symlink correctly, which leads to kernel updates failing (bug #557023)

These all need fixed, but none are -nvidia or -fglrx issues specifically, so I'm closing out the -nvidia task for this bug.

[Bryce Harrington (bryce)]

Hrm, unfortunately even using all the workarounds I still couldn't get fglrx up. Here's what I tried:

[Workaround]
1. Download an ISO (I tested against the alpha-1 iso image)
2. Generate a USB drive using usb-creator. Do not reserve all the remaining space. Leave some amount for system updates (I tested on a 4G drive, reserving 1G to persistent storage and the remaining ~2G for system updates, but other quantities may work as well). This works around bug #562312.
3. Boot the USB drive, select "Try Ubuntu" (the Live CD environment)
4. Delete the broken /vmlinuz symlink and create /vmlinuz as a new symlink to /cdrom/casper/vmlinuz /vmlinuz. This works around bug #557023.
5. Do an update (via running 'Check' in Update Manager, or just 'sudo apt-get update' on the command line). Otherwise jockey will fail with error about not finding expected versions of Qt or other packages.
6. Now run 'Additional Hardware' (aka jockey), and click 'Activate'. Some stuff is downloaded and installed... time passes... then it says it is successfully installed.
7. Reboot (I had to manually select boot from USB in BIOS for some reason; didn't have to do that before.)

Everything seemed fine up through this point. The boot graphics were different than previously, making me think that fglrx had loaded. However no good... checking dmesg shows that radeon was selected, no fglrx at all. So I think there's more at play here, but my guess is that it has to do with the way the filesystems are set up more than it being an issue with -nvidia or -fglrx particularly. So I'll leave the -nvidia task closed, but if anyone identifies something that definitely needs done on either driver, please feel free to reopen.

[Martin Pitt (pitti)]

Bryce Harrington [2010-12-15 0:24 -0000]:
> 1. usb-creator allows people to specify too large a proportion of usb
> drive space to the persistent file, leaving an insufficient amount to do
> initrd updates (bug #562312).

I don't understand this part -- where else would the live system write
updated files if not to the persistent file? The original squashfs is
read-only.

> 2. jockey does not detect the limited system space issue (this bug)

I think we really ought to just disable installing big drivers such as
nvidia or fglrx on the live system. It was never meant to actually
work there.

Thanks for the analysis,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[libertypo (libertypo)]

For what it matters, the .29 version of nvidia-current installs flawlessly

[Bryce Harrington (bryce)]

Martin, yeah you're right that it's read-only, but it's trying to update the initrd file... which lives outside the persistent file. Thus the bug (I think...)

I've made a bunch of installation attempts, but it always fails. Your suggestion to just disable installation and mark it unsupported sounds like a reasonable solution to me.

[Bryce Harrington (bryce)]

I'm bumping up the milestone on this, due to the expectation that we'll be getting a lot of testers with usb keys following alpha-2. We're already getting a lot of dupes for this bug, and I think after alpha-2 it'll turn into a flood.

[Bryce Harrington (bryce)]

@libertypo, where did you get a .29 driver? Please elaborate in more detail how you got it working.

[Robert Hooker (sarvatt)]

libertypo: I'm guessing you installed from nvidia.com directly? Can you attach your /var/log/Xorg.0.log and dmesg after booting that?

[Bryce Harrington (bryce)]

Ah, Sarvatt pointed out it must have been taken from nvidia.com, so would not have DKMS or update-initramfs (I guess because it has no gl?) So unfortunately I don't think knowledge about .29 installation helps diagnose this problem.

[Launchpad Janitor (janitor)]

This bug was fixed in the package jockey - 0.7-0ubuntu1

---------------
jockey (0.7-0ubuntu1) natty; urgency=low

  * New upstream release:
    - Add support for repository fingerprints, and retrieve them from
      openprinting.org when available.
    - Allow binary drivers if they have a valid GPG fingerprint on a trusted
      https:// site. This is a lot weaker than our usual archive trust chain,
      but as it's very hard to get a chain of trust to printer driver vendors,
      relying on good SSL certificates is the next best step, and still much
      better than what the average user does when searching and downloading a
      driver by himself.
    - Add API for retrieving and installing a GPG key based on a fingerprint.
    - Merge add_repository() into install_package(), which is much more
      practical for verifying whether a package in a new repository ist
      trustworthy.
  * jockey/oslib.py, tests/oslib.py: Update for merged add_repository(), and
    implement GPG retrieval and repository trust checking. This now provides
    secure binary third-party drivers. (LP: #604698)
  * tests/oslib.py, test_ubuntu_package_header_modaliases(): Fix typo in
    regular expression which didn't catch "fglrx" before.
  * data/handlers/fglrx.py, data/handlers/nvidia.py: Disable these two
    handlers in a live system environment (if /rofs exists). We will most
    likely run out of RAM trying to download, build, and install all the
    packages in the RAM disk. (LP: #685017)
 -- Martin Pitt <email address hidden> Thu, 06 Jan 2011 19:13:24 +0100