vmbuilder completely fails on Quantal due to kernel pae detection failure

[Robie Basak]

Impact: this appears to stop vmbuilder from creating a Quantal VM completely.

Evidently the kernel package now looks for /proc/cpuinfo in order to check for PAE support (since PAE support is going away in Quantal). I think this is failing because /proc doesn't exist in the environment that vmbuilder is using.

Tested on a fresh Quantal cloud image, dist-upgraded and python-vm-builder installed from the current Quantal archive. The only catch here is that I did it on a virtual image, but I can't see how that would affect this.

Possible solution: mount /proc inside the chroot?

Log:

Selecting previously unselected package linux-image-virtual.
Unpacking linux-image-virtual (from .../linux-image-virtual_3.5.0.10.10_amd64.deb) ...
, stderr: grep: /proc/cpuinfo: No such file or directory
This kernel does not support a non-PAE CPU.
dpkg: error processing /var/cache/apt/archives/linux-image-3.5.0-10-generic_3.5.0-10.10_amd64.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.5.0-10-generic /boot/vmlinuz-3.5.0-10-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.5.0-10-generic /boot/vmlinuz-3.5.0-10-generic
Errors were encountered while processing:
 /var/cache/apt/archives/linux-image-3.5.0-10-generic_3.5.0-10.10_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: python-vm-builder 0.12.4+bzr481-0ubuntu1
ProcVersionSignature: Ubuntu 3.5.0-9.9-generic 3.5.0
Uname: Linux 3.5.0-9-generic x86_64
Architecture: amd64
Date: Thu Aug 16 14:43:35 2012
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 LC_COLLATE=C
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: vm-builder
UpgradeStatus: No upgrade log present (probably fresh install)

[Serge Hallyn (serge-hallyn)]

thanks, confirmed i can reproduce.

[Juha Erkkilä (juha-erkkila)]

I can also reproduce the problem. The problem is that /proc mounts and unmounts before the kernel is installed, and the kernel package assumes /proc is mounted. Here's some log actions:

2012-10-19 15:39:55,701 DEBUG : ['chroot', '/virtualtmp/tmpPq2rS0', 'mount', '-t', 'proc', 'proc', '/proc']
2012-10-19 15:40:09,171 DEBUG : ['umount', '/virtualtmp/tmpPq2rS0/proc']
2012-10-19 15:40:19,965 DEBUG : Unpacking linux-image-3.5.0-17-generic (from .../linux-image-3.5.0-17-generic_3.5.0-17.28_i386.deb) ...

Attached is a patch that worked for me. One can also work around the issue by explicitly installing linux-image-generic at an earlier stage by providing "--addpkg linux-image-generic" option to vmbuilder.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "an ugly hack to work around the problem" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[phil (fongpwf)]

This only happens if you choose the virtual flavor. Or at least if I do --flavour amd64, the VM creation completes.

[phil (fongpwf)]

Nevermind, it fails as there is not amd64 flavor.

[Will Bryant (willbryant)]

I hit this while building a raring VM to test out #1174148.

I was able to workaround by using --addpkg linux-image-generic. The patch above looks like a sensible solution to me though.

[Arto Bendiken (artob)]

Ran into this bug on 12.10, but was able to work around it using "--addpkg linux-image-generic" as described in previous comments.

[Jeff Dickens (loyhz2ayj-jeff)]

Still exists in Trusty Beta 2.

[xlefay (xander-u)]

Same here in 14.04 (final)

[Will Bryant (willbryant)]

Me too. Workaround still works, at least.

[Jason Stephenson (jstephenson)]

I just ran into thins on Trusty.

Using --addpkg linux-image-generic caused the vm to build, but it would never get to a usable state. It sould sit at Starting up... in virt-manager console.

This is on a vmhost upgraded from precise to trusty.

[Jason Stephenson (jstephenson)]

Turns out that my hang up was a bad firstboot script.

I still needed to either apply the patch above to /usr/lib/python2.7/dist-packages/VMBuilder/plugins/ubuntu/dapper.py or use the linux-image-generic trick.

[Brian Candler (b-candler)]

I also confirm the patch in #3 works.

However I'd suggest that rather than patching dapper.py, it would be better to do modify the quantal.py subclass (if that's the first release which requires /proc to be mounted). Otherwise you're going to change the build behaviour for all previous releases too.

[Richard Hansen (rhansen)]

> However I'd suggest that rather than patching dapper.py, it would be
> better to do modify the quantal.py subclass (if that's the first
> release which requires /proc to be mounted). Otherwise you're going
> to change the build behaviour for all previous releases too.

Thanks for the patch; it works like a charm.

Question: Wouldn't it be better to do this:

    self.run_in_target('umount', '/proc')

instead of this:

    run_cmd('umount', '%s/proc' % self.context.chroot_dir)

Then you wouldn't have to import VMBuilder.util.run_cmd. (I haven't
tried it; just an observation.)

[Brian Candler (b-candler)]

Maybe you're right. I just followed the original patch.

[Egon Kidmose (kidmose)]

Confirming issue and '--addpkg linux-image-generic' fix on trusty.

kidmose@radagast:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

kidmose@radagast:~$ uname -a
Linux radagast 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

kidmose@radagast:~$ apt-cache policy ubuntu-vm-builder
ubuntu-vm-builder:
  Installed: 0.12.4+bzr489-0ubuntu2
  Candidate: 0.12.4+bzr489-0ubuntu2
  Version table:
 *** 0.12.4+bzr489-0ubuntu2 0
        500 http://dk.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
        100 /var/lib/dpkg/status

[jlord87 (moro-87)]

I'm on an updated Trusty server and I'm still affected by this bug.

Applying the patch Brian Candler (b-candler) posted with Richard Hansen (a7x) modification result in a correct deploy of the machine.

I don't understand why it hasn't yet been released through system's updates...

[Robie Basak (racb)]

> I don't understand why it hasn't yet been released through system's updates...

This package is in universe. I'm not sure that anyone's looking after it any more. We all moved to using cloud images instead.

See https://wiki.ubuntu.com/StableReleaseUpdates and https://wiki.ubuntu.com/SponsorshipProcess for instructions on preparing an update yourself and having it sponsored, if you'd like to get the package updated in Trusty.

[Will Bryant (willbryant)]

Huh, interesting - docs still say that the cloud images are *made* using this tool, is that no longer true? Is there some replacement?

[Serge Hallyn (serge-hallyn)]

If https://help.ubuntu.com/community/UEC/Images is not out of date, the images are all built on 10.04 using a (presumably modified) older branch of vmbuilder.

[Serge Hallyn (serge-hallyn)]

Is this bug still applicable? Does generating a vivid image on a vivid or trusty host give the same errors?

[Mike Neumann (mike-neumann)]

Bug still exists. I encountered it just now attempting to generate a trusty image on a trusty host. Adding "addpkg = linux-image-generic" to my /etc/vmbuilder.cfg file allowed the process to complete.

[冯宇 (abcfy2)]

Same here. I just add --addpkg linux-generic-lts-vivid to fix this issue.

[Wladimir J. van der Laan (laanwj)]

Had this problem in Wily, creating a Trusty image.
--addpkg linux-image-generic solved the issue.

[Jason Stephenson (jstephenson)]

I'm just adding a comment to let everyone know that the problem still exists on 16.04 Xenial Xerus.

The --add-pkg linux-image-generic workaround still works.

I assume that the patch still works, but I have not tried the patch since I upgraded from Trusty.

[Pontiy_Pilat (p-p)]

>The --add-pkg linux-image-generic workaround still works.

It's not work on Xenial (

[Keith Z-G (keithzg)]

Last time I built a VM I tried using uvtool instead of vmbuilder or using an installer ISO directly, and it seemed to go fine. From a cursory look, it seems like uvt is the current way to build VMs, and the vmbuilder script has fallen entirely by the wayside.

https://help.ubuntu.com/lts/serverguide/cloud-images-and-uvtool.html

[Christian Ehrhardt  (paelzer)]

I'd agree with Keith Z-G that uvtool is one that is more cared about, and then please see comment #19 quoting "This package is in universe. I'm not sure that anyone's looking after it any more. We all moved to using cloud images instead."

I'd set this to a lower importance to reflect that, please feel free to raise again if you totally object to that assessment.