karmic uec builds fail to publish due to 2 installed -ec2 kernels

Bug #524020 reported by Scott Moser on 2010-02-18
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
vm-builder (Ubuntu)
High
Unassigned
Karmic
High
Scott Moser
Lucid
High
Unassigned

Bug Description

build 20100204 is the last karmic uec build to successfully publish to ec2.

The builds fail to publish with:
Thu Feb 18 05:13:57 GMT 2010: publishing -> ubuntu-karmic-daily-i386-server-20100218
must provide kernel for each region
publish-image karmic-server-uec-i386.img for karmic/server/i386 failed
failed daily build publisher: karmic /srv/ec2-images/karmic/20100218

The call chain looks like:
  uec-build-launcher -> publish-build -> publish-build-krd .
publish-build-krd reads the '<prefix>-kernel-info.txt' file in unpacked/ and is confused its content.

The problem is that the build ended up installing 2 versions of the -ec2 kernel (and also 2 versions of -virtual kernel). publish-build-krd expected only 1 of each type.

In investigating *why* this was the case, it can be reproduced by 'apt-get install uec^'. The uec seed data has the released versions of the kernels explicitly listed [1]. See linux-image-2.6.31-14-virtual and linux-image-2.6.31-302-ec2. I believe that linux-image-2.6.31-19-virtual and linux-image-2.6.31-304-ec2 are upgrades to the listed packages.

There has been discussion in ubuntu-devel [2] on how Task: values would be updated with no clear solution. I believe the root issue is that the kernel linux-image packages are not the same package name (because of the ABI bump) and the program consuming or generating output doesn't realize that the -updates version replaces the released version.

This is currently only a problem on karmic, but it will become a problem on lucid as soon as there are kernels with new ABI in -updates.

--
[1] http://people.canonical.com/~ubuntu-archive/germinate-output/ubuntu.karmic/uec
[2] http://irclogs.ubuntu.com/2010/02/18/%23ubuntu-devel.html

Scott Moser (smoser) wrote :

per James in irc:
| it's the machine that the archive actually lives on
| what is happening is that LP is calling apt-ftparchive in such a way that it adds Task to all the packages as appropriate
| it does that using override files
| these are not generated by LP as it works
| the files list "package-name Task foo" stuff
| so we need those updated based on an updated germinate run
| however, I have no idea how to do that
| the files are owned by lp_publish, but that user doesn't have a cron job to do it or anything
| so, I don't know if it is done manually on occasion, or LP handles it out of the path of publishing

summary: - karmic uec builds fail to publish because of 2 intalled kernels
+ karmic uec builds fail to publish due to 2 installed -ec2 kernels
Scott Moser (smoser) wrote :

This can be demonstrated / reproduced from inside an instance with:

$ sudo apt-get update
$ delpkgs=$(dpkg -l "linux-*" | awk '$1 == "ii" {print $2}')
$ sudo apt-get --purge remove ${delpkgs} -y
$ sudo apt-get install uec^

You'll see that the 'install' says:
| The following extra packages will be installed:
| linux-ec2 linux-firmware linux-image-2.6.31-14-virtual
| linux-image-2.6.31-19-virtual linux-image-2.6.31-302-ec2
| linux-image-2.6.31-304-ec2 linux-image-ec2 linux-image-virtual linux-virtual

Colin Watson (cjwatson) wrote :

'apt-get install uec^' looks up all packages with 'uec' in their Task field, and installs them. The Task fields are set by magic automatically generated overrides in the archive, based on germinate output. Sadly, they are not generated properly for karmic-updates.

However, this isn't really the core problem in this case. Putting kernels in the uec seed at all is the core problem, and I must apologise for not noticing this at the time. The reason this is bad is that the kernel ABI changes frequently, and once a release is final we never change its Packages files again! Thus "Task: uec" on linux-image-2.6.31-302-ec2 in dists/karmic/main/binary-*/Packages is frozen for the lifetime of karmic; while technically we *could* regenerate those files, we have never done so for stable Ubuntu releases, preferring to update only karmic-updates and the like, and I expect that things would go wrong if we did - for example, consider what happens when somebody tries to build an image using only karmic and not karmic-updates, or consider that some mirrors may well assume that dists/karmic/ will never change.

My recommendations are as follows:

  * For Karmic, perhaps it is possible to work around this in vm-builder. For example, 'apt-get install uec^ linux-image-2.6.32-302-ec2- linux-image-ec2- linux-ec2-' installs the uec task without the ec2 kernel bits; some elaboration of this might be constructed to use the updated ABI.
  * For Lucid, remove all kernels from the uec seed, and instead install them manually (so 'apt-get install uec^ linux-ec2 linux-virtual' or whatever). It is not safe to put kernels in seeds for which Task fields are generated.
  * We should institute some kind of archive-level test to ensure that kernels never have Task fields.

Scott Moser (smoser) on 2010-02-22
affects: Ubuntu Lucid → vm-builder (Ubuntu Lucid)
Soren Hansen (soren) wrote :

Is there anything to be done in vm-builder to fix this?

Scott Moser (smoser) wrote :

changes were made to vm-builder 0.11 specific at http://bazaar.launchpad.net/~ubuntu-virt/vmbuilder/0.11/revision/367 .

Changed in vm-builder (Ubuntu Karmic):
status: Triaged → Fix Released
Scott Moser (smoser) wrote :

I'll look into getting this into lucid's vm-builder

Scott Moser (smoser) wrote :

Soren,
  Attached is the patch applied to vmbuilder-0.11 moved to vmbuilder trunk.

Scott Moser (smoser) wrote :

Soren, I'm assigning this to you. The patch attached is what was applied to vm-builder 0.11. If you think its reasonable to pull into trunk then do so, otherwise you can mark wontfix.

Changed in vm-builder (Ubuntu Lucid):
assignee: Scott Moser (smoser) → Soren Hansen (soren)
tags: added: patch
Soren Hansen (soren) on 2011-03-15
Changed in vm-builder (Ubuntu):
assignee: Soren Hansen (soren) → nobody
Changed in vm-builder (Ubuntu Lucid):
assignee: Soren Hansen (soren) → nobody
Scott Moser (smoser) on 2011-03-15
Changed in vm-builder (Ubuntu Lucid):
status: Triaged → Fix Released
Serge Hallyn (serge-hallyn) wrote :

@Scott,

this was marked fix released for karmic and lucid. The patch isn't in the upstream tree - does it need to be? If not, should the bug be marked fix released instead of triaged? If so, I'll apply the patch.

Scott Moser (smoser) wrote :

I'm marking this "Wont fix". You could fix it, but really the issue was with karmic's archive. and no need to work around that.

Changed in vm-builder (Ubuntu):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers