vmbuider should Include kernel modules in AMIs for ec2

Bug #429169 reported by Eric Hammond on 2009-09-14
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
VMBuilder
Undecided
Scott Moser
vm-builder (Ubuntu)
Medium
Scott Moser
Declined for Karmic by Mathias Gug

Bug Description

It is fairly standard practice to include kernel modules in images for Amazon EC2, but the most recent Karmic AMIs do not include them:

  ami-a40fefcd
  canonical-alphas-us/karmic-i386-alpha5.1.manifest.xml

  ami-a20fefcb
  canonical-alphas-us/karmic-x86_64-alpha5.1.manifest.xml099720109477

Some other options have been discussed like building kernel modules into the kernel or copying them from initrd into /lib/modules at boot time, but the easiest approach to get things working in the short term is probably just to follow the existing de facto standard and include the kernel modules for the AMI's default kernel. If somebody wants to run it with a different kernel, they are already used to having to install those kernel modules themselves.

Ideally, the addition of the kernel modules would be done by default in the vmbuilder EC2 plugin so that it works when normal users create images with vmbuilder. If it is done some other way, the method should be documented publicly so that users can create working images.

Eric Hammond (esh) on 2009-09-14
affects: ubuntu → vm-builder (Ubuntu)

> Some other options have been discussed like building kernel modules into
> the kernel or copying them from initrd into /lib/modules at boot time,
> but the easiest approach to get things working in the short term is
> probably just to follow the existing de facto standard and include the
> kernel modules for the AMI's default kernel. If somebody wants to run

I know that Eric doesn't really want to hear this, but I'll at least point
it out. Having the modules already installed for the xen kernel is less
than ideal for the uec images.

That said, I absolutely agree that having a full set of modules available
to the user in ec2 (at least comparable to that of the '-virtual' kernel)
is needed.

Matt Zimmerman (mdz) wrote :

On Mon, Sep 14, 2009 at 03:03:49PM -0000, Scott Moser wrote:
> > Some other options have been discussed like building kernel modules into
> > the kernel or copying them from initrd into /lib/modules at boot time,
> > but the easiest approach to get things working in the short term is
> > probably just to follow the existing de facto standard and include the
> > kernel modules for the AMI's default kernel. If somebody wants to run
>
> I know that Eric doesn't really want to hear this, but I'll at least point
> it out. Having the modules already installed for the xen kernel is less
> than ideal for the uec images.

Less than ideal, perhaps, but certainly worth the tradeoff in my opinion.
The only cost should be in storage space, and there presumably aren't very
many modules for the EC2 kernel.

The images should do what is necessary to support both UEC and EC2 very
well, until such time as we find that it would be more effective to maintain
two separate images.

--
 - mdz

> there presumably aren't very many modules for the EC2 kernel.

I guess this raises the questions of what modules we are talking about, and this is an area where I wouldn't know what to cut out. I personally depend on some which may not be considered part of the core modules including: fuse, xfs, dm_mod, sha256, dm_crypt

You'd think that you could cut out things like sound, but I know that some folks are using snd-dummy.ko

I don't know what other kernel modules users might be depending on today in EC2.

> The images should do what is necessary to support both UEC and EC2 very
> well, until such time as we find that it would be more effective to maintain
> two separate images.

I read that to indicate that we should have installed in the image
linux-image-virtual and linux-image-ec2 (or whatever that ends up being
called). According to [1] the installed size of the -virtual kernel (which
we'd probably install for UEC) to be < 8MB, and [2] shows the -ec2 flavor
to be ~ 18M.

When bug 428692 is fixed, I'd expect a smaller amount of modules in the
latter, resulting in a smaller installed size. So, we're looking at less
than 26M installed into a 10GB filesystem. So, I agree, that is probably
a reasonable use of space.

--
[1] http://packages.ubuntu.com/karmic/linux-image-2.6.31-10-virtual
[2] https://edge.launchpad.net/~timg-tpi/+archive/ppa/+build/1235076

On Mon, Sep 14, 2009 at 10:11:51PM -0000, Eric Hammond wrote:
> > there presumably aren't very many modules for the EC2 kernel.
>
> I guess this raises the questions of what modules we are talking about,
> and this is an area where I wouldn't know what to cut out. I personally
> depend on some which may not be considered part of the core modules
> including: fuse, xfs, dm_mod, sha256, dm_crypt

The -virtual kernel, I believe, includes all of the usual stuff that the
-server kernel ships, but subtracts out the hardware drivers (since they are
not used in virtual guests).

I think we should start with the -virtual config, and can add things if
needed from there.

--
 - mdz

On Tue, Sep 15, 2009 at 12:05:15AM -0000, Scott Moser wrote:
> > The images should do what is necessary to support both UEC and EC2 very
> > well, until such time as we find that it would be more effective to maintain
> > two separate images.
>
> I read that to indicate that we should have installed in the image
> linux-image-virtual and linux-image-ec2 (or whatever that ends up being
> called). According to [1] the installed size of the -virtual kernel (which
> we'd probably install for UEC) to be < 8MB, and [2] shows the -ec2 flavor
> to be ~ 18M.
>
> When bug 428692 is fixed, I'd expect a smaller amount of modules in the
> latter, resulting in a smaller installed size. So, we're looking at less
> than 26M installed into a 10GB filesystem. So, I agree, that is probably
> a reasonable use of space.

Agreed.

--
 - mdz

Eric Hammond (esh) on 2009-09-15
Changed in vm-builder (Ubuntu):
status: New → Confirmed
assignee: nobody → Scott Moser (smoser)
Scott Moser (smoser) on 2009-09-15
Changed in vm-builder (Ubuntu):
importance: Undecided → Medium
status: Confirmed → Triaged
Scott Moser (smoser) on 2009-09-22
Changed in vm-builder (Ubuntu):
milestone: none → ubuntu-9.10-beta

I've filed bug 434755 separately, which should be fixed in order to make the ec2 kernel package appropriate for installation in UEC and EC2 instances

Scott Moser (smoser) on 2009-09-23
Changed in vmbuilder:
status: New → In Progress
Changed in vm-builder (Ubuntu):
status: Triaged → In Progress
Changed in vmbuilder:
assignee: nobody → Scott Moser (smoser)
Scott Moser (smoser) on 2009-09-23
Changed in vmbuilder:
status: In Progress → Fix Committed
Scott Moser (smoser) on 2009-09-25
Changed in vm-builder (Ubuntu):
status: In Progress → Fix Committed
Scott Moser (smoser) wrote :

The changes needed are in vmbuilder trunk, and present in the ec2 and uec images. vmbuilder still needs to be released in karmic with these changes.

Changed in vm-builder (Ubuntu):
milestone: ubuntu-9.10-beta → none
summary: - ec2: Include kernel modules in AMIs
+ vmbuider should Include kernel modules in AMIs for ec2
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package vm-builder - 0.11.2-0ubuntu1

---------------
vm-builder (0.11.2-0ubuntu1) karmic; urgency=low

  * New upstream (bugfix-only) release. (LP: #410886, #439868, #426438, #429169, #423497)

 -- Soren Hansen <email address hidden> Thu, 22 Oct 2009 15:07:36 +0200

Changed in vm-builder (Ubuntu):
status: Fix Committed → Fix Released
Soren Hansen (soren) on 2009-10-22
Changed in vmbuilder:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers