options to generate tarballs instead of images

[Paul Larson]

I was wondering if it would be possible to give linaro-media-create options to create tarballs for the boot and rootfs instead of generate an image. I would find this very useful so that I could get this and manually push it to test partitions for automated testing. Also, it seems that some people want to do similar things for setting up things like nfs boot.

[Guilherme Salgado (salgado)]

I think this can be done easily if we do the following:

1. Change both populate_boot() and populate_rootfs() to take an extra argument (a callable) which is called at their very end, but before they call umount on the boot/root partitions. By default this will be None so it won't be called.
2. Add a new argument to l-m-c for it to generate tarballs
3. When the new argument is passed to l-m-c, create an arbitrary file on /tmp and use it as if we were asked to create an image file, but set an atexit handler to remove it upon exit as the file is not what the user wants.
4. Again, when the new argument is given, make l-m-c pass a custom function to populate_boot()/populate_rootfs() that will generate tarballs out of the bootfs/rootfs

This is a bit of a hack, but I think making both populate_boot() and populate_rootfs() extensible (via the extra callable passed to them) might be useful for other things and this approach is much simpler than refactoring things so that we can skip a few steps and save a few seconds when generating tarballs.

[Loïc Minier (lool)]

I checked what populate_rootfs() does, and it:
* sets up stuff to do its work (create directory, mount partition)
* moves the main contents of the rootfs to the mount point
* setups fstab
* setups flash-kernel config
* adds SWAP file
* unmount

I'm guessing that the only thing which would be relevant for Paul and NFS root users would be the SWAP file; the entry with UUID in fstab would be incorrect for Paul and NFS users and NFS users wouldn't want flash-kernel to run, and probably Paul neither.

populate_boot copies and sets up the bootloader stuff, plus kernel and initrds; again, the contents of the boot script is incorrect for NFS users and for Paul. The bootloader would be useful for NFS users though.

I think Guilherme's implementation sounds good, and we probably want to skip some of these steps for a detached tarball.

[Peter Maydell (pmaydell)]

> the contents of the boot script is incorrect for NFS users and for Paul

However typically you want to know what it is because 90% of the options are the same and it's just the root related options you want to change. So it's handy to know what l-m-c would have set it to.

(My use case is qemu-vexpress where at the moment you have to fish the kernel and initrd out of the image, and again the command line is useful to know.)

[Guilherme Salgado (salgado)]

On Thu, 2011-03-03 at 00:21 +0000, Loïc Minier wrote:
> I checked what populate_rootfs() does, and it:
> * sets up stuff to do its work (create directory, mount partition)
> * moves the main contents of the rootfs to the mount point
> * setups fstab
> * setups flash-kernel config
> * adds SWAP file
> * unmount
>
> I'm guessing that the only thing which would be relevant for Paul and
> NFS root users would be the SWAP file; the entry with UUID in fstab
> would be incorrect for Paul and NFS users and NFS users wouldn't want
> flash-kernel to run, and probably Paul neither.
>
> populate_boot copies and sets up the bootloader stuff, plus kernel and
> initrds; again, the contents of the boot script is incorrect for NFS
> users and for Paul. The bootloader would be useful for NFS users
> though.

It was my understanding that Paul was currently generating tarballs
straight out of the generated images, which would mean we wouldn't
necessarily *need* to skip any of the steps currently performed. That's
why I suggested this solution, which is not too intrusive.

If we actually need to skip some steps then we'll have to add code all
around to skip things when generating a tarball and that doesn't sound
like a good idea to me -- I think we should instead refactor the code to
make it easier to select what steps need to be performed.

[Loïc Minier (lool)]

Actually now that I think of it again, setting an UUID in etc/fstab could actually be used by a careful script which would create the target fs with the same UUID, with mkfs -U -- not available on all fses and requires unpacking fstab from tarball, then creating the fs, then unpacking the full tarball. So that weakens my argument a bit.

Anyway, I think Guilherme's approach is the best for now.

[Alan Bennett (akbennett)]

Due to the age of this issue, we are acknowledging that this issue will likely not be fixed, is no longer applicable, or was already fixed by an indirect change. If this issue is still important, please add details and reopen the issue.