clean up root.tar.gz and fast path installer files (mipf rewrite in Python)

[Scott Moser]

on 13.04, after running maas-import-pxe, we end up with 2 copies of the "root.tar.gz" files (that are available for the fast path installer).
example:
 /var/lib/maas/ephemeral/raring/ephemeral/amd64/20130518/dist-root.tar.gz
 /var/lib/maas/tftp/amd64/generic/raring/commissioning/root.tar.gz

those are ~ 290M, so additional copies of all the "latest" is a bit heavy (4 supported releases * X arches).

This whole path isn't settled, as during fast path installer development, we'll need to make a default decision between:
 * using 'cp -a /' from the iscsi ephemeral environment
 * using root.tar.gz (as we're doing now)

And also on how to best distribute those. Right now the user is downloading the ephemeral filesystem image and then using 'uec2roottar' to convert it to .root.tar.gz. The general issue is that the ephemeral environment boots iscsi, and needs to be a "root device" (filesystem in a block device / file served by tgt) while the root.tar.gz file is ideal from a transfer-to-installer perspective. However, these files are identical from a filesystem contents perspective.

Maybe we should open separate bugs, but heres the list of less-than-ideal things right now:
 * 2 copies of .root.tar.gz files
 * maas-import-pxe-files invokes uec2roottar. We'll need to decide what the best way to distribute this is, and this might involve some server side work too.
 * I'm pretty sure maas-import-pxe-files does more work than it needs to when there is nothing to do. We should have a "--force" or something option to it, but largely if there is nothign to do it should return very quickly and do *nothing*. My plan here is to implement this as a simplestreams "mirror" and have a cron job sync and then clean up old images.
 * no cleanup is done of old files (this is especially an issue if we get to a 3 week cadence or the user is using daily streams).

[Andres Rodriguez (andreserl)]

We not only end up with to copies of root.tar.gz but we also end up with two copies of 'initrd' and 'linux' files.

ubuntu@cluster:~$ ls /var/lib/maas/ephemeral/precise/ephemeral/amd64/20121008/
disk.img info initrd.gz linux root.tar.gz tgt.conf

ubuntu@cluster:~$ ls /var/lib/maas/tftp/amd64/generic/precise/xinstall
initrd.gz linux root.tar.gz

[Andres Rodriguez (andreserl)]

two*

[Julian Edwards (julian-edwards)]

Urgh this is nasty.

We have plans to completely re-write the importer script in Python in the near future. That would be an ideal time to address any of these problems, so if you can firm up a plan and document it here, we'll implement it.

Please don't write any more shell, or changes to shell.

[Raphaël Badin (rvb)]

mipf has been rewritten in Python.