add mdadm and lvm2 to cloud image

[Scott Moser]

In order to make use of raid and lvm functionality that is being added to curtin, we need to have mdadm and lvm2 packages available. curtin is able to install the packages, but that is less desireable than having them already available inside the image.

This bug is a feature freeze exception to add mdadm and lvm2 to the wily cloud-image seed [1].

== general risks ==
Adding these packages will add ~ 5M total to the cloud image installed system.

== mdadm risks ==
mdadm recommends 'default-mta', and cloud-images have always been built with 'recommends by default'. So addition of this package to the cloud image means also adding an unconfigured mta (postifx). There is some discussion on this topic from ubuntu-devel at [2].

As a point of reference here, I did a server ISO install with raid, and found that 'default-mta' does *not* get installed. The installer must select to install mdadm but not pull in its Recommends.

Additionally, its possible presence of this package could cause issues. It installs udev rules that could misfire. Any such issues encountered really should be fixed anyway, rather than just hoped to not be hit.

== lvm2 risks ==
Similarly to mdadm, lvm adds udev rules and could raise cause problems.
As with mdadm, any such issues really ought to be fixed any way.

== general information ==
The reason this is proposed for inclusion into the cloud-image is 2 fold
a.) the maas ephemeral images are created from the cloud-image. so packages in the cloud-image are then available in the install environment for maas. We dont want to add differences between these installation sets.
b.) it is not unreasonable to add these packages to the cloud image. It is not at all farfetched that someone would use lvm or raid in a "cloud" environment, so they're not entirely wasted space there.

--
[1] http://bazaar.launchpad.net/~ubuntu-core-dev/ubuntu-seeds/ubuntu.wily/view/head:/cloud-image
[2] http://irclogs.ubuntu.com/2015/08/06/%23ubuntu-devel.html#t22:53

[Martin Pitt (pitti)]

TBH I'm less concerned about this from the POV of a FFE rather than in general. You won't ever need this in almost all cloud environments, and it adds unnecessarily complexity, boot delays, and potential boot blockers. I can't think of a reason why you would set up LVM and mdadm in the cloud guest, rather than on the host?

This is different for using cloud images on real hardware, of course. But for that we are also missing a lot of kernel drivers for that (as the cloud images only have linux-image-virtual, not -generic).

Perhaps it would be time to split this to build two images? One for real clouds, which could then also drop other bits which are rarely needed in the cloud and would better be installed on demand in user_data: accountsservice, ntfs-3g, sosreport, aptitude, vim, etc. And one which are much more like the server images that we have today, with a nice and complete shell CLI environment, a full kernel, mdadm and friends?

[Scott Moser (smoser)]

Martin,
 guests are just systems. People can and do use lvm and raid in cloud guests quite commonly. Its even defined good practice in many cases (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html).

with regard to "cloud images on real hardware". Thats explicitly what maas does. We use the "maas ephemeral image". Which is built starting from the cloud image. we then install the -generic kernel inside, and a very few other packages (cloud-initramfs-dyn-netconf maas-enlist open-iscsi). We could add mdadm and lvm2 to that list. However, we're interested in reducing that delta rather than increasing it.

Wrt boot delays, there should not be significant boot delay (if any) in cases where root= does not require it. Am I wrong in that assumption?

We're really looking to *reduce* deltas and "things" that we produce, rather than create more. I would also probably at some point add 'cloud-initramfs-dyn-netconf' and 'open-iscsi' to the cloud-image and then drop maas-enlist above. Then the only difference between a cloud image and a "real system" from the point of view of a default install would be the kernel selection. We've even investigated if we could reasonably get rid of that difference.

There are costs to having one image that is expected to work in all cases, but there are also lots of benefits.

[Martin Pitt (pitti)]

> Wrt boot delays, there should not be significant boot delay (if any) in cases where root= does not require it. Am I wrong in that assumption?

I'm not concerned about boot delays, but it does add unnecessary complexity both to the boot and to upgrades. From my POV the cloud images already have way too much stuff on it, and this goes further into that direction. Well, I suppose your goals/use cases are different than mine :-) But trying to fit every nice use case on the cloud images will mean that every juju deploy, dist-upgrade, etc. has to carry that cost (extra download, more potential hiccups on upgrades, suid/sgid binaries and potentially open ports which need to be secured (postfix!), etc.

Anyway, this was not a rejection with my release team hat on, just a concern.

[Martin Pitt (pitti)]

> every nice use case

Sorry, that was supposed to mean "niche" use case

[Scott Moser (smoser)]

I generally agree with the "too much stuff".