local provider moar awesome with lxc-clone

[Kapil Thangavelu]

Currently it juju is extracting a cloud image per local unit, ~2m per container. Instead it could create a series template container per template, with an lxc clone hook for new user data. New provisioning requests for that series then pass userdata params via lxc-clone. Total time for provisioning a container then drops down signfiicantly, worst case its a simple copy, best case its a btrfs or lvm snapshot and done in a few seconds and significantly more efficient on disk usage as well.

[Kapil Thangavelu (hazmat)]

its a little unclear how much benefit this will provide from a time perspective (disk efficiency is still a clear win) as a good portion of the time (on an ssd machine) is the relegated to cloudinit setup of the instance.

[Sidnei da Silva (sidnei)]

Seems like overlayfs has issues with not implementing inotify properly (LP:882147), which upstart depends on (LP:1124384).

At this point I'm considering to bail off overlayfs and require either btrfs or lvm, with a fallback to plain copy instead.

[Kapil Thangavelu (hazmat)]

aufs might be a better alternative than overlayfs that should resolve those
issues.

On Fri, Aug 16, 2013 at 9:00 AM, Sidnei da Silva <<email address hidden>
> wrote:

> Seems like overlayfs has issues with not implementing inotify properly
> (LP:882147), which upstart depends on (LP:1124384).
>
> At this point I'm considering to bail off overlayfs and require either
> btrfs or lvm, with a fallback to plain copy instead.
>
> ** Changed in: juju-core
> Importance: Undecided => Medium
>
> ** Changed in: juju-core
> Status: New => Triaged
>
> ** Changed in: juju-core
> Assignee: (unassigned) => Sidnei da Silva (sidnei)
>
> ** Branch linked: lp:~sidnei/golxc/clone-with-backing-store
>
> ** Branch linked: lp:~sidnei/juju-core/lxc-clone-with-overlayfs
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1203291
>
> Title:
> local provider moar awesome with lxc-clone
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1203291/+subscriptions
>

[Kapil Thangavelu (hazmat)]

fwiw, i think this is an important issue for local provider, going to try
and resurrect at the sprint. i currently do a separately mounted ssd as
btrfs (msata), and get 0.4s to create and start a container (via lxc-clone
-s and lxc-start) compared to 3-5m on juju. This sort of
responsive/interactive behavior is exactly the sort of reason people like
tools like docker.

On Fri, Oct 11, 2013 at 7:53 PM, Curtis Hovey <email address hidden> wrote:

> ** Changed in: juju-core
> Importance: Medium => Low
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1203291
>
> Title:
> local provider moar awesome with lxc-clone
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1203291/+subscriptions
>

[Sidnei da Silva (sidnei)]

With https://github.com/lxc/lxc/pull/67 merged, I think we can look at exposing this to juju. In the case of btrfs, the cloning would be automatic if /var/lib/lxc is btrfs AIUI.

For thin-provisioning we'd have to create the initial container into a thin-provisioned pool. We could assume a thin-provisioned lvm pool named 'juju' and check for it's existence, with a way to make it possible to customize that name. btrfs would still have precedence.

[Sidnei da Silva (sidnei)]

Some updates:

As of the merge of https://github.com/lxc/lxc/pull/74 -Bbest will DTRT iff there's an LVM VG named 'lxc' and an LVM thin pool named 'lxc', that is, it will create LVs from the 'lxc' thinpool and snapshots will also be allocated from the thinpool automatically.

If /var/lib/lxc is btrfs-provisioned, then the containers will default to btrfs instead.

The attached golxc branch assumes a release of lxc with the PR above merged in. It will not work on previous versions of lxc. We can either make it backwards-compatible or provide the lxc package in the cloud archive, as discussed previously.