Add an official hook API

Bug #1672436 reported by Barry Warsaw on 2017-03-13
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ubuntu Image
High
Łukasz Zemczak
ubuntu-image (Ubuntu)
Undecided
Unassigned

Bug Description

Currently, to hook into the u-i build process, you have to use the --thru, --until, and --resume options to temporarily stop the state machine, do some out-of-band tweaking, and then resume the state machine to produce the image. While darn handy, and a very useful debugging/testing feature, these options are very problematic for production.

State machine method names and indexes are *not* considered part of the public API so I make no guarantees they won't change, even though I try not to change them arbitrarily.

What we need instead is a collection of official build-process hooks and a mechanism for users to invoke them. Such hooks *would* be part of the public API, with the expected backward compatibility and stability guarantees.

We need to do two things: first, we need to identify the possible hook points where users of u-i might want to modify the image content, etc. They probably will not be as numerous as the number of state machine steps. Second, we need to identify a machinery for actually specifying, locating, and running the hooks.

Steve Langasek (vorlon) wrote :

raised this question privately, but also documenting on the bug: I am not convinced that we want a generic hook mechanism, vs. a limited mechanism to declare files to be copied into the root filesystem.

Barry Warsaw (barry) wrote :

Just as an added use case (also raised privately), I've also used the state machine options to *delete* files before the image was baked. Probably not a great use case, but there it is.

Still, I think a common feature in both cases is the need to manipulate the contents after `snap prepare-image` has run. There could be two hook points here, one that runs on the unpacked contents before they are shuffled around for the images, and a second one that runs after the shuffling but before the disk image files are baked.

On Mon, Mar 13, 2017 at 08:16:15PM -0000, Barry Warsaw wrote:
> Just as an added use case (also raised privately), I've also used the
> state machine options to *delete* files before the image was baked.
> Probably not a great use case, but there it is.

There are not many files on the filesystem in the ubuntu-core case, period;
the only one we've needed to delete, so far, is one that snap prepare-image
will soon stop writing, once cloud-init's ds-identify support has landed. I
don't think we should expect this to be an ongoing production use case for a
hook.

Barry Warsaw (barry) wrote :

On Mar 15, 2017, at 07:22 PM, Steve Langasek wrote:

>There are not many files on the filesystem in the ubuntu-core case, period;
>the only one we've needed to delete, so far, is one that snap prepare-image
>will soon stop writing, once cloud-init's ds-identify support has landed. I
>don't think we should expect this to be an ongoing production use case for a
>hook.

The question then is whether we want a general hook which can be used for
whatever purpose, or a configuration file which describes which files to add
if that's the only use case we think we're ever going to need.

I'm still inclined to provide a more general programmatic hook that can be
used for any purpose. It actually makes ubuntu-image simpler that way too,
because we just need to find the place to add that hook, and then define how
it's used (e.g. can it be a Python callable, or should it be a more generic
subprocess call so the hook can be in any language).

The more limited mechanism means we need to define a format (YAML,
line-oriented text file, etc.), a parser, and the machinery to do the copies,
all of which must be tested and documented. A general hook gives users more
flexibility, and is simpler to code, test, and document in u-i.

Steve Langasek (vorlon) on 2017-03-17
Changed in ubuntu-image:
milestone: none → 1.1
Łukasz Zemczak (sil2100) wrote :

I would personally also opt for a generic hook mechanism. For now we could only concentrate on the hook that would be executed after the rootfs is ready to enable copying/deleting arbitrary files on the filesystem, but the extensible mechanism we could use later on when the need for other hooks arises. I don't think there would be much difference time-wise on doing it one way or another as both solutions require some design and code rewrites.

Changed in ubuntu-image:
assignee: nobody → Łukasz Zemczak (sil2100)
Changed in ubuntu-image:
importance: Undecided → High
Changed in ubuntu-image:
status: New → Confirmed
Changed in ubuntu-image:
status: Confirmed → In Progress
Łukasz Zemczak (sil2100) wrote :

Right now what I have working is a really simple hook mechanism of running scripts in selected defined moments from selected directories (by user + default locations). One hook can have more than one script fired as we'll be running all scripts in hook-name.d if existing. During hook definition additional environment variables can be provided (like, rootfs directory etc.). I'm also writing tests that would make sure no previously defined hooks are easily removed, since we want to make sure they're part of the official API.

My only problem so far is that my code looks somehow ugly, ygh.

Łukasz Zemczak (sil2100) wrote :

Another question while we're at it: should we include any default directories where hooks should be looked for? For now it's completely optional, but maybe we could look in ~/.ubuntu-image/hooks/ or something + in the given directories through command arguments?

Changed in ubuntu-image:
milestone: 1.1 → 1.2
Changed in ubuntu-image:
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-image - 1.3+18.04ubuntu1

---------------
ubuntu-image (1.3+18.04ubuntu1) bionic; urgency=medium

  * Remove the snapcraft dependency by moving the gadget tree priming step away
    from ubuntu-image. (LP: #1734655)
  * Add support for defining the livecd-rootfs configuration path through
    an environment variable. (LP: #1734949)

 -- Łukasz 'sil2100' Zemczak <email address hidden> Mon, 27 Nov 2017 11:12:08 +0100

Changed in ubuntu-image (Ubuntu):
status: New → Fix Released
Changed in ubuntu-image:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers