On Wed, Oct 09, 2019 at 01:57:00PM -0000, Dimitri John Ledkov wrote:
> Sure, and image is configured to use cloud-init with netplan backend,
> which itself uses networkd backend.

The cloud-init generator is shipped by upstream, so it's intended to be
used on distros other than just Ubuntu.  I'm only bringing this up
because there is a non-zero possibility that, in the future, someone
could use netplan to configure the NetworkManager backend in a
configuration that we need to support upstream.

(The generator is templated, so we can easily make distro-specific
changes; this is more of a reminder that we need to be careful about
what is generally useful upstream and what is useful to Ubuntu only.)

> Now, the issues is that of a systemd boot transaction and inability to
> affect it whilst its in flight.
>
> Thus netplan ships a generator, which on boot at generator stage (prior
> to calculating and executing the initial transaction) reads netplan.yaml
> configs and generates appropriate dependencies for a particular backend
> to be invoked.
>
> Similarly cloud-init ships a generator, to hookup all the cloud-init
> units & targets into the initial transaction.
>
> The problem is the one of the integration of the two generators.
>
> cloud-init at generator stage, potentially knows, that it will spit out
> netplan configs & will generate netplan units *just in time* for the
> networking.target to have all the files on disk to have the right
> things.

So, currently, the cloud-init generator only knows about how to detect
the data source that should be used for the platform (as well as
handling kernel cmdline options to modify its behaviour, including
"always disable cloud-init", for example).  The generator is implemented
in shell, as are its dependencies, so that we don't have to pay Python
startup costs to determine that we didn't need to run cloud-init at all.

cloud-init's network backend selection is dynamic.  Once we've
determined the network configuration that should be applied, we detect
what network backends are available on the system and then select the
highest priority one (eni over sysconfig over netplan).  The
availability checks for ENI and netplan are pretty simple: are the
appropriate binaries available and, for ENI, does
/etc/network/interfaces exist.  (Naturally, all this logic is only
implemented in Python at the moment, so we would need to reimplement it
in shell for use in the generator.)

One area of complexity is that the application of network configuration
can be entirely disabled.  I don't, off-hand, know the ways in which we
support doing this, but I think that the cloud-init generator shouldn't
generate any network backend dependencies in those cases.  (Other than
explicitly being disabled, cloud-init expects to generate network
configuration on every first boot.)

> But netplan at generator stage, sees no configs whats so ever, doesn't
> know which backend to use & hook up for the boot target, and thus does
> nothing. When cloud-init calls netplan to generate just in time units,
> netplan does generate dependencies to start the right networking
> backend, but it's too late as the initial boot transaction is already in
> progress and cannot be changed to slot in systemd-networkd & systemd-
> networkd-wait-online to bring up the networking.

Aha, this makes a lot of sense.

> It feels like there should be some integration between cloud-init &
> netplan generators to indicate that "yes we will have networking config"
> and "please hookup <default>, networkd, NetworkManager backends for the
> networking.target"

Yes, I agree that some integration is warranted.  Are generators
ordered?  I'm thinking that cloud-init could do something relatively
simple like write out the name of the backend it expects to use
somewhere, and the netplan generator could add a check for that file to
its backend determination code.  What do you think?