getty.target starts before cloud-config is done

[Dimitri John Ledkov]

On targets that are normally accessed via serial console (i.e. Subiquity, Ubuntu Classic/Core on RPi, etc) cloud-init often does not complete before getty spawns login shell.

This creates subpar user experience, as it appears as if one can login, and should be able to login using cloud-init provided credentials, but in practice cannot.

To mitigate this we have started to add the following override on some of our images:

See: http://launchpadlibrarian.net/461986467/livecd-rootfs_2.636_2.637.diff.gz

+ mkdir -p /etc/systemd/system/cloud-config.service.d
+ cat << EOF > /etc/systemd/system/cloud-config.service.d/getty-wait.conf
+# Wait for cloud-init to finish (creating users, etc.) before running getty
+
+[Unit]
+Before=getty.target
+EOF

Instead, cloud-config.service itself should declare `Before=getty.target`

Use case is monitoring RPi booting on serial console, and loging in once getty is up. With expectation that login succeeds.

Currently login fails, more cloud-config spew appears on screen, then one has to hit enter to realise that login is up, realize that one is now trying to login with empty username, hit enter again, and now type in username & password to finally login.

[Ryan Harper (raharper)]

While the user is created earlier, other modules such as:

locale, set-password, and ssh-import-id
should be run before getty.target for login to be successful.

So I think this is the right change for now; however, if the image also were to need to seed snaps, we're also then blocking login until after snaps have seeded as well.

[Dave Jones (waveform)]

I don't think there's a sensible way of fixing this:

Ideally we'd have getty wait until users exist but there's no trigger / phase of cloud-init that's specific to it creating users. We *could* wait until cloud-init is "done" but in the case of more complex cloud-init configurations that could be an entirely arbitrary (even infinite) delay (waiting for network connections or arbitrary scripts to finish). Meanwhile, the user creation could be complete, but the admin is unable to login to the system to sort out networking (for example) because cloud-init hasn't "finished".

I looked into the possibility of adding conditions (and some extra activation triggers) on getty to cause it to wait until a (non-root) local user appeared in passwd/shadow but this then breaks the scenario where cloud-init is configured *not* to create any local users because (for example) LDAP is going to be used.

I'm leaning towards closing this as "won't fix" at this point, although first I would at least like to look into a first-time boot message warning the user they should wait for cloud-init to complete.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3600

[James Falcon (falcojr)]

related: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2013403