Comment 7 for bug 1848923

Thanks Christian! Interesting, the "ERROR: should execute as the [pollinate] user" looks quite unrelated to any networking setup and smelled like a package postinst/systemd unit bug.

I started trying to reproduce this outside of the Ubuntu CI, with a most naïve

  curl -O https://cloud-images.ubuntu.com/daily/server/groovy/current/groovy-server-cloudimg-amd64.img
  curl -O https://github.com/cockpit-project/bots/raw/master/machine/cloud-init.iso
  qemu-system-x86_64 -enable-kvm -nographic -m 2048 -device virtio-rng-pci -drive file=groovy-server-cloudimg-amd64.img,if=virtio -snapshot -cdrom cloud-init.iso

(that cloud-init is nothing special, just to get root:foobar and admin:foobar accounts, see [1])

But this fails to start ssh.service, and neither does it actually run cloud-init, so there is no way to log in and examine further. But indeed it seems to have started pollinate:

[ OK ] Finished Pollinate to seed…seudo random number generator.
         Starting OpenBSD Secure Shell server...
[ OK ] Started Dispatcher daemon for systemd-networkd.
[FAILED] Failed to start OpenBSD Secure Shell server.
See 'systemctl status ssh.service' for details.

Our CI uses libvirt for everything, which of course makes a CLI reproducer a lot more difficult. It's fairly easy when actually using our python wrappers:

  git clone https://github.com/cockpit-project/bots/
  bots/vm-run ubuntu-stable

(log in as root:foobar)

This downloads the actual image to ~/.cache/cockpit-images and creates a transient domain with a transient overlay. I attach the `virsh dumpxml` output, but of course that has a lot of absolute file system paths in it, UUIDs, and other junk that is specific to my system.

There the pollinate user exists

  # id pollinate
  uid=111(pollinate) gid=1(daemon) groups=1(daemon)

  # systemctl cat pollinate
  [..]
  [Service]
  User=pollinate
  ExecStart=/usr/bin/pollinate
  Type=oneshot

which looks right.

I edited the systemd unit to run pollinate through /bin/sh -ex, and that did reveal something:

Feb 07 09:24:10 ubuntu sh[2449]: + set -e
Feb 07 09:24:10 ubuntu sh[2449]: + set -f
Feb 07 09:24:10 ubuntu sh[2449]: + PKG=pollinate
Feb 07 09:24:10 ubuntu sh[2450]: + mktemp -d -t pollinate.XXXXXXXXXXXX
Feb 07 09:24:10 ubuntu sh[2449]: + TMPDIR=/tmp/pollinate.77BPmzGSfzFm
Feb 07 09:24:10 ubuntu sh[2449]: + trap rm -rf /tmp/pollinate.77BPmzGSfzFm 2>/dev/null || true EXIT HUP INT QUIT TERM
Feb 07 09:24:10 ubuntu sh[2449]: + CACHEDIR=/var/cache/pollinate
Feb 07 09:24:10 ubuntu sh[2449]: + FLAG=/var/cache/pollinate/seeded
Feb 07 09:24:10 ubuntu sh[2449]: + LOG=/var/cache/pollinate/log
Feb 07 09:24:10 ubuntu sh[2451]: + hostname
Feb 07 09:24:10 ubuntu sh[2449]: + HOSTNAME=ubuntu
Feb 07 09:24:10 ubuntu sh[2449]: + STRICT=0
Feb 07 09:24:10 ubuntu sh[2453]: + logger -V
Feb 07 09:24:10 ubuntu sh[2454]: + awk {print $4}
Feb 07 09:24:10 ubuntu sh[2449]: + logger_ver=2.36
Feb 07 09:24:10 ubuntu sh[2449]: + dpkg --compare-versions 2.36 ge 2.26.2
Feb 07 09:24:10 ubuntu sh[2449]: + LOGGER=logger --id=2449
Feb 07 09:24:10 ubuntu sh[2449]: + [ -t 0 ]
Feb 07 09:24:10 ubuntu sh[2449]: + [ -r /etc/default/pollinate ]
Feb 07 09:24:10 ubuntu sh[2449]: + . /etc/default/pollinate
Feb 07 09:24:10 ubuntu sh[2449]: + BINARY=1
Feb 07 09:24:10 ubuntu sh[2449]: + QUIET=0
Feb 07 09:24:10 ubuntu sh[2449]: + WAIT=10
Feb 07 09:24:10 ubuntu sh[2449]: + DEVICE=/dev/urandom
Feb 07 09:24:10 ubuntu sh[2449]: + SERVER=https://entropy.ubuntu.com/
Feb 07 09:24:10 ubuntu sh[2449]: + POOL=
Feb 07 09:24:10 ubuntu sh[2449]: + CURL_OPTS=--cacert /etc/pollinate/entropy.ubuntu.com.pem --capath /dev/null
Feb 07 09:24:10 ubuntu sh[2449]: + [ ! -z ]
Feb 07 09:24:10 ubuntu sh[2449]: + [ -z ]
Feb 07 09:24:10 ubuntu sh[2449]: + [ ! -w /var/cache/pollinate ]
Feb 07 09:24:10 ubuntu sh[2449]: + error should execute as the [pollinate] user
Feb 07 09:24:10 ubuntu sh[2449]: + logger --id=2449 -t pollinate ERROR: should execute as the [pollinate] user

/var/cache/pollinate indeed does not exist.

I added

  CacheDirectory=pollinate

to the systemd unit, and that fixes it.

[1] https://github.com/cockpit-project/bots/blob/master/machine/make-cloud-init-iso