Circular boot dependency cycle with cloud-init et.al.

[Dimitri John Ledkov]

[Impact]

 * xe-deamon.service tries to be an early boot unit; before cloud-init-local / networking is setup
 * However it does not declare DefaultDependencies=no, and thus tries to pull in sysinit.target which should happen after cloud-init-local; yet xe-daemon.service is declared to be before cloud-init-local, thus this is a conflict of interest.
 * xe-deamon should declare DefaultDependencies=no

= Current unit =
[Unit]
Description=Xen Guest Monitoring Agent
After=local-fs.target
Requires=proc-xen.mount
Before=network.target cloud-init.service cloud-init-local.service
ConditionVirtualization=xen
ConditionPathExists=/proc/xen/capabilities

[Service]
ExecStartPre=/usr/sbin/xe-linux-distribution /var/cache/xe-linux-distribution
ExecStart=/usr/sbin/xe-daemon
StandardOutput=journal+console

[Install]
WantedBy=multi-user.target

= Proposed fix =

--- xe-guest-utilities.xe-daemon.service.old 2017-03-03 11:45:48.679700766 +0000
+++ xe-guest-utilities.xe-daemon.service 2017-03-03 11:46:07.923549054 +0000
@@ -1,5 +1,6 @@
 [Unit]
 Description=Xen Guest Monitoring Agent
+DefaultDependencies=no
 After=local-fs.target
 Requires=proc-xen.mount
 Before=network.target cloud-init.service cloud-init-local.service

==

[Test Case]

 * Boot a cloud instance, with xen hypervisor and cloud-init metdata
 * There should be no boot cycles

[Regression Potential]

 * Minimal, currently xe-daemon.service creates dependency cycles which prevent reliable and reproducible boots.

[Other Info]

 * Related bug #1496730

[Dependency cycle]

Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found ordering cycle on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on sockets.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on snapd.socket/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on sysinit.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on cloud-init.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on cloud-init-local.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on xe-daemon.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Breaking ordering cycle by deleting job sockets.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: sockets.target: Job sockets.target/start deleted to break ordering cycle starting with basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found ordering cycle on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on sysinit.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on cloud-init.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on cloud-init-local.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on xe-daemon.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Breaking ordering cycle by deleting job cloud-init.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: cloud-init.service: Job cloud-init.service/start deleted to break ordering cycle starting with basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found ordering cycle on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on sysinit.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on cloud-init-local.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on xe-daemon.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Found dependency on basic.target/start
Mar 03 12:01:15 slmeta1 systemd[1]: basic.target: Breaking ordering cycle by deleting job cloud-init-local.service/start
Mar 03 12:01:15 slmeta1 systemd[1]: cloud-init-local.service: Job cloud-init-local.service/start deleted to break ordering cycle starting with basic.target/start

[Jose L. VG (josvaz)]

I'd vote for the "DefaultDependencies=no" option as per experience with this RAX issue:
https://bugs.launchpad.net/cpc-rax/+bug/1657223

[Dimitri John Ledkov (xnox)]

I am advised that if one wants to communicate with xenstore from cloud-init modules, one needs xe-daemon runnning, hence xe-daemon should be brought up asap.

[Launchpad Janitor (janitor)]

This bug was fixed in the package xe-guest-utilities - 6.2.0-1120+dsf1-0ubuntu4

---------------
xe-guest-utilities (6.2.0-1120+dsf1-0ubuntu4) zesty; urgency=medium

  * debian/xe-guest-utilities.xe-daemon.service: declare
    DefaultDependencies=no to prevent dependency cycles. The unit cannot
    be simultaniously before and after sysinit target. By default all
    units are after sysinit target, yet cloud-init-local is before it. LP:
    #1669755

 -- Dimitri John Ledkov <email address hidden> Fri, 03 Mar 2017 12:33:01 +0000