| 2017-09-15 09:39:12 |
thermoman |
bug |
|
|
added bug |
| 2017-09-15 09:40:55 |
thermoman |
tags |
|
cycle ordering |
|
| 2017-09-15 09:41:28 |
thermoman |
tags |
cycle ordering |
systemd |
|
| 2017-09-15 13:43:08 |
Scott Moser |
cloud-init: importance |
Undecided |
High |
|
| 2017-09-15 13:52:21 |
Scott Moser |
cloud-init: status |
New |
Confirmed |
|
| 2017-09-15 14:13:55 |
Scott Moser |
attachment added |
|
journalctl -o short-precise before https://bugs.launchpad.net/cloud-init/+bug/1717477/+attachment/4950747/+files/journal-short-precise-before.txt |
|
| 2017-09-15 14:14:15 |
Scott Moser |
attachment added |
|
journalctl -o short-precise after https://bugs.launchpad.net/cloud-init/+bug/1717477/+attachment/4950748/+files/journal-short-precise-after.txt |
|
| 2017-09-15 14:17:20 |
Scott Moser |
description |
We're running several machines with
cloud-init_0.7.9-153-g16a7302f-0ubuntu1~16.04.2
without problems.
Just upgraded all machines to
cloud-init_0.7.9-233-ge586fe35-0ubuntu1~16.04.1
and rebooted them all.
All machines report ordering cycles in their dmesg, resulting in systemd breaking the
loop by NOT starting some important services, e.g. mouting local filesystems:
Sep 14 15:43:52.487945 noname systemd[1]: networking.service: Found ordering cycle on networking.service/start
Sep 14 15:43:52.487952 noname systemd[1]: networking.service: Found dependency on local-fs.target/start
Sep 14 15:43:52.487960 noname systemd[1]: networking.service: Found dependency on home.mount/start
Sep 14 15:43:52.487968 noname systemd[1]: networking.service: Found dependency on systemd-fsck@dev-disk-by\x2dlabel-Home.service/start
Sep 14 15:43:52.487975 noname systemd[1]: networking.service: Found dependency on cloud-init.service/start
Sep 14 15:43:52.487982 noname systemd[1]: networking.service: Found dependency on networking.service/start
Sep 14 15:43:52.488297 noname systemd[1]: networking.service: Breaking ordering cycle by deleting job local-fs.target/start
Sep 14 15:43:52.488306 noname systemd[1]: local-fs.target: Job local-fs.target/start deleted to break ordering cycle starting with networking.service/start
% cat /etc/fstab
LABEL=cloudimg-rootfs / ext4 defaults,discard 0 1
LABEL=Home /home xfs defaults,logbufs=8 0 2
In this case /home isn't mounted as a result of systemd breaking the loop, resulting in services depending on /home not being started.
1. Tell us your cloud provider
AWS
2. dpkg-query -W -f='${Version}' cloud-init
0.7.9-233-ge586fe35-0ubuntu1~16.04.1
3. Any appropriate cloud-init configuration you can provide us
Nothing special - worked with 0.7.9-153-g16a7302f-0ubuntu1~16.04.2 on all machines without hassle.
The problem is this change:
diff -uaNr 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
--- 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 1970-01-01 01:00:00.000000000 +0100
+++ 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 2017-07-28 22:28:47.000000000 +0200
@@ -0,0 +1,2 @@
+[Unit]
+After=cloud-init.service
WORKAROUND
==========
I just did a
rm /lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
on all machines and rebooted them: no more dependency loops reported, everything works again. |
We're running several machines with
cloud-init_0.7.9-153-g16a7302f-0ubuntu1~16.04.2
without problems.
Just upgraded all machines to
cloud-init_0.7.9-233-ge586fe35-0ubuntu1~16.04.1
and rebooted them all.
All machines report ordering cycles in their dmesg, resulting in systemd breaking the
loop by NOT starting some important services, e.g. mouting local filesystems:
Sep 14 15:43:52.487945 noname systemd[1]: networking.service: Found ordering cycle on networking.service/start
Sep 14 15:43:52.487952 noname systemd[1]: networking.service: Found dependency on local-fs.target/start
Sep 14 15:43:52.487960 noname systemd[1]: networking.service: Found dependency on home.mount/start
Sep 14 15:43:52.487968 noname systemd[1]: networking.service: Found dependency on systemd-fsck@dev-disk-by\x2dlabel-Home.service/start
Sep 14 15:43:52.487975 noname systemd[1]: networking.service: Found dependency on cloud-init.service/start
Sep 14 15:43:52.487982 noname systemd[1]: networking.service: Found dependency on networking.service/start
Sep 14 15:43:52.488297 noname systemd[1]: networking.service: Breaking ordering cycle by deleting job local-fs.target/start
Sep 14 15:43:52.488306 noname systemd[1]: local-fs.target: Job local-fs.target/start deleted to break ordering cycle starting with networking.service/start
% cat /etc/fstab
LABEL=cloudimg-rootfs / ext4 defaults,discard 0 1
LABEL=Home /home xfs defaults,logbufs=8 0 2
In this case /home isn't mounted as a result of systemd breaking the loop, resulting in services depending on /home not being started.
1. Tell us your cloud provider
AWS
2. dpkg-query -W -f='${Version}' cloud-init
0.7.9-233-ge586fe35-0ubuntu1~16.04.1
3. Any appropriate cloud-init configuration you can provide us
Nothing special - worked with 0.7.9-153-g16a7302f-0ubuntu1~16.04.2 on all machines without hassle.
The problem is this change:
diff -uaNr 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
--- 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 1970-01-01 01:00:00.000000000 +0100
+++ 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 2017-07-28 22:28:47.000000000 +0200
@@ -0,0 +1,2 @@
+[Unit]
+After=cloud-init.service
WORKAROUND
==========
I just did a
rm /lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
on all machines and rebooted them: no more dependency loops reported, everything works again.
Related bugs:
* bug 1686514: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks
* bug 1691489: fstab entries written by cloud-config may not be mounted |
|
| 2017-09-15 14:18:52 |
Scott Moser |
bug task added |
|
cloud-init (Ubuntu) |
|
| 2017-09-15 14:18:58 |
Scott Moser |
cloud-init (Ubuntu): status |
New |
Confirmed |
|
| 2017-09-15 14:19:02 |
Scott Moser |
cloud-init (Ubuntu): importance |
Undecided |
High |
|
| 2017-09-15 14:19:09 |
Scott Moser |
nominated for series |
|
Ubuntu Artful |
|
| 2017-09-15 14:19:09 |
Scott Moser |
bug task added |
|
cloud-init (Ubuntu Artful) |
|
| 2017-09-15 14:19:09 |
Scott Moser |
nominated for series |
|
Ubuntu Xenial |
|
| 2017-09-15 14:19:09 |
Scott Moser |
bug task added |
|
cloud-init (Ubuntu Xenial) |
|
| 2017-09-15 14:19:09 |
Scott Moser |
nominated for series |
|
Ubuntu Zesty |
|
| 2017-09-15 14:19:09 |
Scott Moser |
bug task added |
|
cloud-init (Ubuntu Zesty) |
|
| 2017-09-15 14:19:18 |
Scott Moser |
cloud-init (Ubuntu Xenial): status |
New |
Confirmed |
|
| 2017-09-15 14:19:21 |
Scott Moser |
cloud-init (Ubuntu Zesty): status |
New |
Confirmed |
|
| 2017-09-15 14:19:24 |
Scott Moser |
cloud-init (Ubuntu Xenial): importance |
Undecided |
High |
|
| 2017-09-15 14:19:26 |
Scott Moser |
cloud-init (Ubuntu Zesty): importance |
Undecided |
High |
|
| 2017-09-15 14:19:35 |
Scott Moser |
tags |
systemd |
regression-released systemd |
|
| 2017-09-15 14:20:24 |
Scott Moser |
tags |
regression-released systemd |
regression-release systemd |
|
| 2017-09-15 16:04:47 |
Balint Reczey |
merge proposal linked |
|
https://code.launchpad.net/~rbalint/cloud-init/+git/cloud-init/+merge/330842 |
|
| 2017-09-15 19:17:04 |
Scott Moser |
description |
We're running several machines with
cloud-init_0.7.9-153-g16a7302f-0ubuntu1~16.04.2
without problems.
Just upgraded all machines to
cloud-init_0.7.9-233-ge586fe35-0ubuntu1~16.04.1
and rebooted them all.
All machines report ordering cycles in their dmesg, resulting in systemd breaking the
loop by NOT starting some important services, e.g. mouting local filesystems:
Sep 14 15:43:52.487945 noname systemd[1]: networking.service: Found ordering cycle on networking.service/start
Sep 14 15:43:52.487952 noname systemd[1]: networking.service: Found dependency on local-fs.target/start
Sep 14 15:43:52.487960 noname systemd[1]: networking.service: Found dependency on home.mount/start
Sep 14 15:43:52.487968 noname systemd[1]: networking.service: Found dependency on systemd-fsck@dev-disk-by\x2dlabel-Home.service/start
Sep 14 15:43:52.487975 noname systemd[1]: networking.service: Found dependency on cloud-init.service/start
Sep 14 15:43:52.487982 noname systemd[1]: networking.service: Found dependency on networking.service/start
Sep 14 15:43:52.488297 noname systemd[1]: networking.service: Breaking ordering cycle by deleting job local-fs.target/start
Sep 14 15:43:52.488306 noname systemd[1]: local-fs.target: Job local-fs.target/start deleted to break ordering cycle starting with networking.service/start
% cat /etc/fstab
LABEL=cloudimg-rootfs / ext4 defaults,discard 0 1
LABEL=Home /home xfs defaults,logbufs=8 0 2
In this case /home isn't mounted as a result of systemd breaking the loop, resulting in services depending on /home not being started.
1. Tell us your cloud provider
AWS
2. dpkg-query -W -f='${Version}' cloud-init
0.7.9-233-ge586fe35-0ubuntu1~16.04.1
3. Any appropriate cloud-init configuration you can provide us
Nothing special - worked with 0.7.9-153-g16a7302f-0ubuntu1~16.04.2 on all machines without hassle.
The problem is this change:
diff -uaNr 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
--- 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 1970-01-01 01:00:00.000000000 +0100
+++ 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 2017-07-28 22:28:47.000000000 +0200
@@ -0,0 +1,2 @@
+[Unit]
+After=cloud-init.service
WORKAROUND
==========
I just did a
rm /lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
on all machines and rebooted them: no more dependency loops reported, everything works again.
Related bugs:
* bug 1686514: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks
* bug 1691489: fstab entries written by cloud-config may not be mounted |
http://pad.lv/1717477
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1717477
=== Begin SRU Template ===
[Impact]
Cloud-init's inclusion of a systemd drop-in file
/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
Caused a regression on systems that had entries in /etc/fstab
that were not authored by cloud-init (specifically that did not have
something like 'x-systemd.requires=cloud-init.service' in their
filesystem options.
[Test Case]
The test can be done on any cloud that has space to put a non-root
filesystem.
a.) launch instance
b.) upgrade to cloud-init to -updates pocket
c.) create a filesystem and put it in /etc/fstab
bdev="/dev/sdb1"
mkdir -p /mnt
mkfs.ext4 -F "$bdev"
echo "$bdev /mnt auto defaults 0 2" >> /etc/fstab
reboot
d.) see mention of 'ordering cycle' in journal
$ journalctl -o short-precise | grep -i ordering.cycle
Sep 15 14:08:48.331033 xenial-20170911-174122 systemd[1]: local-fs.target: Found ordering cycle on local-fs.target/start
Sep 15 14:08:48.331097 xenial-20170911-174122 systemd[1]: local-fs.target: Breaking ordering cycle by deleting job mnt.mount/start
Sep 15 14:08:48.331108 xenial-20170911-174122 systemd[1]: mnt.mount: Job mnt.mount/start deleted to break ordering cycle starting with local-fs.target/start
e.) upgrade to proposed
f.) reboot
g.) expect no mention of ordering cycle as seen in 'd'
$ journalctl -o short-precise | grep -i ordering.cycle || echo "no cycles"
no cycles
[Regression Potential]
This change will mean that bug 1691489 is present again.
That bug is much less severe and affects a much smaller set of users.
[Other Info]
Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=a2f8ce9c80
=== End SRU Template ===
We're running several machines with
cloud-init_0.7.9-153-g16a7302f-0ubuntu1~16.04.2
without problems.
Just upgraded all machines to
cloud-init_0.7.9-233-ge586fe35-0ubuntu1~16.04.1
and rebooted them all.
All machines report ordering cycles in their dmesg, resulting in systemd breaking the
loop by NOT starting some important services, e.g. mouting local filesystems:
Sep 14 15:43:52.487945 noname systemd[1]: networking.service: Found ordering cycle on networking.service/start
Sep 14 15:43:52.487952 noname systemd[1]: networking.service: Found dependency on local-fs.target/start
Sep 14 15:43:52.487960 noname systemd[1]: networking.service: Found dependency on home.mount/start
Sep 14 15:43:52.487968 noname systemd[1]: networking.service: Found dependency on systemd-fsck@dev-disk-by\x2dlabel-Home.service/start
Sep 14 15:43:52.487975 noname systemd[1]: networking.service: Found dependency on cloud-init.service/start
Sep 14 15:43:52.487982 noname systemd[1]: networking.service: Found dependency on networking.service/start
Sep 14 15:43:52.488297 noname systemd[1]: networking.service: Breaking ordering cycle by deleting job local-fs.target/start
Sep 14 15:43:52.488306 noname systemd[1]: local-fs.target: Job local-fs.target/start deleted to break ordering cycle starting with networking.service/start
% cat /etc/fstab
LABEL=cloudimg-rootfs / ext4 defaults,discard 0 1
LABEL=Home /home xfs defaults,logbufs=8 0 2
In this case /home isn't mounted as a result of systemd breaking the loop, resulting in services depending on /home not being started.
1. Tell us your cloud provider
AWS
2. dpkg-query -W -f='${Version}' cloud-init
0.7.9-233-ge586fe35-0ubuntu1~16.04.1
3. Any appropriate cloud-init configuration you can provide us
Nothing special - worked with 0.7.9-153-g16a7302f-0ubuntu1~16.04.2 on all machines without hassle.
The problem is this change:
diff -uaNr 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
--- 153/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 1970-01-01 01:00:00.000000000 +0100
+++ 233/lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf 2017-07-28 22:28:47.000000000 +0200
@@ -0,0 +1,2 @@
+[Unit]
+After=cloud-init.service
WORKAROUND
==========
I just did a
rm /lib/systemd/system/systemd-fsck@.service.d/cloud-init.conf
on all machines and rebooted them: no more dependency loops reported, everything works again.
Related bugs:
* bug 1686514: Azure: cloud-init does not handle reformatting GPT partition ephemeral disks
* bug 1691489: fstab entries written by cloud-config may not be mounted |
|
| 2017-09-15 19:32:20 |
Scott Moser |
cloud-init (Ubuntu Xenial): status |
Confirmed |
In Progress |
|
| 2017-09-15 19:32:23 |
Scott Moser |
cloud-init (Ubuntu Zesty): status |
Confirmed |
In Progress |
|
| 2017-09-15 19:32:27 |
Scott Moser |
cloud-init (Ubuntu Artful): status |
Confirmed |
In Progress |
|
| 2017-09-15 19:32:33 |
Scott Moser |
cloud-init (Ubuntu Xenial): assignee |
|
Scott Moser (smoser) |
|
| 2017-09-15 19:32:36 |
Scott Moser |
cloud-init (Ubuntu Zesty): assignee |
|
Scott Moser (smoser) |
|
| 2017-09-15 21:18:20 |
Launchpad Janitor |
cloud-init (Ubuntu Artful): status |
In Progress |
Fix Released |
|
| 2017-09-18 15:37:14 |
Łukasz Zemczak |
cloud-init (Ubuntu Zesty): status |
In Progress |
Fix Committed |
|
| 2017-09-18 15:37:16 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2017-09-18 15:37:21 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2017-09-18 15:37:23 |
Łukasz Zemczak |
tags |
regression-release systemd |
regression-release systemd verification-needed verification-needed-zesty |
|
| 2017-09-18 15:39:29 |
Łukasz Zemczak |
cloud-init (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2017-09-18 15:39:35 |
Łukasz Zemczak |
tags |
regression-release systemd verification-needed verification-needed-zesty |
regression-release systemd verification-needed verification-needed-xenial verification-needed-zesty |
|
| 2017-09-23 02:36:04 |
Scott Moser |
cloud-init: status |
Confirmed |
Fix Released |
|
| 2017-10-02 19:45:57 |
Scott Moser |
tags |
regression-release systemd verification-needed verification-needed-xenial verification-needed-zesty |
regression-release systemd verification-done-xenial verification-needed verification-needed-zesty |
|
| 2017-10-02 19:57:29 |
Scott Moser |
tags |
regression-release systemd verification-done-xenial verification-needed verification-needed-zesty |
regression-release systemd verification-done-xenial verification-done-zesty |
|
| 2017-10-05 15:45:40 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2017-10-05 15:55:42 |
Launchpad Janitor |
cloud-init (Ubuntu Zesty): status |
Fix Committed |
Fix Released |
|
| 2017-10-05 16:50:25 |
Launchpad Janitor |
cloud-init (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2023-05-11 05:51:40 |
James Falcon |
bug watch added |
|
https://github.com/canonical/cloud-init/issues/3000 |
|
| 2023-05-11 06:21:52 |
thermoman |
removed subscriber thermoman |
|
|
|
