iscsid.service fails to start in container, results in failed dist-upgrade later on

[David Britton]

I am getting a transient error installing open-iscsi in a xenial lxd container. Sometimes it works fine, but occasionally, it fails like this (apt-get -f install usually fixes it up). As you can see in the syslog, the daemon fails to start, but then later starts fine. I think it might have something to do with the refresh cycle that is going on at the same time the iscsi daemon is starting, but I'm not really sure:

Easier reading: http://paste.ubuntu.com/23659441/

Error installing iscsi:

017-12-20 12:33:54 INFO install Setting up open-iscsi (2.0.873+git0.3b4b4500-14ubuntu3.3) ...
2016-12-20 12:33:55 INFO install Job for iscsid.service failed because a configured resource limit was exceeded. See "systemctl status iscsid.service" and "journalctl -xe" for details.
2016-12-20 12:33:55 INFO install invoke-rc.d: initscript iscsid, action "restart" failed.
2016-12-20 12:33:55 INFO install dpkg: error processing package open-iscsi (--configure):
2016-12-20 12:33:55 INFO install subprocess installed post-installation script returned error exit status 1

Syslog, from the end of cloud-init until the shutdown of the system:

Dec 20 12:33:24 juju-e08f21-0-lxd-2 cloud-init[387]: Cloud-init v. 0.7.8 finished at Tue, 20 Dec 2016 12:33:24 +0000. Datasource DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]. Up 76.0 seconds
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Main process exited, code=exited, status=1/FAILURE
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Failed to start Execute cloud user/final scripts.
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Unit entered failed state.
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: cloud-final.service: Failed with result 'exit-code'.
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Reached target Cloud-init target.
Dec 20 12:33:24 juju-e08f21-0-lxd-2 systemd[1]: Startup finished in 1min 15.898s.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Starting LXD - network bridge...
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Started LXD - network bridge.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Starting LXD - main daemon...
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 lxd[8960]: t=2016-12-20T12:33:25+0000 lvl=warn msg="Per-container AppArmor profiles are disabled because LXD is running in an unprivileged container."
Dec 20 12:33:25 juju-e08f21-0-lxd-2 lxd[8960]: t=2016-12-20T12:33:25+0000 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 9h 57min 27.662713s random time.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 5h 57min 51.806386s random time.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 1h 24min 59.817093s random time.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-unit-postgresql-0.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Started juju unit agent for postgresql/0.
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /user.slice: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /init.scope: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-null.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-fuse.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-.lxd\x2dmounts.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-full.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-update-utmp.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/console-getty.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init-local.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-urandom.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rsyslog.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-lxd.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rc-local.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-random.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/networking.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/snapd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug-tracing.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/polkitd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/-.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lvm2-monitor.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/mdadm.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dbus.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ufw.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/irqbalance.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/resolvconf.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-cpuinfo.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-sys-fs-binfmt_misc.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/var-lib-lxcfs.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-tty.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-stat.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-containers.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-logind.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/system-getty.slice: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udevd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apparmor.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-diskstats.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-machine-0-lxd-2.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ondemand.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-zero.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-swaps.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-uptime.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-net-tun.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-meminfo.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/atd.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cron.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apport.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journal-flush.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/accounts-daemon.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-user-sessions.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-mqueue.mount: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udev-trigger.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-random-seed.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journald.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-tmpfiles-setup.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-config.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init.service: Operation not permitted
Dec 20 12:33:25 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-fs-fuse-connections.mount: Operation not permitted
Dec 20 12:33:30 juju-e08f21-0-lxd-2 systemd[1]: Started LXD - main daemon.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 5h 57min 26.697000s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 17min 59.238219s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 23min 5.325331s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 6h 10min 27.192344s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 26min 50.022137s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 2h 27min 27.974037s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 3h 5min 31.681366s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 3h 45min 18.755704s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 56min 58.814178s random time.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Stopped iSCSI initiator daemon (iscsid).
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/iscsid.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Starting iSCSI initiator daemon (iscsid)...
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /user.slice: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /init.scope: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-meminfo.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apparmor.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-stat.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-fuse.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dbus.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-machine-0-lxd-2.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/system-getty.slice: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-swaps.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-unit-postgresql-0.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-config.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-zero.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/networking.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-random.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-tty.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/polkitd.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rsyslog.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udev-trigger.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-fs-fuse-connections.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udevd.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/mdadm.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init-local.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-diskstats.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/console-getty.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lvm2-monitor.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug-tracing.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-user-sessions.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journald.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-logind.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-update-utmp.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-uptime.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/atd.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-full.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/var-lib-lxcfs.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-sys-fs-binfmt_misc.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cron.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-containers.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/accounts-daemon.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apport.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journal-flush.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/-.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-cpuinfo.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-urandom.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-random-seed.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/snapd.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rc-local.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-.lxd\x2dmounts.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-net-tun.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-tmpfiles-setup.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ondemand.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/resolvconf.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-mqueue.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-lxd.mount: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ufw.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/irqbalance.service: Operation not permitted
Dec 20 12:33:55 juju-e08f21-0-lxd-2 iscsid: iSCSI logger with pid=14329 started!
Dec 20 12:33:55 juju-e08f21-0-lxd-2 iscsid: iSCSI daemon with pid=14330 started!
Dec 20 12:33:55 juju-e08f21-0-lxd-2 iscsid: failed to mlockall, exiting...
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: PID 14330 read from file /run/iscsid.pid does not exist or is a zombie.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: Failed to start iSCSI initiator daemon (iscsid).
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Unit entered failed state.
Dec 20 12:33:55 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Failed with result 'resources'.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 6h 58min 13.024241s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 37min 56.289578s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 2h 35min 8.104459s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 8h 30min 27.002498s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 1h 52min 4.623955s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 22min 53.196363s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 2h 3min 11.839965s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 25min 2.285778s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 1h 21min 1.034405s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Stopped iSCSI initiator daemon (iscsid).
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/iscsid.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Starting iSCSI initiator daemon (iscsid)...
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /user.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /init.scope: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udevd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-mqueue.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-update-utmp.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dbus.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cron.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-cpuinfo.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ufw.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-meminfo.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init-local.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/-.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-user-sessions.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apport.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/console-getty.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/polkitd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-random.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-unit-postgresql-0.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lvm2-monitor.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-machine-0-lxd-2.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journald.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-tmpfiles-setup.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/atd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-net-tun.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apparmor.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ondemand.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug-tracing.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-diskstats.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rsyslog.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-urandom.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-fs-fuse-connections.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/resolvconf.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/accounts-daemon.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-stat.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-random-seed.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-tty.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-sys-fs-binfmt_misc.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-logind.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-full.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-lxd.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-swaps.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journal-flush.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-fuse.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-zero.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-uptime.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rc-local.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/var-lib-lxcfs.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/snapd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-null.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/irqbalance.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/mdadm.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-.lxd\x2dmounts.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-config.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udev-trigger.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-containers.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/system-getty.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/networking.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsid: iSCSI logger with pid=14549 started!
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Failed to read PID from file /run/iscsid.pid: Invalid argument
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsid: iSCSI daemon with pid=14550 started!
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsid: failed to mlockall, exiting...
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Started iSCSI initiator daemon (iscsid).
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Main process exited, code=exited, status=1/FAILURE
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsiadm[14552]: iscsiadm: can not connect to iSCSI daemon (111)!
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsiadm[14552]: iscsiadm: initiator reported error (20 - could not connect to iscsid)
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsiadm[14552]: iscsiadm: Could not stop iscsid. Trying sending iscsid SIGTERM or SIGKILL signals manually
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Unit entered failed state.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Failed with result 'exit-code'.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Reloading.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: apt-daily.timer: Adding 3h 14min 31.201253s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 4h 37min 20.488280s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: snapd.refresh.timer: Adding 3h 43min 16.817460s random time.
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/iscsid.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Starting iSCSI initiator daemon (iscsid)...
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /user.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /init.scope: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-cpuinfo.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cron.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-mqueue.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-zero.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/snapd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rsyslog.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/console-getty.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/mdadm.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/irqbalance.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-null.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journal-flush.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-stat.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-random.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-urandom.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apparmor.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-unit-postgresql-0.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-diskstats.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udev-trigger.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-fuse.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-user-sessions.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-net-tun.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-config.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-logind.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/networking.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/system-getty.slice: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-full.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/jujud-machine-0-lxd-2.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-containers.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lvm2-monitor.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-lxd.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-swaps.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/var-lib-lxcfs.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-tmpfiles-setup.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-fs-fuse-connections.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/polkitd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-uptime.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-udevd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/rc-local.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-update-utmp.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dbus.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/apport.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/cloud-init-local.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-tty.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-journald.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ssh.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-meminfo.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/systemd-random-seed.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/-.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ondemand.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/accounts-daemon.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/dev-.lxd\x2dmounts.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/proc-sys-fs-binfmt_misc.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/ufw.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/lxd-bridge.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/sys-kernel-debug-tracing.mount: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/resolvconf.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/atd.service: Operation not permitted
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsid: iSCSI logger with pid=14601 started!
Dec 20 12:34:05 juju-e08f21-0-lxd-2 iscsid: iSCSI daemon with pid=14602 started!
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: PID 14602 read from file /run/iscsid.pid does not exist or is a zombie.
Dec 20 12:34:06 juju-e08f21-0-lxd-2 iscsid: failed to mlockall, exiting...
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: Failed to start iSCSI initiator daemon (iscsid).
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Unit entered failed state.
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: iscsid.service: Failed with result 'resources'.
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: Failed to reset devices.list on /system.slice/open-iscsi.service: Operation not permitted
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: Starting Login to default iSCSI targets...
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: open-iscsi.service: Control process exited, code=exited status=3
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: Failed to start Login to default iSCSI targets.
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: open-iscsi.service: Unit entered failed state.
Dec 20 12:34:06 juju-e08f21-0-lxd-2 systemd[1]: open-iscsi.service: Failed with result 'exit-code'.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[1]: Created slice User Slice of ubuntu.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[1]: Starting User Manager for UID 1000...
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[1]: Started Session c1 of user ubuntu.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Sockets.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Timers.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Paths.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Basic System.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Default.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[14731]: Startup finished in 11ms.
Dec 20 12:34:13 juju-e08f21-0-lxd-2 systemd[1]: Started User Manager for UID 1000.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[1]: Stopping User Manager for UID 1000...
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Reached target Shutdown.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Stopped target Default.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Stopped target Basic System.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Stopped target Sockets.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Stopped target Paths.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Stopped target Timers.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Starting Exit the Session...
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[14731]: Received SIGRTMIN+24 from PID 14777 (kill).
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[1]: Stopped User Manager for UID 1000.
Dec 20 12:34:15 juju-e08f21-0-lxd-2 systemd[1]: Removed slice User Slice of ubuntu.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[1]: Created slice User Slice of ubuntu.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[1]: Starting User Manager for UID 1000...
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[1]: Started Session c2 of user ubuntu.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Reached target Paths.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Reached target Timers.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Reached target Sockets.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Reached target Basic System.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Reached target Default.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[14786]: Startup finished in 10ms.
Dec 20 12:34:17 juju-e08f21-0-lxd-2 systemd[1]: Started User Manager for UID 1000.

Related bugs:
 * bug 1755858: iscsid autostarts on all servers when it has nothing to do

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[haolin zhang (zhanghaolin)]

I do not mean to change the status. How can I change it back?

[Jason Hobbs (jason-hobbs)]

We hit this doing an apt update in a lxd container: http://paste.ubuntu.com/25233711/

[Paco Avila (monkiki)]

Me to: update in a LXD container. I don't know if the safest option is to remove this package.

Any tip?

[Alexander Karelas (karjala)]

Me too: dist-upgrade in both of my LXD containers. In a VPS somewhere. I don't know if I need this package.

[Jason Hobbs (jason-hobbs)]

We're still hitting this in LXD.

[Jason Hobbs (jason-hobbs)]

We're actually seeing iscsi errors show up as soon as the container starts; I guess the iscsid service never starts properly and so when the charm does an apt-get install of some unrelated packages, something is trying to set up that iscsi service and it's failing again.

I've attached logs from a container where this happened. Excerpt from syslog:

http://paste.ubuntu.com/p/vwqmRkGzdP/

[Steve Langasek (vorlon)]

Fixing this properly should mean that open-iscsi is socket-activated in stable releases the same as it currently is in cosmic. This should resolve the problems including on package upgrade since the postinst will no longer be trying to start the daemon that shouldn't be started.

[Scott Moser (smoser)]

I had hoped to reproduce this so that I could be sure that I actually fixed it.
What I did to attempt that was
 - download the 20180816 xenial daily build that had 2.0.873+git0.3b4b4500-14ubuntu3.4.
 - add to lxc
   lxc image import xenial-server-cloudimg-amd64-lxd.tar.xz xenial-server-cloudimg-amd64.squashfs --alias xenial-20180816
 - start instance: lxc launch xenial-20180816 xtest
 - enter: lxc exec xtest /bin/bash
 - update, upgrade ...

that did not reproduce. :-(

Attached is a screen log.

[Scott Moser (smoser)]

@Jason,
I'm guessing that you ran some script to collect the logs that you collected (open-iscsi-errors.tar.gz). There is a lot of good information there, thank you.

Please add collection of /etc/cloud/build.info to the script that collects.

[Scott Moser (smoser)]

I cannot seem to re-create this, so I'm not really interested in SRU'ing a fix that we think might solve the problem without the ability to verify that.

Here is what I'm doing to attempt to recreate
 http://paste.ubuntu.com/p/STVg62nGPb/

I'm through 432 containers 10 at a time right now on my system without any recreate.

I'm not opposed to SRU'ing the change to start open-iscsi only when necessary, but that should be done on its own merit, with its own test cases and such. I'd suggest another bug for that rather than taking over this one.

[David Britton (dpb)]

I also cannot repro this failure. I can trigger the upgrade, and use a script similar to what smoser had, but each time, the upgrade works fine. I went through 250 iterations of the script without failure.

Could someone please attach a reproducer, or give access to a live system that has just shown the failure?

[Scott Moser (smoser)]

2016-12-20 12:33:55 INFO install Job for iscsid.service failed because a configured resource limit was exceeded. See "systemctl status

^ if that is the source of the issue, then this is "working as designed". If the container had some memory or other constraints and open-iscsi failed to restart as a result, then there isn't really much that can be done here.

[Scott Moser (smoser)]

If we were to change the behavior of open-iscsi to be socket activated, then that should happen under bug 1755858.

[Chris Gregan (cgregan)]

Just saw this one again after a while.