ppp files are created one level too deep

[Simon Eisenmann]

Starting with ubuntu-core 5 a systemd service pppd-dns.service is started on boot and it fails as the corresponding file /etc/ppp/ip-down.d/0000usepeerdns does not exist.

[Simon Eisenmann (longsleep)]

Details:

~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2015-09-22 18:07:22 UTC; 6min ago
  Process: 644 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=203/EXEC)
 Main PID: 644 (code=exited, status=203/EXEC)

[Simon Eisenmann (longsleep)]

I did some digging and found that the ppp files are one level too deep

find /etc/ppp/ -type f
/etc/ppp/ppp/ip-down.d/000resolvconf
/etc/ppp/ppp/ip-down.d/0000usepeerdns
/etc/ppp/ppp/pap-secrets
/etc/ppp/ppp/ipv6-up
/etc/ppp/ppp/ip-up
/etc/ppp/ppp/options
/etc/ppp/ppp/ip-down
/etc/ppp/ppp/peers/provider
/etc/ppp/ppp/ipv6-down
/etc/ppp/ppp/chap-secrets
/etc/ppp/ppp/ip-up.d/000resolvconf
/etc/ppp/ppp/ip-up.d/0000usepeerdns

[Michael Vogt (mvo)]

This looks like a issue with the writable-paths in some way, both /etc/ppp and /etc/modprobe.d are at the wrong level. /writable/system-data/etc/modprobe.d/modprobe.d contains also a wrong level. Both of these files are marked as:
"""
/etc/modprobe.d/ auto persistent transition none
/etc/ppp/ auto persistent tran
"""
Note the trailing "/". I suspect this confuses the writable-path code in the initramfs.

[Michael Vogt (mvo)]

This is fixed for new image with the latest ubuntu-core-config upload. We also need a fixup script for existing users, I will add that next.

[Michael Vogt (mvo)]

The workaround is now also uploaded.

[Leo Arias (elopio)]

I can confirm that the files are at the right level using 15.04 edge #179 on kvm.
But if I update from #178 to #179, the service still shows Active: failed and the ppp files are one level too deep.

[Leo Arias (elopio)]

No wait. It says failed, but the files are at the right level.

(amd64)ubuntu@localhost:~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2015-09-23 20:17:44 UTC; 3min 58s ago
  Process: 603 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=203/EXEC)
 Main PID: 603 (code=exited, status=203/EXEC)

$ find /etc/ppp/ -type f
/etc/ppp/ip-up
find: `/etc/ppp/peers': Permission denied
/etc/ppp/chap-secrets
/etc/ppp/ip-up.d/000resolvconf
/etc/ppp/ip-up.d/0000usepeerdns
/etc/ppp/ipv6-up
/etc/ppp/pap-secrets
/etc/ppp/options
/etc/ppp/ip-down
/etc/ppp/ip-down.d/000resolvconf
/etc/ppp/ip-down.d/0000usepeerdns
/etc/ppp/ipv6-down

[Leo Arias (elopio)]

After a restart, the service seems to be working.

So, #178 fails -> #180 fails -> restart service works.

(amd64)ubuntu@localhost:~$ sudo journalctl -u pppd-dns.service
-- Logs begin at Wed 2015-09-23 20:17:43 UTC, end at Wed 2015-09-23 20:23:57 UTC. --
Sep 23 20:17:44 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service: main process exited, code=exited, status=203/EXEC
Sep 23 20:17:44 localhost.localdomain systemd[1]: Failed to start Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..
Sep 23 20:17:44 localhost.localdomain systemd[1]: Unit pppd-dns.service entered failed state.
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service failed.

(amd64)ubuntu@localhost:~$ sudo systemctl restart pppd-dns.service

(amd64)ubuntu@localhost:~$ sudo journalctl -u pppd-dns.service
-- Logs begin at Wed 2015-09-23 20:17:43 UTC, end at Wed 2015-09-23 20:25:22 UTC. --
Sep 23 20:17:44 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service: main process exited, code=exited, status=203/EXEC
Sep 23 20:17:44 localhost.localdomain systemd[1]: Failed to start Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..
Sep 23 20:17:44 localhost.localdomain systemd[1]: Unit pppd-dns.service entered failed state.
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service failed.
Sep 23 20:25:19 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:25:19 localhost.localdomain systemd[1]: Started Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..

(amd64)ubuntu@localhost:~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Wed 2015-09-23 20:25:19 UTC; 21s ago
  Process: 893 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=0/SUCCESS)
 Main PID: 893 (code=exited, status=0/SUCCESS)

[Michael Vogt (mvo)]

This sounds a lot like ppp is started before the snappy workarounds run. We need to run the fixes earlier I guess.

[Leo Arias (elopio)]

flashed 15.04 edge #178, the ppp service doesn't start. updated to #182, reboot, the service starts. All good now!

Thanks longsleep for the report. Thanks mvo for the fix. Beers for everybody!

[Leo Arias (elopio)]

ugh, there's something wrong with this in the alpha.

Flashed 15.04 alpha #11, the ppp service doesn't start. updated to alpha #12, reboot and it still doesn't start. Restart the service and it works. What could be missing in the alpha #12 that's present in edge #182?

[Michael Vogt (mvo)]

Thanks Leo! I found the issue why the workaround was not correctly applied in the right order and fixed with with the latest image. The fix should be in 15.04-alpha r185.