isc-dhcp-server: Can't create PID file /run/dhcp-server/dhcpd.pid: Permission denied

[Ralf G. R. Bergs]

Just upgraded from 14-10 to 15-04, and now see the following in syslog:

Apr 26 10:50:08 server kernel: [70470.960718] audit: type=1400 audit(1430045408.725:8): apparmor="DENIED" operation="capable" profile="/usr/sbin/dhcpd" pid=8619 comm="dhcpd" capability=1 capname="dac_ove
rride"
Apr 26 10:50:08 server sh[8619]: Can't create PID file /run/dhcp-server/dhcpd.pid: Permission denied.

Description: Ubuntu 15.04
Release: 15.04

isc-dhcp-server:
  Installed: 4.3.1-5ubuntu2

----------- 8x -----------------

# cat /etc/default/isc-dhcp-server

# Defaults for isc-dhcp-server initscript
# sourced by /etc/init.d/isc-dhcp-server
# installed at /etc/default/isc-dhcp-server by the maintainer scripts

#
# This is a POSIX shell fragment
#

# Path to dhcpd's config file (default: /etc/dhcp/dhcpd.conf).
#DHCPD_CONF=/etc/dhcp/dhcpd.conf

# Path to dhcpd's PID file (default: /var/run/dhcpd.pid).
#DHCPD_PID=/var/run/dhcpd.pid

# Additional options to start dhcpd with.
# Don't use options -cf or -pf here; use DHCPD_CONF/ DHCPD_PID instead
#OPTIONS=""

# On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
# Separate multiple interfaces with spaces, e.g. "eth0 eth1".
INTERFACES=""

----------- 8x -----------------

# ls -la /var/run
lrwxrwxrwx 1 root root 4 Oct 24 2013 /var/run -> /run

# ls -la /run/dhcp-server/
total 0
drwxr-xr-x 2 dhcpd dhcpd 40 Apr 26 10:59 .
drwxr-xr-x 34 root root 1060 Apr 26 11:33 ..

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jared Fernandez (jared-fernandez)]

I don't think it was correct to mark this as a duplicate of #985417. That bug affected Precise and Quantal and was fixed back in 2012. This bug is currently affecting users of Utopic in 2015.

[Jared Fernandez (jared-fernandez)]

s/Utopic/Vivid/

[Jared Fernandez (jared-fernandez)]

Still occurs in Wily.

[Stephen Pape (srpape)]

It looks like the line in /lib/systemd/system/isc-dhcp-server.service:

    exec dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf $CONFIG_FILE $INTERFACES'

 is hardcoded and ignores the DHCPD_PID variable from /etc/default/isc-dhcp-server.

  I think it should be:

    exec dhcpd -user dhcpd -group dhcpd -f -4 -pf $DHCPD_PID -cf $CONFIG_FILE $INTERFACES'

Then there are other issues with the file, like it always tries to run:

    ExecStartPre=/bin/chown dhcpd:dhcpd /run/dhcp-server

regardless of the specified path. Also, if you don't set the variable in /etc/default/isc-dhcp-server, no default is given and the service fails to start.

[Stephen Pape (srpape)]

Even after fixing the above problem, AppArmor seems to prevent a PID file from being written. I'm not very familiar with it, but I added:

capability dac_override,

to /etc/apparmor.d/usr.sbin.dhcpd near the other capabilities, restarted the service, and now I have a PID file.

[Honsan (honoratosantos-ti)]

I'm newbie on linux and I had this problem too. I think I solved it that way:

[Raphaël Jacquot (sxpert+ubuntu)]

here's the contents of a working isc-dhcp-server.service

[Jason Hunter (jhunterwu)]

I don't understand how this bug can just stand open? It's not even assigned? I'm even seeing this in 15.10. Why are things like this not being fixed? If Rahaels file is working, then why isn't it included when I apt upgrade?

[Simon Arlott (sa.me.uk)]

The directory /run/dhcp-server is already owned by dhcpd:dhcpd, but dhcpd is writing to it as root which requires CAP_DAC_OVERRIDE. The directory needs to be owned by root:root if dhcpd is going to write the PID file as root.

[Jason Penney (jpenney+ubuntu)]

This also affects isc-dhcp-server6 in the same way. It's service file would also need to be updated similarly.

[Wladimir Mutel (mwg)]

Why not fix this bug in Ubuntu Bionic where it still happens ?

[Nicorac (nicorac)]

Ubuntu 18.04 here, no apparmor.

Directory /run is on tempfs, so it is empty at start.
I suppose PID file /run/dhcp-server/dhcpd.pid can't be created because directory /run/dhcp-server is missing at startup and I haven't found where it will be created.

I've fixed systemd unit "/lib/systemd/system/isc-dhcp-server.service" by creating the needed "/run/dhcp-server" folder before starting dhcpd:
-----------------
[Unit]
Description=ISC DHCP IPv4 server
Documentation=man:dhcpd(8)
Wants=network-online.target
After=network-online.target
After=time-sync.target
ConditionPathExists=/etc/default/isc-dhcp-server
ConditionPathExists=|/etc/ltsp/dhcpd.conf
ConditionPathExists=|/etc/dhcp/dhcpd.conf

[Service]
EnvironmentFile=/etc/default/isc-dhcp-server
RuntimeDirectory=dhcp-server
# The leases files need to be root:dhcpd even when dropping privileges
ExecStartPre=/bin/mkdir -p /run/dhcp-server
ExecStart=/bin/sh -ec '\
    CONFIG_FILE=/etc/dhcp/dhcpd.conf; \
    if [ -f /etc/ltsp/dhcpd.conf ]; then CONFIG_FILE=/etc/ltsp/dhcpd.conf; fi; \
    [ -e /var/lib/dhcp/dhcpd.leases ] || touch /var/lib/dhcp/dhcpd.leases; \
    chown root:dhcpd /var/lib/dhcp /var/lib/dhcp/dhcpd.leases; \
    chmod 775 /var/lib/dhcp ; chmod 664 /var/lib/dhcp/dhcpd.leases; \
    if [ ! -d /run/dhcp-server ]; then mkdir -p /run/dhcp-server ; chown dhcpd /run/dhcp-server ; chmod 775 /run/dhcp-server; fi; \
    exec dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf $CONFIG_FILE $INTERFACES'

[Install]
WantedBy=multi-user.target
-----------------

Still need to fix hardcoded PID filename instead of /etc/default/isc-dhcp-server value...

[Nicorac (nicorac)]

After a few investigation I found that my previous fix does not fix the bug at boot, at least on my Ubuntu 18.04 server.

There's no need to manually create the /run/dhcp-server folder since dhcpd automatically creates the needed PID file parent folders by itself.

It only happens at boot so the bug shoud be somewhere else, maybe something in startup sequence/dependencies.
With the original .service file, PID file /run/dhcp-server/dhcp.pid is not created (but server is running); manually restarting the service after boot will create the PID file correctly.

I've workarounded the bug by hardcoding PID file names WITHOUT folder:
/lib/systemd/system/isc-dhcp-server.service ==> "-pf /run/dhcp.pid"
/lib/systemd/system/isc-dhcp-server6.service ==> "-pf /run/dhcp6.pid"

and it works correctly after boot.

[Le corre (mulbzh)]

same problem for me with ubuntu 18. The PID is not create !

no update for this case ?

[Jeff Reeves (seveerrm)]

I'm encountering the same thing on a fresh installation of Ubuntu 20.04.

Here is what happens right after a reboot:
jeff@bridges:~$ ll /run/ | grep dhcp
jeff@bridges:~$ sudo systemctl status isc-dhcp-server
[sudo] password for jeff:
● isc-dhcp-server.service - ISC DHCP IPv4 server
     Loaded: loaded (/lib/systemd/system/isc-dhcp-server.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2020-04-28 23:59:43 MST; 11s ago
       Docs: man:dhcpd(8)
   Main PID: 1374 (dhcpd)
      Tasks: 4 (limit: 19047)
     Memory: 6.9M
     CGroup: /system.slice/isc-dhcp-server.service
             └─1374 dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf

Apr 28 23:59:44 bridges dhcpd[1374]:
Apr 28 23:59:44 bridges dhcpd[1374]: Listening on LPF/enp2s0/2c:60:0c:c6:a4:be/192.168.1.0/24
Apr 28 23:59:44 bridges sh[1374]: Listening on LPF/enp2s0/2c:60:0c:c6:a4:be/192.168.1.0/24
Apr 28 23:59:44 bridges sh[1374]: Sending on LPF/enp2s0/2c:60:0c:c6:a4:be/192.168.1.0/24
Apr 28 23:59:44 bridges sh[1374]: Sending on Socket/fallback/fallback-net
Apr 28 23:59:44 bridges sh[1374]: Can't create PID file /run/dhcp-server/dhcpd.pid: No such file or directory.
Apr 28 23:59:44 bridges dhcpd[1374]: Sending on LPF/enp2s0/2c:60:0c:c6:a4:be/192.168.1.0/24
Apr 28 23:59:44 bridges dhcpd[1374]: Sending on Socket/fallback/fallback-net
Apr 28 23:59:44 bridges dhcpd[1374]: Can't create PID file /run/dhcp-server/dhcpd.pid: No such file or directory.
Apr 28 23:59:44 bridges dhcpd[1374]: Server starting service.
jeff@bridges:~$ sudo systemctl restart isc-dhcp-server
jeff@bridges:~$ ll /run/ | grep dhcp
drwxr-xr-x 2 root root 60 Apr 29 00:00 dhcp-server/
jeff@bridges:~$ ll /run/dhcp-server/
total 4
drwxr-xr-x 2 root root 60 Apr 29 00:00 ./
drwxr-xr-x 39 root root 1140 Apr 29 00:00 ../
-rw-r--r-- 1 root root 5 Apr 29 00:00 dhcpd.pid

I can hardcode the PID file names like Nicorac above mentions, but I'd rather things just work out of the box without needing to apply random workarounds like that.

[Jeff Reeves (seveerrm)]

So I found a comment at https://unix.stackexchange.com/a/533900/253840

> My problem was that I had two services using same RuntimeDirectory (isc-dhcp-server and isc-dhcp-server6), but I configured only one to work. So when the second one died, its runtime directory got removed, making it a problem for the first service.

Since I have only the IPv4 addresses configured as well, I decided to look into this.

When I tested this out by changing the RuntimeDirectory to "dhcp-server6" and the exec line's PID file parameter to "-pf /run/dhcp-server6/dhcpd6.pid", everything worked as it should on a reboot. The PID file was created for /run/dhcp-server/dhcpd.pid successfully.

[Bob Morane (bobmorane06)]

Same problem encountered on Ubuntu 20.04.

Fixed after I ran
`sudo systemctl mask isc-dhcp-server6.service`

[LyckeleB (lyckeleb)]

I had two problems :
   1) no PID-file created and
   2) I couldn't assign a NIC to "isc-dhcp-server"aka "dhcpd".

It turned out that the PID-file has to be "/var/run/dhcp-server/dhcpd.pid". If not, apparmor wouldn't allow the creation of the PID-file.

And the the file defining the originally installed dhcpd-service script ("/usr/lib/systemd/system/isc-dhcp-server.service") is having other erroneous fixed paths.

Solution
---------
So I copied "/usr/lib/systemd/system/isc-dhcp-server.service" to "/etc/systemd/system/" and modified it into the one shown below. This solved my problems.

[Unit]
Description=ISC DHCP IPv4 server
Documentation=man:dhcpd(8)
Wants=network-online.target
After=network-online.target
After=time-sync.target
ConditionPathExists=/etc/default/isc-dhcp-server
ConditionPathExists=|/etc/ltsp/dhcpd.conf
ConditionPathExists=|/etc/dhcp/dhcpd.conf

[Service]
EnvironmentFile=/etc/default/isc-dhcp-server
RuntimeDirectory=dhcp-server
# The leases files need to be root:dhcpd even when dropping privileges
ExecStart=/bin/sh -ec '\
    CONFIG_FILE=/etc/dhcp/dhcpd.conf; \
    if [ -f /etc/ltsp/dhcpd.conf ]; then CONFIG_FILE=/etc/ltsp/dhcpd.conf; fi; \
    if [ ! -d /var/run/dhcp-server ]; then mkdir -p /var/run/dhcp-server ; chown dhcpd:dhcpd /var/run/dhcp-server ; chmod u=rwx,go=rx /var/run/dhcp-server ; fi; \
    if [ ! "$DHCPDv4_PID" ] ; then DHCPDv4_PID=/var/run/dhcp-server/dhcpd.pid; fi; \
    [ -e /var/lib/dhcp/dhcpd.leases ] || touch /var/lib/dhcp/dhcpd.leases; \
    chown root:dhcpd /var/lib/dhcp /var/lib/dhcp/dhcpd.leases; \
    chmod 775 /var/lib/dhcp ; chmod 664 /var/lib/dhcp/dhcpd.leases; \
    exec dhcpd -user dhcpd -group dhcpd -f -4 -pf $DHCPDv4_PID -cf $CONFIG_FILE $INTERFACESv4'

[Install]
WantedBy=multi-user.target

[LyckeleB (lyckeleb)]

However, the service still couldn't create a PID-file after a reboot. So I tweaked a little bit longer.

It turns out that the isc-dhcp-server is started before the temporary filesystem "/run".
The lines "Wants=local-fs.target", "After=local-fs.target" and "sleep 3 ; \" solves that.

Security by apparmor requires the creation of the PID-file in "/{,var/}run/{,dhcp-server/}dhcpd{,6}.pid rw,". The lines
    "if [ ! -d /run/dhcp-server ]; then mkdir -p /run/dhcp-server; fi; \"
    "chown root:dhcpd /run/dhcp-server; \"
    "chmod ug=rwx,o=rx /run/dhcp-server; \"
    "sleep 1; \"
enables this.

This tweaking resulted in the following "/etc/systemd/system/isc-dhcp-server.services" file.

========================================================

[Unit]
Description=ISC DHCP IPv4 server
Documentation=man:dhcpd(8)
Wants=network-online.target
Wants=local-fs.target
After=network-online.target
After=local-fs.target
After=time-sync.target
ConditionPathExists=/etc/default/isc-dhcp-server
ConditionPathIsMountPoint=/run
ConditionPathExists=|/etc/ltsp/dhcpd.conf
ConditionPathExists=|/etc/dhcp/dhcpd.conf

[Service]
EnvironmentFile=/etc/default/isc-dhcp-server
RuntimeDirectory=dhcp-server
# The leases files need to be root:dhcpd even when dropping privileges
ExecStart=/bin/sh -ec '\
    sleep 3 ; \
    CONFIG_FILE=/etc/dhcp/dhcpd.conf; \
    if [ -f /etc/ltsp/dhcpd.conf ]; then CONFIG_FILE=/etc/ltsp/dhcpd.conf; fi; \
    if [ ! -d /run/dhcp-server ]; then mkdir -p /run/dhcp-server; fi; \
    chown root:dhcpd /run/dhcp-server; \
    chmod ug=rwx,o=rx /run/dhcp-server; \
    sleep 1; \
    if [ ! "$DHCPDv4_PID" ] ; then DHCPDv4_PID=/run/dhcp-server/dhcpd.pid; fi; \
    [ -e /var/lib/dhcp/dhcpd.leases ] || touch /var/lib/dhcp/dhcpd.leases; \
    chown root:dhcpd /var/lib/dhcp /var/lib/dhcp/dhcpd.leases; \
    chmod 775 /var/lib/dhcp ; chmod 664 /var/lib/dhcp/dhcpd.leases; \
    exec dhcpd -user dhcpd -group dhcpd -f -4 -pf $DHCPDv4_PID -cf $CONFIG_FILE $INTERFACESv4'

[Install]
WantedBy=multi-user.target

==================================================

I hope this helps. Any suggestion to improve this service-file is appreciated :)

[dienteperro (dienteperro1207)]

This bug exists in Ubuntu 20.04 focal.

[CvB (cvb-kruemel)]

Same here with 22.04 still. The solution by @lyckeleb appears to be working.

[Yuriy Tabolin (olddanmer)]

Have the same bug is in Ubuntu 20.04.5 and isc-dhcp-server 4.4.1-2.1ubuntu5.20.04.4

It's very sad that bug is still haven't been fixed since 2015(!) year.

For myself I solve it with rc.local:
sleep 5
/usr/bin/systemctl restart isc-dhcp-server