MAAS

maas-dhcp-2.0.0b3+ does not start missing dhcpd.conf dhcpd-interfaces /run/maas not writable by dhcpd

Bug #1592540 reported by Jeff Kowalczyk on 2016-06-14

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	MAAS	Won't Fix	Undecided	Unassigned

Bug Description

Package maas-dhcp should include and configure usable default config files to allow it to start.

We have a fresh install of Ubuntu 16.04 LTS with MAAS PPA maas/next. Our first version installed was MAAS and specifically maas-dhcp-2.0.0~beta3+bzr4941-0ubuntu1, currently newer versions, see package info at bottom.

After install, the dhcp server does not start. It seems the following items are missing, at a minimum:

1. Missing /var/lib/maas/dhcpd-interfaces. A default file should be installed containing a generic entry to listen on all interfaces. Let the user customize the entry if limiting to one or more interfaces is desired.

$ sudo service maas-dhcpd start
$ sudo service maas-dhcpd status
● maas-dhcpd.service - MAAS instance of ISC DHCP server for IPv4
   Loaded: loaded (/lib/systemd/system/maas-dhcpd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2016-06-14 11:32:15 PDT; 4min 59s ago
Condition: start condition failed at Tue 2016-06-14 11:37:11 PDT; 3s ago
           ConditionPathExists=/var/lib/maas/dhcpd-interfaces was not met
     Docs: man:dhcpd(8)
Main PID: 30586 (code=exited, status=1/FAILURE)

Jun 14 11:32:15 maas systemd[1]: Starting MAAS instance of ISC DHCP server for IPv4...
Jun 14 11:32:15 maas systemd[1]: Started MAAS instance of ISC DHCP server for IPv4.
Jun 14 11:32:15 maas systemd[1]: maas-dhcpd.service: Main process exited, code=exited, status=1/FAILURE
Jun 14 11:32:15 maas systemd[1]: maas-dhcpd.service: Unit entered failed state.
Jun 14 11:32:15 maas systemd[1]: maas-dhcpd.service: Failed with result 'exit-code'.
Jun 14 11:36:08 maas systemd[1]: Stopped MAAS instance of ISC DHCP server for IPv4.

I have entered an interface name as the only content of the requested file:

$ cat /var/lib/maas/dhcpd-interfaces
enp3s0

2. Missing /var/lib/maas/dhcpd.conf. A default file should be installed containing a generic minimum config to boot:

$ sudo service maas-dhcpd start
$ sudo service maas-dhcpd status
● maas-dhcpd.service - MAAS instance of ISC DHCP server for IPv4
   Loaded: loaded (/lib/systemd/system/maas-dhcpd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2016-06-14 11:32:15 PDT; 6min ago
Condition: start condition failed at Tue 2016-06-14 11:38:47 PDT; 24s ago
           ConditionPathExists=/var/lib/maas/dhcpd.conf was not met
     Docs: man:dhcpd(8)
Main PID: 30586 (code=exited, status=1/FAILURE)

Using a blank dhcpd.conf, which I presume is not valid dhcpd configuration:

$ sudo touch /var/lib/maas/dhcpd.conf

The service will not start:

$ sudo service maas-dhcpd status
● maas-dhcpd.service - MAAS instance of ISC DHCP server for IPv4
   Loaded: loaded (/lib/systemd/system/maas-dhcpd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2016-06-14 12:33:19 PDT; 2s ago
     Docs: man:dhcpd(8)
  Process: 3706 ExecStart=/bin/sh -ec INTERFACES=$(cat /var/lib/maas/dhcpd-interfaces); LEASES_FILE=/var/lib/maas/dhcp/dhcpd.leases; [ -e $LEASES_FILE ] |
  Process: 3702 ExecStartPre=/bin/mkdir -p /var/lib/maas/dhcp (code=exited, status=0/SUCCESS)
  Process: 3697 ExecStartPre=/bin/mkdir -p /run/maas/dhcp (code=exited, status=0/SUCCESS)
Main PID: 3706 (code=exited, status=1/FAILURE)

Jun 14 12:33:18 maas systemd[1]: Starting MAAS instance of ISC DHCP server for IPv4...
Jun 14 12:33:19 maas systemd[1]: Started MAAS instance of ISC DHCP server for IPv4.
Jun 14 12:33:19 maas dhcpd[3706]: Wrote 0 leases to leases file.
Jun 14 12:33:19 maas systemd[1]: maas-dhcpd.service: Main process exited, code=exited, status=1/FAILURE
Jun 14 12:33:19 maas systemd[1]: maas-dhcpd.service: Unit entered failed state.

or more precisely it exits immediately with exit code 1. Expanding out the last line of the service file:

$ sudo dhcpd -user dhcpd -group dhcpd -f -q -4 -pf /run/maas/dhcp/dhcpd.pid -cf /var/lib/maas/dhcpd.conf -lf /var/lib/maas/dhcp/dhcpd.leases enp3s0
$ echo $?
1

An example showing basic valid config for a range should be included in the package installed documentation. The current documentation https://maas.ubuntu.com/docs/dhcpsnippets.html does not provide any examples of what should go into a valid snippet, link to pages with more information.

3. /run/maas/dhcp is owned by root, but not writable by user dhcpd

Inspecting the service definition, it clearly states that the PID file will be written by user dhcpd. The directory as created by the run attempts above is owned by root, and does not have the necessary permissions for user dhcpd to write to the directory:

$ ls -al /run/maas/dhcp
total 0
drwxr-xr-x 2 root root 40 Jun 14 12:33 .
drwxr-xr-x 4 root root 80 Jun 14 12:33 ..

$ cat /lib/systemd/system/maas-dhcpd.service
[Unit]
Description=MAAS instance of ISC DHCP server for IPv4
Documentation=man:dhcpd(8)
Wants=network-online.target
After=network-online.target
After=time-sync.target
BindsTo=maas-rackd.service
ConditionPathExists=/var/lib/maas/dhcpd.conf
ConditionPathExists=/var/lib/maas/dhcpd-interfaces

[Service]
# Allow dhcp server to write lease and pid file as 'dhcpd' user
ExecStartPre=/bin/mkdir -p /run/maas/dhcp
# The leases files need to be root:dhcpd even when dropping privileges
ExecStartPre=/bin/mkdir -p /var/lib/maas/dhcp
# Start the daemon
ExecStart=/bin/sh -ec '\
    INTERFACES=$(cat /var/lib/maas/dhcpd-interfaces); \
    LEASES_FILE=/var/lib/maas/dhcp/dhcpd.leases; \
    [ -e $LEASES_FILE ] || touch $LEASES_FILE; \
    /usr/sbin/maas-dhcp-helper clean $LEASES_FILE; \
    chown root:dhcpd /var/lib/maas/dhcp $LEASES_FILE; \
    chmod 775 /var/lib/maas/dhcp ; chmod 664 $LEASES_FILE; \
    exec dhcpd -user dhcpd -group dhcpd -f -q -4 -pf /run/maas/dhcp/dhcpd.pid \
        -cf /var/lib/maas/dhcpd.conf -lf $LEASES_FILE $INTERFACES'

[Install]
WantedBy=multi-user.target

Workaround attempted, unsuccessful:

$ sudo chmod 775 /run/maas/dhcp
$ ls -al /run/maas/dhcp
total 0
drwxrwxr-x 2 root root 40 Jun 14 12:33 .
drwxr-xr-x 4 root root 80 Jun 14 12:33 ..
$ sudo service maas-dhcpd start
$ sudo service maas-dhcpd status
● maas-dhcpd.service - MAAS instance of ISC DHCP server for IPv4
   Loaded: loaded (/lib/systemd/system/maas-dhcpd.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2016-06-14 12:40:45 PDT; 1s ago
     Docs: man:dhcpd(8)
  Process: 4132 ExecStart=/bin/sh -ec INTERFACES=$(cat /var/lib/maas/dhcpd-interfaces); LEASES_FILE=/var/lib/maas/dhcp/dhcpd.leases; [ -e $LEASES_FILE ] |
  Process: 4128 ExecStartPre=/bin/mkdir -p /var/lib/maas/dhcp (code=exited, status=0/SUCCESS)
  Process: 4125 ExecStartPre=/bin/mkdir -p /run/maas/dhcp (code=exited, status=0/SUCCESS)
Main PID: 4132 (code=exited, status=1/FAILURE)

Jun 14 12:40:45 maas systemd[1]: Starting MAAS instance of ISC DHCP server for IPv4...
Jun 14 12:40:45 maas systemd[1]: Started MAAS instance of ISC DHCP server for IPv4.
Jun 14 12:40:45 maas dhcpd[4132]: Wrote 0 leases to leases file.
Jun 14 12:40:45 maas systemd[1]: maas-dhcpd.service: Main process exited, code=exited, status=1/FAILURE
Jun 14 12:40:45 maas systemd[1]: maas-dhcpd.service: Unit entered failed state.
Jun 14 12:40:45 maas systemd[1]: maas-dhcpd.service: Failed with result 'exit-code'.

Currently installed versions:

$ dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-====================================-============-=================================================
ii maas 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS server common files
ii maas-dhcp 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS DHCP server
ii maas-dns 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS DNS server
ii maas-proxy 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS Caching Proxy
ii maas-rack-controller 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all Rack Controller for MAAS
ii maas-region-api 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all Region controller API service for MAAS
ii maas-region-controller 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all Region Controller for MAAS
un maas-region-controller-min <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisioningserver <none> <none> (no description available)
ii python3-django-maas 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.0.0~beta7+bzr5112-0ubuntu1~xenial1 all MAAS server provisioning libraries (Python 3)

Tags:

Revision history for this message

Jeff Kowalczyk (jfkw) wrote on 2016-06-14:

contents of /var/log/maas/* Edit (267.6 KiB, application/x-tar)

Revision history for this message

Blake Rouse (blake-rouse) wrote on 2016-06-14:

This is explicitly designed this way. MAAS allows you to have multiple DHCP servers managed by different rack controllers. The DHCP server can quickly be moved from one rack controller to another and MAAS will handle all of this. It will render the correct configuration on the new rack controller and remove the old configuration from the previous rack controller. This prevents administrators from starting the DHCP server when it is now being managed by another rack controller.

Also MAAS does not run a DHCP server by default so it cannot ship with a DHCP server configuration that can run. It only runs a DHCP server when you tell MAAS to turn it on.

The DHCP service also cannot be running unless rackd is running that is why maas-dhcpd is BindTo=maas-rackd. This is because the DHCP server sends notification messages to the maas-rackd process when it commits, releases, or expires a lease.

You need to configure MAAS using the API or UI to turn on the DHCP service. MAAS will then monitor that service, handle notifications, and control/ensure that service is doing what it should be doing.

Changed in maas:
status:	New → Won't Fix

Revision history for this message

Jeff Kowalczyk (jfkw) wrote on 2016-06-14:

Thank you for educating me on that. Can you point me to where in the MAAS 2.0 controller UI the DHCP service can be enabled?

And for the CLI option, can you provide any instructions targeting 2.0 equivalent to:
https://maas.ubuntu.com/docs/maascli.html#node-group-interface

Revision history for this message

Andres Rodriguez (andreserl) wrote on 2016-06-15: Re: [Bug 1592540] Re: maas-dhcp-2.0.0b3+ does not start missing dhcpd.conf dhcpd-interfaces /run/maas not writable by dhcpd

Hi Jeff,

You can look at:

http://maas.ubuntu.com/docs2.0/rack-configuration.html#enabling-a-dhcp-on-a-vlan-optional-ha

The CLI documentation is still in process of being updated.

Thanks!

On Tue, Jun 14, 2016 at 7:01 PM, Jeff Kowalczyk <email address hidden>
wrote:

> Thank you for educating me on that. Can you point me to where in the
> MAAS 2.0 controller UI the DHCP service can be enabled?
>
> And for the CLI option, can you provide any instructions targeting 2.0
> equivalent to:
> https://maas.ubuntu.com/docs/maascli.html#node-group-interface
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1592540
>
> Title:
> maas-dhcp-2.0.0b3+ does not start missing dhcpd.conf dhcpd-interfaces
> /run/maas not writable by dhcpd
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1592540/+subscriptions
>