2.1 DHCP reported as enabled but not running

[John George]

After installing and configuring 2.1.0~beta1+bzr5411-0ubuntu1~16.04.1, DHCP is reported by the GUI as enabled but it's really not.
Please see attached logs.

$ systemctl -a | grep maas
  maas-dhcpd.service loaded inactive dead MAAS instance of ISC DHCP server for IPv4
  maas-dhcpd6.service loaded inactive dead MAAS instance of ISC DHCP server for IPv6
  maas-proxy.service loaded active running MAAS Proxy
  maas-rackd.service loaded active running MAAS Rack Controller
  [e-mail] loaded active running MAAS Region Controller (Worker 1)
  [e-mail] loaded active running MAAS Region Controller (Worker 2)
  [e-mail] loaded active running MAAS Region Controller (Worker 3)
  [e-mail] loaded active running MAAS Region Controller (Worker 4)
  maas-regiond.service loaded active exited MAAS Region Controller
  system-maas\x2dregiond\x2dworker.slice loaded active active system-maas\x2dregiond\x2dworker.slice

[Andres Rodriguez (andreserl)]

John,

Can you provide an step by step to reproduce? The only thing I see is that the rack failed to import images, but that may be causing your rack to not work.

Also, can you confirm:

1. DHCP was enabled for the VLAN
2. The UI shows DHCP is active
3. Have you waited a few to see if the rack start dhcp ? there's some situations in which the rack will wait a few seconds to start DHCP.

[John George (jog)]

>1. DHCP was enabled for the VLAN
Here is evidence from the our Jenkins job, that DHCP was enabled:
2016-09-29-21:23:54 INFO 'maas maas vlan update fabric-0 untagged dhcp_on=true primary_rack=gckmmf'
2016-09-29-21:23:56 INFO {
    "mtu": 1500,
    "id": 5001,
    "dhcp_on": true,
    "vid": 0,
    "name": "untagged",
    "secondary_rack": null,
    "external_dhcp": null,
    "fabric": "fabric-0",
    "primary_rack": "gckmmf",
    "resource_uri": "/MAAS/api/2.0/vlans/5001/"
}
2016-09-29-21:23:56 INFO
2016-09-29-21:23:56 DEBUG {
    "name": "untagged",
    "vid": 0,
    "dhcp_on": true,
    "mtu": 1500,
    "primary_rack": "gckmmf",
    "external_dhcp": null,
    "resource_uri": "/MAAS/api/2.0/vlans/5001/",
    "id": 5001,
    "secondary_rack": null,
    "fabric": "fabric-0"
}
>2. The UI shows DHCP is active
Yes, in the UI DHCP was listed as enabled. I also tried disabling and then re-enabling DHCP.
>3. Have you waited a few to see if the rack start dhcp ? there's some situations in which the rack will wait a few seconds to start DHCP.
Yes, the server had been running for many minutes, while we investigated. I also manually tried restarting maas-rackd.service and then maas-dhcpd.service (e.g. sudo systemctl restart maas-rackd)

[Andres Rodriguez (andreserl)]

John,

Can you please attach the systemd journal for Maas-dhcp? Also confirm you
are using 2.1a4?

On Thursday, September 29, 2016, John George <email address hidden>
wrote:

> >1. DHCP was enabled for the VLAN
> Here is evidence from the our Jenkins job, that DHCP was enabled:
> 2016-09-29-21:23:54 INFO 'maas maas vlan update fabric-0 untagged
> dhcp_on=true primary_rack=gckmmf'
> 2016-09-29-21:23:56 INFO {
> "mtu": 1500,
> "id": 5001,
> "dhcp_on": true,
> "vid": 0,
> "name": "untagged",
> "secondary_rack": null,
> "external_dhcp": null,
> "fabric": "fabric-0",
> "primary_rack": "gckmmf",
> "resource_uri": "/MAAS/api/2.0/vlans/5001/"
> }
> 2016-09-29-21:23:56 INFO
> 2016-09-29-21:23:56 DEBUG {
> "name": "untagged",
> "vid": 0,
> "dhcp_on": true,
> "mtu": 1500,
> "primary_rack": "gckmmf",
> "external_dhcp": null,
> "resource_uri": "/MAAS/api/2.0/vlans/5001/",
> "id": 5001,
> "secondary_rack": null,
> "fabric": "fabric-0"
> }
> >2. The UI shows DHCP is active
> Yes, in the UI DHCP was listed as enabled. I also tried disabling and then
> re-enabling DHCP.
> >3. Have you waited a few to see if the rack start dhcp ? there's some
> situations in which the rack will wait a few seconds to start DHCP.
> Yes, the server had been running for many minutes, while we investigated.
> I also manually tried restarting maas-rackd.service and then
> maas-dhcpd.service (e.g. sudo systemctl restart maas-rackd)
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1629142
>
> Title:
> 2.1 DHCP reported as enabled but not running
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1629142/+subscriptions
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Blake Rouse (blake-rouse)]

2016-09-29 21:20:47 [-] Failed configuring DHCP on rack controller 'id:1'.
 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 434, in errback
     self._startRunCallbacks(fail)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
     self._runCallbacks()
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
     current.result = callback(current.result, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1184, in gotResult
     _inlineCallbacks(r, g, deferred)
 --- <exception caught here> ---
   File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
     result = result.throwExceptionIntoGenerator(g)
   File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
     return g.throw(self.type, self.value, self.tb)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 640, in configure_dhcp
     config = yield deferToDatabase(get_dhcp_configuration, rack_controller)
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 246, in inContext
     result = inContext.theWork()
   File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
     inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 834, in callInContext
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 211, in wrapper
     return func(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 603, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python3/dist-packages/maasserver/utils/orm.py", line 422, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python3.5/contextlib.py", line 30, in inner
     return func(*args, **kwds)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 556, in get_dhcp_configuration
     ntp_servers = get_ntp_server_addresses_for_rack(rack_controller)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 266, in get_ntp_server_addresses_for_rack
     rack_addresses, get_space_id_and_family)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 265, in <dictcomp>
     for space_id_and_family, group in groupby(
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 264, in <genexpr>
     (ip for _, _, ip in group), key=IPAddress)
   File "/usr/lib/python3/dist-packages/maasserver/dhcp.py", line 260, in get_space_id_and_family
     return space_id, IPNetwork(cidr).version
   File "/usr/lib/python3/dist-packages/netaddr/ip/__init__.py", line 923, in __init__
   ...

[Blake Rouse (blake-rouse)]

Looks like the cidr for the subnet is None, which is not possible, so something really strange is going on here.