[2.4] Rack controller interfaces don't have a subnet associated with them

[Eric Vasquez]

MAAS is failing to start DHCP with the following error:
["dhcp can only be turned on when a primary rack controlleris set."]

The console output contains debug info about the rack controllers.
Apparently there are no subnets associated with them, even though the subnets are specified in the infra.

Console failure message here:
https://pastebin.canonical.com/p/t3g9ZQW7PS/

[Jason Hobbs (jason-hobbs)]

reproduced again on 2.4.0~beta2-6783-g2739ac4-0ubuntu1~18.04.1~20180402~ubuntu18.04.1

[Andres Rodriguez (andreserl)]

Hi Jason/Eric,

Does this happen before or after you move the subnet from one VLAN to the other ?

[Mike Pontillo (mpontillo)]

A few things:

(0) Can you attach the output of the following:

    sudo maas-region dbshell
    \pset pager off
    SELECT * from maas_support__node_networking;
    SELECT f.name, v.vid, s.cidr FROM maasserver_fabric f, maasserver_vlan v, maasserver_subnet s WHERE v.fabric_id = f.id AND s.vlan_id = v.id;
    \q

(1) Can you check if restarting MAAS allows the infrastructure to be properly re-discovered, thus working around the issue?

(2) Can you attach the output of `sudo maas-rack support-dump --networking`?

Thanks.

[Jason Hobbs (jason-hobbs)]

re #3 This happens before we move subnets.

re #4.0: We hit this during automated testing. Please provide something that can be run non-interactively.

re #4.1 We've already restarted maas-regiond and maas-rackd before we've got this far

re #4.2: Ok, we can add that.

[Mike Pontillo (mpontillo)]

Here's a script that automates the SQL queries I mentioned in my previous comment.

https://paste.ubuntu.com/p/qGjx339BBV/

[Andres Rodriguez (andreserl)]

setting this bug report to incomplete because the requested information has not been provided

[Andres Rodriguez (andreserl)]

 hostname | ifid | name | type | mac_address | ip | alloc_type | cidr | vid | fabric
----------+------+--------+----------+-------------------+----+------------+------+-----+--------
 leafeon | 1 | eno1 | physical | 1c:98:ec:21:98:54 | | | | |
 leafeon | 2 | eno2 | physical | 1c:98:ec:21:98:55 | | | | |
 leafeon | 3 | eno3 | physical | 1c:98:ec:21:98:56 | | | | |
 leafeon | 4 | eno4 | physical | 1c:98:ec:21:98:57 | | | | |
 leafeon | 5 | eno49 | physical | 8c:dc:d4:b3:08:0c | | | | |
 leafeon | 6 | eno50 | physical | 8c:dc:d4:b3:08:0d | | | | |
 leafeon | 7 | ens1f0 | physical | 00:11:0a:68:67:c8 | | | | |
 leafeon | 8 | ens1f1 | physical | 00:11:0a:68:67:c9 | | | | |
 meinfoo | 9 | eno1 | physical | 14:02:ec:42:28:bc | | | | |
 meinfoo | 10 | eno2 | physical | 14:02:ec:42:28:bd | | | | |
 meinfoo | 11 | eno3 | physical | 14:02:ec:42:28:be | | | | |
 meinfoo | 12 | eno4 | physical | 14:02:ec:42:28:bf | | | | |
 meinfoo | 13 | eno49 | physical | 5c:b9:01:9d:09:fc | | | | |
 meinfoo | 14 | eno50 | physical | 5c:b9:01:9d:09:fd | | | | |
 meinfoo | 15 | ens1f0 | physical | 00:11:0a:66:2d:6c | | | | |
 meinfoo | 16 | ens1f1 | physical | 00:11:0a:66:2d:6d | | | | |
 swoobat | 17 | eno1 | physical | 14:02:ec:40:fd:9c | | | | |
 swoobat | 18 | eno2 | physical | 14:02:ec:40:fd:9d | | | | |
 swoobat | 19 | eno3 | physical | 14:02:ec:40:fd:9e | | | | |
 swoobat | 20 | eno4 | physical | 14:02:ec:40:fd:9f | | | | |
 swoobat | 21 | eno49 | physical | 5c:b9:01:9b:56:64 | | | | |
 swoobat | 22 | eno50 | physical | 5c:b9:01:9b:56:65 | | | | |
 swoobat | 23 | ens1f0 | physical | 00:11:0a:68:6e:b4 | | | | |
 swoobat | 24 | ens1f1 | physical | 00:11:0a:68:6e:b5 | | | | |
(24 rows)

  name | vid | cidr
---------+------+-----------------
 default | 0 | 10.244.40.0/21
 default | 2736 | 192.168.36.0/26
 default | 2733 | 192.168.33.0/24
 default | 2734 | 10.244.32.0/21
 default | 2735 | 192.168.35.0/26
(5 rows)

[Andres Rodriguez (andreserl)]

### get_ip_addr() ###
{'broam': {'bridged_interfaces': ['eno1'],
           'flags': ['BROADCAST', 'MULTICAST', 'UP', 'LOWER_UP'],
           'index': 10,
           'inet': ['10.244.40.31/21'],
           'mac': 'd2:66:98:72:48:73',
           'name': 'broam',
           'settings': {'group': 'default',
                        'mtu': '1500',
                        'qdisc': 'noqueue',
                        'qlen': '1000',
                        'state': 'UP'},
           'type': 'ethernet.bridge'},
 'eno1': {'flags': ['BROADCAST', 'MULTICAST', 'UP', 'LOWER_UP'],
          'index': 2,
          'mac': '14:02:ec:42:28:bc',
          'name': 'eno1',
          'settings': {'group': 'default',
                       'master': 'broam',
                       'mtu': '1500',
                       'qdisc': 'mq',
                       'qlen': '1000',
                       'state': 'UP'},
          'type': 'ethernet.physical'},
 'eno2': {'flags': ['BROADCAST', 'MULTICAST'],
          'index': 3,
          'mac': '14:02:ec:42:28:bd',
          'name': 'eno2',
          'settings': {'group': 'default',
                       'mtu': '1500',
                       'qdisc': 'noop',
                       'qlen': '1000',
                       'state': 'DOWN'},
          'type': 'ethernet.physical'},
 'eno3': {'flags': ['BROADCAST', 'MULTICAST'],
          'index': 5,
          'mac': '14:02:ec:42:28:be',
          'name': 'eno3',
          'settings': {'group': 'default',
                       'mtu': '1500',
                       'qdisc': 'noop',
                       'qlen': '1000',
                       'state': 'DOWN'},
          'type': 'ethernet.physical'},
 'eno4': {'flags': ['BROADCAST', 'MULTICAST'],
          'index': 6,
          'mac': '14:02:ec:42:28:bf',
          'name': 'eno4',
          'settings': {'group': 'default',
                       'mtu': '1500',
                       'qdisc': 'noop',
                       'qlen': '1000',
                       'state': 'DOWN'},
          'type': 'ethernet.physical'},
 'eno49': {'flags': ['BROADCAST', 'MULTICAST', 'UP', 'LOWER_UP'],
           'index': 4,
           'inet': ['10.245.219.23/20'],
           'mac': '5c:b9:01:9d:09:fc',
           'name': 'eno49',
           'settings': {'group': 'default',
                        'mtu': '1500',
                        'qdisc': 'mq',
                        'qlen': '1000',
                        'state': 'UP'},
           'type': 'ethernet.physical'},
 'eno50': {'flags': ['BROADCAST', 'MULTICAST'],
           'index': 7,
           'mac': '5c:b9:01:9d:09:fd',
           'name': 'eno50',
           'settings': {'group': 'default',
                        'mtu': '1500',
                        'qdisc': 'noop',
                        'qlen': '1000',
                        'state': 'DOWN'},
           'type': 'ethernet.physical'},
 'ens1f0': {'flags': ['BROADCAST', 'MULTICAST'],
            'index': 8,
            'mac': '00:11:0a:66:2d:6c',
            'name': 'ens1f0',
            'settings': {'group': 'default',
                         'mtu': '1500',
                         'qdisc': 'noop',
                         'qlen': '1000',
    ...

[Andres Rodriguez (andreserl)]

in another environment

2018-04-09 19:07:25 provisioningserver.utils.services: [critical] Failed to update and/or record network interface configuration: Connection was closed cleanly.; interfaces: {'eno1': {'type': 'physical', 'index': 2, 'links': [], 'enabled': True, 'parents': [], 'source': 'ipaddr', 'mac_address': 'ec:a8:6b:fd:ac:70', 'monitored': True}, 'usb0': {'type': 'physical', 'index': 4, 'links': [], 'enabled': True, 'parents': [], 'source': 'ipaddr', 'mac_address': '00:0e:c6:88:b7:9f', 'monitored': True}, 'br1': {'type': 'bridge', 'index': 5, 'links': [{'mode': 'static', 'address': '192.168.1.13/24', 'gateway': '192.168.1.1'}], 'enabled': True, 'parents': ['eno1'], 'source': 'ipaddr', 'mac_address': '62:32:23:3b:3d:c8', 'monitored': False}, 'br0': {'type': 'bridge', 'index': 6, 'links': [{'mode': 'static', 'address': '10.90.90.1/24'}, {'mode': 'static', 'address': '192.168.100.1/24'}], 'enabled': True, 'parents': ['usb0'], 'source': 'ipaddr', 'mac_address': 'f2:a0:90:00:88:34', 'monitored': False}, 'wlp2s0': {'type': 'physical', 'index': 11, 'links': [], 'enabled': False, 'parents': [], 'source': 'ipaddr', 'mac_address': 'c4:d9:87:5b:42:62', 'monitored': False}}
        Traceback (most recent call last):
        Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.

[Andres Rodriguez (andreserl)]

We believe this issue has been fixed with as a side effect. Could you please test the latest MAAS for dailybuils?

[1]: https://code.launchpad.net/~blake-rouse/maas/+git/maas/+merge/342811

[Adam Collard (adam-collard)]

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

MAAS Team