MAAS

MAAS uses incorrect source IP address when running rndc - Internal server error

Bug #1466162 reported by mahmoh
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Invalid
Medium
Unassigned

Bug Description

Attempting to commission fails:

The commission action for 2 nodes failed with error: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1: rndc: connection to remote host closed This may indicate that * the remote server is using an older version of the command protocol, * this host is not authorized to connect, * the clocks are not synchronized, or * the key is invalid.

The commission action for 2 nodes failed with error: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1: rndc: recv failed: connection reset
Add machine

$ dpkg -l | grep maas
ii maas 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server all-in-one metapackage
ii maas-cli 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS command line API tool
ii maas-cluster-controller 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server cluster controller
ii maas-common 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server common files
ii maas-dhcp 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS DHCP server
ii maas-dns 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS DNS server
ii maas-proxy 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS Caching Proxy
ii maas-region-controller 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server complete region controller
ii maas-region-controller-min 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS Server minimum region controller
ii python-django-maas 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server Django web framework
ii python-maas-client 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS python API client
ii python-maas-provisioningserver 1.8.0~rc3+bzr4000-0ubuntu1~trusty1 all MAAS server provisioning libraries

$ Jun 17 13:20:42 maas maas.node: [INFO] junior-spot: Status transition from NEW to COMMISSIONING
Jun 17 13:20:42 maas maas.node: [INFO] brilliant-snail: Status transition from NEW to COMMISSIONING
Jun 17 13:20:42 maas maas.node: [INFO] tedious-fruit: Status transition from NEW to COMMISSIONING
Jun 17 13:20:42 maas maas.node: [WARNING] junior-spot: Unrecognised power type.
Jun 17 13:20:42 maas maas.node: [INFO] sharp-calculator: Status transition from NEW to COMMISSIONING
Jun 17 13:20:42 maas maas.node: [WARNING] brilliant-snail: Unrecognised power type.
Jun 17 13:20:42 maas maas.node: [WARNING] sharp-calculator: Unrecognised power type.
Jun 17 13:20:42 maas maas.dns: [INFO] Generating new DNS zone file for maas
Jun 17 13:20:42 maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: connection to remote host closed#012This may indicate that#012* the remote server is using an older version of the command protocol,#012* this host is not authorized to connect,#012* the clocks are not synchronized, or#012* the key is invalid.
Jun 17 13:20:42 maas maas.dns: [INFO] Generating new DNS zone file for maas
Jun 17 13:20:42 maas maas.node: [INFO] junior-spot: Status transition from COMMISSIONING to NEW
Jun 17 13:20:42 maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: recv failed: connection reset
Jun 17 13:20:42 maas maas.node: [ERROR] junior-spot: Could not start node for commissioning:
2015-06-17 13:20:42 [-] Error on request (24) node.action: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: connection to remote host closed
 This may indicate that
 * the remote server is using an older version of the command protocol,
 * this host is not authorized to connect,
 * the clocks are not synchronized, or
 * the key is invalid.
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 783, in __bootstrap
     self.__bootstrap_inner()
   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 763, in run
     self.__target(*self.__args, **self.__kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 404, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 300, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 312, in apply
     dns_update_zones_now(zones_to_update.values())
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 155, in dns_update_zones_now
     bind_reload_zone(zone.zone_name)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/actions.py", line 95, in bind_reload_zone
     execute_rndc_command(("reload", zone_name))
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/config.py", line 170, in execute_rndc_command
     call_and_check(rndc_cmd)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/shell.py", line 146, in call_and_check
     raise ExternalProcessError(process.returncode, command, output=stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: connection to remote host closed
 This may indicate that
 * the remote server is using an older version of the command protocol,
 * this host is not authorized to connect,
 * the clocks are not synchronized, or
 * the key is invalid.

Jun 17 13:20:42 maas maas.dns: [INFO] Generating new DNS zone file for maas
Jun 17 13:20:42 maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: connection to remote host closed#012This may indicate that#012* the remote server is using an older version of the command protocol,#012* this host is not authorized to connect,#012* the clocks are not synchronized, or#012* the key is invalid.
Jun 17 13:20:42 maas maas.node: [INFO] tedious-fruit: Status transition from COMMISSIONING to NEW
Jun 17 13:20:42 maas maas.dns: [INFO] Generating new DNS zone file for maas
Jun 17 13:20:43 maas maas.node: [ERROR] tedious-fruit: Could not start node for commissioning:
Jun 17 13:20:43 maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: recv failed: connection reset
Jun 17 13:20:43 maas maas.node: [INFO] sharp-calculator: Status transition from COMMISSIONING to NEW
2015-06-17 13:20:43 [-] Error on request (26) node.action: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: recv failed: connection reset
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 783, in __bootstrap
     self.__bootstrap_inner()
   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 763, in run
     self.__target(*self.__args, **self.__kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 404, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 300, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 312, in apply
     dns_update_zones_now(zones_to_update.values())
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 155, in dns_update_zones_now
     bind_reload_zone(zone.zone_name)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/actions.py", line 95, in bind_reload_zone
     execute_rndc_command(("reload", zone_name))
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/config.py", line 170, in execute_rndc_command
     call_and_check(rndc_cmd)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/shell.py", line 146, in call_and_check
     raise ExternalProcessError(process.returncode, command, output=stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: recv failed: connection reset

Jun 17 13:20:43 maas maas.node: [ERROR] sharp-calculator: Could not start node for commissioning:
2015-06-17 13:20:43 [-] Error on request (25) node.action: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: connection to remote host closed
 This may indicate that
 * the remote server is using an older version of the command protocol,
 * this host is not authorized to connect,
 * the clocks are not synchronized, or
 * the key is invalid.
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 783, in __bootstrap
     self.__bootstrap_inner()
   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 763, in run
     self.__target(*self.__args, **self.__kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 404, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 300, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 312, in apply
     dns_update_zones_now(zones_to_update.values())
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 155, in dns_update_zones_now
     bind_reload_zone(zone.zone_name)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/actions.py", line 95, in bind_reload_zone
     execute_rndc_command(("reload", zone_name))
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/config.py", line 170, in execute_rndc_command
     call_and_check(rndc_cmd)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/shell.py", line 146, in call_and_check
     raise ExternalProcessError(process.returncode, command, output=stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: connection to remote host closed
 This may indicate that
 * the remote server is using an older version of the command protocol,
 * this host is not authorized to connect,
 * the clocks are not synchronized, or
 * the key is invalid.

Jun 17 13:20:43 maas maas.node: [INFO] brilliant-snail: Status transition from COMMISSIONING to NEW
Jun 17 13:20:43 maas maas.node: [ERROR] brilliant-snail: Could not start node for commissioning:
2015-06-17 13:20:43 [-] Error on request (23) node.action: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: recv failed: connection reset
 Traceback (most recent call last):
   File "/usr/lib/python2.7/threading.py", line 783, in __bootstrap
     self.__bootstrap_inner()
   File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
     self.run()
   File "/usr/lib/python2.7/threading.py", line 763, in run
     self.__target(*self.__args, **self.__kwargs)
 --- <exception caught here> ---
   File "/usr/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 191, in _worker
     result = context.call(ctx, function, *args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
     return func(*args,**kw)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 404, in call_within_transaction
     return func_outside_txn(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/orm.py", line 300, in retrier
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/django/db/transaction.py", line 339, in inner
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 312, in apply
     dns_update_zones_now(zones_to_update.values())
   File "/usr/lib/python2.7/dist-packages/maasserver/utils/__init__.py", line 229, in call_with_lock
     return func(*args, **kwargs)
   File "/usr/lib/python2.7/dist-packages/maasserver/dns/config.py", line 155, in dns_update_zones_now
     bind_reload_zone(zone.zone_name)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/actions.py", line 95, in bind_reload_zone
     execute_rndc_command(("reload", zone_name))
   File "/usr/lib/python2.7/dist-packages/provisioningserver/dns/config.py", line 170, in execute_rndc_command
     call_and_check(rndc_cmd)
   File "/usr/lib/python2.7/dist-packages/provisioningserver/utils/shell.py", line 146, in call_and_check
     raise ExternalProcessError(process.returncode, command, output=stderr)
 provisioningserver.utils.shell.ExternalProcessError: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:
 rndc: recv failed: connection reset

Revision history for this message
Raphaël Badin (rvb) wrote :

Looks like the MAAS-controller DNS server is either down or not working properly. Can you please have a look at /var/log/syslog and see if you can spot the problem in there (and attach /var/log/syslog to this bug report)?

Maybe this is bug 1413388 (have a look at https://bugs.launchpad.net/maas/+bug/1413388/comments/8 too). This bug occurs when a configuration of the DNS server conflicts with the DNS config snippet that MAAS uses.

Changed in maas:
status: New → Incomplete
importance: Undecided → Critical
Revision history for this message
mahmoh (mahmoh) wrote :

1.8.0+bzr4001-0ubuntu2~trusty1:

/var/log/syslog:
...
Jul 21 22:36:09 addc-maas named[5691]: rejected command channel message from 172.27.80.1#42064
Jul 21 22:36:09 addc-maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: recv failed: connection reset
Jul 21 22:38:41 addc-maas maas.node: [INFO] card-node2: Status transition from NEW to COMMISSIONING
Jul 21 22:38:41 addc-maas maas.dns: [INFO] Generating new DNS zone file for maas
Jul 21 22:38:41 addc-maas named[5691]: rejected command channel message from 172.27.80.1#58801
Jul 21 22:38:41 addc-maas maas.dns: [ERROR] Reloading BIND zone u'maas' failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1:#012rndc: recv failed: connection reset
Jul 21 22:38:41 addc-maas maas.node: [INFO] card-node2: Status transition from COMMISSIONING to NEW
Jul 21 22:38:41 addc-maas maas.node: [ERROR] card-node2: Could not start node for commissioning:
Jul 21 22:40:34 addc-maas maas.power: [INFO] card-node2: Power state has changed from unknown to on.
Jul 21 22:41:34 addc-maas named[5691]: error (network unreachable) resolving 'org/DNSKEY/IN': 2001:500:48::1#53

Changed in maas:
status: Incomplete → New
Revision history for this message
mahmoh (mahmoh) wrote :

P.S. These were both clean installs of MAAS, setup the MAAS server as the router with forward and MASQ, interface eth0 (external) and p4p1 (internal MAAS dhcp/dns).

Revision history for this message
Mike Pontillo (mpontillo) wrote :

Thanks for the logs. It looks like (for some reason) BIND is refusing the RPC calls from its rndc client.

It might narrow it down if you can check the output of the following commands:

(1) sudo /usr/sbin/rndc -b 172.27.80.1 status
(2) sudo /usr/sbin/rndc -c /etc/bind/maas/rndc.conf.maas status
(3) sudo /usr/sbin/rndc -b 127.0.0.1 -c /etc/bind/maas/rndc.conf.maas status

If you take a look at /etc/bind/maas/rndc.conf.maas, you'll notice there is a secret key inside, which MAAS must use to authenticate with the BIND server. (This file is included in /etc/bind/maas/named.conf.maas, which should be included in named.conf.local.) You'll also notice that it includes access control to ensure that it *only* accepts connections from 127.0.0.1. One potential oddity here is that the source address seems to be an IP address other than 127.0.0.1. One approach I can think of is to add it to the access list in rndc.conf.maas, but it might be preferred to create an /etc/bind/rndc.conf such as:

options {
    default-server localhost;
};

Then try running command (2) above to see if it can communicate.

I'm curious what configuration might be causing this to happen, but ultimately it might be good for MAAS to add the "-b <source-address>" command to force binding to localhost (or, in the case that the MAAS region server is [one day] distributed and not on the same host as named, use the authorized IP address .)

Did you do any configuration tweaks to the DNS server (or maybe firewall, or address rewriting rules?) that may be relevant here?

summary: - commissioning error
+ MAAS uses incorrect source IP address when running rndc
Revision history for this message
mahmoh (mahmoh) wrote : Re: MAAS uses incorrect source IP address when running rndc

MAAS/node/ Commission error:

Node failed to be commissioned, because of the following error: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload maas` returned non-zero exit status 1: rndc: recv failed: connection reset

Tried to comment our query and cache per comment pointing to bug above and no change.

Revision history for this message
mahmoh (mahmoh) wrote :

$ sudo /usr/sbin/rndc -b 172.27.80.1 status
version: 9.9.5-3ubuntu0.3-Ubuntu <id:f9b8a50e>
CPUs found: 4
worker threads: 4
UDP listeners per interface: 4
number of zones: 101
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is OFF
recursive clients: 0/0/1000
tcp clients: 0/100
server is up and running
ubuntu@addc-maas:/etc/bind/maas$ sudo /usr/sbin/rndc -c /etc/bind/maas/rndc.conf.maas status
rndc: recv failed: connection reset
ubuntu@addc-maas:/etc/bind/maas$ sudo /usr/sbin/rndc -b 127.0.0.1 -c /etc/bind/maas/rndc.conf.maas status
rndc: connection to remote host closed
This may indicate that
* the remote server is using an older version of the command protocol,
* this host is not authorized to connect,
* the clocks are not synchronized, or
* the key is invalid.

Revision history for this message
mahmoh (mahmoh) wrote :

Enabling DNS again in Cluster, syslog:

Jul 22 00:30:54 addc-maas maas: [WARNING] Failed to create Network when adding/editing cluster interface maas-p4p1 with error [{'ip': [u'Network with this Ip already exists.'], 'name': [u'Network with this Name already exists.']}]. This is OK if it already exists.
Jul 22 00:30:55 addc-maas named[14217]: rejected command channel message from 172.27.80.1#38651
Jul 22 00:30:55 addc-maas maas.dns: [ERROR] Reloading BIND failed: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload` returned non-zero exit status 1:#012rndc: recv failed: connection reset

Revision history for this message
mahmoh (mahmoh) wrote :

$ sudo /usr/sbin/rndc -b 172.27.80.1 status
WARNING: key file (/etc/bind/rndc.key) exists, but using default configuration file (/etc/bind/rndc.conf)
rndc: get default key: not found
ubuntu@addc-maas:/var/log/maas$ sudo /usr/sbin/rndc -c /etc/bind/maas/rndc.conf.maas status
rndc: connection to remote host closed
This may indicate that
* the remote server is using an older version of the command protocol,
* this host is not authorized to connect,
* the clocks are not synchronized, or
* the key is invalid.
ubuntu@addc-maas:/var/log/maas$ sudo /usr/sbin/rndc -b 127.0.0.1 -c /etc/bind/maas/rndc.conf.maas status
rndc: connection to remote host closed
This may indicate that
* the remote server is using an older version of the command protocol,
* this host is not authorized to connect,
* the clocks are not synchronized, or
* the key is invalid.
ubuntu@addc-maas:/var/log/maas$ /sbin/ip route get 127.0.0.1
local 127.0.0.1 dev lo src 127.0.0.1
    cache <local>

Revision history for this message
mahmoh (mahmoh) wrote :

$ cat /etc/bind/rndc.conf
options { default-server 127.0.0.1; };

Revision history for this message
mahmoh (mahmoh) wrote :

nat:
Chain PREROUTING (policy ACCEPT 1242 packets, 101K bytes)
 pkts bytes target prot opt in out source destination

Chain INPUT (policy ACCEPT 817 packets, 69978 bytes)
 pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 1821 packets, 117K bytes)
 pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
 2246 148K MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0

filter:
Chain INPUT (policy ACCEPT 1114K packets, 1281M bytes)
 pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
  466 34440 ACCEPT all -- p4p1 eth0 172.27.0.0/16 0.0.0.0/0 ctstate NEW
 1470 408K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT 865K packets, 4332M bytes)
 pkts bytes target prot opt in out source destination

Revision history for this message
mahmoh (mahmoh) wrote :

# sudo netstat -anp | grep LISTEN | grep named
tcp 0 0 192.168.3.147:53 0.0.0.0:* LISTEN 14217/named
tcp 0 0 172.27.80.1:53 0.0.0.0:* LISTEN 14217/named
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 14217/named
tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN 14217/named
tcp 0 0 127.0.0.1:954 0.0.0.0:* LISTEN 14217/named
tcp6 0 0 :::53 :::* LISTEN 14217/named

summary: - MAAS uses incorrect source IP address when running rndc
+ MAAS uses incorrect source IP address when running rndc - Internal
+ server error
Revision history for this message
Mike Pontillo (mpontillo) wrote :

I spent some time troubleshooting this with mahmoh, and it looks like the problem is an broad MASQERADE rule which is rewriting the rndc packets' such that the source address is no longer 127.0.0.1. So, not a MAAS bug, but something to look out for...

Changed in maas:
status: New → Triaged
importance: Critical → Undecided
Revision history for this message
Mike Pontillo (mpontillo) wrote :

Setting this to Undecided so we can discuss whether or not we want to try to detect this condition.

Revision history for this message
mahmoh (mahmoh) wrote :

This is how I apparently "fixed" it by adding interface restrictions (ie. --in-... --out-...):

iptables -A FORWARD --in-interface p4p1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

iptables -t nat -A POSTROUTING --out-interface eth0 -j MASQUERADE

Revision history for this message
mahmoh (mahmoh) wrote :

^with the gracious troubleshooting help of mpontillo.

Gavin Panella (allenap)
Changed in maas:
status: Triaged → New
Ricardo Bánffy (rbanffy)
Changed in maas:
importance: Undecided → Medium
Ricardo Bánffy (rbanffy)
Changed in maas:
status: New → Triaged
Revision history for this message
Andres Rodriguez (andreserl) wrote :

I believe that in the latest versions of MAAS, this has now been fixed. If you believe this issue still exists, please re-open this bug report or file a new one.

Changed in maas:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.