juju-core

Bug #1534795
Comment #31

Comment 31 for bug 1534795

Revision history for this message

Andrew McDermott (frobware) wrote on 2016-01-20:

#31

I want to focus and (D) as a) that's both proposed and currently tested and b) I believe is correct.

In the following examples I'm going to cut this down to 2 interfaces:

On MAAS I have deployed a node whose original /e/n/i looks like:

  ubuntu@icky-son:~$ sudo cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
      dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
dns-search maas19

which is similar to case (D).

I deploy the node and in /etc/resolv.conf I see:

  ubuntu@icky-son:~$ sudo cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 10.17.20.200
  nameserver 192.168.1.1
  search maas19

I bring down eth1:

  ubuntu@icky-son:~$ sudo ifdown eth1
  ubuntu@icky-son:~$ cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 10.17.20.200

Note the nominally global 'nameserver 192.168.1.1' has disappeared.

Ditto again for eth0:

  ubuntu@icky-son:~$ sudo ifdown eth0
  ubuntu@icky-son:~$ cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

And 10.17.20.200 has now gone.

So the question is why would `ifdown eth1' remove the global nameserver?

Answer: perhaps it's not actually global.

Let's see if we can prove this; let's now comment all dns-nameserver entries out and reboot.

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

#dns-nameservers 192.168.1.1
#dns-search maas19

On reboot eth0 AND eth1 are marked 'UP' because they have an 'auto' stanza entry:

  $ubuntu@icky-son:~$ ip -d link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0
  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:1c:f1:5b brd ff:ff:ff:ff:ff:ff promiscuity 0
  3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:dd:5e:b3 brd ff:ff:ff:ff:ff:ff promiscuity 0

And, as expected, since we commented out the `dns-nameservers' option we have no nameserver entries:

  ubuntu@icky-son:~$ cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

Let's take down eth1:

  ubuntu@icky-son:~$ sudo ifdown eth1
  ubuntu@icky-son:~$ ip -d link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0
  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:1c:f1:5b brd ff:ff:ff:ff:ff:ff promiscuity 0
  3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
      link/ether 52:54:00:dd:5e:b3 brd ff:ff:ff:ff:ff:ff promiscuity 0

And edit /etc/network/interfaces so that we have:

  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
#dns-search maas19

And bring it back up:

  ubuntu@icky-son:~$ sudo ifup eth1
  ubuntu@icky-son:~$ cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 192.168.1.1

We get a nameserver in /etc/resolv.conf. This tells me that the dns-nameserver entry is associated with the interface and is not a global setting.

In example (D) that Ryan posted we would get a populated /etc/resolv.conf from eth3 -- eth3 now has indented dns-* options.

Let's futz some more with /etc/network/interfaces.

Let's move the global dns-nameservers entry to the beginning of the file, but be first we do:

  ubuntu@icky-son:~$ sudo ifdown eth1
  ubuntu@icky-son:~$ sudo cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

Note: no DNS info. Our new /etc/network/interfaces looks like:

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  dns-nameservers 192.168.1.1
  dns-search maas19

  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

And we bring up eth1:

  ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:1: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Whoa! - ifup cannot parse the file!

Let's shuffle our global dns-nameserver options around a bit:

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  dns-nameservers 192.168.1.1
  iface eth1 inet manual
      mtu 1500

dns-search maas19

Bring eth1 back up:

  ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:9: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Yikes - still cannot parse /e/n/i.

Let's introduce a new top-level stanza (source-directory):

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

source-directory /tmp/foo.d

dns-nameservers 192.168.1.1
dns-search maas19

And bring up eth1 again:

  ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:14: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Let's move the source-directory stanza directly after the dns-nameservers option:

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

  dns-nameservers 192.168.1.1
  source-directory /tmp/foo.d
  dns-search maas19

Bring eth1 up:

  ubuntu@icky-son:~$ sudo ifdown eth1
  /etc/network/interfaces:14: misplaced option
  ifdown: couldn't read interfaces file "/etc/network/interfaces"

Still busted.

And finally, moving source-directory to EOF:

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  # dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

  dns-nameservers 192.168.1.1
  dns-search maas19
  source-directory /tmp/foo.d

Bring eth1 up:

BINGO! -- Parses OK and we have a populated /etc/resolv.conf.

From this I conclude there's no such thing as global dns-* options. The indentation is misleading. The only was I am able to get the dns-* entries to work is if they come after the 'iface' stanza, and this is pretty much what the resolvconf(8) manpage specifies. And looking at the list of stanza names in interfaces(5) dns-* is not mentioned at all.

Right now I believe the way (D) comes out post juju adding the bridge is correct; both from an indentation POV and that by virtue of 'eth3' being 'auto' means that when `ifup -a' runs, the 'dns-nameservers' options will be picked out by resolvconf and those will end up in /etc/resolv.conf.

There was some concern that if the MAAS setup had no default nameserver entry set we would get the equivalent /e/n/i:

  ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

  auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
dns-search maas19

Note the now MISSING dns-nameservers option in eth0.

Let's reboot and see what /etc/resolv.conf looks like:

  ubuntu@icky-son:~$ uptime
   14:59:48 up 0 min, 1 user, load average: 0.00, 0.00, 0.00
  ubuntu@icky-son:~$ cat /etc/resolv.conf
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 192.168.1.1
  search maas19

which all seems fine to me.

I want to focus and (D) as a) that's both proposed and currently tested and b) I believe is correct.

In the following examples I'm going to cut this down to 2 interfaces:

On MAAS I have deployed a node whose original /e/n/i looks like:

ubuntu@icky-son:~$ sudo cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
      dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
  dns-search maas19

which is similar to case (D).

I deploy the node and in /etc/resolv.conf I see:

ubuntu@icky-son:~$ sudo cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 10.17.20.200
  nameserver 192.168.1.1
  search maas19

I bring down eth1:

ubuntu@icky-son:~$ sudo ifdown eth1 
  ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 10.17.20.200

Note the nominally global 'nameserver 192.168.1.1' has disappeared.

Ditto again for eth0:

ubuntu@icky-son:~$ sudo ifdown eth0
  ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

And 10.17.20.200 has now gone.

So the question is why would `ifdown eth1' remove the global nameserver?

Answer: perhaps it's not actually global.

Let's see if we can prove this; let's now comment all dns-nameserver entries out and reboot.

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

#dns-nameservers 192.168.1.1
  #dns-search maas19

On reboot eth0 AND eth1 are marked 'UP' because they have an 'auto' stanza entry:

$ubuntu@icky-son:~$ ip -d link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 
  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:1c:f1:5b brd ff:ff:ff:ff:ff:ff promiscuity 0 
  3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:dd:5e:b3 brd ff:ff:ff:ff:ff:ff promiscuity 0

And, as expected, since we commented out the `dns-nameservers' option we have no nameserver entries:

ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

Let's take down eth1:

ubuntu@icky-son:~$ sudo ifdown eth1
  ubuntu@icky-son:~$ ip -d link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 
  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
      link/ether 52:54:00:1c:f1:5b brd ff:ff:ff:ff:ff:ff promiscuity 0 
  3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
      link/ether 52:54:00:dd:5e:b3 brd ff:ff:ff:ff:ff:ff promiscuity 0

And edit /etc/network/interfaces so that we have:

auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
  #dns-search maas19

And bring it back up:

ubuntu@icky-son:~$ sudo ifup eth1
  ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 192.168.1.1

We get a nameserver in /etc/resolv.conf. This tells me that the dns-nameserver entry is associated with the interface and is not a global setting.

In example (D) that Ryan posted we would get a populated /etc/resolv.conf from eth3 -- eth3 now has indented dns-* options.

Let's futz some more with /etc/network/interfaces.

Let's move the global dns-nameservers entry to the beginning of the file, but be first we do:

ubuntu@icky-son:~$ sudo ifdown eth1
  ubuntu@icky-son:~$ sudo cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

Note: no DNS info. Our new /etc/network/interfaces looks like:

ubuntu@icky-son:~$ cat /etc/network/interfaces
  dns-nameservers 192.168.1.1
  dns-search maas19

auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

And we bring up eth1:

ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:1: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Whoa! - ifup cannot parse the file!

Let's shuffle our global dns-nameserver options around a bit:

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  dns-nameservers 192.168.1.1
  iface eth1 inet manual
      mtu 1500

dns-search maas19

Bring eth1 back up:

ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:9: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Yikes - still cannot parse /e/n/i.

Let's introduce a new top-level stanza (source-directory):

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

source-directory /tmp/foo.d

dns-nameservers 192.168.1.1
  dns-search maas19

And bring up eth1 again:

ubuntu@icky-son:~$ sudo ifup eth1
  /etc/network/interfaces:14: misplaced option
  ifup: couldn't read interfaces file "/etc/network/interfaces"

Let's move the source-directory stanza directly after the dns-nameservers option:

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
  source-directory /tmp/foo.d
  dns-search maas19

Bring eth1 up:

ubuntu@icky-son:~$ sudo ifdown eth1
  /etc/network/interfaces:14: misplaced option
  ifdown: couldn't read interfaces file "/etc/network/interfaces"

Still busted.

And finally, moving source-directory to EOF:

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
  #    dns-nameservers 10.17.20.200
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
  dns-search maas19
  source-directory /tmp/foo.d

Bring eth1 up:

ubuntu@icky-son:~$ sudo ifup eth1
  ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 192.168.1.1
  search maas19

BINGO! -- Parses OK and we have a populated /etc/resolv.conf.

There was some concern that if the MAAS setup had no default nameserver entry set we would get the equivalent /e/n/i:

ubuntu@icky-son:~$ cat /etc/network/interfaces
  auto eth0
  iface eth0 inet static
      gateway 10.17.20.1
      address 10.17.20.211/24
      mtu 1500

auto eth1
  iface eth1 inet manual
      mtu 1500

dns-nameservers 192.168.1.1
  dns-search maas19

Note the now MISSING dns-nameservers option in eth0.

Let's reboot and see what /etc/resolv.conf looks like:

ubuntu@icky-son:~$ uptime
   14:59:48 up 0 min,  1 user,  load average: 0.00, 0.00, 0.00
  ubuntu@icky-son:~$ cat /etc/resolv.conf 
  # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
  #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
  nameserver 192.168.1.1
  search maas19

which all seems fine to me.