dns-sortlist not recognized in network-config

[James Gibbins]

I'm using LXD with Ubuntu images, and cloud-init to do initial setup of container. I'm not sure, but I believe the issue is in cloud-init, but would appreciate a pointer to the right location if not. Currently using network-config meta data, cloud-init takes the following config and converts correctly:

version: 1
config:
 - type: physical
   name: eth0
 - type: nameserver
   address: ['xx.xx.xx.xx']
   search: ['example.local', 'example.org']

However including the following line is not recognized:

   sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx']

Please add support for this standard nameserver option.

[James Gibbins (jfgibbins)]

Beuller?

[James Gibbins (jfgibbins)]

Is this site even monitored?

[James Gibbins (jfgibbins)]

3 months and not a single response?

[Scott Moser (smoser)]

'sortlist' is new to me.. I've just never ran into it before.

I can understand the usefulness of it, and you're desire to feed
that information into cloud-init.

The problem that I see with it is that I'm not sure what to do with
that information on a 17.10 ubuntu system or any other system using
systemd-resolv.

I don't want to add support to the networking configuration syntax
that I can't support in each renderer.

As someone with experience here, do you have suggestions? Do
you know how systemd-resolv handles sortorder? or is there an
equivalent?

I've marked this 'incomplete', feel free to set it back to 'new' if
you answer the question, or even if you dont' have an answer :)

Thanks, and sorry for the slow reply.

Scott

[James Gibbins (jfgibbins)]

Trying to do some testing real quick and research on systemd-resolve. Unfortunately it doesn't seem there is a cloud-init enabled version of artful yet on ubuntu: to really test. I have looked at what systemd-resolve is doing with it on 16.10 and earlier, and systemd-resolve seems to be oblivious to it, but then the system seems oblivious to systemd-resolve. I think given that 16.04 LTS still has 3.5 years of support, it still has a long life, and that adding sortlist, which I think may just be a one line change(?) will add the functionality for those who want/need it, but come systemd-resolve coming in to effect, I will have to wait til cloud-init artful is available to test. I will however continue to test against 17.10 as I'm able to get a cloud-init version functioning on lxd. I can't imagine Stephane will be far off on that one. He's usually pretty on the ball.

And no worries about the slow reply. I never use launchpad, so maybe there is a way to ping you, but I don't know it. I'll have to remember you are on github though, and can ping you through that if really needed. Let me know what you think, but I think long term, it adds, and my instinct tells me no harm, but will test more.

[James Gibbins (jfgibbins)]

While I joked about it on github, it looks like sortlist from resolv may very well be superceded by RFC 3484 when falling under systemd-resolve.

[James Gibbins (jfgibbins)]

hmm...launchpad doesn't let me edit, so adding this from systemd-resolve docs:

Alternatively, /etc/resolv.conf may be managed by other packages, in which case systemd-resolved will read it for DNS configuration data. In this mode of operation systemd-resolved is consumer rather than provider of this configuration file.

[James Gibbins (jfgibbins)]

Hi Scott,Not sure if launchpad sends you notice of updates or not.  Let me know if it does.  Was surprised that this message came from your actual address, so you sent, or that's just how launchpad works, but added some comments on the ticket for you to see.  I might be able to use packer to generate my own cloud-init version of artful til Stephane and the guys have a chance to add it to the ubuntu: image server.  If so, I'll test with a system running systemd-resolve, not just for sort list, but the networking functions in general.  My containers tend to be a little more complex than the average regarding network configs, ovs, multi-home, custom routing, vlans, etc.  Blame it on my 30 years of networking, lol.
James

      From: Scott Moser <email address hidden>
 To: <email address hidden>
 Sent: Wednesday, October 18, 2017 11:01 AM
 Subject: [Bug 1704402] Re: dns-sortlist not recognized in network-config

'sortlist' is new to me.. I've just never ran into it before.

I can understand the usefulness of it, and you're desire to feed
that information into cloud-init.

The problem that I see with it is that I'm not sure what to do with
that information on a 17.10 ubuntu system  or any other system using
systemd-resolv.

I don't want to add support to the networking configuration syntax
that I can't support in each renderer.

As someone with experience here, do you have suggestions?  Do
you know how systemd-resolv handles sortorder? or is there an
equivalent?

I've marked this 'incomplete', feel free to set it back to 'new' if
you answer the question, or even if you dont' have an answer :)

Thanks, and sorry for the slow reply.

Scott

** Changed in: cloud-init (Ubuntu)
      Status: New => Incomplete

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1704402

Title:
  dns-sortlist not recognized in network-config

Status in cloud-init package in Ubuntu:
  Incomplete

Bug description:
  I'm using LXD with Ubuntu images, and cloud-init to do initial setup
  of container.  I'm not sure, but I believe the issue is in cloud-init,
  but would appreciate a pointer to the right location if not.
  Currently using network-config meta data, cloud-init takes the
  following config and converts correctly:

  version: 1
  config:
  - type: physical
    name: eth0
  - type: nameserver
    address: ['xx.xx.xx.xx']
    search: ['example.local', 'example.org']

  However including the following line is not recognized:

    sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx']

  Please add support for this standard nameserver option.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1704402/+subscriptions

[Scott Moser (smoser)]

James,

there are images in the ubuntu-daily: image repo, just not the ubuntu image repo.

$ lxc launch ubuntu-daily:artful a1
Creating a1
Starting a1
$ lxc exec a1 -- ls -l /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Oct 17 21:19 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf
$ lxc exec a1 -- cat /etc/resolv.conf# This file is managed by man:systemd-resolved(8). Do not edit.
#
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53

search lxd

[Scott Moser (smoser)]

(I do get email on bug comments, but as you've seen sometimes I miss them).

[James Gibbins (jfgibbins)]

Appreciate that.  I always forget about daily.  So spun it up, and due to netplan, everything networking broke, as in network-config was ignored, and cloud-init couldn't use any of it.  I'm guessing that netplan is supported using Networking Config Version 2 — Cloud-Init 17.1 documentation?  I currently use Version 1.  FYI, sortlist under the old method, ie version 1, does no harm under netplan.  It in fact does the same as the rest of the network config, nothing, lol.

|
| |
Networking Config Version 2 — Cloud-Init 17.1 documentation
   | |

  |

      From: Scott Moser <email address hidden>
 To: <email address hidden>
 Sent: Wednesday, October 18, 2017 1:11 PM
 Subject: [Bug 1704402] Re: dns-sortlist not recognized in network-config

James,

there are images in the ubuntu-daily: image repo, just not the ubuntu
image repo.

$ lxc launch  ubuntu-daily:artful a1
Creating a1
Starting a1
$ lxc exec a1 -- ls -l /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Oct 17 21:19 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf
$ lxc exec a1 -- cat /etc/resolv.conf# This file is managed by man:systemd-resolved(8). Do not edit.
#
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53

search lxd

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1704402

Title:
  dns-sortlist not recognized in network-config

Status in cloud-init package in Ubuntu:
  New

Bug description:
  I'm using LXD with Ubuntu images, and cloud-init to do initial setup
  of container.  I'm not sure, but I believe the issue is in cloud-init,
  but would appreciate a pointer to the right location if not.
  Currently using network-config meta data, cloud-init takes the
  following config and converts correctly:

  version: 1
  config:
  - type: physical
    name: eth0
  - type: nameserver
    address: ['xx.xx.xx.xx']
    search: ['example.local', 'example.org']

  However including the following line is not recognized:

    sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx']

  Please add support for this standard nameserver option.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1704402/+subscriptions

[James Gibbins (jfgibbins)]

Well, build with the ubuntu-daily, and there weren't even any files at /var/lib/cloud. I put up a post in lxc/lxd to inquire whether the image actually supports cloud-init under lxd with the new version. network-config, user-data, vendor-data, etc were all missing and non-functioning under lxd. Obviously more things have changed, as no adjustments were needed for 16.10 or 17.04. Very sad. :(

[Scott Moser (smoser)]

James, sorry, can you show what you were doing ?

ubuntu-daily definitely does work, with user-data and meta-data and such.
I just filed bug 1724634 a few minutes ago using --config=user.user-data=...

[James Gibbins (jfgibbins)]

Hey Scott, was going back and forth a bit with Stephane. Something goofy stil, as I was using ubuntu-daily:artful, which should have actually failed, cause daily doesn't have artful, but has 17.10, and images:, is the opposite. so even though I specified ubuntu-daily, it seemed to still be using the artful from images despite my command. I do have working instances of artful/17.10, whichever, lol, but cloud-enabled, yeah!!! It thankfully renders my version1 systax to netplan, nice job, I was worried I was going to have to rewrite tons of network stuff. Manually edit netplan to include sortlist, since cloud-init doesn't. Restart, etc, netplan/systemd-resolve completely ignores it as I was expecting. It does seem that netplan and systemd-resolve is going to defer to /etc/gai.conf which follows RFC 3484 as I mentioned previously. But that's going to be a whole other issue, and frankly, until I'm stuck using 17.10 and later, I really don't want to brush up on yet another RFC. I'm not that bored.

In summary, sortlist, great for 17.04 and prior, which means useful on 16.04 LTS until 20.10. Completely ignored and harmless in 17.10 and if ubuntu keeps netplan, going forward. I know for my own uses, 16.04 will be around and in use for a long time. Even now, coming up on 3.5 years for 14.04, some apps and guides still aren't too happy about being on 16.04. I have a few 14.04s still left supporting a few apps to which I've never gotten cloud-init to handle networking well on it, so just manually edit or push a file to overwrite for those few instances.

[James Gibbins (jfgibbins)]

Damn, I just realized, netplan doesn't use ifup and ifdown now, does it? That's going to completely break my source-based routing. The important part is as follows:

IF_GATEWAY="$(echo "$IF_ADDRESS" | cut -d. -f1-3).254"
ip route flush table "$IFACE"
ip route add default via "$IF_GATEWAY" table "$IFACE"
ip rule del lookup "$IFACE" || true
ip rule add from "$IF_ADDRESS" lookup "$IFACE"

That doesn't by chance translate to netplan and cloud-init, does it? I can find routes: to, via, but nothing for adding to table.

[James Gibbins (jfgibbins)]

Hey Scott,
Been taking a deep dive into the netplan package, a la 17.10. Less than 20 minutes review, and can already see 1/2 dozen things that aren't implemented and will completely break networking in just my environment. Given its current state, I can't believe they even allowed it into 17.10 for testing. It is woefully lacking so many functions it shouldn't have even been considered. Hopefully, come 18.04, they'll drop the package and attempt it again later, say 19.04 or 19.10. It's interesting in concept, but at this point, it's alpha code at best, and needs work to get to beta.

[James Gibbins (jfgibbins)]

Hi Scott,I know you're aware of and dealing with netplan issues, but not if you're directly involved with it.  But netplan has serious deficiencies and nowhere near as inclusive or as capable as the current system.  Functions such as source-based routing, open vswitch support, etc.  One release before an LTS seems far too late to introduce such a half-baked and untested package that is so central to network connectivity.  Please tell me their not seriously considering going forward with it for the next LTS.  If so, I may have to stay on 16.04 til 20.04 comes along, assuming they get it working by then.James

      From: Scott Moser <email address hidden>
 To: <email address hidden>
 Sent: Thursday, October 19, 2017 11:26 AM
 Subject: [Bug 1704402] Re: dns-sortlist not recognized in network-config

** Also affects: cloud-init
  Importance: Undecided
      Status: New

** Changed in: cloud-init
      Status: New => Confirmed

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1704402

Title:
  dns-sortlist not recognized in network-config

Status in cloud-init:
  Confirmed
Status in cloud-init package in Ubuntu:
  New

Bug description:
  I'm using LXD with Ubuntu images, and cloud-init to do initial setup
  of container.  I'm not sure, but I believe the issue is in cloud-init,
  but would appreciate a pointer to the right location if not.
  Currently using network-config meta data, cloud-init takes the
  following config and converts correctly:

  version: 1
  config:
  - type: physical
    name: eth0
  - type: nameserver
    address: ['xx.xx.xx.xx']
    search: ['example.local', 'example.org']

  However including the following line is not recognized:

    sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx']

  Please add support for this standard nameserver option.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1704402/+subscriptions

[David Britton (dpb)]

Hi James -- do you have an example #cloud-config that you can attach that sets up source-based routing (please sanitize first if required)?

Thanks.

[James Gibbins (jfgibbins)]

Hi David,This ticket is for the source-list, which is for dns resolver and the config is as follows.
  version: 1
  config:
  - type: physical
    name: eth0
  - type: nameserver
    address: ['xx.xx.xx.xx']    sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx'] I believe the only thing that is needed for cloud-init is to add the keyword "sort-list" to the case list, so that it is written to /etc/resolv.conf.
As for source-based routing, there is no config recognized by cloud-init.  It's just straight network statements, name, ip, netmask, etc.  The work is done by a shell script in the if-up.d directory as follows:
#!/bin/sh
set -e
if [ "$METHOD" = loopback ]; then
    exit 0
elif [ "$METHOD" = dhcp ]; then
    exit 0
elif [ "$METHOD" = static ]; then
    if [ "$IF_GATEWAY" ]; then
echo "exiting..."
        exit 0
    fi
fi
IF_GATEWAY="$(echo "$IF_ADDRESS" | cut -d. -f1-3).254"
ip route flush table "$IFACE"
ip route add default via "$IF_GATEWAY" table "$IFACE"
ip rule del lookup "$IFACE" || true
ip rule add from "$IF_ADDRESS" lookup "$IFACE"

Basically, if the interface is a loopback, dhcp, or already has a default gateway, it does nothing. For any other interface, it adds a default route to the route table causing all responses to inbound traffic to use the same interface to respond on. The email I sent late last night to your email, discusses what I've been able to find, in terms of using systemd-networkd, but requires ver 235, as well as some additional work in netplan and cloud-init to recognize all the values.

P.S. systemd 235, which I saw is proposed for bionic beaver, adds quite a few nice functions, including "vrf".

      From: David Britton <email address hidden>
 To: <email address hidden>
 Sent: Wednesday, November 1, 2017 4:50 PM
 Subject: [Bug 1704402] Re: dns-sortlist not recognized in network-config

Hi James -- do you have an example #cloud-config that you can attach
that sets up source-based routing (please sanitize first if required)?

Thanks.

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1704402

Title:
  dns-sortlist not recognized in network-config

Status in cloud-init:
  Confirmed
Status in cloud-init package in Ubuntu:
  New

Bug description:
  I'm using LXD with Ubuntu images, and cloud-init to do initial setup
  of container.  I'm not sure, but I believe the issue is in cloud-init,
  but would appreciate a pointer to the right location if not.
  Currently using network-config meta data, cloud-init takes the
  following config and converts correctly:

  version: 1
  config:
  - type: physical
    name: eth0
  - type: nameserver
    address: ['xx.xx.xx.xx']
    search: ['example.local', 'example.org']

  However including the following line is not recognized:

    sortlist: ['xx.xx.xx.xx', 'xx.xx.xx.xx']

  Please add support for this standard nameserver option.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1704402/+subscriptions

[Dan Watkins (oddbloke)]

Dropping this from the cloud-init Ubuntu package so we're just tracking it in one place.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2953