By default DNSSEC is enabled with automatic keys

[Brad Marshall]

The default setting for DNSSEC in MaaS is to enable it with automatic keys - I'd argue that it will break DNS for more setups than it fixes. If you have ingress and egress filtering (a very common situation in corporate environments) and don't have DNSSEC on your upstream servers (more common than you'd ideally want), you need to explicitly disable this setting.

I would suggest a better default would be to disable DNSSEC - anyone who has set it up will know, and can flip the setting over to what they need. Current statistics from http://stats.labs.apnic.net/dnssec show that at the very best, 18% of DNS queries in the Americas use DNSSEC (a lot of that can be attributed to Google's public DNS server, which supports it).

This testing was done with MaaS 1.8 from the stable PPA, and the latest Trusty.

$ lsb_release -d
Description: Ubuntu 14.04.3 LTS

$ dpkg-query -W maas
maas 1.8.2+bzr4041-0ubuntu1~trusty1

Please let us know if you have any further questions.

[Mike Pontillo (mpontillo)]

I can't dispute the data, but I disagree with the suggested fix.

Reasoning:

(1) MAAS "takes over" existing BIND configuration files in order to provide DNS.
(2) If a user was running BIND on the machine that is becoming the region controller, (or running a previous version of MAAS) they may expect DNSSEC to work properly. (as per the BIND default)
(3) Due to (1) and (2), if we disable DNSSEC by default, then a properly configured upstream DNSSEC will *SILENTLY BREAK* when installing MAAS.

So we have two options:

Option A: Keep existing behavior, which properly migrates BIND's default values for DNSSEC configuration to MAAS.
Option B: Change existing behavior as suggested, which will silently break DNSSEC configurations upon installation of MAAS, and open a potential attack vector for security-conscious organizations running MAAS.

Perhaps a compromise solution would be for a future version of MAAS to include a guided install process, which could detect this situation and ask the user what to do.

[Mike Pontillo (mpontillo)]

Adding the `bind` package. MAAS simply uses the value from the BIND configuration files in order to determine what to configure in MAAS. So this cannot change in MAAS unless a decision is made in BIND, or the value is left out of the configuration file by default, allowing MAAS to make its own decision.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robie Basak (racb)]

So it sounds like the bug is really that DNSSEC is enabled with automatic keys (note that I don't actually know what that means) and the reporter thinks that it shouldn't be because of his use case where this default breaks things?

[Mike Pontillo (mpontillo)]

Yes, that seems to be the argument. I would like to understand why it seems to be that many environments are set up with a forwarder that does not support DNSSEC. (is this by choice? is it a particular vendor, or old DNS server which does not forward the queries properly? misconfigured firewall rules?)

There are three possible values for the BIND dnssec-validation option: 'yes', 'no', and 'auto'.

By saying "enabled with automatic keys", we just mean the default value of "dnssec-validation auto;" in the BIND configuration file.

See also: http://users.isc.org/~jreed/dnssec-guide/dnssec-guide.html#dnssec-validation-explained

[LaMont Jones (lamont)]

It is defaulted to "auto" because more and more of the internet _IS_ enabling DNSSEC: all delegations from the root are signed, and most registries will take care of getting the DS RRsets into the parent zone.

The only way to actually fix some of the DNS cache poisoning attacks is to enable DNSSEC. That the upstream forwarder doesn't support dnssec is a configuration bug in the upstream forwarder. I'm disinclined to make the default be less secure, in order to "support" broken upstream forwarders. But I'll stop short of marking it Won't Fix, at least for now.

[Andres Rodriguez (andreserl)]

I tend to agree with LaMont here...

[Mike Pontillo (mpontillo)]

+1 to the above comments; I tend to think this is a "Won't Fix" in the bind9 package. I would stop short of making it "Opinion", because it does not appear to be open for discussion.

[Christian Reis (kiko)]

This continues to cause us pain, though; I just spent an hour looking through a DNS issue to remember that this was the root cause. We should either change the default or make it so we can detect when the forwarder won't handle DNSSEC and correctly disable it -- a failure in the field due to this is unacceptable.

[Christian Reis (kiko)]

(In other words, MAAS should do something like the described in https://docs.menandmice.com/display/MM/How+to+test+DNSSEC+validation when the forwarder is configured, and tweak the setting automatically with a warning).

[Andreas Hasenack (ahasenack)]

@Lamont, I have had to disable this option when the upstream is a canonical.com dns. Something we can do there perhaps?

[LaMont Jones (lamont)]

The current state of the DNS is that the root zone is signed, and EVERYTHING delegated from it is signed by the root zone. Once you get below that, the lack of signatures on a zone is left as an exercise for the admins of that zone. (example.com can be delegated from the [signed] COM zone without being signed, and that's all good and fine and DNSSEC=auto handles that just fine.)

What doesn't work is when the admin chooses to use an undelegated top-level domain (TLD), which won't be signed by the root key, and therefore fails DNSSEC validation.

Especially given the recent changes in what constitutes a valid TLD, the admin choosing to use a TLD oftheir own choosing is hoping from their hearts that there will never be sufficient demand for that TLD to cause it to be creeated and subdomains sold therein by various registries. Because when that happens, and their users want to access things in that newly-created TLD, then they will get to go and change all of their domain names to avoid that.

Properly delegating children (whether that is published publicly or not) from domain names that are actually under the control of the admin is the only sane way of doing this.

[LaMont Jones (lamont)]

Having said that, if MAAS is going to continue to defualt to, and RECOMMMEND that users use, a TLD of "maas", then MAAS should default it's setting to "dnssec no" in the case where the default domain is still named "maas" -- Open to discussion is whether this should be thru chaning the global setting, or overridden at named.conf generation time.

[Christian Reis (kiko)]

I'm curious whether this affects *.maas, when in fact the MAAS server itself is primary for that domain. In other words, the request should not ever hit the forwarder, should it?

Or does BIND do a root check anyway to ensure that we are indeed allowed to be primary for that TLD?

If we are, is there a way to say "this is not an official TLD" that wouldn't trigger the problem?

[LaMont Jones (lamont)]

When the config has "dnssec-validation auto;", BIND attempts to validate all answers, and stops checking once it gets to a DNS zone that has no signatures. Categorically, all TLDs have signatures, so the lack of a signature on the .maas DNS zone is fatal to the DNSSEC validation.

If the domain in question were "maas.example.com", then the validation would proceed as follows (while being a bit handwavy about the specifics of requests, I admit):
1) verify the .com signature
2) verify the .example.com signature (if any, otherwise declare success: done validation)
3) verify the .maas.example.com signature (if any, otherwise declare success: done validation)
4) verify the signature for machine.maas.example.com (if any, otherwise declare success: done validation)

BIND doesn't care if you're a primary or a secondary or caching - it's validating the RRset, not the server.
I am not aware of any distinction in BIND wrt "not an official TLD" -- Note also that if you wanted to set that, you would have to set it as an enterprise wide configuration option on every resolver, which does not scale.

As far as saner default MAAS domain names, that warrants more discussion. A non-exhaustive list:
1) default to a subdomain of the the FQDN of the region, minus the first lable. (mymaashost.example.com becomes maas.example.com) Pro: The admin is very likely the admin of example.com. Con: the second time you do this, you have duplicate domains, and if they ever need to talk to each other, there is a rename...
2) default to the FQDN of the region (mymaashost.example.com becomes the default domain name) Pro: we have good confidence that we OWN this point in the DNS tree, and its descendants. Con: upstream quite possibly controls our address set, and delegation at this point means that we become authoritative for the addresses, while upstream needs them for glue. Changing the upstream-facing IP of the server becomes a more difficult process
3) default to maas.FQDN of the region (mymaashost.example.com becomes maas.mymaashost.example.com) Pro: we definitely control everything from that down. Cons: likely redundant.

Whatever the domain is, we should (at least in the docs) be telling the admin what to add to the parent DNS zone to properly delegate (and maybe even have secondaries..) the DNS for the MAAS region. At what point DNSSEC signatures stop is left as a discussion for upstream DNS. MAAS generating signatures is the subject of https://bugs.launchpad.net/maas/+bug/1620662

[LaMont Jones (lamont)]

And after more investigation, I can see that what I really need is a failing system where I can poke at it to understand what is going on -- the whole all-TLDs-are-signed-and-that's-why doesn't seem to pan out as I tried to reproduce the issue.