Bind9 (8.04) not returning 'ad' flag when dnssec is enabled

[buecking]

Binary package hint: bind9

% lsb_release -rd
Description: Ubuntu 8.04
Release: 8.04

% apt-cache policy bind9
bind9:
  Installed: 1:9.4.2-10
  Candidate: 1:9.4.2-10
  Version table:
 *** 1:9.4.2-10 0
        500 http://ubuntu-ashisuto.ubuntulinux.jp hardy/main Packages
        100 /var/lib/dpkg/status

% cat /etc/resolv.conf
nameserver 127.0.0.1
options edns0

When running dig against dns server w/DNSSEC enabled it is expected that
named should return the ad flag for authenticated records; however, this
system is not returning the correct response. If I query asking for
+dnssec the ad flag is properly returned - as expected.

Without the ad flag I am not able to use ssh VerifyHostKeyDNS.

I have two systems with identical named configs. System A is a NetBSD
machine running Bind 9.4.2 built against OpenSSL 0.9.8d 28 Sep 2006, and
System B Ubuntu 8.04 running Bind 9.4.2 built against OpenSSL 0.9.8g 19
Oct 2007.

If I dig @system-a foo.example.com A the ad flag is return; but as I
mentioned above if I dig @system-b foo.example.com A the ad flag is not
returned even though the configurations are exactly the same.

When quering for an SSHFP record both servers, a and b, return the same
SSHFP record in the results.

[LaMont Jones (lamont)]

9.4.2 rc1 introduced the following change:
  2249. [bug] Only set Authentic Data bit if client requested DNSSEC, per RFC 3655 [RT #17175]

What you're seeing here is that the AD bit was redefined here: http://www.ietf.org/rfc/rfc3655.txt

[buecking (buecking)]

Thanks for your response.

> What you're seeing here is that the AD bit was redefined here:
> http://www.ietf.org/rfc/rfc3655.txt

That is why options edns0 is defined, so that the client is forced to
ask for the AD bit. Who do you suggest I talk to about this?

Thanks,
--
Bryan Buecking http://www.starling-software.com

On Wed, Jul 02, 2008 at 12:06:46PM -0000, LaMont Jones wrote:
> 9.4.2 rc1 introduced the following change:
> 2249. [bug] Only set Authentic Data bit if client requested DNSSEC, per RFC 3655 [RT #17175]
>
> ** Changed in: bind9 (Ubuntu)
> Assignee: (unassigned) => LaMont Jones (lamont)
> Status: New => Invalid
>
> --
> Bind9 (8.04) not returning 'ad' flag when dnssec is enabled
> https://bugs.launchpad.net/bugs/242956
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “bind9” source package in Ubuntu: Invalid
>
> Bug description:
> Binary package hint: bind9
>
> % lsb_release -rd
> Description: Ubuntu 8.04
> Release: 8.04
>
> % apt-cache policy bind9
> bind9:
> Installed: 1:9.4.2-10
> Candidate: 1:9.4.2-10
> Version table:
> *** 1:9.4.2-10 0
> 500 http://ubuntu-ashisuto.ubuntulinux.jp hardy/main Packages
> 100 /var/lib/dpkg/status
>
> % cat /etc/resolv.conf
> nameserver 127.0.0.1
> options edns0
>
> When running dig against dns server w/DNSSEC enabled it is expected that
> named should return the ad flag for authenticated records; however, this
> system is not returning the correct response. If I query asking for
> +dnssec the ad flag is properly returned - as expected.
>
> Without the ad flag I am not able to use ssh VerifyHostKeyDNS.
>
> I have two systems with identical named configs. System A is a NetBSD
> machine running Bind 9.4.2 built against OpenSSL 0.9.8d 28 Sep 2006, and
> System B Ubuntu 8.04 running Bind 9.4.2 built against OpenSSL 0.9.8g 19
> Oct 2007.
>
> If I dig @system-a foo.example.com A the ad flag is return; but as I
> mentioned above if I dig @system-b foo.example.com A the ad flag is not
> returned even though the configurations are exactly the same.
>
> When quering for an SSHFP record both servers, a and b, return the same
> SSHFP record in the results.

[buecking (buecking)]

BIND 9 uses EDNS0 (RFC2671) to advertise its receive buffer size.
It also sets an EDNS flag bit in queries to indicate that it wishes to
receive DNSSEC responses; this flag bit usage is not yet standardized,
but we hope it will be.

[buecking (buecking)]

Moving this issue. When "options edns0" is turned on (usually in
/etc/resolv.conf), ssh doesn't see it, and fails to request a DNSSEC
response, which in turn leads to SSHFP records being considered
insecure.

[Curt Sampson (cjs-cynic)]

Note also that the definition of RES_USE_DNSSEC (defined to be 0x00200000) must be added to /usr/include/resolv.h in order for OpenSSH's openbsd-compat/getrrsetbyname.c to be able to set that bit when edns0 is set.

[Curt Sampson (cjs-cynic)]

It gets worse; I tried the above, and getrrsetbyname still doesn't get back a record with the AD bit set. (I verified that RES_USE_DNSSEC was set in the options passed to res_query.) Is the resolver broken?

[Jonathan Stewart (reaper-fudo)]

In my (limited) experience, the server only responds with the AD bit set which it can validate the DNSSEC records on the domain. As there is no root key in the DNS now, this means you must configure trust anchors on your recursive nameserver.

My question would be: is your recursive DNS server actually able to validate the DNSSEC records? If you operate the server, you should be able to examine the dnssec logs and determine if the nameserver is able to validate the DNSSEC records.

[Curt Sampson (cjs-cynic)]

As related in the original ticket description, all this stuff is working fine under NetBSD; the issue is not with the data, but with Ubuntu. In particular, there's certainly an issue with Ubuntu glibc that will simply not allow it to check the AD bit.

While we only reported this recently, I'm wondering if this bug has languished for years in glibc because nobody understands what the AD bit is used for. Let me give you one example of the current situation:

 1. Try to ssh to foo.cynic.net, with authentication forwarding.
 2. OpenSSH looks up the IP address in DNS, but this has been intercepted by the attacker.
 3. The resolver cannot authenticate so we carry on.
 4. Look up the SSHFP record in DNS, which may also have been intercepted by the attacker.
 5. The resolver cannot authenticate this, so OpenSSH (correctly) refuses to use it.
 6. User gets an, "Unknown host, fingerprint is blah blah" message.

At this point, it goes one of two different ways.

  7. User foolishly says, "it's ok, continue", and connects to hostile system.
  8. Hostile system uses user's SSH authentication channel to log in to and subvert other systems to which the user has access.
  9. User a few seconds later, realizes he's on some weird system and logs out, but the damage is done.

Or, if the user is a bit smarter:

 7. User says, "no don't connect to an untrusted host.
 8. User tries to find some out-of-band way to figure out the fingerprint of the host he really wants to connect to.
 9. User then realizes that he's under attack.

We get around this by manually generating and copying around (to /etc/ssh/ssh_known_hosts) a file of public keys for our systems. This costs us time and money.

[Rob Gallagher (rob-gallagher)]

Not present in the stub resolver on jaunty either. I can understand the reluctance to add it in due to potential DNSSEC breakage, but it is a user-configurable option in resolv.conf so it shouldn't be too much of a risk.

Could it be added in for jaunty ?

[buecking (buecking)]

It would be very cool if someone could get the AD bit parsing done in the resolver library before
the next release. I believe this is the only thing stopping us from using DNSSEC as outlined above.

[Jeff Enns (cyberpenguinks)]

I'm just checking in to see if this is still a problem in the latest release. Could you let me know one way or another if you have tested this bug on 9.04. Thank you.

[buecking (buecking)]

Still broken.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 9.04
Release: 9.04
Codename: jaunty

[Wolfgang Nagele (wnagele)]

The only system i could get this working at the moment was OpenBSD. To enable
this i had to provide 'edns0' as an option in resolv.conf[1].

I have attached a PCAP (openbsd.pcap) generated with tcpdump. If you observe it
(for instance with Wireshark) you will see that the request for the SSHFP
records has the DO bit set in the EDNS0 section of the packet and the response
has the AD bit set in the packet header.

[1] http://<email address hidden>/msg11176.html

[Curt Sampson (cjs-cynic)]

I'm glad to hear you had good success with this with OpenBSD. It's also been working on NetBSD for many, many years now.

[Hauke Lampe (hauke)]

The problem lies within the libc stub resolver. I patched mine, so it always sets the AD flag in queries, which prompts the recursor to set AD in response if DNSSEC validation succeeded.

Verbose explanations (one of my seldom public blog posts ;) and small patch here:
http://bd.hauke-lampe.de/dnssec/how-to-get-dnssec-ad-flag-with-glibc.html

[spbike (bill-broadley)]

I hope this gets fixed. I was trying to setup a hardy server running bind9 and DNSSEC using the ISC DLV service. I couldn't figure out why all DNS queries came back as unauthenticated, thus DKIM (signed email), ssh, and dig all think that the DNS info returned is not trustworthy.

[Wolfgang Nagele (wnagele)]

RedHat has some quite simple patches that were used to enable this: https://bugzilla.redhat.com/show_bug.cgi?id=205842

Maybe somebody wants to port this into the Ubuntu source?

[Hauke Lampe (hauke)]

I now use a backport to glibc 2.9 of the RedHat patches in 2.11 instead of my own patch set.

See https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/288011/comments/3

OpenSSH still needs recompiling and "options edns0" in /etc/resolv.conf if you want it to check DNSSEC flags on SSHFP host fingerprint lookups.

[Chuck Short (zulcss)]

Hi,

Thanks for the bug report, it might be considered for future releases of Ubuntu.

Regards
chuck

[Hauke Lampe (hauke)]

Followup: Thanks to the nice PPA system, I now have binary packages available for libc and openssh:
https://launchpad.net/~hauke/+archive/dnssec-enabled (jaunty and karmic tested, hardy untested)

http://bd.hauke-lampe.de/dnssec/dnssec-enabled-ubuntu-packages.html

I imagine Ubuntu >= 10.04 will use glib2.11+ and fully support DNSSEC in the resolver.