xmpp4moz doesn't read SRV DNS records

[Matěj Cepl]

Having a JID <email address hidden>, xmpp4moz should (according to 14.3. of RFC3920) resolve the hostname of the server using SRV service:

[matej@viklef ~]$ host -t SRV _xmpp-client._tcp.ceplovi.cz
_xmpp-client._tcp.ceplovi.cz has SRV record 0 0 5222 luther.ceplovi.cz.
[matej@viklef ~]$

xmpp4moz certainly doesn't resolves server name with SRV (so it has special type of GMail account distinct from Jabber account, which is unnecessary when SRV records are followed).

[Leger-formerly-netscape (leger-formerly-netscape)]

Bulk move of all Networking-Core (to be deleted component) bugs to new
Networking component.

[Gordon-formerly-netscape (gordon-formerly-netscape)]

Help wanted.

[Mark-andrews (mark-andrews)]

See also Internet Draft draft-andrews-http-srv-00

[Djb-bugzilla (djb-bugzilla)]

It takes time to check for SRV records. Be careful with features that
can degrade the user experience! I'd like to see a study of SRV's impact
on web-page load times.

[Kcd (kcd)]

It takes time when an HTTP request times out because the server is down or
overloaded as well! Any testing which attempts to assess impact on "user
experience" should take into account not only the "best case" scenarios but
also the load-balancing and failover benefits of SRV. We live, after all, in an
imperfect world...

[Evertonm (evertonm)]

Please guys, we need this improvement on our browser of choice.

SRV RRs are very useful; such enhancement is _innovation_. Just take a look at
the specs.

There is no good reason to not include this (other than allocating some
development resources).

[Benc-meer (benc-meer)]

mass move, v2.
qa to me.

[Benc-meer (benc-meer)]

Can we target this (1.0 or future?)

[M438urxuqu85001 (m438urxuqu85001)]

Some corrections to the previous remarks:
 - Browsing is improved: you'd like the web robust like email? SRV do that!
 - Goal of supporting SRV records is support load balanced & backed-up servers.
 - RFC 2052 is superseded by RFC 2782
 - Dealing with SRV records does not take more time, it uses one DNS lookup
 - People are working on library support to automate SRV records
 - Targeting it at v1.0 seems smart; being able to claim robustness is nice ;-)
All in more detail on http://dns.vanrein.org/srv (or send me email and ask)

[Dnewman (dnewman)]

I am taking ownership of this RFE. I have been looking for a programming
project for some time now and this one is very interesting to me. I'm thinking
that all protocols that Mozilla supports should use SRV records, i.e. HTTP,
NNTP, FTP, POP, IMAP, LDAP, SMTP, GOPHER (does anyone still use this?). I will
be creating an outline of the plan for comments.

[Njs-netscape (njs-netscape)]

Why was I copied on this bug when it was updated? the update was sent to

<email address hidden>? Why?

By the way, the RFC is more than six months old - draft RFCs more than
six months old are questionable - I'd contact the authors to check on their
current felings before writing anything.

[M438urxuqu85001 (m438urxuqu85001)]

Nick/others,

RFC 2052, the originally mentioned topic, is indeed over 6 months, but you may
also have noticed my earlier remark that it was replaced with RFC 2782. This one
is on the Standards Track, and STD 1 specifies its status, so rather than making
an RFC look suspicious, I'd like to see a quote from STD 1.
Allow me to demonstrate: it is listed under the heading of
        3.4. Proposed Standard Protocols
And that is a result of a specific action by the IESG [see RFC 2026, sec 4.1.1],
so it definately makes sense to implement this particular feature.

Hope this helps to take your worries away,

Rick van Rein.

[Dnewman (dnewman)]

This past weekend I was working on some "proof of concept" code to kind of hash
out how it is going to work and some things occured to me that maybe others can
comment on.

If this was a simple ftp client, such as ncftp, then when I initiate a
connection to domain.org I could do a simple DNS lookup of _ftp._tcp.domain.org
and plow through the list of SRV records, sorted by priority and picked at
random by weight within the same priority. When a connection failes, remove it
from the list and try again. Once a successful connection has been established
the data concerning SRV records could be thrown away.

The web, by contrast, is a completely different paradigm. Instead of having
one continuous connection throughout the life of the visit, a seperate
connection is used for each "transaction". And each transaction may be
comprised of multiple connections when a particular page contains multiple
objects.

This is where my experience with Cisco Local Directors, ArrowPoints and Central
Dispatch comes in and confuses the issue. These devices are keeping running
tabs on the services that they are watching. For example the Local Director
will mark a service as bad when it gets a failed connection. Even though the
connection would finally make it over to a good server the user experienced a
delay assuming the server was off the network and had to wait for TCP
timeouts. From that point on the LD would direct traffic to services it knew
were good. Every once in a while it would try the bad one just for yuks to see
if it had been fixed.

So assuming we don't want the user to get hung up on a down server every time
they traverse the service list we need to keep a running tab on sevices we've
tried in the past. So does this mean that once we have queried the SRV record
we keep it cached within mozilla until the TTL runs out? Or do we make actual
DNS queries every time and run through the same algorithm to choose a server?
Or we could keep a running list of servers we've connected to and how
successful we were at it in a seperate list. Then we could do seperate SRV
queries every time and compare what we get back to our running list. We would
then try a server marked as down only if we had no other alternatives.

That then brings up the issue of how long do we keep the information around
for? I suppose it is no different than a history, but it eats up memory.

[Evertonm (evertonm)]

> (...) For example the Local Director
> will mark a service as bad when it gets a failed connection. Even though the
> connection would finally make it over to a good server the user experienced a
> delay assuming the server was off the network and had to wait for TCP
> timeouts. From that point on the LD would direct traffic to services it knew
> were good. (...)

Only the connection which "detected" the failed service hangs?

> So assuming we don't want the user to get hung up on a down server every time
> they traverse the service list we need to keep a running tab on sevices we've
> tried in the past. (...)

I haven't understood why the user will always get hung up on a
failed service. The LD is not going to deliver newer connections
to a healthy server after one failure?

[Dnewman (dnewman)]

>> (...) For example the Local Director
>> will mark a service as bad when it gets a failed connection. Even
>> though the connection would finally make it over to a good server
>> the user experienced a delay assuming the server was off the network
>> and had to wait for TCP timeouts. From that point on the LD would
>> direct traffic to services it knew were good. (...)
>
> Only the connection which "detected" the failed service hangs?

Correct.

>> So assuming we don't want the user to get hung up on a down server
>> every time they traverse the service list we need to keep a running
>> tab on sevices we've tried in the past. (...)

> I haven't understood why the user will always get hung up on a
> failed service. The LD is not going to deliver newer connections
> to a healthy server after one failure?

That part I was describing was without LDs and so strictly inside of
the browser. If we requery DNS every time and have to run down the
list servers listed for the service and there is a down server, the
user will get hung up on that down server. It will eventually time
out and the browser will try the next server, but the user will have
to wait. Which is why the implementation might want to cache the
DNS data internal to the browser and keep track of which servers are
up and which ones are down.

Or something else I had thought of, but didn't want to mention
because I didn't like it, was to initiate connections to all servers
listed and use the one that responds first. Or if there are, for the
sake of argument, 20 objects on a particular page and connections
were opened to 4 differant servers, then each connection can serve 5
objects, assuming each server is weighted equally, through the use of
persistant connections.

[Evertonm (evertonm)]

> That part I was describing was without LDs and so strictly inside of
> the browser. If we requery DNS every time and have to run down the
> list servers listed for the service and there is a down server, the
> user will get hung up on that down server. It will eventually time
> out and the browser will try the next server, but the user will have
> to wait. Which is why the implementation might want to cache the
> DNS data internal to the browser and keep track of which servers are
> up and which ones are down.

So you're talking about timeout on requests for the TCP connections.

I'm now wondering whether such "healthy cache" breaks the SRV behavior
as specified by RFC2782. The thing explicitily states:

"for each address record found, try to connect to the (protocol,
address, service)."

While the cache seems a smart approach, it would be a shame to have
the awesome SRV feature not adhering to the proper standard. How
about to make the "healthy cache" an optional, user-tunable feature
and let it disabled by default?

[Jo-hermans (jo-hermans)]

Changing title because RFC2052bis was superseeded a long time ago by RFC2782

[Jo-hermans (jo-hermans)]

*** Bug 162798 has been marked as a duplicate of this bug. ***

[Mozilla-elektrono (mozilla-elektrono)]

I'm disappointed that support for SRV's hasn't made it into Mozilla after three
years. There are are very useful device and are now being used by a number of
services: Jabber, LDAP, Kerberos, Windows 2000, etc...

The main reason I want SRV RRs is so I can have multiple SSL Apache virtual
hosts with one IP. Name-based virtual hosts don't work with SSL, but port-based
virtual hosts would :-)

This document answers many of the questions asked above about how to go about
implimenting http with srv:
http://www.globecom.net/ietf/draft/draft-andrews-http-srv-01.html

It is technically expired, but it's the closest thing to a standard I could find.

[Rick-vanrein (rick-vanrein)]

I have built a working library that supports SRV records in a generic
way. This library is intended for simple applications that want to
"grab a good connection quickly" so it may be unsuitable for Mozilla,
but at least it is example code, open source so ready for copy/paste.

The library can be found on
  http://libsrv.sf.net
and it works on Linux, OpenBSD, FreeBSD, MacOS X.
Any Mozilla coders out there who would like a quick-n-easy way to
get popular? ;-)

[Benc-meer (benc-meer)]

There are lots of issues related to caching and DNS, but I don't think that
necessarily should be a bottleneck to adding this support.

What you probably need to do is look at implementing this with a pref, and then
checking it in as off by default. You should also carefully research any
dependent changes to existing code, because you have to sell this to the module
owners.

After some bake time to make sure the existence of the feature is not causing
regressions, you can have individual users turn it on and see how it works.

Don't worry about caching now, it seems like the problems described w/ the
network load distributors would be issues for normal DNS as well.

[Leknor (leknor)]

I'd like to see this implemented so that things like mail server configuration
can be more automatic.

[Mbabcock-mozilla (mbabcock-mozilla)]

Is David still working on this bug or should this bug's assignment be changed?

Just curious.

Also consider looking at DNS caching discussions from bug 109313

[Jo-hermans (jo-hermans)]

see also bug 173804 (RendezVous/ZeroConf support)

[Adam-cypherspace (adam-cypherspace)]

What's the status on this bug? Is anyone working on it?

It would be a a very useful feature, but the bug seems to be rather old.

Adam

[Njs-netscape (njs-netscape)]

As long as the RFC was 2052, it was an "Experimental" RFC. RFC 2782 is
Standards Track. This should mean that the priority is higher. I guess that I
would like to hear about what the possibilities of supporting this are.

I did a search on "RFC 2782 Support Library" in google and found
http://www.vovida.org/document/vocaldoc/util/srv_readme.txt. There seems to be
other support code that could be "borrowed" running around as well.

[Alexey Zakhlestin (indeyets)]

There is more, actually:
This is an lgpl library which can deal with SRV records withoud dependency on
bind - http://faerion.sourceforge.net/doc/rascal/

currently, it can only be downloaded using CVS, but author will release it as a
package in the near future.

the exact function is, possibly, this:
http://faerion.sourceforge.net/doc/rascal/rascal_connect_service.html

[Everton da Silva Marques (evertonsm)]

There is also ruliwrap:
http://weinholt.se/hacks/

ruliwrap is a wrapper for getaddrinfo() that uses libruli to seamlessly
add SRV record support to programs that use getaddrinfo(3).

[Jo-hermans (jo-hermans)]

*** Bug 293701 has been marked as a duplicate of this bug. ***

[D7dvttp02 (d7dvttp02)]

At the risk of triggering stupendous wrath, but maybe some real discussion, too,
I want to suggest a dastardly hack in lieu of the DNS SRV support envisioned in
this bug report. It seems clear that:

- DNS SRV records have been around for a long time -- even this bug has been
around for a long time, but nobody seems to be implementing it for HTTP, even
though may other, newer protocols have been supporting the DNS SRV records long
since. Even Microsoft can't seem to implement this thing.

- A careful look at the name resolution process might offer an explanation of
why the current SRV proposed standard is not getting implemented in the world of
the interactive web. No matter how you look at it, if you have to query for an
SRV record, and then query for an A record, you have two round trips through the
DNS. Even if everything is working perfectly, the name resolution process will
take (nominally) twice as long if you have to go through an SRV resolution step.
 You could imagine fixing the problem by allowing IP addresses in SRV records
(this is not what the present standard says, and there are in general good
reasons for that), but even then, it is hard to optimize the process of looking
for records that usually won't be there. Negative caching is a hard problem.
(In other words, you have added a full-scale, uncacheable DNS query to every
lousy URL you want to resolve. A web browser that needs to build a complex page
from several URLs would have a hard time presenting a reasonably responsive
experience to the user even if all servers that need to say "no SRV record here"
are configured properly and don't drop the unknown request on the floor.)

- Doing SRV lookups is hard from a programming perspective. It goes against the
grain of the facilities provided in the standard C library. You almost need to
provide an SRV lookup library, and the more libraries you depend on, the less
portable your result.

- Mozilla is the cutting edge of browser development. If we could find a better
way to do it and write an RFC and maybe get Microsoft on board if it was simple
enough, we would have taken care of 95% of browsers out there and the rest would
take care of themselves.

- DNS SRV is wonderful for load balancing, but it's overkill if you just want to
run on an alternate port. (If we can consider the space of TCP and UDP ports to
be part of the address space of an IPv4 network, instead of worrying about
exhausting a 32 bit space, we now have 48 bits, which is a fair amount of
breathing room for the time between now and when IPv6 happens.)

I'm going to propose a simple flag that could be inserted in a set of DNS A
records to indicate that the name describes a service instead of a host and
provide the port where the service can be connected to. It will have the
advantages that:

- It is less than a dozen lines to add (or remove) in only one place,
- It is ridiculously portable,
- It imposes negligible performance/bandwidth/processing penalty/overhead,
- It solves the problem of naming a service on an alternate port, and
- It is safe in that it will not be triggered unexpectedly (keep reading).

Disadvantages:
- I'm proposing it here for the fi...

[Alexey Zakhlestin (indeyets)]

Bugzilla is not a place for creating new RFCs. sorry. You should try NNTP.
also, You are not right, about need of several DNS requests in process of SRV
queries. DNS answer includes "additional" section which already holds the
required data.
for example:

$ host -a -t SRV _irc._tcp.forestnet.org
Trying "_irc._tcp.forestnet.org"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5356
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;_irc._tcp.forestnet.org. IN SRV

;; ANSWER SECTION:
_irc._tcp.forestnet.org. 3600 IN SRV 1 0 6667 irc.forestnet.org.

;; AUTHORITY SECTION:
forestnet.org. 3600 IN NS ns1.cwn.ru.
forestnet.org. 3600 IN NS ve208.msm.ru.

;; ADDITIONAL SECTION:
irc.forestnet.org. 3600 IN A 81.9.48.3
irc.forestnet.org. 3600 IN A 195.91.162.198
irc.forestnet.org. 3600 IN A 213.248.60.142
ve208.msm.ru. 3600 IN A 213.248.60.142

Received 190 bytes from 192.168.0.1#53 in 725 ms

portability is not a big question too. The libraries which were mentioned
earlier are VERY portable.

p.s. just to clear things up. Rascal is BSD-licensed now:
http://faerion.sourceforge.net/librascal.html.var

[Jo-hermans (jo-hermans)]

I have worked with SRV records before (they're used in SIP networks, see
RFC3263), and I can tell you that multiple round-trip is not an issue at all.
SRV and A records will be returned in 1 packet, if there's enough room (that's
where RFC2671 comes in handy).

Note that 127.0.0.1/8 was originally a routable network. I still have customers
that use them, both on private networks and as 'internal' addresses (only
routable in 1 host). But it becomes harder and harder to use them, because lots
of routers and IP-stacks are now hardcoding them (due to RFC3330).

[Numbski (numbski)]

Well, as I'm seeing it right now, we have portable libraries, and we have
emerging technologies like Coral Cache that stand to make excellent use of it,
but no one can use it because the browsers aren't embracing it. :(

Is there really anything left about this whole process that is left to debate,
or is it really a matter of lack of a coder wanting to work on it? If it is the
latter, then I think we're almost approaching the point of needing to place a
bounty on it. Truly (affoardable) redundant systems are just dying for this
technology. You can place a load balancer in your data center all day long, but
if your load balancers fail, you're down. If you can implement SRV, then you
fail to another system, and if configured correctly, you can even pull from
Coral Cache.

Anyway, thoughts? Should be be starting a bounty on this?

Here are some pertinent links:

http://www.ietf.org/rfc/rfc2782.txt

Implementation Examples (not limited to http)
http://www.voip-info.org/tiki-index.php?page=DNS%20SRV
http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy/admingd/ver2_1/ddns.htm

[Bill+mozilla-bugzilla (bill+mozilla-bugzilla)]

I made a Mozillzine forum for people who want to continue non-development discussion:
  http://forums.mozillazine.org/viewtopic.php?p=1676455#1676455

[Christoph Anton Mitterer (calestyo)]

In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
each case, no matter if there are performance questions or anything like this.
The reson is: RFC2782 is a proposed standard according to the IETF.

Are there any current developments regarding SRV RRs in Mozilla products?

[Numbski (numbski)]

(In reply to comment #35)
> In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
> each case, no matter if there are performance questions or anything like this.
> The reson is: RFC2782 is a proposed standard according to the IETF.
>
> Are there any current developments regarding SRV RRs in Mozilla products?

I don't think there is, however just for grins, some pseudo-code for someone
familiar enough with the zilla sources, here are a pair of methods to give a try:

presuming someone has typed in a plain url of the type http://www.domain.tld,
which of course presumes port 80:

Rather than simply attempting to connect to www.domain.tld on port 80, first
attempt to get _http._tcp.www.domain.tld. Respect whatever is returned
(although to be honest, I'm not sure what happens at this point. I really
should try this from telnet sometime to see exactly what occurs here).

Second would be to see if there is a dns library floating around out there that
is lightweight and can handle srv records. When the user hits enter or clicks
the 'go' button, do an actual dns resolution before proceeding, again presuming
_http._tcp.www.domain.tld. Respect what is returned.

I believe the crux of the matter is timeouts. How long do you timeout on trying
srv's? If you go the second route and do a full name resolution and you get
valid srv data back before then reverting to traditional name resolution, I'd
then proceed with the correct priority site. Should that time out, then we
would stray from tradidition a bit. Rather than giving the usual timeout
message, give a new error:

"The requested page could not be retrieved, however the site has specified n
additional backup pages. Would you like to try the next one now?"
[Please take me to the next one.] [No, Thanks though.]
Don't ask me this again.[x]

Just a couple of thoughts on how to deal with it. The user experience wouldn't
be damaged in any way. The srv record timeouts could be adjusted through
about:config, and if done correctly an extension writer could probably extend
things a bit and make them prettier.

[Thomas Butter (tbutter)]

I do not think that the retry logic has to be changed, since browsers should
already try all returned A records. SRV just adds ordering to that logic.

I can offer some SRV query code if anyone wants to adapt it mozilla, but it is
quite easy anyway.

[Elmar-ludwig (elmar-ludwig)]

*** Bug 319939 has been marked as a duplicate of this bug. ***

[Timeless-bemail (timeless-bemail)]

<email address hidden>: lgpl code isn't acceptable for mozilla, please don't suggest it.
<email address hidden>: if you're really interested, i'd suggest you try doing it, build mozilla, hop on irc and ask for help. offering sample code isn't ilkely to get this very far.

personally, this whole thing seems very annoying. i don't feel like reading the rfc, but i would like a basic specification that explains what to do when:
1. there's no SRV record
2. there are 20 alternates in the SRV record and all of them are bad.
3. general cases in between.

The spec should list the series of requests (dns, http, other) as well as wait points and expected time spent at each step. the cost (size) of each inbound and outbound request should be listed, and a comparisoon should be made against today's browsing behaviors.

the following browsing behaviors are available today:
1. host doesn't respond, user tries a different link to a different server
2. user tries google cache for link
3. user tries coral cache for link

[Mozilla-krellis (mozilla-krellis)]

(In reply to comment #39)
> personally, this whole thing seems very annoying. i don't feel like reading the
> rfc, but i would like a basic specification that explains what to do when:
> 1. there's no SRV record
> 2. there are 20 alternates in the SRV record and all of them are bad.
> 3. general cases in between.

To be frank, "seems very annoying" and "don't feel like reading the rfc" seems like a pretty poor attitude to take. If you don't want to take the time to form an educated opinion on the bug, no less perform any work on it, that's certainly your right, but if that's going to be the case, why not just leave it alone?

Anyway, to answer these three questions:

1) If there's no SRV record, everything works as it does now, you look up an address record and make the request.

2) This exact case is not addressed in RFC2782 or Mark Andrews' HTTP SRV draft. I would see two possibilities - either the normal "this request timed out" error is displayed immediately, or a fallback to the standard A record / port 80 behavior takes place. I think the former would be more "correct", as specification of an SRV record says "if you speak SRV, you should use this".

3) I'm not sure what you mean by general cases in between.

If there are any developers out there interested in implementing this functionality (to be contributed back to Mozilla, if it will be accepted) under contract, please contact me off-bug, as my company is interested in sponsoring this work.

[L. David Baron (dbaron)]

This enhancement request depends on having DNSSEC (see http://www.dnssec.net/ ) support, which we don't currently have, although there is a project at http://www.dnssec-tools.org/ working on adding some to Mozilla. It is in some ways similar to bug 242867, but would be even easier for users.

I propose that Thunderbird implement a not-yet-existant standard for auto-configuration of email via DNS which would operate only when the DNS records could be verified via DNSSEC. It could be a single DNS record that defines what servers can be used for handling email from a domain so that a user doesn't have to enter the information into the account manager. For example, given a DNSSEC-verified DNS record that looked something like "imaps:outgoing.example.com;smtptls:incoming.example.com;ldaps:directory.example.com" the user could enter "<email address hidden>" in the account wizard and everything else could be filled in for him. This is one step less than what bug 242867 would get you.

I'm not sure exactly what information would go in such an autoconfiguration format, but at least some of the advantages include:

 * Making it much easier for users to start using Thunderbird.

 * Getting users to use secure SMTP and secure IMAP (either via STARTTLS or via the SSL-specific ports) rather than insecure (which greatly reduces the chances of their password being sniffed).

 * getting users to use authenticated SMTP (preferably secure, and preferably on the smtps (465) or submission (587) port), which can help make it easier for servers to block things spammers do.

The autoconfiguration format would probably also need to contain information about how to form the IMAP and SMTP login name from the user part of the email address, i.e., whether the user should log in as "john" or "<email address hidden>"; this might be specified as "%u", "%u@%d", or "%<email address hidden>", or something like that.

It probably also needs some additional information that I'm not thinking about.

[Mscott-mozilla (mscott-mozilla)]

Thanks for taking the time to file this in such detail David.

I'll put this in the Thunderbird 3 bucket for right now while we watch dnssec get added to mozilla (is there a bug for that part of this feature?)

[L. David Baron (dbaron)]

I couldn't find a bug on DNSSEC.

[Tom-germeau (tom-germeau)]

Where should the dns srv code be added?
nspr or necko only?

I would like to add support using OS API calls and avoid dns libs
DnsQuery (win2k+)
res_query (linux, osx)

Is this enough platform reach to add it to mozilla?

[James-mcininch (james-mcininch)]

(In reply to comment #41)
> Where should the dns srv code be added?
> nspr or necko only?

I'm not sure about that. However, I have been playing with SRV using the POSIX APIs and I'll be willing to contribute the code to Mozilla. I'm not sure if the code will work unchanged under Windows without Cygwin, though it might with MS' pseudo-POSIX libraries.

[Jo-hermans (jo-hermans)]

*** Bug 356150 has been marked as a duplicate of this bug. ***

[Tom-germeau (tom-germeau)]

Created an attachment (id=242044)
OSX, Windows and (most probably) Linux using OS API calls

quote:

Here is a patch that adds SRV support using OS specific api calls
(making use of local cache etc..).
bug 14328

I'm trying to make use of existing interfaces which has the following
concequences:
- the PRNextAddr doesn't take a port or hostname so the only way to
get that info is through PRNextAddrAsString which returns
"hostname:port".
- people advised to to put the code in netwerk/dns so the code could
be easier approved.. but at the end i think the code belongs in the
nspr.

This patch works, but is not really cleaned because of the large
chance of having to refactor it

[Tom-germeau (tom-germeau)]

Created an attachment (id=242046)
Test script for the previous patch

[Usenet-tonal (usenet-tonal)]

Given the need elsewhere (see bug 356104) to support not just one, but several other DNS record types, I think it would make sense not to implement the lowest-level part of this with a single routine that can look up (potentially multiple) records of various types, including TXT and NAPTR records. This could also potentially be extended to support other record types in future.

For example, NAPTR records are essental for DDDS, and TXT records are becoming important in supporting a number of new protocol extensions such as SPF and DomainKeys.

If this was also accessible as an XPCOM service, this would be very useful for adding new functionality from Javascript as well.

[Tom-germeau (tom-germeau)]

> I think it would make sense not to implement the
> lowest-level part of this with a single routine that can look up (potentially
> multiple) records of various types, including TXT and NAPTR records. This could
> also potentially be extended to support other record types in future.

As the RR spec is designed to be extensible. nsIDnsRecord should be able to hold any type of RR. So we can gradually add more RR types _when needed_.

> If this was also accessible as an XPCOM service, this would be very useful for
> adding new functionality from Javascript as well.

ResolveSRV (and the async one) make use of nsIDnsRecord and are accessible from js.

[Usenet-tonal (usenet-tonal)]

Believe me, NAPTR and TXT record support are needed right now.

TXT record support is currently needed for frameworks like SPF and DomainKeys. They would also be useful for a number of more exotic services.

NAPTR records are needed right now if you're going to implement any servive that uses DDDS; for example ENUM services and SIP.

MX and PTR records would be useful, too: consider things like performing double-reverse-lookups.

If these two record types were also supported, say with resolveTXT, resolveNAPTR, resolveMX and resolvePTR methods, this would also close bug 356104 (at least for the time being).

However, what I was proposing was more generic. By implementing an interface like

nsIIDNService::resolveWithRecordType(name, record_type)

where "record_type" is a string in the obvious format, and returns an iterator in exactly the same way as the proposed resolveSRV method, this would remove the need for a method per record type, thus allowing extension of the repertoire of supported records in future without changing the interface.

(I'm not sure what the behaviour should be if a particular record type was not supported, though; presumably something should be chosen which maximizes the possibilities for forwards and backwards compatibility)

resolveSRV could then be implemented on top of this, without any loss of generality or efficiency.

[Tom-germeau (tom-germeau)]

resolve() can stay to resolve cname and A records (and backwards compat)
a method probably called resolveWithType(...) could implement all other extensions.

Probably we need to extend nsIDnsRecord to hold key/value data (propertybag) of the different RR types (for example weight, priority, type, ip, hostname) or have them as a bunch of typed members.
You can see what kind of members we need looking at the DNS RR types here http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dns/dns/dns_structures.asp

[Usenet-tonal (usenet-tonal)]

I completely agree: in which case, all records looked up with the new interfaces (even A records) should return property-bags, even if they only return a single field.

If property-bags are too complex, simple lists of strings, or strings and ints, would also be perfectly adequate.

Needless to say, these more complex return values should only apply to the new resolveWithType() interfaces, so that the existing resolve() interfaces remain 100% compatible with existing implementations.

[Darin-moz (darin-moz)]

(From update of attachment 242044)
Can you please document the API proposed for nsIDNSService?

[Tom-germeau (tom-germeau)]

(In reply to comment #51)
> (From update of attachment 242044 [edit])
> Can you please document the API proposed for nsIDNSService?
>

what about this?
This approach adds 2 methods and an interface per record type.
I added SRV and TXT as examples. The old A record stays untouched.

interface nsIDNSService : nsISupports
{
    /* TXT Record */
    nsIDNSTXTListener resolveTXT(in AUTF8String aHostName);

    nsICancelable asyncResolveTXT(in AUTF8String aHostName,
                                  in nsIDNSTXTListener aListener,
                                  in nsIEventTarget aListenerTarget);
    /* SRV Record */
    nsIDNSSRVListener resolveSRV(in AUTF8String aHostName,
                                 in AUTF8String aService,
                                 in AUTF8String aProtocol);

    nsICancelable asyncResolveSRV(in AUTF8String aHostName,
                                  in AUTF8String aService,
                                  in AUTF8String aProtocol,
                                  in nsIDNSSRVListener aListener,
                                  in nsIEventTarget aListenerTarget);
    /* A Record (unchanged) */
    nsICancelable asyncResolve(in AUTF8String aHostName,
                               in unsigned long aFlags,
                               in nsIDNSListener aListener,
                               in nsIEventTarget aListenerTarget);

    nsIDNSRecord resolve(in AUTF8String aHostName,
                         in unsigned long aFlags);
};

interface nsIDNSTXTRecord : nsISupports
{
    readonly attribute ACString text;
    boolean hasMore();
    void rewind();
};

interface nsIDNSSRVRecord : nsISupports
{
    /* weight and priority are just informational because
     * nsIDNSSRVRecord sorts the records beforehand
     */
    readonly attribute long weight;
    readonly attribute long priority;
    readonly attribute ACString host;
    boolean hasMore();
    void rewind();
};

[Thomas Butter (tbutter)]

What happened to that patch? What would be needed to get SRV support commited?

[Ben-bucksch (ben-bucksch)]

Nice idea.

> Getting users to use secure SMTP and secure IMAP

How do you set up the finer details of the account (see Server Settings, including esp. Advanced... for IMAP)? Outlook Express config files contain a lot of info (usually 10-20 lines long, and a lots more optional).

> information about how to form the IMAP and SMTP login name from the user part
> of the email address

There are quite a number of large ISPs where the login name has no relation to the email address. IIRC that the case for both T-Online (Germany, maybe 50% market share) and France Telecom / Orange (maybe 80% market share).

[Ben-bucksch (ben-bucksch)]

> login name has no relation to the email address

Sorry, more concretely: login name is numeric and generated by ISP, e.g. 858675467; email address username part can be freely chosen by user (e.g. ben.bucksch).

[Volkmarkostka (volkmarkostka)]

Because i filed a related bug ( #393372 ) i just want to ask if anyone was able to get SRV records at all.
I tried DnsQuery on W2K and WXP and asked sites like myspace, youtube, google and microsoft. All of them returned several A records but no SRV records.

[Thomas Butter (tbutter)]

Those sites don't publish SRV records. You could try "_http._tcp.butter.eu" which has one.

[Volkmarkostka (volkmarkostka)]

Thanks. That works (got one SRV record).
But what is the sense in implementing this when no major sites are offering such records? Fit for the future? You have still to check for A records.

[Adam-cypherspace (adam-cypherspace)]

comment 56 says "hat is the sense in implementing this when no major sites are offering such records? Fit for the future?"

Well SRV records are clearly a good idea, they are a more generic version of the MX record applying to mail receipt. The problem is chicken and egg ... you cant expect any sites to support when the browsers dont support it. So if we start with firefox supporting SRV, then sites will have an interest to provide SRV records, and sites will work more reliably for browsers supporting SRV records, and then IE will want to add SRV record support to catchup with firefox.

[Jo-hermans (jo-hermans)]

Just wait until networks like Akamai start using SRV records to replace their current short-TTL records. Or websites with massive load balancing (Youtube, Gogoel, etc ...)

[Volkmarkostka (volkmarkostka)]

Thanks. Got it.

[higuita (higuita)]

i work with a hardware load balancer to balance several web servers and i can tell that i would prefer SRV to do that job and dump that horrible hack that is the load balancer...

but why would i try to configure the SRV records and even worst, tune then, if no users will use it, or even i cant test it with a browser

browsers need to support this first... if its supported, people will use it as its usefull, alot simpler and cheap than load balancers

[Nikolay Shopik (shopik)]

Most great idea for next major release. DNS SRV records do half of this job and this is where it should starts. Mostly in controlled environments (enterprise, university, etc) this do great job not everything like ISP hooks but most and with almost no efforts at all.
Probably I ask one coder to contribute code as extension for TB.

[Sgautherie-bz (sgautherie-bz)]

Boris, Christian, Darin:
any hope to get a review in time for Gecko 1.9 ?

[Bzbarsky (bzbarsky)]

I don't think I'm qualified to review this code.

I also think you should have poked the reviewers about 6 months back. At this stage, even if it had reviews it might not get accepted -- 1.9 is locking down.

[Rodc (rodc)]

I too would very much like to see this implemented in the very near future. The sooner the better in my books as I think srv records are way past due on the net and and this 8 year old enhancement request really needs to be implemented.

[Tom-germeau (tom-germeau)]

I have this code running and stable in our mozilla app for a long time now. With new linux/osx/windows code. Although I'm using the GetNextAddrAsString hack as in the current patch.

So I'm just waiting until someone with authority in this area can tell me if the interface I proposed is the right way to go. I'm not going to rewrite our own code to a new interface if no one here cares.

[Mike Connor (mconnor)]

We've shipped something like 11 major releases (and over a hundred minor releases) without this, not going to block.

[Nikolay Shopik (shopik)]

current implementation for next generation of TB lies here
http://wiki.mozilla.org/Thunderbird:Autoconfiguration
also maybe bug 242867 dup of this one, I'm not sure about this!

[Mike Connor (mconnor)]

This is entirely inappropriate to take in a security/stability release.

[Sgautherie-bz (sgautherie-bz)]

(In reply to comment #66)
> This is entirely inappropriate to take in a security/stability release.

I thought it was debated that the follow-up 1.9.0.x/1.9.1 could be more than that.
Anyway, 'wanted‑next=?' then?

[Nikolay Shopik (shopik)]

Wayne, could you put url in comment #6 to bug url

[Eyardeni (eyardeni)]

Created an attachment (id=322140)
Linux, Windows and (probably) OSX SRV/NAPTR support

[Eyardeni (eyardeni)]

I have submitted the patch in Attachment #322140 for adding SRV/NAPTR support.

I have added resolveWithType(...) and asyncResolveWithType(...) API.
The sync API returns a nsISimpleEnumerator that is used as an iterator to the resolved records that could be of type nsIDNSSRVRecord/nsIDNSNAPTRRecord or nsIDNSRecord.

[Eyardeni (eyardeni)]

Created an attachment (id=322495)
 Linux, Windows and OSX SRV/NAPTR support

I have attached a modified patch that includes OSX compilation fix, libxul build compilation fix and a better way for implementing the nsISimpleEnumerator interface for the new listener.

[Mkmelin+mozilla (mkmelin+mozilla)]

Please leave setting the Priority to assignees.

[Nikolay Shopik (shopik)]

*** Bug 242867 has been marked as a duplicate of this bug. ***

[Morz (morz)]

Taken from my comment to bug 389275, this proposal has the following disadvantages:

- Uses DNSSEC, which is not widely deployed and has well-known deployment and operational issues (like DNS root and most TLDs not being signed).

- Requires an effort of DNS coordination when a single e-mail provider wants to
deliver automation for accounts under multiple domains, as opposed to providing
auto-config for a single (own) domain.

- Forces the user to enter his own data, and hence does not cover the important
case where you provide a configuration file with ALL the account's
configuration (including e-mail address, IMAP/POP user id, default folder
names, and perhaps even password), to prevent users from making mistakes.

[Nikolay Shopik (shopik)]

David can we have this in beta1 or beta2? This is one of major thing we like to introduce in TB3. But its seems it need lots work and probably can't make it for b1.

[Ben-bucksch (ben-bucksch)]

Nikolay, I planned to start with this soonish.

[David-ascher (david-ascher)]

FYI, there is some forward motion on the dialog for auto-configuration w/o DNS/DNSSEC, as per the design here:

  http://clarkbw.net/designs/account-wizard/account-dialog/account-dialog-2.html

in bug 422814. Bienvenu has some of the backend code working, and I've got a start on the front-end (I'll attach a screenshot of the WIP).

IMO, that is relatively low-hanging fruit, based on port sniffing & heuristics for domain names. There are a bunch of added layers that we can put on, including DNS/DNSSEC-based approaches, hosting a database of known configurations, etc.

[David-ascher (david-ascher)]

Created attachment 333621
screenshot of WIP, with no logic behind it yet

[Dmose (dmose)]

Note that the DNS stuff we have in necko is not really a resolver, but just a wrapper around the host OS getaddrinfo() implementation. In order to do anything interesting that's DNS-based, we'd need a real resolver in Gecko, which is a non-trivial amount of work. If someone wants to do that, we should investigate the license & code compatibility of the various available open-source resolvers. For the Tb3 time frame, though, doing something DNS-based seems a little impractical.

[Ben-bucksch (ben-bucksch)]

FYI, I am planning to fetch the configuration from a webservice database via http for now, not DNS. The most I may do via DNS is to lookup a certain predefined hostname. See https://wiki.mozilla.org/Thunderbird:Autoconfiguration, Section "Proposal", Step 2 ("contact a mail configuration server of the provider"), Alternative 2: "just try to contact https://autoconfig.emailaddressdomain/mail/mozilla.xml?emailaddress=emailaddress and see whether that host/URL exists"

[Nikolay Shopik (shopik)]

Thanks David, I also keep track of bug 422814.
Ben, that's great, I'll be glad to do QA for this.

[David-ascher (david-ascher)]

The DNS/DNSSEC parts of auto-configuration will most likely not make 3, but other forms of autoconfig likely will.

[Dmose (dmose)]

(In reply to comment #16)
> FYI, I am planning to fetch the configuration from a webservice database via
> http for now, not DNS. The most I may do via DNS is to lookup a certain
> predefined hostname.

Comment 0 is pretty clear about the scope of this bug. It sounds like the work you're doing should be tracked in another bug.

[Ben-bucksch (ben-bucksch)]

OK, filed bug 450710 about what I want to implement.

[Ben-bucksch (ben-bucksch)]

*** Bug 411898 has been marked as a duplicate of this bug. ***

[Matěj Cepl (mcepl)]

Having a JID <email address hidden>, xmpp4moz should (according to 14.3. of RFC3920) resolve the hostname of the server using SRV service:

[matej@viklef ~]$ host -t SRV _xmpp-client._tcp.ceplovi.cz
_xmpp-client._tcp.ceplovi.cz has SRV record 0 0 5222 luther.ceplovi.cz.
[matej@viklef ~]$

xmpp4moz certainly doesn't resolves server name with SRV (so it has special type of GMail account distinct from Jabber account, which is unnecessary when SRV records are followed).

[Massimiliano Mirra (bard-hyperstruct)]

xmpp4moz cannot support DNS SRV until Mozilla does. https://bugzilla.mozilla.org/show_bug.cgi?id=14328

[Sgautherie-bz (sgautherie-bz)]

Unassigning from current assignee, who did not comment since 2001' comment 15.

[Sgautherie-bz (sgautherie-bz)]

(In reply to comment #62)
> I don't think I'm qualified to review this code.

(13 months later)

Boris, do you know who could review this ?
You, Christian and Darin are the only ones listed on http://www.mozilla.org/owners.html...

[Sgautherie-bz (sgautherie-bz)]

(From update of attachment 322495)
Dave, Boris suggested me to try asking you...

[Mike-complexplastics (mike-complexplastics)]

Is this patch going to make it to Stable release anytime soon? I am no programmer and have no clue what to do with this patch. I would like my web sites to have backup/load balancing features without the need to rely on hardware devices that can break down/may be off the Internet cause of a route failure.

[Mozilla7 (mozilla7)]

The proposal "that Thunderbird implement a not-yet-existant [sic] standard" is fine and all, but there is an existent standard that we should be seriously looking at, already supported in Outlook 2007, and theoretically also in Apple Mail on Mac OS X 10.6 "Snow Leopard" and the iPhone (although I have filed bugs with Apple because I can't get it to work).

Word document: http://office.microsoft.com/download/afile.aspx?AssetID=AM102105061033
Exchange 2007 whitepaper: http://technet.microsoft.com/en-us/library/bb332063.aspx
Autodiscover reference: http://msdn.microsoft.com/en-us/library/aa581522.aspx

[Ben-bucksch (ben-bucksch)]

First, Exchange autodiscover is not a standard by any means. It's only supported by Microsoft. Second, we have been looking at Autodiscover closely, even discussing with Microsoft at length. Third, Autodiscover does not work via DNS. That's what this bug is about.

[Ben-bucksch (ben-bucksch)]

s/only supported by Microsoft/only supported by Microsoft Exchange (server side)/

[Mozilla7 (mozilla7)]

Most of the documentation for server-side implementation is for Exchange, but I've implemented it myself with a Perl script. I've seen a PHP script for it floating around as well.

[Smoerk (smoerk)]

today: 10th anniversary of the bug. let's party!

[L-wood (l-wood)]

RFC2502bis referenced in the description has now been published:
http://tools.ietf.org/html/rfc2782

Some internet-drafts related to SRV and DNS records:
as mentioned above (expired):
http://tools.ietf.org/html/draft-andrews-http-srv

for determining transport - HTTP has been run over SCTP and TCP, IPv4 and v6; how would you know which to use? Possible methods include DNS record lookups and specifying the transport direct in the URI.
http://tools.ietf.org/html/draft-natarajan-http-over-sctp
http://tools.ietf.org/html/draft-yourtchenko-tran-announce-dns
http://tools.ietf.org/html/draft-wing-http-new-tech
http://tools.ietf.org/html/draft-wood-tae-specifying-uri-transports
http://tools.ietf.org/html/draft-jennings-http-srv
http://tools.ietf.org/html/draft-faltstrom-uri

[Smoerk (smoerk)]

(In reply to comment #76)
> RFC2502bis referenced in the description has now been published:
> http://tools.ietf.org/html/rfc2782

it's already in the title :-)

> Some internet-drafts related to SRV and DNS records:
> as mentioned above (expired):
> http://tools.ietf.org/html/draft-andrews-http-srv
>
> for determining transport - HTTP has been run over SCTP and TCP, IPv4 and v6;
> how would you know which to use? Possible methods include DNS record lookups
> and specifying the transport direct in the URI.
> http://tools.ietf.org/html/draft-natarajan-http-over-sctp
> http://tools.ietf.org/html/draft-yourtchenko-tran-announce-dns
> http://tools.ietf.org/html/draft-wing-http-new-tech
> http://tools.ietf.org/html/draft-wood-tae-specifying-uri-transports
> http://tools.ietf.org/html/draft-jennings-http-srv
> http://tools.ietf.org/html/draft-faltstrom-uri

Thanks for the links. I have skimmed through the drafts and I don't see any advantage over SRV / rfc2782. Things like http+srv are a usability nightmare, others trying to implement something similar, but with less flexibility. rfc2782 is a simple and good standard.

I still wonder, why nobody is pushing this for firefox.

[Nikolay Shopik (shopik)]

This still give us no clue who are qualified to review patch, while owners pages suggest its Christian Biesinger, Darin Fisher, Boris Zbarsky.

[Andre Schild (a-schild)]

Yes, of course exchange auto discovery is not a standard, but it is a specification which is used in outlook 2007, apple mail and iPhone.

Why not use it, instead of building our own ?

Here a example of a PHP (serverside) script for generation the config file: http://www.andrewyager.com/content/view/52/27/

It is very simple to configure the web server to provide such a script.

[Ben-bucksch (ben-bucksch)]

Andre,
a) This is offtopic here, because this bug is specifically about DNS, and the MS protocol is based on XML. (Outlook can also look up DNS SRV records, but only when explicitly enabled by pref.)
b) Microsoft itself dropped the protocol that Exchange 2007 uses, because they didn't like it anymore. Exchange 2010 uses a different, SOAP-based protocol.
c) Both protocols require the user not only to know the username in advance, but also to authenticate to Exchange, before the server returns any configuration (even though it's just hostnames etc.). I find myself often not knowing how exactly my username is expected (with @ or //, with or without domain, email address or other username), so that alone is a no-go. We go from only realname, email address and password.

That the XML-based protocol has already been implemented in Thunderbird 3.
This bug is about finding servers via DNS.

[Mozilla7 (mozilla7)]

I filed bug 521538 for supporting Microsoft's autodiscovery protocol; please go there for that. :-)

[Morz (morz)]

(In reply to comment #27)
> That the XML-based protocol has already been implemented in Thunderbird 3.

Could you post an URL with the specification for such protocol?

[Ben-bucksch (ben-bucksch)]

I did, in comment 16. Please read before writing, thanks.

[Morz (morz)]

(In reply to comment #30)
> I did, in comment 16. Please read before writing, thanks.

The stuff in https://wiki.mozilla.org/Thunderbird:Autoconfiguration has some implementation notes, but is not a specification.

[Ben-bucksch (ben-bucksch)]

That and <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat>
(linked there) is sufficient to implement it.

[Morz (morz)]

(In reply to comment #32)
> That and
> <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat>
> (linked there) is sufficient to implement it.

I am aware of the subpages, but:

1) All the IMAP stuff is missing.
2) A lot of things are undefined or unclear, including very basic things (for example the defaults handling, or how to deal with multiple identities).
3) "All settings and enum values" are also in the TODO list, which does not sound as "sufficient to implement it".

In any case, apparently it has the same design faults that were pointed out for the Autoconfig RDF files in TB2, years ago. For example, linking the identity of the email provider with the domain of such provider (although this is also unclear, as it is unknown the role of the 'id' attribute in the 'emailProvider' element).

If this has been already implemented following the lines shown in these notes, IMHO this is a lost opportunity for a highly important feature.

[Ben-bucksch (ben-bucksch)]

morz, please email me about concrete problems you see (in more detail than above). I don't want to respond here, because it would be offtopic.

[Bubblboy (bubblboy)]

Another poor soul pleading for updates on this bug. Given the context (a *ten year old* bug) I would personally argue that this is not even bad form anymore.

Please, Mozilla, please fix this. This is the one thing preventing people from being able to set up cheap hosting easily. The only functionally equivalent alternative is lots of heartbeat hosts and very low TTL DNS records, which is a strain on both administration of those hosts and on the DNS system.

Everybody will benefit from this. I beg of you, somebody, please at least tell us what is going on here. Why is nobody working on it?

To put my money where my mouth is: I am willing to actually pay money to whoever decides to implement this. Not much, but if a few other people chip in I bet we could make at least a modest sum. It will definitely save me the hosting costs, so I think it would be worth it.

[Kjetil Torgrim Homme (kjetilho)]

Hraban, the reason this bug is unresolved is that there is no standard for HTTP in SRV. someone needs to write an RFC and get it approved. Mark Andrews wrote a draft, but it never reached RFC status.

the SRV spec says that a standard must explicitly support SRV, you can't just apply it to a random protocol. it also says that the name from IANA assigned numbers should be used, but the registration for port 80 specifies several names: "http", "www" and "www-http". in other words, it's not clear if you should query for _http._tcp or _www._tcp -- this is why a formal specification is needed.

IETF is open for anyone to submit a draft. it just takes some time and interest for writing formal documents and engaging in IETF discussions.

[Bubblboy (bubblboy)]

Kjetil,

You are bringer of terrible news. I hate to say that this seems to be right... before we can use this:

1) The SRV draft must become a real standard.
2) The HTTP standard must explicitly specify how to use it.
3) Some browser must support it.

I thought step 3 alone was daunting enough, but this... What a shame.

Regardless, thank you for your very clarifying post. I think I can finally move on now. ;)

Greetings,

Hraban Luyat

[Mysidia (mysidia)]

It appears clear that 'http' is the preferred service name for an interactive browser such as Firefox that is used for human-readable content.
As listed:

DNS-SD Service types
http://www.dns-sd.org/ServiceTypes.html

[Ben-bucksch (ben-bucksch)]

After many years, DNSSEC is starting to be deployed. ICANN and IETF plan that in 2 months from now, the first public root server publishes a DNSSEC-signed root. TLD .org is already signed.

[Chris-newman (chris-newman)]

I'm voting for the bug, but only on the condition it uses a standard mechanism such as the approaching finalized IETF proposal:

  http://tools.ietf.org/html/draft-daboo-srv-email

I don't see that it's necessary to wait for DNSSEC to implement that. DNSSEC will deploy when it deploys and we'll get the extra security bump when it shows up.

I consider central-server based architectures such as the Mozilla ISPDB to be a poor design that introduces a single-point of failure. One customer is correctly worried about the extra dependency that creates.

[Jesse-thompson (jesse-thompson)]

Am I the only one who thinks that Mozilla's new centralized approach of client autoconfiguration is a poor idea?

https://wiki.mozilla.org/Thunderbird/ISPDB/Requirements
http://ispdb.mozillamessaging.com/
https://wiki.mozilla.org/MailServerList

It allows for unverified users to submit the information (which has already happened for our domain, but they mostly got it right.)

It is also kind of a pain to manage if you host lots of domains (we host 270 domains.)

It also makes mozilla.org a dependency, which leads to concerns about their reliability and longevity.

How can we ensure that this mozilla-centric model won't interfere with other clients that want to implement autoconfiguration. If we have to backfill this information for 270 domains, I don't want to do it again for each client that chooses to implement their own centralized database.

[Jesse-thompson (jesse-thompson)]

Coincidentally, the XMPP folks are struggling with the same problem. XMPP client SRV records work wonderfully, but they rely on the server using Start-TLS with a signed certificate that matches the virtual domain. This has been a complaint of mine for some time, because obtaining certificates for all of our 270 domains is the biggest obstacle to enabling all of the email domains within our chat service. They have recognized the shortfall in this methodology (no doubt Google Apps Talk had something to do with this) and are working on the following draft to solve the problem.

http://tools.ietf.org/html/draft-hildebrand-dna-00

[Nikolay Shopik (shopik)]

(In reply to comment #36)
> I'm voting for the bug, but only on the condition it uses a standard mechanism
> such as the approaching finalized IETF proposal:
>
> http://tools.ietf.org/html/draft-daboo-srv-email

I do agree with you Chris, but you may look into bug 356104 and bug 14328, these are showstoppers. Necko doesn't have code right now to resolve SRV records.

[Ben-bucksch (ben-bucksch)]

> Am I the only one who thinks that Mozilla's new centralized approach of
> client autoconfiguration is a poor idea?

No, you're not. It wasn't intended for the use case you mention. Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

[L. David Baron (dbaron)]

(In reply to comment #36)
> I don't see that it's necessary to wait for DNSSEC to implement that. DNSSEC
> will deploy when it deploys and we'll get the extra security bump when it shows
> up.

I'd agree that it's not necessary to wait for DNSSEC if all servers found in this manner require SSL/TLS (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match).

[Jesse-thompson (jesse-thompson)]

> (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match)

Requiring that certs match the domain will be a show stopper for services that host lots of domains. We are an EDU, and we host 270 domains. Just think about services that host thousands of domains!

Supposing it is practical for email service administrators to maintain hundreds or thousands of valid signed matching certificates for domains that they don't own - it isn't practical, IMHO - do we know if the major MTAs support the ability to present a different cert for the individual domain during the Start-TLS negotiation? If not, that will also be a show stopper.

Is it feasible to abandon plain old TLS? Outlook still seems to work better with port 465/TLS. Start-TLS would be a new requirement since the domain needs to be known before the server can present a matching cert.

Take a look at http://tools.ietf.org/html/draft-hildebrand-dna-00 for some ideas on other methods of asserting domain ownership without the need for DNSSEC.

[Jesse-thompson (jesse-thompson)]

> Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

Yes, step 2 is a more rational approach. I still think that step 3 is problematic just in its existence. I see that there is mailing list http://groups.google.com/group/ispdb so I'll voice my concerns there.

[Brett Zamir (brettz9)]

I'd also like to see it, specifically with XPCOM access for extension use and use in other protocols like XMPP...

[Ben-bucksch (ben-bucksch)]

The patch here is great, because it shows how to make arbitrary DNS queries, but it unfortunately limits the API to SRV and TXT. In bug 342242, we need MX, too. So, I filed bug 545866 for an IDL API to allow arbitrary DNS record types to be queried from within Mozilla. This bug could then use this API.

[Usenet-tonal (usenet-tonal)]

See bug 14328 for the work being done on extending DNS support to include the resolution of SRV records, which seems to me to be closely related to the work being discussed here.

[Ben-bucksch (ben-bucksch)]

Yes, I filed bug 545866 (arbitrary DNS queries in Mozilla) based on that and made it block this bug.

[Ben-bucksch (ben-bucksch)]

Narrowing bug summary to DNS SRV records, to differentiate from DNS MX, which I filed as bug 551519.

[Nikolay Shopik (shopik)]

*** Bug 536915 has been marked as a duplicate of this bug. ***

[Nikolay Shopik (shopik)]

draft-daboo-srv-email is now http://tools.ietf.org/html/rfc6186

[Ludovic-mozilla (ludovic-mozilla)]

*** Bug 675476 has been marked as a duplicate of this bug. ***

[WhyNotHugo (whynothugo)]

If this is implemented, I'd suggest it be in compliance with rfc6186.

I don't see DNSSEC as a requirement, but rather an extra.
There's really little harm that can be done on a DNSSEC-less scenario that can't be done right now. And even if the SRV records are on a DNS domains that implements DNSSEC, the actual IMAP/SMTP servers might be in DNSSEC-less domains.

[WhyNotHugo (whynothugo)]

Any updates on this issue? The improvements to user experience by implementing this are huge:

Now:
User need to know email, username, password, smtp server, imap server, smtp port, imap port.

With RFC6186
User needs to know email, password.

[Ben-bucksch (ben-bucksch)]

Just FYI:
RFC 6186 doesn't give username.
The following method gives username and a number of other parameters, and it works today, see
https://developer.mozilla.org/en/Thunderbird/Autoconfiguration

[WhyNotHugo (whynothugo)]

There are some severe downsides to the ISBDB:

 1) It's controlled by Mozilla, instead of each ISP.
 2) It won't work on LANs.
 3) It requires intervention on Behalf of mozilla to update information.
 4) Only thunderbird uses this information. If each email client had it's own non-standard mechanism, then ISPs would have a huge burden to update each of them with new information when they change their email server.

The other mechanism:
 1) Shares point (4).
 2) Is inheretly insecure (why HTTP instead of HTTPS?)

While I have nothing in particular AGAINST maintaining support for mozilla's in-house mechanism, I belive that supporting standardized mechanisms (especially RFCs) is ultimately important.

[Ben-bucksch (ben-bucksch)]

> 1) It's controlled by Mozilla, instead of each ISP.
> 2) It won't work on LANs.
> 3) It requires intervention on Behalf of mozilla to update information.

Not true for the ISP fetch.

> 2) Is inheretly insecure ... HTTP

So is DNS (at least without DNSSEC)

[Bugmail-asutherland (bugmail-asutherland)]

(In reply to Hugo Osvaldo Barrera from comment #53)
> 4) Only thunderbird uses this information. If each email client had it's
> own non-standard mechanism, then ISPs would have a huge burden to update
> each of them with new information when they change their email server.

Evolution uses it too, although it looks like they host their own copy of the database. Check out:
http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
which uses a (sadly non-SSL) base url of:
http://api.gnome.org/evolution/autoconfig/1.1/

The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.

Note: Neither this nor I believe Ben's comments are meant to suggest that we don't want to support DNS SRV. Unfortunately, it's still a non-trivial undertaking to get the support in the gecko platform that has not been prioritized on the platform side. For Firefox OS v2 we are hoping to get it prioritized.

[WhyNotHugo (whynothugo)]

(In reply to Ben Bucksch (:BenB) from comment #54)
> > 2) Is inheretly insecure ... HTTP
>
> So is DNS (at least without DNSSEC)

So rfc6186 provides secure autoconfiguration for domains that implement DNSSEC. There's no way to achieve secure autoconfiguration with this mechanism, since HTTP provides no security mechanism.

However, my strongest point is it's non-standardness, while rfc6186 is standard, and grants total control (and optional security) to the ISP.
Also, there's no reason both autoconfiguration mechanisms can't exist.

(In reply to Andrew Sutherland (:asuth) from comment #55)
> (In reply to Hugo Osvaldo Barrera from comment #53)
> > 4) Only thunderbird uses this information. If each email client had it's
> > own non-standard mechanism, then ISPs would have a huge burden to update
> > each of them with new information when they change their email server.
>
> Evolution uses it too, although it looks like they host their own copy of
> the database. Check out:
> http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
> which uses a (sadly non-SSL) base url of:
> http://api.gnome.org/evolution/autoconfig/1.1/

Sadly, this is a perfect example of how rfc6186 is superior, since every email server's configuration is in it's owner's DNS server.

>
> The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.
>
>
> Note: Neither this nor I believe Ben's comments are meant to suggest that we
> don't want to support DNS SRV. Unfortunately, it's still a non-trivial
> undertaking to get the support in the gecko platform that has not been
> prioritized on the platform side. For Firefox OS v2 we are hoping to get it
> prioritized.

Actually, there's an alternate fix for this issue; if Mozilla's ISPDB contains no information related to a particular domain, it could query it using rfc6186, and return that.

This means that Thunderbird would still [indirectly] resolve using rfc6186, with no actual changes to the client code.

[Alexey Zakhlestin (indeyets)]

Shouldn't this depend on bug 14328 instead of 545866?

[chrysn (chrysn)]

With the DNS-SD support brought with Flyweb in Firefox, the lack of DNS support in Gecko should have been mitigated. The security implications of using DNS data do not seem worse than those of using the current HTTP based mechanism.

[Noah (noahod)]

I would love to have this implemented in Thunderbird, as a sysadmin that would like to make it easier for our users to connect to our service.

Regarding the security context of this, I actually think it's a great improvement, because it means people are more likely to automatically set up encrypted mail through imap(S) than if they have to manually enter the details.

2. As pointed out the MiTM risk is no worse than the current http method, and the guessing method (which actually uses the wrong server for us)

3. The MITM risk is only at setup time. If the user doesn't use autoconfig, and ends up using unencrypted mail, they are at MiTM risk everywhere they go, every coffee shop wifi, etc. Since the MitM risk is only at setup time, it's extremely hard for an attacker to exploit, because they have to convince the user to delete and recreate the connection in order to be able to hijack the autoconfigure request.

4. Doesn't the current UX flow mean that the users password is only sent to the server *AFTER* the autoconfigure completes and the server details are shown to them? What is the problem here? Doesn't the user get a chance to verify the server details?

[Ben-bucksch (ben-bucksch)]

If you want to make it easy for users, you can already do that today with http instead of DNS SRV:
https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration

[W-jan-k (w-jan-k)]

(In reply to Ben Bucksch (:BenB) from comment #60)
> If you want to make it easy for users, you can already do that today with http

Literally "http://". Bug 986967 is about that and is a real security problem.

[Ilf-6 (ilf-6)]

RFC 6186 was published in 2011, but it remained only an option. In January 2018, RFC 8314 was published: https://tools.ietf.org/html/rfc8314

This explicitly sais:

> User-configurable MUAs SHOULD support the use of [RFC6186] for account setup.

https://tools.ietf.org/html/rfc8314#section-5.1

So please, let's follow the RFC and provide this option in Thunderbird.

Thanks, and keep up the good work!

[Ben-bucksch (ben-bucksch)]

Although RFC8314 is on the standards track, it's more a "best practices" document than an actual protocol definition.

It's true that Thunderbird does not implement configuration via DNS SRV (this bug here).
There are several fundamental reasons:

1. Arbitrary DNS lookups - bug 545866 / bug 14328 - are not implemented by Gecko.
   I've spent a lot of time implementing it, over 8 years ago (April 2010),
   only for the patch to be ignored and not reviewed. Forgive me when I'm a little less excited today.

2. DNSSEC is very hard to administrate without shooting yourself in the foot, and consequently,
   it's not widely deployed on second level domains. APNIC says:
   "Unfortunately, our recent study showed that DNSSEC deployment is still very low
   (only ~1% of .com, .net and .org domains deploy DNSSEC)"
   https://blog.apnic.net/2017/12/06/dnssec-deployment-remains-low/
   On most domains, DNS is completely insecure. I don't see it gaining quickly, either.
   APNIC <https://stats.labs.apnic.net/dnssec/XA> actually shows declining deployment in the last year.
   Without DNSSEC, DNS SRV is no more secure than http. In that light, comment 61 is a little ironic.

3. DNS SRV does not give us some critical information we need to configure the accounts.
   Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or
   even something like EXAMPLE\\bbob? Get it wrong, and the configuration doesn't work.
   Sure, we can try them all. And risk authentication failures and - worst case - triggering
   3-failures-in-a-row-and-you're-out security mechanisms.
   We're back to guessing and trail&error.
   That defeats the entire point of getting an authoritative configuration that will definitely work.
   If we were to support DNS SRV, then only as a second-class, fallback, legacy format.

DNS SRV is not widely used now. Why trying to establish a mechanism that has known shortcomings?

I think people here want this, because they want a standards-based mechanism. I fully concur with that.
But I'd rather make a new standard than implement a bad standard that nobody else implements anyway.

[Ben-bucksch (ben-bucksch)]

On the up side: Thunderbird already implements most of what RFC8314 requests, including most SHOULDs. In fact, it seems RFC8314 took Thunderbird as reference implementation, and basically says "Everybody should do what Thunderbird does" ;-) .

[Chris-newman (chris-newman)]

The goal in RFC 8314 was to reasonably promote security/interoperability improvements without forcing a lot of software to be declared broken. It wasn't actually based on what Thunderbird does, but if that's the impression, that's good.

The IETF has backed SRV records in general and email+SRV in particular so the spec followed that trend on the auto-configuration front. If the various ad-hoc proposals for HTTP+XML email auto-configuration were to come up with a single compromise proposal and bring that to the IETF, I wouldn't expect major problems moving that to the standards track, but unless that's done, DNS SRV is the only standard for email account auto-configuration and Thunderbird has a standards-gap by not implementing it.

[William66750 (william66750)]

It's easier to add a dns field, than to configure via a web server
We will hope that the RFC will be applied

[Martin (martin22)]

For the record (and potential inspiration), implementation of rfc6186 in kmail is done: https://github.com/k9mail/k-9/pull/4719

[O-esa (o-esa)]

After 14 years this is still open. What I can't figure out is why not having DNSSEC with RFC 6186 is seen as such a big problem while the currently implemented [Autoconfiguration](https://wiki.mozilla.org/Thunderbird:Autoconfiguration) feature gets the information from plain HTTP URLs. Both methods are identically vulnerable to MitM.

[Ben-bucksch (ben-bucksch)]

Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

[O-esa (o-esa)]

(In reply to Ben Bucksch (:BenB) from comment #69)
> Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

Thanks, Ben! Some good and valid arguments there. It's just that while the current approach might have some clear advantages over RFC 6186, it doesn't seem any better security wise – quite the opposite. Or does the Autoconfiguration perhaps support HTTPS and HSTS (RFC 6797) with preloading, which would make forcing TLS possible?

[Zocker-net (zocker-net)]

(In reply to Ben Bucksch (:BenB) from comment #63)
> 3. DNS SRV does not give us some critical information we need to configure the accounts. Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or even something like EXAMPLE\\bbob? Get it wrong, and the configuration
> doesn't work.
> Sure, we can try them all. And risk authentication failures and - worst
> case - triggering
> 3-failures-in-a-row-and-you're-out security mechanisms.
> We're back to guessing and trail&error.
> That defeats the entire point of getting an authoritative configuration
> that will definitely work.
> If we were to support DNS SRV, then only as a second-class, fallback,
> legacy format.

This question is already addressed by RFC 6186, see under Section 4:
> When a user identifier is required, MUAs MUST first use the full email address provided by the user, and if that results in an authentication failure, SHOULD fall back to using the "local-part" extracted from the email address. This is in line with the guidance outlined in Section 5. If both these user identifiers result in authentication failure, the MUA SHOULD prompt the user for a valid identifier.

According to Section 5, service providers SHOULD implement their service so that either the mail address or the "local-part" only is sufficient for authentication. So this should work for most cases.

I propose that for the username Thunderbird should do according to the RFC. If both variants fail, Thunderbird could then fallback to first try out the other auto configuration methods. If any other auto configuration method are configured, Thunderbird could use those and ignore the DNS SRV records. If only the DNS SRV are configured, it could ask the user. If such rare case happens, users should probably know which username they should use. If not, it is still better to only ask for the username instead of currently not implementing DNS SRV because it might not work perfectly and ask the user for the complete server configuration.

To me, it sounds like you want users to have a great experience with Thunderbird. However, I think users will now go to a mail provider which claims to support auto configuration according to common standard (referring to RFC 6186) and they will see that most mail applications support his mail provider out of box, but not Thunderbird, which in my opinion could lead that users might not want to use Thunderbird "because it does not just work".

[Lubomir Spacek (spacek-6)]

Hello,

at what stage is the implementation of RFC6186? It seems like it still doesn't work. It would really make life easier for us ISPs.

[Karana-8 (karana-8)]

bug 1852752 seems to be necessary before this can be done

[Vseerror-i (vseerror-i)]

(In reply to Selek Respa from comment #73)
> bug 1852752 seems to be necessary before this can be done