xmpp4moz doesn't read SRV DNS records

Bug #308181 reported by Matěj Cepl
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mozilla Thunderbird
Confirmed
Unknown
SamePlace
Invalid
Undecided
Unassigned

Bug Description

Having a JID <email address hidden>, xmpp4moz should (according to 14.3. of RFC3920) resolve the hostname of the server using SRV service:

[matej@viklef ~]$ host -t SRV _xmpp-client._tcp.ceplovi.cz
_xmpp-client._tcp.ceplovi.cz has SRV record 0 0 5222 luther.ceplovi.cz.
[matej@viklef ~]$

xmpp4moz certainly doesn't resolves server name with SRV (so it has special type of GMail account distinct from Jabber account, which is unnecessary when SRV records are followed).

Revision history for this message
In , Leger-formerly-netscape (leger-formerly-netscape) wrote :

Bulk move of all Networking-Core (to be deleted component) bugs to new
Networking component.

Revision history for this message
In , Gordon-formerly-netscape (gordon-formerly-netscape) wrote :

Help wanted.

Revision history for this message
In , Mark-andrews (mark-andrews) wrote :

See also Internet Draft draft-andrews-http-srv-00

Revision history for this message
In , Djb-bugzilla (djb-bugzilla) wrote :

It takes time to check for SRV records. Be careful with features that
can degrade the user experience! I'd like to see a study of SRV's impact
on web-page load times.

Revision history for this message
In , Kcd (kcd) wrote :

It takes time when an HTTP request times out because the server is down or
overloaded as well! Any testing which attempts to assess impact on "user
experience" should take into account not only the "best case" scenarios but
also the load-balancing and failover benefits of SRV. We live, after all, in an
imperfect world...

Revision history for this message
In , Evertonm (evertonm) wrote :

Please guys, we need this improvement on our browser of choice.

SRV RRs are very useful; such enhancement is _innovation_. Just take a look at
the specs.

There is no good reason to not include this (other than allocating some
development resources).

Revision history for this message
In , Benc-meer (benc-meer) wrote :

mass move, v2.
qa to me.

Revision history for this message
In , Benc-meer (benc-meer) wrote :

Can we target this (1.0 or future?)

Revision history for this message
In , M438urxuqu85001 (m438urxuqu85001) wrote :

Some corrections to the previous remarks:
 - Browsing is improved: you'd like the web robust like email? SRV do that!
 - Goal of supporting SRV records is support load balanced & backed-up servers.
 - RFC 2052 is superseded by RFC 2782
 - Dealing with SRV records does not take more time, it uses one DNS lookup
 - People are working on library support to automate SRV records
 - Targeting it at v1.0 seems smart; being able to claim robustness is nice ;-)
All in more detail on http://dns.vanrein.org/srv (or send me email and ask)

Revision history for this message
In , Dnewman (dnewman) wrote :

I am taking ownership of this RFE. I have been looking for a programming
project for some time now and this one is very interesting to me. I'm thinking
that all protocols that Mozilla supports should use SRV records, i.e. HTTP,
NNTP, FTP, POP, IMAP, LDAP, SMTP, GOPHER (does anyone still use this?). I will
be creating an outline of the plan for comments.

Revision history for this message
In , Njs-netscape (njs-netscape) wrote :

Why was I copied on this bug when it was updated? the update was sent to

<email address hidden>? Why?

By the way, the RFC is more than six months old - draft RFCs more than
six months old are questionable - I'd contact the authors to check on their
current felings before writing anything.

Revision history for this message
In , M438urxuqu85001 (m438urxuqu85001) wrote :

Nick/others,

RFC 2052, the originally mentioned topic, is indeed over 6 months, but you may
also have noticed my earlier remark that it was replaced with RFC 2782. This one
is on the Standards Track, and STD 1 specifies its status, so rather than making
an RFC look suspicious, I'd like to see a quote from STD 1.
Allow me to demonstrate: it is listed under the heading of
        3.4. Proposed Standard Protocols
And that is a result of a specific action by the IESG [see RFC 2026, sec 4.1.1],
so it definately makes sense to implement this particular feature.

Hope this helps to take your worries away,

Rick van Rein.

Revision history for this message
In , Dnewman (dnewman) wrote :

This past weekend I was working on some "proof of concept" code to kind of hash
out how it is going to work and some things occured to me that maybe others can
comment on.

If this was a simple ftp client, such as ncftp, then when I initiate a
connection to domain.org I could do a simple DNS lookup of _ftp._tcp.domain.org
and plow through the list of SRV records, sorted by priority and picked at
random by weight within the same priority. When a connection failes, remove it
from the list and try again. Once a successful connection has been established
the data concerning SRV records could be thrown away.

The web, by contrast, is a completely different paradigm. Instead of having
one continuous connection throughout the life of the visit, a seperate
connection is used for each "transaction". And each transaction may be
comprised of multiple connections when a particular page contains multiple
objects.

This is where my experience with Cisco Local Directors, ArrowPoints and Central
Dispatch comes in and confuses the issue. These devices are keeping running
tabs on the services that they are watching. For example the Local Director
will mark a service as bad when it gets a failed connection. Even though the
connection would finally make it over to a good server the user experienced a
delay assuming the server was off the network and had to wait for TCP
timeouts. From that point on the LD would direct traffic to services it knew
were good. Every once in a while it would try the bad one just for yuks to see
if it had been fixed.

So assuming we don't want the user to get hung up on a down server every time
they traverse the service list we need to keep a running tab on sevices we've
tried in the past. So does this mean that once we have queried the SRV record
we keep it cached within mozilla until the TTL runs out? Or do we make actual
DNS queries every time and run through the same algorithm to choose a server?
Or we could keep a running list of servers we've connected to and how
successful we were at it in a seperate list. Then we could do seperate SRV
queries every time and compare what we get back to our running list. We would
then try a server marked as down only if we had no other alternatives.

That then brings up the issue of how long do we keep the information around
for? I suppose it is no different than a history, but it eats up memory.

Revision history for this message
In , Evertonm (evertonm) wrote :

> (...) For example the Local Director
> will mark a service as bad when it gets a failed connection. Even though the
> connection would finally make it over to a good server the user experienced a
> delay assuming the server was off the network and had to wait for TCP
> timeouts. From that point on the LD would direct traffic to services it knew
> were good. (...)

Only the connection which "detected" the failed service hangs?

> So assuming we don't want the user to get hung up on a down server every time
> they traverse the service list we need to keep a running tab on sevices we've
> tried in the past. (...)

I haven't understood why the user will always get hung up on a
failed service. The LD is not going to deliver newer connections
to a healthy server after one failure?

Revision history for this message
In , Dnewman (dnewman) wrote :

>> (...) For example the Local Director
>> will mark a service as bad when it gets a failed connection. Even
>> though the connection would finally make it over to a good server
>> the user experienced a delay assuming the server was off the network
>> and had to wait for TCP timeouts. From that point on the LD would
>> direct traffic to services it knew were good. (...)
>
> Only the connection which "detected" the failed service hangs?

Correct.

>> So assuming we don't want the user to get hung up on a down server
>> every time they traverse the service list we need to keep a running
>> tab on sevices we've tried in the past. (...)

> I haven't understood why the user will always get hung up on a
> failed service. The LD is not going to deliver newer connections
> to a healthy server after one failure?

That part I was describing was without LDs and so strictly inside of
the browser. If we requery DNS every time and have to run down the
list servers listed for the service and there is a down server, the
user will get hung up on that down server. It will eventually time
out and the browser will try the next server, but the user will have
to wait. Which is why the implementation might want to cache the
DNS data internal to the browser and keep track of which servers are
up and which ones are down.

Or something else I had thought of, but didn't want to mention
because I didn't like it, was to initiate connections to all servers
listed and use the one that responds first. Or if there are, for the
sake of argument, 20 objects on a particular page and connections
were opened to 4 differant servers, then each connection can serve 5
objects, assuming each server is weighted equally, through the use of
persistant connections.

Revision history for this message
In , Evertonm (evertonm) wrote :

> That part I was describing was without LDs and so strictly inside of
> the browser. If we requery DNS every time and have to run down the
> list servers listed for the service and there is a down server, the
> user will get hung up on that down server. It will eventually time
> out and the browser will try the next server, but the user will have
> to wait. Which is why the implementation might want to cache the
> DNS data internal to the browser and keep track of which servers are
> up and which ones are down.

So you're talking about timeout on requests for the TCP connections.

I'm now wondering whether such "healthy cache" breaks the SRV behavior
as specified by RFC2782. The thing explicitily states:

"for each address record found, try to connect to the (protocol,
address, service)."

While the cache seems a smart approach, it would be a shame to have
the awesome SRV feature not adhering to the proper standard. How
about to make the "healthy cache" an optional, user-tunable feature
and let it disabled by default?

Revision history for this message
In , Jo-hermans (jo-hermans) wrote :

Changing title because RFC2052bis was superseeded a long time ago by RFC2782

Revision history for this message
In , Jo-hermans (jo-hermans) wrote :

*** Bug 162798 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Mozilla-elektrono (mozilla-elektrono) wrote :

I'm disappointed that support for SRV's hasn't made it into Mozilla after three
years. There are are very useful device and are now being used by a number of
services: Jabber, LDAP, Kerberos, Windows 2000, etc...

The main reason I want SRV RRs is so I can have multiple SSL Apache virtual
hosts with one IP. Name-based virtual hosts don't work with SSL, but port-based
virtual hosts would :-)

This document answers many of the questions asked above about how to go about
implimenting http with srv:
http://www.globecom.net/ietf/draft/draft-andrews-http-srv-01.html

It is technically expired, but it's the closest thing to a standard I could find.

Revision history for this message
In , Rick-vanrein (rick-vanrein) wrote :

I have built a working library that supports SRV records in a generic
way. This library is intended for simple applications that want to
"grab a good connection quickly" so it may be unsuitable for Mozilla,
but at least it is example code, open source so ready for copy/paste.

The library can be found on
  http://libsrv.sf.net
and it works on Linux, OpenBSD, FreeBSD, MacOS X.
Any Mozilla coders out there who would like a quick-n-easy way to
get popular? ;-)

Revision history for this message
In , Benc-meer (benc-meer) wrote :

There are lots of issues related to caching and DNS, but I don't think that
necessarily should be a bottleneck to adding this support.

What you probably need to do is look at implementing this with a pref, and then
checking it in as off by default. You should also carefully research any
dependent changes to existing code, because you have to sell this to the module
owners.

After some bake time to make sure the existence of the feature is not causing
regressions, you can have individual users turn it on and see how it works.

Don't worry about caching now, it seems like the problems described w/ the
network load distributors would be issues for normal DNS as well.

Revision history for this message
In , Leknor (leknor) wrote :

I'd like to see this implemented so that things like mail server configuration
can be more automatic.

Revision history for this message
In , Mbabcock-mozilla (mbabcock-mozilla) wrote :

Is David still working on this bug or should this bug's assignment be changed?

Just curious.

Also consider looking at DNS caching discussions from bug 109313

Revision history for this message
In , Jo-hermans (jo-hermans) wrote :

see also bug 173804 (RendezVous/ZeroConf support)

Revision history for this message
In , Adam-cypherspace (adam-cypherspace) wrote :

What's the status on this bug? Is anyone working on it?

It would be a a very useful feature, but the bug seems to be rather old.

Adam

Revision history for this message
In , Njs-netscape (njs-netscape) wrote :

As long as the RFC was 2052, it was an "Experimental" RFC. RFC 2782 is
Standards Track. This should mean that the priority is higher. I guess that I
would like to hear about what the possibilities of supporting this are.

I did a search on "RFC 2782 Support Library" in google and found
http://www.vovida.org/document/vocaldoc/util/srv_readme.txt. There seems to be
other support code that could be "borrowed" running around as well.

Revision history for this message
In , Alexey Zakhlestin (indeyets) wrote :

There is more, actually:
This is an lgpl library which can deal with SRV records withoud dependency on
bind - http://faerion.sourceforge.net/doc/rascal/

currently, it can only be downloaded using CVS, but author will release it as a
package in the near future.

the exact function is, possibly, this:
http://faerion.sourceforge.net/doc/rascal/rascal_connect_service.html

Revision history for this message
In , Everton da Silva Marques (evertonsm) wrote :

There is also ruliwrap:
http://weinholt.se/hacks/

ruliwrap is a wrapper for getaddrinfo() that uses libruli to seamlessly
add SRV record support to programs that use getaddrinfo(3).

Revision history for this message
In , Jo-hermans (jo-hermans) wrote :

*** Bug 293701 has been marked as a duplicate of this bug. ***

Revision history for this message
In , D7dvttp02 (d7dvttp02) wrote :
Download full text (7.4 KiB)

At the risk of triggering stupendous wrath, but maybe some real discussion, too,
I want to suggest a dastardly hack in lieu of the DNS SRV support envisioned in
this bug report. It seems clear that:

- DNS SRV records have been around for a long time -- even this bug has been
around for a long time, but nobody seems to be implementing it for HTTP, even
though may other, newer protocols have been supporting the DNS SRV records long
since. Even Microsoft can't seem to implement this thing.

- A careful look at the name resolution process might offer an explanation of
why the current SRV proposed standard is not getting implemented in the world of
the interactive web. No matter how you look at it, if you have to query for an
SRV record, and then query for an A record, you have two round trips through the
DNS. Even if everything is working perfectly, the name resolution process will
take (nominally) twice as long if you have to go through an SRV resolution step.
 You could imagine fixing the problem by allowing IP addresses in SRV records
(this is not what the present standard says, and there are in general good
reasons for that), but even then, it is hard to optimize the process of looking
for records that usually won't be there. Negative caching is a hard problem.
(In other words, you have added a full-scale, uncacheable DNS query to every
lousy URL you want to resolve. A web browser that needs to build a complex page
from several URLs would have a hard time presenting a reasonably responsive
experience to the user even if all servers that need to say "no SRV record here"
are configured properly and don't drop the unknown request on the floor.)

- Doing SRV lookups is hard from a programming perspective. It goes against the
grain of the facilities provided in the standard C library. You almost need to
provide an SRV lookup library, and the more libraries you depend on, the less
portable your result.

- Mozilla is the cutting edge of browser development. If we could find a better
way to do it and write an RFC and maybe get Microsoft on board if it was simple
enough, we would have taken care of 95% of browsers out there and the rest would
take care of themselves.

- DNS SRV is wonderful for load balancing, but it's overkill if you just want to
run on an alternate port. (If we can consider the space of TCP and UDP ports to
be part of the address space of an IPv4 network, instead of worrying about
exhausting a 32 bit space, we now have 48 bits, which is a fair amount of
breathing room for the time between now and when IPv6 happens.)

I'm going to propose a simple flag that could be inserted in a set of DNS A
records to indicate that the name describes a service instead of a host and
provide the port where the service can be connected to. It will have the
advantages that:

- It is less than a dozen lines to add (or remove) in only one place,
- It is ridiculously portable,
- It imposes negligible performance/bandwidth/processing penalty/overhead,
- It solves the problem of naming a service on an alternate port, and
- It is safe in that it will not be triggered unexpectedly (keep reading).

Disadvantages:
- I'm proposing it here for the fi...

Read more...

Revision history for this message
In , Alexey Zakhlestin (indeyets) wrote :

Bugzilla is not a place for creating new RFCs. sorry. You should try NNTP.
also, You are not right, about need of several DNS requests in process of SRV
queries. DNS answer includes "additional" section which already holds the
required data.
for example:

$ host -a -t SRV _irc._tcp.forestnet.org
Trying "_irc._tcp.forestnet.org"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5356
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;_irc._tcp.forestnet.org. IN SRV

;; ANSWER SECTION:
_irc._tcp.forestnet.org. 3600 IN SRV 1 0 6667 irc.forestnet.org.

;; AUTHORITY SECTION:
forestnet.org. 3600 IN NS ns1.cwn.ru.
forestnet.org. 3600 IN NS ve208.msm.ru.

;; ADDITIONAL SECTION:
irc.forestnet.org. 3600 IN A 81.9.48.3
irc.forestnet.org. 3600 IN A 195.91.162.198
irc.forestnet.org. 3600 IN A 213.248.60.142
ve208.msm.ru. 3600 IN A 213.248.60.142

Received 190 bytes from 192.168.0.1#53 in 725 ms

portability is not a big question too. The libraries which were mentioned
earlier are VERY portable.

p.s. just to clear things up. Rascal is BSD-licensed now:
http://faerion.sourceforge.net/librascal.html.var

Revision history for this message
In , Jo-hermans (jo-hermans) wrote :

I have worked with SRV records before (they're used in SIP networks, see
RFC3263), and I can tell you that multiple round-trip is not an issue at all.
SRV and A records will be returned in 1 packet, if there's enough room (that's
where RFC2671 comes in handy).

Note that 127.0.0.1/8 was originally a routable network. I still have customers
that use them, both on private networks and as 'internal' addresses (only
routable in 1 host). But it becomes harder and harder to use them, because lots
of routers and IP-stacks are now hardcoding them (due to RFC3330).

Revision history for this message
In , Numbski (numbski) wrote :

Well, as I'm seeing it right now, we have portable libraries, and we have
emerging technologies like Coral Cache that stand to make excellent use of it,
but no one can use it because the browsers aren't embracing it. :(

Is there really anything left about this whole process that is left to debate,
or is it really a matter of lack of a coder wanting to work on it? If it is the
latter, then I think we're almost approaching the point of needing to place a
bounty on it. Truly (affoardable) redundant systems are just dying for this
technology. You can place a load balancer in your data center all day long, but
if your load balancers fail, you're down. If you can implement SRV, then you
fail to another system, and if configured correctly, you can even pull from
Coral Cache.

Anyway, thoughts? Should be be starting a bounty on this?

Here are some pertinent links:

http://www.ietf.org/rfc/rfc2782.txt

Implementation Examples (not limited to http)
http://www.voip-info.org/tiki-index.php?page=DNS%20SRV
http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy/admingd/ver2_1/ddns.htm

Revision history for this message
In , Bill+mozilla-bugzilla (bill+mozilla-bugzilla) wrote :

I made a Mozillzine forum for people who want to continue non-development discussion:
  http://forums.mozillazine.org/viewtopic.php?p=1676455#1676455

Revision history for this message
In , Christoph Anton Mitterer (calestyo) wrote :

In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
each case, no matter if there are performance questions or anything like this.
The reson is: RFC2782 is a proposed standard according to the IETF.

Are there any current developments regarding SRV RRs in Mozilla products?

Revision history for this message
In , Numbski (numbski) wrote :

(In reply to comment #35)
> In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
> each case, no matter if there are performance questions or anything like this.
> The reson is: RFC2782 is a proposed standard according to the IETF.
>
> Are there any current developments regarding SRV RRs in Mozilla products?

I don't think there is, however just for grins, some pseudo-code for someone
familiar enough with the zilla sources, here are a pair of methods to give a try:

presuming someone has typed in a plain url of the type http://www.domain.tld,
which of course presumes port 80:

Rather than simply attempting to connect to www.domain.tld on port 80, first
attempt to get _http._tcp.www.domain.tld. Respect whatever is returned
(although to be honest, I'm not sure what happens at this point. I really
should try this from telnet sometime to see exactly what occurs here).

Second would be to see if there is a dns library floating around out there that
is lightweight and can handle srv records. When the user hits enter or clicks
the 'go' button, do an actual dns resolution before proceeding, again presuming
_http._tcp.www.domain.tld. Respect what is returned.

I believe the crux of the matter is timeouts. How long do you timeout on trying
srv's? If you go the second route and do a full name resolution and you get
valid srv data back before then reverting to traditional name resolution, I'd
then proceed with the correct priority site. Should that time out, then we
would stray from tradidition a bit. Rather than giving the usual timeout
message, give a new error:

"The requested page could not be retrieved, however the site has specified n
additional backup pages. Would you like to try the next one now?"
[Please take me to the next one.] [No, Thanks though.]
Don't ask me this again.[x]

Just a couple of thoughts on how to deal with it. The user experience wouldn't
be damaged in any way. The srv record timeouts could be adjusted through
about:config, and if done correctly an extension writer could probably extend
things a bit and make them prettier.

Revision history for this message
In , Thomas Butter (tbutter) wrote :

I do not think that the retry logic has to be changed, since browsers should
already try all returned A records. SRV just adds ordering to that logic.

I can offer some SRV query code if anyone wants to adapt it mozilla, but it is
quite easy anyway.

Revision history for this message
In , Elmar-ludwig (elmar-ludwig) wrote :

*** Bug 319939 has been marked as a duplicate of this bug. ***

Revision history for this message
In , Timeless-bemail (timeless-bemail) wrote :

<email address hidden>: lgpl code isn't acceptable for mozilla, please don't suggest it.
<email address hidden>: if you're really interested, i'd suggest you try doing it, build mozilla, hop on irc and ask for help. offering sample code isn't ilkely to get this very far.

personally, this whole thing seems very annoying. i don't feel like reading the rfc, but i would like a basic specification that explains what to do when:
1. there's no SRV record
2. there are 20 alternates in the SRV record and all of them are bad.
3. general cases in between.

The spec should list the series of requests (dns, http, other) as well as wait points and expected time spent at each step. the cost (size) of each inbound and outbound request should be listed, and a comparisoon should be made against today's browsing behaviors.

the following browsing behaviors are available today:
1. host doesn't respond, user tries a different link to a different server
2. user tries google cache for link
3. user tries coral cache for link

Revision history for this message
In , Mozilla-krellis (mozilla-krellis) wrote :

(In reply to comment #39)
> personally, this whole thing seems very annoying. i don't feel like reading the
> rfc, but i would like a basic specification that explains what to do when:
> 1. there's no SRV record
> 2. there are 20 alternates in the SRV record and all of them are bad.
> 3. general cases in between.

To be frank, "seems very annoying" and "don't feel like reading the rfc" seems like a pretty poor attitude to take. If you don't want to take the time to form an educated opinion on the bug, no less perform any work on it, that's certainly your right, but if that's going to be the case, why not just leave it alone?

Anyway, to answer these three questions:

1) If there's no SRV record, everything works as it does now, you look up an address record and make the request.

2) This exact case is not addressed in RFC2782 or Mark Andrews' HTTP SRV draft. I would see two possibilities - either the normal "this request timed out" error is displayed immediately, or a fallback to the standard A record / port 80 behavior takes place. I think the former would be more "correct", as specification of an SRV record says "if you speak SRV, you should use this".

3) I'm not sure what you mean by general cases in between.

If there are any developers out there interested in implementing this functionality (to be contributed back to Mozilla, if it will be accepted) under contract, please contact me off-bug, as my company is interested in sponsoring this work.

Changed in sameplace:
status: New → Invalid
83 comments hidden view all 163 comments
Revision history for this message
In , Nikolay Shopik (shopik) wrote :

(In reply to comment #36)
> I'm voting for the bug, but only on the condition it uses a standard mechanism
> such as the approaching finalized IETF proposal:
>
> http://tools.ietf.org/html/draft-daboo-srv-email

I do agree with you Chris, but you may look into bug 356104 and bug 14328, these are showstoppers. Necko doesn't have code right now to resolve SRV records.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

> Am I the only one who thinks that Mozilla's new centralized approach of
> client autoconfiguration is a poor idea?

No, you're not. It wasn't intended for the use case you mention. Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

Revision history for this message
In , L. David Baron (dbaron) wrote :

(In reply to comment #36)
> I don't see that it's necessary to wait for DNSSEC to implement that. DNSSEC
> will deploy when it deploys and we'll get the extra security bump when it shows
> up.

I'd agree that it's not necessary to wait for DNSSEC if all servers found in this manner require SSL/TLS (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match).

Revision history for this message
In , Jesse-thompson (jesse-thompson) wrote :

> (i.e., the connection is configured to fail if TLS can't be initiated or if the cert doesn't match)

Requiring that certs match the domain will be a show stopper for services that host lots of domains. We are an EDU, and we host 270 domains. Just think about services that host thousands of domains!

Supposing it is practical for email service administrators to maintain hundreds or thousands of valid signed matching certificates for domains that they don't own - it isn't practical, IMHO - do we know if the major MTAs support the ability to present a different cert for the individual domain during the Start-TLS negotiation? If not, that will also be a show stopper.

Is it feasible to abandon plain old TLS? Outlook still seems to work better with port 465/TLS. Start-TLS would be a new requirement since the domain needs to be known before the server can present a matching cert.

Take a look at http://tools.ietf.org/html/draft-hildebrand-dna-00 for some ideas on other methods of asserting domain ownership without the need for DNSSEC.

Revision history for this message
In , Jesse-thompson (jesse-thompson) wrote :

> Please see https://wiki.mozilla.org/Thunderbird:Autoconfiguration and bug 534722.

Yes, step 2 is a more rational approach. I still think that step 3 is problematic just in its existence. I see that there is mailing list http://groups.google.com/group/ispdb so I'll voice my concerns there.

Revision history for this message
In , Brett Zamir (brettz9) wrote :

I'd also like to see it, specifically with XPCOM access for extension use and use in other protocols like XMPP...

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

The patch here is great, because it shows how to make arbitrary DNS queries, but it unfortunately limits the API to SRV and TXT. In bug 342242, we need MX, too. So, I filed bug 545866 for an IDL API to allow arbitrary DNS record types to be queried from within Mozilla. This bug could then use this API.

Revision history for this message
In , Usenet-tonal (usenet-tonal) wrote :

See bug 14328 for the work being done on extending DNS support to include the resolution of SRV records, which seems to me to be closely related to the work being discussed here.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

Yes, I filed bug 545866 (arbitrary DNS queries in Mozilla) based on that and made it block this bug.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

Narrowing bug summary to DNS SRV records, to differentiate from DNS MX, which I filed as bug 551519.

Revision history for this message
In , Nikolay Shopik (shopik) wrote :

*** Bug 536915 has been marked as a duplicate of this bug. ***

Changed in thunderbird:
importance: Unknown → Wishlist
status: Unknown → Confirmed
Revision history for this message
In , Nikolay Shopik (shopik) wrote :

draft-daboo-srv-email is now http://tools.ietf.org/html/rfc6186

Revision history for this message
In , Ludovic-mozilla (ludovic-mozilla) wrote :

*** Bug 675476 has been marked as a duplicate of this bug. ***

Revision history for this message
In , WhyNotHugo (whynothugo) wrote :

If this is implemented, I'd suggest it be in compliance with rfc6186.

I don't see DNSSEC as a requirement, but rather an extra.
There's really little harm that can be done on a DNSSEC-less scenario that can't be done right now. And even if the SRV records are on a DNS domains that implements DNSSEC, the actual IMAP/SMTP servers might be in DNSSEC-less domains.

Revision history for this message
In , WhyNotHugo (whynothugo) wrote :

Any updates on this issue? The improvements to user experience by implementing this are huge:

Now:
User need to know email, username, password, smtp server, imap server, smtp port, imap port.

With RFC6186
User needs to know email, password.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

Just FYI:
RFC 6186 doesn't give username.
The following method gives username and a number of other parameters, and it works today, see
https://developer.mozilla.org/en/Thunderbird/Autoconfiguration

Revision history for this message
In , WhyNotHugo (whynothugo) wrote :

There are some severe downsides to the ISBDB:

 1) It's controlled by Mozilla, instead of each ISP.
 2) It won't work on LANs.
 3) It requires intervention on Behalf of mozilla to update information.
 4) Only thunderbird uses this information. If each email client had it's own non-standard mechanism, then ISPs would have a huge burden to update each of them with new information when they change their email server.

The other mechanism:
 1) Shares point (4).
 2) Is inheretly insecure (why HTTP instead of HTTPS?)

While I have nothing in particular AGAINST maintaining support for mozilla's in-house mechanism, I belive that supporting standardized mechanisms (especially RFCs) is ultimately important.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

> 1) It's controlled by Mozilla, instead of each ISP.
> 2) It won't work on LANs.
> 3) It requires intervention on Behalf of mozilla to update information.

Not true for the ISP fetch.

> 2) Is inheretly insecure ... HTTP

So is DNS (at least without DNSSEC)

Revision history for this message
In , Bugmail-asutherland (bugmail-asutherland) wrote :

(In reply to Hugo Osvaldo Barrera from comment #53)
> 4) Only thunderbird uses this information. If each email client had it's
> own non-standard mechanism, then ISPs would have a huge burden to update
> each of them with new information when they change their email server.

Evolution uses it too, although it looks like they host their own copy of the database. Check out:
http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
which uses a (sadly non-SSL) base url of:
http://api.gnome.org/evolution/autoconfig/1.1/

The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.

Note: Neither this nor I believe Ben's comments are meant to suggest that we don't want to support DNS SRV. Unfortunately, it's still a non-trivial undertaking to get the support in the gecko platform that has not been prioritized on the platform side. For Firefox OS v2 we are hoping to get it prioritized.

Revision history for this message
In , WhyNotHugo (whynothugo) wrote :

(In reply to Ben Bucksch (:BenB) from comment #54)
> > 2) Is inheretly insecure ... HTTP
>
> So is DNS (at least without DNSSEC)

So rfc6186 provides secure autoconfiguration for domains that implement DNSSEC. There's no way to achieve secure autoconfiguration with this mechanism, since HTTP provides no security mechanism.

However, my strongest point is it's non-standardness, while rfc6186 is standard, and grants total control (and optional security) to the ISP.
Also, there's no reason both autoconfiguration mechanisms can't exist.

(In reply to Andrew Sutherland (:asuth) from comment #55)
> (In reply to Hugo Osvaldo Barrera from comment #53)
> > 4) Only thunderbird uses this information. If each email client had it's
> > own non-standard mechanism, then ISPs would have a huge burden to update
> > each of them with new information when they change their email server.
>
> Evolution uses it too, although it looks like they host their own copy of
> the database. Check out:
> http://git.gnome.org/browse/evolution/tree/mail/e-mail-autoconfig.c
> which uses a (sadly non-SSL) base url of:
> http://api.gnome.org/evolution/autoconfig/1.1/

Sadly, this is a perfect example of how rfc6186 is superior, since every email server's configuration is in it's owner's DNS server.

>
> The upcoming Firefox OS Gaia e-mail client also uses the ISP database too.
>
>
> Note: Neither this nor I believe Ben's comments are meant to suggest that we
> don't want to support DNS SRV. Unfortunately, it's still a non-trivial
> undertaking to get the support in the gecko platform that has not been
> prioritized on the platform side. For Firefox OS v2 we are hoping to get it
> prioritized.

Actually, there's an alternate fix for this issue; if Mozilla's ISPDB contains no information related to a particular domain, it could query it using rfc6186, and return that.

This means that Thunderbird would still [indirectly] resolve using rfc6186, with no actual changes to the client code.

Revision history for this message
In , Alexey Zakhlestin (indeyets) wrote :

Shouldn't this depend on bug 14328 instead of 545866?

Revision history for this message
In , chrysn (chrysn) wrote :

With the DNS-SD support brought with Flyweb in Firefox, the lack of DNS support in Gecko should have been mitigated. The security implications of using DNS data do not seem worse than those of using the current HTTP based mechanism.

Revision history for this message
In , Noah (noahod) wrote :

I would love to have this implemented in Thunderbird, as a sysadmin that would like to make it easier for our users to connect to our service.

Regarding the security context of this, I actually think it's a great improvement, because it means people are more likely to automatically set up encrypted mail through imap(S) than if they have to manually enter the details.

2. As pointed out the MiTM risk is no worse than the current http method, and the guessing method (which actually uses the wrong server for us)

3. The MITM risk is only at setup time. If the user doesn't use autoconfig, and ends up using unencrypted mail, they are at MiTM risk everywhere they go, every coffee shop wifi, etc. Since the MitM risk is only at setup time, it's extremely hard for an attacker to exploit, because they have to convince the user to delete and recreate the connection in order to be able to hijack the autoconfigure request.

4. Doesn't the current UX flow mean that the users password is only sent to the server *AFTER* the autoconfigure completes and the server details are shown to them? What is the problem here? Doesn't the user get a chance to verify the server details?

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

If you want to make it easy for users, you can already do that today with http instead of DNS SRV:
https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration

Revision history for this message
In , W-jan-k (w-jan-k) wrote :

(In reply to Ben Bucksch (:BenB) from comment #60)
> If you want to make it easy for users, you can already do that today with http

Literally "http://". Bug 986967 is about that and is a real security problem.

Revision history for this message
In , Ilf-6 (ilf-6) wrote :

RFC 6186 was published in 2011, but it remained only an option. In January 2018, RFC 8314 was published: https://tools.ietf.org/html/rfc8314

This explicitly sais:

> User-configurable MUAs SHOULD support the use of [RFC6186] for account setup.

https://tools.ietf.org/html/rfc8314#section-5.1

So please, let's follow the RFC and provide this option in Thunderbird.

Thanks, and keep up the good work!

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

Although RFC8314 is on the standards track, it's more a "best practices" document than an actual protocol definition.

It's true that Thunderbird does not implement configuration via DNS SRV (this bug here).
There are several fundamental reasons:

1. Arbitrary DNS lookups - bug 545866 / bug 14328 - are not implemented by Gecko.
   I've spent a lot of time implementing it, over 8 years ago (April 2010),
   only for the patch to be ignored and not reviewed. Forgive me when I'm a little less excited today.

2. DNSSEC is very hard to administrate without shooting yourself in the foot, and consequently,
   it's not widely deployed on second level domains. APNIC says:
   "Unfortunately, our recent study showed that DNSSEC deployment is still very low
   (only ~1% of .com, .net and .org domains deploy DNSSEC)"
   https://blog.apnic.net/2017/12/06/dnssec-deployment-remains-low/
   On most domains, DNS is completely insecure. I don't see it gaining quickly, either.
   APNIC <https://stats.labs.apnic.net/dnssec/XA> actually shows declining deployment in the last year.
   Without DNSSEC, DNS SRV is no more secure than http. In that light, comment 61 is a little ironic.

3. DNS SRV does not give us some critical information we need to configure the accounts.
   Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or
   even something like EXAMPLE\\bbob? Get it wrong, and the configuration doesn't work.
   Sure, we can try them all. And risk authentication failures and - worst case - triggering
   3-failures-in-a-row-and-you're-out security mechanisms.
   We're back to guessing and trail&error.
   That defeats the entire point of getting an authoritative configuration that will definitely work.
   If we were to support DNS SRV, then only as a second-class, fallback, legacy format.

DNS SRV is not widely used now. Why trying to establish a mechanism that has known shortcomings?

I think people here want this, because they want a standards-based mechanism. I fully concur with that.
But I'd rather make a new standard than implement a bad standard that nobody else implements anyway.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

On the up side: Thunderbird already implements most of what RFC8314 requests, including most SHOULDs. In fact, it seems RFC8314 took Thunderbird as reference implementation, and basically says "Everybody should do what Thunderbird does" ;-) .

Revision history for this message
In , Chris-newman (chris-newman) wrote :

The goal in RFC 8314 was to reasonably promote security/interoperability improvements without forcing a lot of software to be declared broken. It wasn't actually based on what Thunderbird does, but if that's the impression, that's good.

The IETF has backed SRV records in general and email+SRV in particular so the spec followed that trend on the auto-configuration front. If the various ad-hoc proposals for HTTP+XML email auto-configuration were to come up with a single compromise proposal and bring that to the IETF, I wouldn't expect major problems moving that to the standards track, but unless that's done, DNS SRV is the only standard for email account auto-configuration and Thunderbird has a standards-gap by not implementing it.

Revision history for this message
In , William66750 (william66750) wrote :

It's easier to add a dns field, than to configure via a web server
We will hope that the RFC will be applied

Changed in thunderbird:
importance: Wishlist → Medium
Revision history for this message
In , Martin (martin22) wrote :

For the record (and potential inspiration), implementation of rfc6186 in kmail is done: https://github.com/k9mail/k-9/pull/4719

Revision history for this message
In , O-esa (o-esa) wrote :

After 14 years this is still open. What I can't figure out is why not having DNSSEC with RFC 6186 is seen as such a big problem while the currently implemented [Autoconfiguration](https://wiki.mozilla.org/Thunderbird:Autoconfiguration) feature gets the information from plain HTTP URLs. Both methods are identically vulnerable to MitM.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

Revision history for this message
In , O-esa (o-esa) wrote :

(In reply to Ben Bucksch (:BenB) from comment #69)
> Esa, I answered that 3 years ago in comment 63. I mentioned 3 separate reasons. Each reason *on its own* is a serious problem.

Thanks, Ben! Some good and valid arguments there. It's just that while the current approach might have some clear advantages over RFC 6186, it doesn't seem any better security wise – quite the opposite. Or does the Autoconfiguration perhaps support HTTPS and HSTS (RFC 6797) with preloading, which would make forcing TLS possible?

Revision history for this message
In , Zocker-net (zocker-net) wrote :

(In reply to Ben Bucksch (:BenB) from comment #63)
> 3. DNS SRV does not give us some critical information we need to configure the accounts. Specifically, which form the username needs to be transmitted. Is it bbob, or <email address hidden>, or even something like EXAMPLE\\bbob? Get it wrong, and the configuration
> doesn't work.
> Sure, we can try them all. And risk authentication failures and - worst
> case - triggering
> 3-failures-in-a-row-and-you're-out security mechanisms.
> We're back to guessing and trail&error.
> That defeats the entire point of getting an authoritative configuration
> that will definitely work.
> If we were to support DNS SRV, then only as a second-class, fallback,
> legacy format.

This question is already addressed by RFC 6186, see under Section 4:
> When a user identifier is required, MUAs MUST first use the full email address provided by the user, and if that results in an authentication failure, SHOULD fall back to using the "local-part" extracted from the email address. This is in line with the guidance outlined in Section 5. If both these user identifiers result in authentication failure, the MUA SHOULD prompt the user for a valid identifier.

According to Section 5, service providers SHOULD implement their service so that either the mail address or the "local-part" only is sufficient for authentication. So this should work for most cases.

I propose that for the username Thunderbird should do according to the RFC. If both variants fail, Thunderbird could then fallback to first try out the other auto configuration methods. If any other auto configuration method are configured, Thunderbird could use those and ignore the DNS SRV records. If only the DNS SRV are configured, it could ask the user. If such rare case happens, users should probably know which username they should use. If not, it is still better to only ask for the username instead of currently not implementing DNS SRV because it might not work perfectly and ask the user for the complete server configuration.

To me, it sounds like you want users to have a great experience with Thunderbird. However, I think users will now go to a mail provider which claims to support auto configuration according to common standard (referring to RFC 6186) and they will see that most mail applications support his mail provider out of box, but not Thunderbird, which in my opinion could lead that users might not want to use Thunderbird "because it does not just work".

Changed in thunderbird:
importance: Medium → Unknown
Revision history for this message
In , Lubomir Spacek (spacek-6) wrote :

Hello,

at what stage is the implementation of RFC6186? It seems like it still doesn't work. It would really make life easier for us ISPs.

Revision history for this message
In , Karana-8 (karana-8) wrote :

bug 1852752 seems to be necessary before this can be done

Revision history for this message
In , Vseerror-i (vseerror-i) wrote :

(In reply to Selek Respa from comment #73)
> bug 1852752 seems to be necessary before this can be done

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

> MUAs MUST first use the full email address provided by the user, and if that results in an authentication failure, SHOULD fall back to using the "local-part" extracted from the email address

a) This is not good enough, because it's guessing / try&error.
b) We also need to know the authentication method: Password or OAuth2 or CRAM MD5 or SCRAM or one of the other SASL mechanisms. Together with the 2 username forms, there are too many permutations (2 * 5 = 10, just for these 2 factors), and we cannot try them all.
c) The servers may block the user account after too many failed login attempts, so even one pass with all the permutations might lock the user out and make even the working (!) config fail as well. Which, together, makes guessing fairly futile.

We need both username form and authentication method, and we need to be *certain* about both of them.

Revision history for this message
In , Ben-bucksch (ben-bucksch) wrote :

If the reasoning is about DNS SRV being a standard: Autoconfig is already a quasi-standard today, supported by over 10 email clients and many servers and server-side software, and we (i.e. the people at the IETF and me) are currently in the process of standardizing it as IETF standard.

Displaying first 40 and last 40 comments. View all 163 comments or add a comment.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.