Possible exim retry bug (Re: master mail problems -- help needed)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
exim4 (Debian) |
Fix Released
|
Unknown
|
|||
exim4 (Ubuntu) |
Fix Released
|
High
|
Daniel Silverstone |
Bug Description
Automatically imported from Debian bug report #342619 http://
In Debian Bug tracker #342619, Jeroen van Wolffelaar (jeroenvw) wrote : [Mailer-Daemon@master.debian.org: Mail delivery failed: returning message to sender] | #1 |
Debian Bug Importer (debzilla) wrote : | #2 |
Automatically imported from Debian bug report #342619 http://
Debian Bug Importer (debzilla) wrote : | #3 |
Message-ID: <email address hidden>
Date: Fri, 9 Dec 2005 01:12:56 +0100
From: Jeroen van Wolffelaar <email address hidden>
To: <email address hidden>
Subject: Possible exim retry bug (Re: master mail problems -- help needed)
Package: exim4-daemon-heavy
Version: 4.50-8
Severity: serious
On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
> * Lionel Elie Mamane:
>
> > On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
> >
> >> The fact that my primary MX is only available through IPv6, and that
> >> this is the case for other people who're having problems too might
> >> then be a better chance at being the problem.
> >
> > My primary MX is IPv6-only, too. I don't have detected a problem yet :)
>
> Do you receive lots of mail from master.debian.org, and would you
> notice the bounces? Mail from Debian mailing lists come directly from
> murphy.debian.org, which does not seem to have the problem.
>
> You also have one IPv4-only MX, which might be enough to prevent the
> Exim bug[1] from occurring.
>
> [1] I'm not sure if it's a Exim's fault, it's only a hunch.
I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
bug has been witnessed positively both on master.d.o and on
one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
related (or maybe there are two bugs).
The situation on my mailserver was that the primary MX had a long term
unavailability and was way past cutoff time, but the secondary MX worked
fine. However, for some reason, what suddenly happened was that all the
mail queued for the domain in question got bounced for reason of having
a extended time of being unreacheable, past the retry time. Obviously,
that's bogus, as the secundary MX wasn't past cutoff yet.
I've meant to look into the code for this, but didn't yet get around to
it. If someone wants to do so, please -- I seriously suspect that Exim
in Sarge has a serious bug in there somewhere, it's showing up with this
IPv6 and IPv4 multihomed MX's too, after all.
I think this is a serious bug, as it can cause mail to get lost
(bouncing a mail for no good reason at all in some very common
situations like the IPv6 vs IPv4 multimhomed MX's)
Log snippets:
# Primary (long time unreacheable) MX is shrek.vanschaik.tk, secundary
# reacheable MX is mailrelay.
Last succesful delivery:
2005-11-30 17:49:53 1EhV6R-0000uq-Qg shrek.vanschaik.tk [81.207.193.3]:
Connection timed out
2005-11-30 17:50:02 1EhV6R-0000uq-Qg => <email address hidden>
<email address hidden> R=dnslookup_
H=mailrelay.
Message accepted for delivery" QT=3m19s
First failure:
2005-11-30 18:35:41 1EhVnA-0002GK-L1 shrek.vanschaik.tk [81.207.193.3]:
No route to host
2005-11-30 18:35:41 1EhVnA-0002GK-L1 == <email address hidden>
<email address hidden> R=dnslookup_
(113): No route to host
2005-11-30 18:35:41 1EhVnA-0002GK-L1 ** <email address hidden>
<email address hidden>: retry timeout exceeded
Second failure:
2005-11-30 18:36:43 1EhVrp-0001pB-Jw ** <email address hidden>
<email address hidden> R=dnslookup_
Debian Bug Importer (debzilla) wrote : | #4 |
Message-ID: <email address hidden>
Date: Fri, 9 Dec 2005 01:45:40 +0100
From: Jeroen van Wolffelaar <email address hidden>
To: <email address hidden>
Subject: [<email address hidden>: Mail delivery failed: returning message to sender]
Heh. Example:
----- Forwarded message from Mail Delivery System <email address hidden> -----
Return-path: <>
Envelope-to: <email address hidden>
Delivery-date: Fri, 09 Dec 2005 01:42:04 +0100
Received: from 220pc220.sshunet.nl
([145.97.220.220] helo=mordor.
by a-eskwadraat.nl with esmtps (TLS-1.
(Exim 4.50)
id 1EkWKq-0004xe-EP
for <email address hidden>; Fri, 09 Dec 2005 01:42:04 +0100
Received: from master.debian.org ([146.82.138.7])
by mordor.
id 1EkWKp-0001Ke-I1
for <email address hidden>; Fri, 09 Dec 2005 01:42:03 +0100
Received: from Debian-exim by master.debian.org with local (Exim 4.50)
id 1EkWKo-00040c-IO
for <email address hidden>; Thu, 08 Dec 2005 18:42:02 -0600
X-Failed-
Auto-Submitted: auto-generated
From: Mail Delivery System <email address hidden>
To: <email address hidden>
Subject: Mail delivery failed: returning message to sender
Message-Id: <email address hidden>
Date: Thu, 08 Dec 2005 18:42:02 -0600
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
<email address hidden>
retry time not reached for any host after a long failure period
------ This is a copy of the message, including all the headers. ------
Return-path: <email address hidden>
Received: from a-eskwadraat.nl ([131.211.39.72])
by master.debian.org with esmtp (Exim 4.50)
id 1EkWKn-00040K-Cd; Thu, 08 Dec 2005 18:42:01 -0600
Received: from jeroen by a-eskwadraat.nl with local (Exim 4.50)
id 1EkWKh-0004xF-Q0; Fri, 09 Dec 2005 01:41:55 +0100
Date: Fri, 9 Dec 2005 01:41:55 +0100
To: Florian Weimer <email address hidden>
Cc: Lionel Elie Mamane <email address hidden>,
Wouter Verhelst <email address hidden>,
Romain Francoise <email address hidden>,
<email address hidden>
Subject: Re: master mail problems -- help needed
Message-ID: <email address hidden>
References: <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-
In-Reply-To: <email address hidden>
User-Agent: Mutt/1.5.9i
From: Jeroen van Wolffelaar <email address hidden>
On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
> * Lionel Elie Mamane:
>
> > On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
> >
> >> The fact that my primary MX is only available through IPv6, and that
> >> this is the case for other people who're having problems too might
> >> then be a better chance at...
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed) | #5 |
tags #342619 help moreinfo
thanks
On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
> > * Lionel Elie Mamane:
> >
> > > On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
> > >
> > >> The fact that my primary MX is only available through IPv6, and that
> > >> this is the case for other people who're having problems too might
> > >> then be a better chance at being the problem.
> > >
> > > My primary MX is IPv6-only, too. I don't have detected a problem yet :)
> >
> > Do you receive lots of mail from master.debian.org, and would you
> > notice the bounces? Mail from Debian mailing lists come directly from
> > murphy.debian.org, which does not seem to have the problem.
> >
> > You also have one IPv4-only MX, which might be enough to prevent the
> > Exim bug[1] from occurring.
> >
> > [1] I'm not sure if it's a Exim's fault, it's only a hunch.
>
> I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> bug has been witnessed positively both on master.d.o and on
> one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> related (or maybe there are two bugs).
I have monitored Florian Weimer's attempts to debug. The main stopper
seems to be that it is extremely hard to get ahold of Master's logs.
> The situation on my mailserver was that the primary MX had a long term
> unavailability and was way past cutoff time, but the secondary MX worked
> fine. However, for some reason, what suddenly happened was that all the
> mail queued for the domain in question got bounced for reason of having
> a extended time of being unreacheable, past the retry time. Obviously,
> that's bogus, as the secundary MX wasn't past cutoff yet.
If you experience this for the next time, I'd like to see the output
of exinext $DOMAIN.
> I think this is a serious bug, as it can cause mail to get lost
> (bouncing a mail for no good reason at all in some very common
> situations like the IPv6 vs IPv4 multimhomed MX's)
Agreed. But since the bug doesn't happen for me, I need cooperation -
either yours or DSA's to be able to Debug on master.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
Debian Bug Importer (debzilla) wrote : | #6 |
Message-ID: <email address hidden>
Date: Fri, 9 Dec 2005 07:56:09 +0100
From: Marc Haber <email address hidden>
To: Jeroen van Wolffelaar <email address hidden>, <email address hidden>
Subject: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
tags #342619 help moreinfo
thanks
On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
> > * Lionel Elie Mamane:
> >
> > > On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
> > >
> > >> The fact that my primary MX is only available through IPv6, and that
> > >> this is the case for other people who're having problems too might
> > >> then be a better chance at being the problem.
> > >
> > > My primary MX is IPv6-only, too. I don't have detected a problem yet :)
> >
> > Do you receive lots of mail from master.debian.org, and would you
> > notice the bounces? Mail from Debian mailing lists come directly from
> > murphy.debian.org, which does not seem to have the problem.
> >
> > You also have one IPv4-only MX, which might be enough to prevent the
> > Exim bug[1] from occurring.
> >
> > [1] I'm not sure if it's a Exim's fault, it's only a hunch.
>
> I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> bug has been witnessed positively both on master.d.o and on
> one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> related (or maybe there are two bugs).
I have monitored Florian Weimer's attempts to debug. The main stopper
seems to be that it is extremely hard to get ahold of Master's logs.
> The situation on my mailserver was that the primary MX had a long term
> unavailability and was way past cutoff time, but the secondary MX worked
> fine. However, for some reason, what suddenly happened was that all the
> mail queued for the domain in question got bounced for reason of having
> a extended time of being unreacheable, past the retry time. Obviously,
> that's bogus, as the secundary MX wasn't past cutoff yet.
If you experience this for the next time, I'd like to see the output
of exinext $DOMAIN.
> I think this is a serious bug, as it can cause mail to get lost
> (bouncing a mail for no good reason at all in some very common
> situations like the IPv6 vs IPv4 multimhomed MX's)
Agreed. But since the bug doesn't happen for me, I need cooperation -
either yours or DSA's to be able to Debug on master.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : | #7 |
On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> bug has been witnessed positively both on master.d.o and on
> one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> related (or maybe there are two bugs).
It might be DNS related. Can you comment about
http://
please?
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Jeroen van Wolffelaar (jeroenvw) wrote : | #8 |
On Sat, Dec 17, 2005 at 04:07:51PM +0100, Marc Haber wrote:
> On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> > I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> > bug has been witnessed positively both on master.d.o and on
> > one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> > related (or maybe there are two bugs).
>
> It might be DNS related. Can you comment about
> http://
> please?
That sounds very much like the cause of this bug. It's also possible
even for IPv4 issues like the one in my original bugreport, as also in
that case, TTL's were distinct, and indeed, the situation could arrise
that a DNS query would give only partial results in the additional
section, because of the setup of the involved DNS servers.
So, it seems like the logic error is in that the additional DNS section
is believed to be complete, which is not at all guaranteed to be true --
it can be partial for any reason.
Thanks a lot for finding the (IMHO) most logical cause of this bug,
which according to Ockham's razor, is the cause of this bug. I tried
looking into the code etc, but simply failed to find the reason and
wouldn't easily have found this at all...
If you have any further questions, please ask,
--Jeroen
--
Jeroen van Wolffelaar
<email address hidden> (also for Jabber & MSN; ICQ: 33944357)
http://
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : | #9 |
On Sat, Dec 17, 2005 at 04:17:45PM +0100, Jeroen van Wolffelaar wrote:
> On Sat, Dec 17, 2005 at 04:07:51PM +0100, Marc Haber wrote:
> > On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> > > I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> > > bug has been witnessed positively both on master.d.o and on
> > > one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> > > related (or maybe there are two bugs).
> >
> > It might be DNS related. Can you comment about
> > http://
> > please?
>
> That sounds very much like the cause of this bug. It's also possible
> even for IPv4 issues like the one in my original bugreport, as also in
> that case, TTL's were distinct, and indeed, the situation could arrise
> that a DNS query would give only partial results in the additional
> section, because of the setup of the involved DNS servers.
I agree with that. I filed the message for exim-users before reviewing
your bug report, so I'll have to mention the possibility of this
happening in a v4-only setup later. But first I'll wait for Philip to
comment.
> So, it seems like the logic error is in that the additional DNS section
> is believed to be complete, which is not at all guaranteed to be true --
> it can be partial for any reason.
Right.
> Thanks a lot for finding the (IMHO) most logical cause of this bug,
Florian had a big part in this, we spent the better part of this
afternoon over DNS queries and DNS output on IRC. Without his help,
this wouldn't have been possible.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Jeroen van Wolffelaar (jeroenvw) wrote : retitle 342619 to DNS logic error: Delivers to MX query's additional section only, causing mailloss | #10 |
retitle 342619 DNS logic error: Delivers to MX query's additional section only, causing mailloss
Debian Bug Importer (debzilla) wrote : | #11 |
Message-ID: <email address hidden>
Date: Sat, 17 Dec 2005 16:07:51 +0100
From: Marc Haber <email address hidden>
To: Jeroen van Wolffelaar <email address hidden>, <email address hidden>,
<email address hidden>
Cc: Marc Haber <email address hidden>
Subject: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> bug has been witnessed positively both on master.d.o and on
> one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> related (or maybe there are two bugs).
It might be DNS related. Can you comment about
http://
please?
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
Debian Bug Importer (debzilla) wrote : | #12 |
Message-ID: <email address hidden>
Date: Sat, 17 Dec 2005 16:17:45 +0100
From: Jeroen van Wolffelaar <email address hidden>
To: Marc Haber <email address hidden>
Cc: Jeroen van Wolffelaar <email address hidden>, <email address hidden>,
<email address hidden>
Subject: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
On Sat, Dec 17, 2005 at 04:07:51PM +0100, Marc Haber wrote:
> On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> > I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> > bug has been witnessed positively both on master.d.o and on
> > one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> > related (or maybe there are two bugs).
>
> It might be DNS related. Can you comment about
> http://
> please?
That sounds very much like the cause of this bug. It's also possible
even for IPv4 issues like the one in my original bugreport, as also in
that case, TTL's were distinct, and indeed, the situation could arrise
that a DNS query would give only partial results in the additional
section, because of the setup of the involved DNS servers.
So, it seems like the logic error is in that the additional DNS section
is believed to be complete, which is not at all guaranteed to be true --
it can be partial for any reason.
Thanks a lot for finding the (IMHO) most logical cause of this bug,
which according to Ockham's razor, is the cause of this bug. I tried
looking into the code etc, but simply failed to find the reason and
wouldn't easily have found this at all...
If you have any further questions, please ask,
--Jeroen
--
Jeroen van Wolffelaar
<email address hidden> (also for Jabber & MSN; ICQ: 33944357)
http://
Debian Bug Importer (debzilla) wrote : | #13 |
Message-ID: <email address hidden>
Date: Sat, 17 Dec 2005 16:45:32 +0100
From: Marc Haber <email address hidden>
To: Jeroen van Wolffelaar <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
On Sat, Dec 17, 2005 at 04:17:45PM +0100, Jeroen van Wolffelaar wrote:
> On Sat, Dec 17, 2005 at 04:07:51PM +0100, Marc Haber wrote:
> > On Fri, Dec 09, 2005 at 01:12:56AM +0100, Jeroen van Wolffelaar wrote:
> > > I'm quite sure it's an exim bug, but haven't quite nailed it yet. The
> > > bug has been witnessed positively both on master.d.o and on
> > > one mailserver I maintain. Interestingly, it doesn't seem to be IPv6
> > > related (or maybe there are two bugs).
> >
> > It might be DNS related. Can you comment about
> > http://
> > please?
>
> That sounds very much like the cause of this bug. It's also possible
> even for IPv4 issues like the one in my original bugreport, as also in
> that case, TTL's were distinct, and indeed, the situation could arrise
> that a DNS query would give only partial results in the additional
> section, because of the setup of the involved DNS servers.
I agree with that. I filed the message for exim-users before reviewing
your bug report, so I'll have to mention the possibility of this
happening in a v4-only setup later. But first I'll wait for Philip to
comment.
> So, it seems like the logic error is in that the additional DNS section
> is believed to be complete, which is not at all guaranteed to be true --
> it can be partial for any reason.
Right.
> Thanks a lot for finding the (IMHO) most logical cause of this bug,
Florian had a big part in this, we spent the better part of this
afternoon over DNS queries and DNS output on IRC. Without his help,
this wouldn't have been possible.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
Debian Bug Importer (debzilla) wrote : | #14 |
Message-Id: <email address hidden>
Date: Sat, 17 Dec 2005 17:05:13 +0100
From: Jeroen van Wolffelaar <email address hidden>
To: <email address hidden>
Subject: retitle 342619 to DNS logic error: Delivers to MX query's additional section only,
causing mailloss
retitle 342619 DNS logic error: Delivers to MX query's additional section only, causing mailloss
In Debian Bug tracker #342619, Jeroen van Wolffelaar (jeroenvw) wrote : Re: [exim] Potential logic error in retry handling for IPv4+IPv6 hosts | #15 |
(Please cc me on all replies, I'm not subscribed)
On Sat, Dec 17, 2005 at 05:32:15PM +0100, Florian Weimer wrote:
> * Marc Haber:
>
> > This is, btw, not an ipv6 issue exclusively, it might happen in
> > ipv4-only setups as well. See Debian Bug #342619 for another example.
>
> I'm not sure if it's the same bug, and I wouldn't be surprised if the
> behavior was deliberate in that case (after all, the whole "long
> failure period" business is there to generate immediate bounces, so
> that users won't have to wait for five days until they are told about
> their mistake).
>
> In the example in the bug report, we there are two A RRs:
>
> mailrelay.
> mailrelay.
>
> But resolvers MUST cache the whole set of records and expire them at
> the same time. If the resolver fails to do this properly and provides
> a wrong view on DNS, there is no workaround on Exim's side.
The DNS setup changed in the meanwhile, because of, as I now know, a wrong
guess at the cause of the failure. The old setup had MX's from multiple
different zone's, and the one MX that had a long failure was also served
the DNS server of the mail server, but the secundary MX's to which the mail
should have been delivered to, was only remotely DNS-served -- so the DNS
server in question at times only had the broken MX cached (well, was
authoritive for it, even), and only that was in the additional section --
the IP addresses of the working MX's were out of the cache.
The only solution seems to me to actively query for all A (and AAAA)
records of all MX's before determining that no MX's are available for
delivery -- the additional DNS section is not to be trusted to ever give an
exhaustive list of IP addresses to try. I'd even say that it needs to
happen at every delivery attempt after delivery is found to be unsuccesful
to all the MX's in the additional section, because it can happen that some
MX's are more often in it than others. And you don't want delivery to fail
just because at the ultimate attempt all MX's happen to be down -- but some
were up in the past 4 days.
Note that RFC 974, MAIL ROUTING AND THE DOMAIN SYSTEM from 1986 (predating IPv6
by 12 years) explicitely warns against wrong handling of the DNS additional
section for MX queries:
| The incomplete data problem also requires some care when handling
| domain queries. If the answer section of a query is incomplete
| critical MX RRs may be left out. This may result in mail looping, or
| in a message being mistakenly labelled undeliverable. As a result,
| mailers may only accept responses from the domain system which have
| complete answer sections. Note that this entire problem can be
| avoided by only using virtual circuits for queries, but since this
| situation is likely to be very rare and datagrams are the preferred
| way to interact with the domain system, implementors should probably
| just ensure that their mailer will repeat a query with virtual
| circuits should the truncation bit ever be set.
Even though the RFC only mentions problems regarding to DNS datagram
truncation, and does not mention...
Debian Bug Importer (debzilla) wrote : | #16 |
Message-ID: <email address hidden>
Date: Mon, 19 Dec 2005 03:33:32 +0100
From: Jeroen van Wolffelaar <email address hidden>
To: Florian Weimer <email address hidden>
Cc: Marc Haber <email address hidden>, <email address hidden>,
<email address hidden>
Subject: Re: [exim] Potential logic error in retry handling for IPv4+IPv6 hosts
(Please cc me on all replies, I'm not subscribed)
On Sat, Dec 17, 2005 at 05:32:15PM +0100, Florian Weimer wrote:
> * Marc Haber:
>
> > This is, btw, not an ipv6 issue exclusively, it might happen in
> > ipv4-only setups as well. See Debian Bug #342619 for another example.
>
> I'm not sure if it's the same bug, and I wouldn't be surprised if the
> behavior was deliberate in that case (after all, the whole "long
> failure period" business is there to generate immediate bounces, so
> that users won't have to wait for five days until they are told about
> their mistake).
>
> In the example in the bug report, we there are two A RRs:
>
> mailrelay.
> mailrelay.
>
> But resolvers MUST cache the whole set of records and expire them at
> the same time. If the resolver fails to do this properly and provides
> a wrong view on DNS, there is no workaround on Exim's side.
The DNS setup changed in the meanwhile, because of, as I now know, a wrong
guess at the cause of the failure. The old setup had MX's from multiple
different zone's, and the one MX that had a long failure was also served
the DNS server of the mail server, but the secundary MX's to which the mail
should have been delivered to, was only remotely DNS-served -- so the DNS
server in question at times only had the broken MX cached (well, was
authoritive for it, even), and only that was in the additional section --
the IP addresses of the working MX's were out of the cache.
The only solution seems to me to actively query for all A (and AAAA)
records of all MX's before determining that no MX's are available for
delivery -- the additional DNS section is not to be trusted to ever give an
exhaustive list of IP addresses to try. I'd even say that it needs to
happen at every delivery attempt after delivery is found to be unsuccesful
to all the MX's in the additional section, because it can happen that some
MX's are more often in it than others. And you don't want delivery to fail
just because at the ultimate attempt all MX's happen to be down -- but some
were up in the past 4 days.
Note that RFC 974, MAIL ROUTING AND THE DOMAIN SYSTEM from 1986 (predating IPv6
by 12 years) explicitely warns against wrong handling of the DNS additional
section for MX queries:
| The incomplete data problem also requires some care when handling
| domain queries. If the answer section of a query is incomplete
| critical MX RRs may be left out. This may result in mail looping, or
| in a message being mistakenly labelled undeliverable. As a result,
| mailers may only accept responses from the domain system which have
| complete answer sections. Note that this entire problem can be
| avoided by only using virtual circuits for queries, but since this
| situation i...
In Debian Bug tracker #342619, Lionel Elie Mamane (lionel-mamane) wrote : master mail problems: A temporary workaround | #17 |
On Fri, Dec 09, 2005 at 01:41:55AM +0100, Jeroen van Wolffelaar wrote:
> On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
>> * Lionel Elie Mamane:
>>> On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
>>>> The fact that my primary MX is only available through IPv6, and
>>>> that this is the case for other people who're having problems too
>>>> might then be a better chance at being the problem.
>>> My primary MX is IPv6-only, too. I don't have detected a problem
>>> yet :)
>> Do you receive lots of mail from master.debian.org, and would you
>> notice the bounces? Mail from Debian mailing lists come directly
>> from murphy.debian.org, which does not seem to have the problem.
>> You also have one IPv4-only MX, which might be enough to prevent the
>> Exim bug[1] from occurring.
>> [1] I'm not sure if it's a Exim's fault, it's only a hunch.
> I've filed #342619 on the strong suspicion something fishy is going
> on in exim, even though I don't know for sure what's going on
> exactly.
Just a note for fellow DDs that are hit by the problem: A way to
temporarily fix it is to SSH to master and force resolution of your
domain's MXs; for example if your MXs are "foo.example.com" and
"bar.example.org" then do:
for qtype in A AAAA; do for mx in foo.example.com bar.example.org; do
dig -t "${qtype}" "${mx}"; done; done
(Hmm... Thinking now that doing the AAAA resolution is probably not
necessary.)
This fix will hold for as long as this information is in master's DNS
cache, thus for at most the TTL of your MX's A entries, maybe shorter
if master's bind decides to throw the information away from the cache
sooner.
Good luck!
--
Lionel
Debian Bug Importer (debzilla) wrote : | #18 |
Message-ID: <email address hidden>
Date: Mon, 19 Dec 2005 08:26:38 +0100
From: Lionel Elie Mamane <email address hidden>
To: Jeroen van Wolffelaar <email address hidden>, <email address hidden>
Cc: Florian Weimer <email address hidden>, Wouter Verhelst <email address hidden>,
Romain Francoise <email address hidden>, <email address hidden>
Subject: master mail problems: A temporary workaround
On Fri, Dec 09, 2005 at 01:41:55AM +0100, Jeroen van Wolffelaar wrote:
> On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote:
>> * Lionel Elie Mamane:
>>> On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
>>>> The fact that my primary MX is only available through IPv6, and
>>>> that this is the case for other people who're having problems too
>>>> might then be a better chance at being the problem.
>>> My primary MX is IPv6-only, too. I don't have detected a problem
>>> yet :)
>> Do you receive lots of mail from master.debian.org, and would you
>> notice the bounces? Mail from Debian mailing lists come directly
>> from murphy.debian.org, which does not seem to have the problem.
>> You also have one IPv4-only MX, which might be enough to prevent the
>> Exim bug[1] from occurring.
>> [1] I'm not sure if it's a Exim's fault, it's only a hunch.
> I've filed #342619 on the strong suspicion something fishy is going
> on in exim, even though I don't know for sure what's going on
> exactly.
Just a note for fellow DDs that are hit by the problem: A way to
temporarily fix it is to SSH to master and force resolution of your
domain's MXs; for example if your MXs are "foo.example.com" and
"bar.example.org" then do:
for qtype in A AAAA; do for mx in foo.example.com bar.example.org; do
dig -t "${qtype}" "${mx}"; done; done
(Hmm... Thinking now that doing the AAAA resolution is probably not
necessary.)
This fix will hold for as long as this information is in master's DNS
cache, thus for at most the TTL of your MX's A entries, maybe shorter
if master's bind decides to throw the information away from the cache
sooner.
Good luck!
--
Lionel
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed) | #19 |
tags #342619 - moreinfo
tags #342619 confirmed
thanks
On Sat, Dec 17, 2005 at 04:17:45PM +0100, Jeroen van Wolffelaar wrote:
> If you have any further questions, please ask,
Upstream has indicated that exim will be changed to ignore all data
sent to it in an additional section and that the patch is probably
trivial. His planning for january 2006 is already complete though, so
there won't be a patch until the end of the month.
If Debian wants to have this patch earlier, please submit a patch - I
don't have time and expertise to fix this myself.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
Debian Bug Importer (debzilla) wrote : | #20 |
Message-ID: <email address hidden>
Date: Mon, 2 Jan 2006 21:01:52 +0100
From: Marc Haber <email address hidden>
To: Jeroen van Wolffelaar <email address hidden>, <email address hidden>,
<email address hidden>
Cc: Marc Haber <email address hidden>
Subject: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
tags #342619 - moreinfo
tags #342619 confirmed
thanks
On Sat, Dec 17, 2005 at 04:17:45PM +0100, Jeroen van Wolffelaar wrote:
> If you have any further questions, please ask,
Upstream has indicated that exim will be changed to ignore all data
sent to it in an additional section and that the patch is probably
trivial. His planning for january 2006 is already complete though, so
there won't be a patch until the end of the month.
If Debian wants to have this patch earlier, please submit a patch - I
don't have time and expertise to fix this myself.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Re: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed) | #21 |
On Mon, Jan 02, 2006 at 09:01:52PM +0100, Marc Haber wrote:
> If Debian wants to have this patch earlier, please submit a patch - I
> don't have time and expertise to fix this myself.
Florian has submitted a patch, and I have made it available.
The Debian archive has exim4 4.60-2, which has the patch in
debian/
not applied though since it needs more testing.
http://
exim4 4.60-2+zg1, which has been built against sid with the patch enabled
exim4 4.60-2~zg1, which has been built against sarge with the patch
exim4 4.50-8sarge1~pre1, which is exim 4.50-8 from sarge with the
candidate for sarge-volatile and stable-
Please test at your convenience.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
Debian Bug Importer (debzilla) wrote : | #22 |
Message-ID: <email address hidden>
Date: Thu, 12 Jan 2006 14:52:33 +0100
From: Marc Haber <email address hidden>
To: <email address hidden>, <email address hidden>
Cc: Jeroen van Wolffelaar <email address hidden>
Subject: Re: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed)
On Mon, Jan 02, 2006 at 09:01:52PM +0100, Marc Haber wrote:
> If Debian wants to have this patch earlier, please submit a patch - I
> don't have time and expertise to fix this myself.
Florian has submitted a patch, and I have made it available.
The Debian archive has exim4 4.60-2, which has the patch in
debian/
not applied though since it needs more testing.
http://
exim4 4.60-2+zg1, which has been built against sid with the patch enabled
exim4 4.60-2~zg1, which has been built against sarge with the patch
exim4 4.50-8sarge1~pre1, which is exim 4.50-8 from sarge with the
candidate for sarge-volatile and stable-
Please test at your convenience.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Re: Re: Re: Bug#342619: Possible exim retry bug (Re: master mail problems -- help needed) | #23 |
tags #342619 forwarded http://
thanks
On Thu, Jan 12, 2006 at 02:52:33PM +0100, Marc Haber wrote:
> On Mon, Jan 02, 2006 at 09:01:52PM +0100, Marc Haber wrote:
> > If Debian wants to have this patch earlier, please submit a patch - I
> > don't have time and expertise to fix this myself.
>
> Florian has submitted a patch, and I have made it available.
Florians patch is still pending upstream approval. To speed things up,
I have created an entry in upstream's bugzilla.
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : | #24 |
forwarded #342619 http://
thanks
*groan*
Greetings
Marc
--
-------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 72739834
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 72739835
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Bug#342619: fixed in exim4 4.60-4 | #25 |
Source: exim4
Source-Version: 4.60-4
We believe that the bug you reported is fixed in the latest version of
exim4, which is due to be installed in the Debian FTP archive:
exim4-base_
to pool/main/
exim4-config_
to pool/main/
exim4-daemon-
to pool/main/
exim4-daemon-
to pool/main/
exim4_4.
to pool/main/
exim4_4.60-4.dsc
to pool/main/
exim4_4.
to pool/main/
eximon4_
to pool/main/
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Marc Haber <email address hidden> (supplier of updated exim4 package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.7
Date: Wed, 22 Feb 2006 10:30:16 +0000
Source: exim4
Binary: eximon4 exim4-daemon-custom exim4-daemon-heavy exim4-base exim4 exim4-daemon-light exim4-config
Architecture: source i386 all
Version: 4.60-4
Distribution: unstable
Urgency: low
Maintainer: Exim4 Maintainers <email address hidden>
Changed-By: Marc Haber <email address hidden>
Description:
exim4 - metapackage to ease exim MTA (v4) installation
exim4-base - support files for all exim MTA (v4) packages
exim4-config - configuration for the exim MTA (v4)
exim4-daemon-heavy - exim MTA (v4) daemon with extended features, including exiscan-ac
exim4-daemon-light - lightweight exim MTA (v4) daemon
eximon4 - monitor application for the exim MTA (v4) (X11 interface)
Closes: 342619
Changes:
exim4 (4.60-4) unstable; urgency=low
.
* add rationale to README.Debian explaining why using system
passwords for SMTP AUTH is a bad idea.
* streamline configuration to decrease differences to upstream default
example, and to adopt new things that were added since we last
looked there.
* Do not set inst_aliases for installation, this only affects
example.conf anyway.
* fail build if upstream's example configuration has changed.
* fix NEWS confusion. Thanks to Andreas for spotting this.
* exim4-base.
cleaned to avoid language confusion.
* document tls on connect in README.Debian.
* use adduser --quiet instead of > /dev/null in *.postinst.
* Add require_files directive to userforward router to avoid errors
when mailing uucp@hostname.
* Add comment about setting up TLS in conf.d/
to keep people from blindly allowing cl...
In Debian Bug tracker #342619, Marc Haber (mh+debian-packages) wrote : Bug#342619: fixed in exim4 4.50-8sarge1 | #26 |
Source: exim4
Source-Version: 4.50-8sarge1
We believe that the bug you reported is fixed in the latest version of
exim4, which is due to be installed in the Debian FTP archive:
exim4-base_
to pool/main/
exim4-config_
to pool/main/
exim4-daemon-
to pool/main/
exim4-daemon-
to pool/main/
exim4_4.
to pool/main/
exim4_4.
to pool/main/
exim4_4.
to pool/main/
eximon4_
to pool/main/
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to <email address hidden>,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Marc Haber <email address hidden> (supplier of updated exim4 package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing <email address hidden>)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.7
Date: Wed, 1 Mar 2006 15:24:51 +0000
Source: exim4
Binary: eximon4 exim4-daemon-custom exim4-daemon-heavy exim4-base exim4 exim4-daemon-light exim4-config
Architecture: source i386 all
Version: 4.50-8sarge1
Distribution: stable
Urgency: low
Maintainer: Exim4 Maintainers <email address hidden>
Changed-By: Marc Haber <email address hidden>
Description:
exim4 - metapackage to ease exim MTA (v4) installation
exim4-base - support files for all exim MTA (v4) packages
exim4-config - configuration for the exim MTA (v4)
exim4-daemon-heavy - exim MTA (v4) daemon with extended features, including exiscan-ac
exim4-daemon-light - lightweight exim MTA (v4) daemon
eximon4 - monitor application for the exim MTA (v4) (X11 interface)
Closes: 342619
Changes:
exim4 (4.50-8sarge1) stable; urgency=low
.
* Replace 37_dns_
37_
Closes: #342619.
Files:
5a6cefb583d711
d3f3db2a55d6a8
d43d92c8dcbcb9
2f87ae6c13c011
80332d57360730
ba5ea026849b5b
ed81f1a9a070af
11ded685cfac9c
Matt Zimmerman (mdz) wrote : | #27 |
This is fixed in Debian now, and we should merge the fix
Changed in exim4: | |
assignee: | nobody → dsilvers |
Daniel Silverstone (dsilvers) wrote : | #28 |
Accepted:
OK: exim4_4.
-> Component: main Section: mail
OK: exim4_4.
Format: 1.7
Date: Thu, 13 Apr 2006 11:32:52 +0100
Source: exim4
Binary: eximon4 exim4-daemon-custom exim4-daemon-heavy exim4-base exim4 exim4-daemon-light exim4-config
Architecture: source
Version: 4.60-3ubuntu3
Distribution: dapper
Urgency: low
Maintainer: Exim4 Maintainers <email address hidden>
Changed-By: Daniel Silverstone <email address hidden>
Description:
exim4 - metapackage to ease exim MTA (v4) installation
exim4-base - support files for all exim MTA (v4) packages
exim4-config - configuration for the exim MTA (v4)
exim4-
exim4-daemon-heavy - exim MTA (v4) daemon with extended features, including exiscan-ac
exim4-daemon-light - lightweight exim MTA (v4) daemon
eximon4 - monitor application for the exim MTA (v4) (X11 interface)
Changes:
exim4 (4.60-3ubuntu3) dapper; urgency=low
.
* Replace 37_dns_
sanctioned patch for this behaviour and turn it on.
Closes: launchpad #26820
Files:
8c6c362359cb50
f4f0197e533468
Changed in exim4: | |
status: | Unconfirmed → Fix Released |
In Debian Bug tracker #342619, Bts-link-upstream (bts-link-upstream) wrote : [bts-link] source package exim4 | #29 |
# bts-link upstream status pull for source package exim4
# see http://
#
user <email address hidden>
tags 222540 + upstream
usertags 222540 + status-NEW
usertags 240883 + status-RESOLVED resolution-WONTFIX
usertags 268083 + status-RESOLVED resolution-FIXED
tags 280282 + upstream
usertags 280282 + status-NEW
tags 297915 + upstream
usertags 297915 + status-NEW
usertags 302600 + status-NEW
usertags 303130 + status-NEW
tags 315032 + fixed-upstream
usertags 315032 + status-RESOLVED resolution-FIXED
tags 342619 + upstream fixed-upstream
usertags 342619 + status-RESOLVED resolution-FIXED
thanks
Heh. Example:
----- Forwarded message from Mail Delivery System <email address hidden> -----
Return-path: <> wolffelaar. nl ident=Debian-exim) 0:RSA_AES_ 256_CBC_ SHA:32) wolffelaar. nl with esmtp (Exim 4.50) Recipients: <email address hidden>
Envelope-to: <email address hidden>
Delivery-date: Fri, 09 Dec 2005 01:42:04 +0100
Received: from 220pc220.sshunet.nl
([145.97.220.220] helo=mordor.
by a-eskwadraat.nl with esmtps (TLS-1.
(Exim 4.50)
id 1EkWKq-0004xe-EP
for <email address hidden>; Fri, 09 Dec 2005 01:42:04 +0100
Received: from master.debian.org ([146.82.138.7])
by mordor.
id 1EkWKp-0001Ke-I1
for <email address hidden>; Fri, 09 Dec 2005 01:42:03 +0100
Received: from Debian-exim by master.debian.org with local (Exim 4.50)
id 1EkWKo-00040c-IO
for <email address hidden>; Thu, 08 Dec 2005 18:42:02 -0600
X-Failed-
Auto-Submitted: auto-generated
From: Mail Delivery System <email address hidden>
To: <email address hidden>
Subject: Mail delivery failed: returning message to sender
Message-Id: <email address hidden>
Date: Thu, 08 Dec 2005 18:42:02 -0600
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
<email address hidden>
retry time not reached for any host after a long failure period
------ This is a copy of the message, including all the headers. ------
Return-path: <email address hidden> Disposition: inline
Received: from a-eskwadraat.nl ([131.211.39.72])
by master.debian.org with esmtp (Exim 4.50)
id 1EkWKn-00040K-Cd; Thu, 08 Dec 2005 18:42:01 -0600
Received: from jeroen by a-eskwadraat.nl with local (Exim 4.50)
id 1EkWKh-0004xF-Q0; Fri, 09 Dec 2005 01:41:55 +0100
Date: Fri, 9 Dec 2005 01:41:55 +0100
To: Florian Weimer <email address hidden>
Cc: Lionel Elie Mamane <email address hidden>,
Wouter Verhelst <email address hidden>,
Romain Francoise <email address hidden>,
<email address hidden>
Subject: Re: master mail problems -- help needed
Message-ID: <email address hidden>
References: <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden> <email address hidden>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-
In-Reply-To: <email address hidden>
User-Agent: Mutt/1.5.9i
From: Jeroen van Wolffelaar <email address hidden>
On Thu, Dec 08, 2005 at 10:33:54PM +0100, Florian Weimer wrote: debian. org...
> * Lionel Elie Mamane:
>
> > On Thu, Dec 08, 2005 at 09:30:52PM +0100, Wouter Verhelst wrote:
> >
> >> The fact that my primary MX is only available through IPv6, and that
> >> this is the case for other people who're having problems too might
> >> then be a better chance at being the problem.
> >
> > My primary MX is IPv6-only, too. I don't have detected a problem yet :)
>
> Do you receive lots of mail from master.debian.org, and would you
> notice the bounces? Mail from Debian mailing lists come directly from
> murphy.