[gutsy] fetchmail: mail is permanently LOST when root disk is full and retrieved through POP3

[Christopher Barrington-Leigh]

Binary package hint: fetchmail

I'm using the latest Gutsy. Fetchmail retrieves my mail to a local /var partition which has ample space, using POP3.
But when the / partition became full, fetchmailconf retrieved mail from the server, complained that there was no space to save it, but let the server believe the mail had been successfully downloaded!

If the mail is not successfully saved, it should not be deleted from the server, ie should not be acknowledged as retrieved. This is a serious error, as it means mail is permanently lost and unacknowledged.

I can imagine this may even represent a security vulnerability, since emails may contain server alerts.

[Christopher Barrington-Leigh (cpbl)]

I can't do it, but shouldn't this be labeled important, even before it's confirmed by someone else?
Realistically, confirmation by someone other than me may require a deliberate test.
I don't think even one more person should lose email irretrievably this way!
c

[Gerald Zehetner (zege)]

I can confirm that, lost more than 100 e-mails.

zege

[Gerald Zehetner (zege)]

Set status to confirmed

[Matthias Andree (matthias-andree)]

In order to debug this upstream, please provide a trace along with all information listed in
http://www.fetchmail.info/fetchmail-FAQ.html#G3

fetchmail is designed to *NOT* mark the messages as read if the local MTA properly reports temporary failure, for instance, through SMTP codes between 400 and 499 inclusively, such as 421 disk full or the MDA (if you use --mda) through EX_TEMPFAIL from /usr/include/sysexits.h - this is 75 on all machines I know. Fetchmail may report 19200 instead. SMTP requires that servers return 4XX codes for temporary conditions such as disk full.

IF however your MDA exits with 1 (procmail for instance is notorious for all sorts of incapabilities in error handling), or your SMTP server exits with a 5XX code, the fault is outside fetchmail and this bug report then needs to be reassigned to the actual culprit (package).

IF the upstream server marks message for deletion implicitly on retrieval, that's also not a fetchmail bug; particularly, with POP3, fetchmail does not mark messages as deleted if it encounters temporary errors. If the server overrides this, please complain to your ISP to stop doing that and make their POP3 service conformant to RFC-1939.

[Christopher Barrington-Leigh (cpbl)]

I am using postfix with fetchmail pop3'ing from google. I do not know how to trace this. And it has just happened again.

[Matthias Andree (matthias-andree)]

I (as fetchmail maintainer) do believe that you're seeing what you're reporting, however:

Someone MUST provide the information requested in http://www.fetchmail.info/fetchmail-FAQ.html#G3 (particularly #6!) FROM A FETCHMAIL RUN THAT LOSES MAIL in the described way (disk full). Unless I have such information, I can neither tell if the bug is in fetchmail, in the server, or your configuration, or in the software that fetchmail passes the mail on to. And if it's inside fetchmail, there is no hint as to where exactly the problem is without such information.

So do stop wasting time writing "it happened to me too" posts, but do read http://www.fetchmail.info/fetchmail-FAQ.html#G3 and do provide the requested information. Send yourself test mail from a different account if needed.

[rusivi2 (rusivi2-deactivatedaccount)]

We'd like to figure out what's causing this bug for you, but we haven't heard back from you in a while. Could you please provide the requested information? Thanks!

[Christopher Barrington-Leigh (cpbl)]

Hi. Sorry! I would prefer to follow through on this! But not wanting to waste Matthias Andree's time any more, I'll have to decline and desist with the bug. I'm swamped and haven't a testbed whose disk I can fill up just now. I'll only write again if that changes.

[Launchpad Janitor (janitor)]

[Expired for fetchmail (Ubuntu) because there has been no activity for 60 days.]

[Matthias Andree (matthias-andree)]

I have recently come across a failure mode with POP3, and that is if servers still provide the "LAST" command. If fetchmail uses that, mail can be lost because the server will log the retrieval attempt no matter if it's successful. The workaround is to use --uidl on the command line, or add "uidl" as server option, as in "poll server.example.org proto pop3 uidl user foo password bar is baz here ssl sslcertck". fetchmail 7 will no longer support "LAST" because it's been obsolete for way more than a decade and exhibits this nasty "you cannot retry" behaviour.