fetchmail: emails having an error during import are set as SEEN

[Guewen Baconnier @ Camptocamp]

The fetchmail module imports emails in OpenERP from IMAP or POP3 servers. It browses the UNSEEN emails in the mailbox, then imports them in OpenERP, eventually creating associated records (claims, leads, ...).
This bug happens with IMAP and POP3, but slightly differently.

Scenario:

I setup OpenERP to create claims in OpenERP for every mail that is received in a mailbox.
For some reason, some mails fails to be imported. However, OpenERP set them as SEEN (for IMAP; it deletes them on a POP server), so they They will never be imported again because they are now read on the server. So we can't know which emails have generated a claim and which have not.

Workaround: set all the emails to unread from the inbox, they will be imported again and OpenERP will ignore the duplicates using the message-id.

Expected:
It let them as UNSEEN when it could not import them so we have more chance to detect the failure, and as soon as the origin of the former failure is corrected, the emails are imported again.

[Amit Dodiya (OpenERP) (ado-openerp)]

Hello,

This issue is fixed with following branch:
branch: lp:~openerp-dev/openobject-addons/7.0-opw-605667-ado
revision-id: <email address hidden>
revision-no: 9963

Soon our experts will review and merge it with stable 7.0 addons.

Regards,
Amit

[Martin Trigaux (OpenERP) (mat-openerp)]

Hello,

The issue with this behaviour is that, in case of problematic email (e.g. malformed email), we will try to fetch it again every 10 minutes. If importing your email has failed at one point, it's very likely that it will still fail at the next import, as well as the 143 next runs the same day.
Each email would be leaving an error in your logs. You could quickly accumulate the invalid emails having many errors each 10 minutes. Even more if you run a saas platform with many instances all together.

I agree it's quite radical to erase the email in the case of POP but IMAP leaves it on your server for further inspection (one more reason to use IMAP instead of POP).

What do you think ?

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

That's exactly what I would expect: every 10 minutes I will see the
reminder that 10 mails have failed, you can't miss one line saying that
n mails have failed and this is an incentive to fix the issues (in our
cases, we see more often problems with configuration errors in aliases
than malformed emails). So, once the configuration is fixed, the emails
are imported correctly.

Currently, once the configuration is fixed, we don't know which emails could not be imported (our only solution is to
select all emails and reset to unread, which is very intensive and could
create duplicates if the message-id is missing) (and no, digging in the
logs to search the emails and put them one by one to unread is not a
solution). With this change, our client just have to connect on the
webmail to see which emails had problems (and eventually contact
directly their customer).

Deleting POP emails is insane. But of course we use IMAP.

After some thoughts, it appears that there is different figures here:
 - An email cannot be processed because it is malformed
 - An email cannot be processed due to an internal error in OpenERP (for
instance, it can't create a record due to a missing default value, or
access rights problems)

Maybe it would be interesting to differentiate them, I don't know if
this is possible though. At least, it seems logical to retry the emails
from the second category because the cause of the failure could have
been fixed in OpenERP.