offlineimap 6.3.3 deleting messages erroneously

[Greg Grossmeier]

I routinely discover that offlineimap decides to delete a large chunk of my "LISTS" folder (quick large, as you can imagine (>42,000 messages). It deletes over a thousand messages, and each time this happens it deletes more and more.

I researched/debugged, and this is the same issues as described here:
https://bugzilla.redhat.com/show_bug.cgi?id=708898
and here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=629515

An update to 6.3.4 (what is in Precise) fixes the issue.

[Impact]
Data loss, potentially forever if not monitored. This happens about daily for me.

[Development Fix]
upgrade to using libimap2 version 2.24

[Stable Fix]
What is needed:
Relevant upstream commit: https://github.com/nicolas33/offlineimap/commit/520e39d35536172d03b22d012cf4d8828687f2ff

[Test Case]
1) Install offlineimap 6.3.3
2) sync a mailbox with a large number of messages (>6,000)
3) continue to run offlineimap for a while (a day-ish?) until the server sends a response that imaplib2 version 2.22 doesn't like
4) observe the deletion of messages.
5) upgrade to imaplib2 version 2.24
6) continue syncing
7) observe no loss of data due to server messages

[Regression Potential]
The change only touches the imap code, but does hit a few places in the code that are non obvious, so it may cause some slight change in behavior. Worth the risk given the potential for data loss.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: offlineimap 6.3.3-3
ProcVersionSignature: Ubuntu 3.0.0-12.20-generic 3.0.4
Uname: Linux 3.0.0-12-generic x86_64
ApportVersion: 1.23-0ubuntu3
Architecture: amd64
Date: Sun Oct 30 14:08:08 2011
EcryptfsInUse: Yes
PackageArchitecture: all
SourcePackage: offlineimap
UpgradeStatus: No upgrade log present (probably fresh install)

[Clint Byrum (clint-fewbar)]

Fix uploaded to oneiric-proposed

[Martin Pitt (pitti)]

Hello Greg, or anyone else affected,

Accepted offlineimap into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Michael Hudson-Doyle (mwhudson)]

I have installed the new version from oneiric-proposed. I'll try to follow up here in a few days, but if I don't come bearing screaming rage, it probably works ok :-)

[Clint Byrum (clint-fewbar)]

Thanks Michael. I've been running the version from proposed for the last 36 hours or so, and the one I compiled from the same source as well. I have not seen any mass-deletes on my two 20,000+ email IMAP accounts, and everything else is still working well. Since I'm the original uploader to oneiric-proposed, I don't think my verification counts, so please Michael keep us posted over the next 5 days. I'd like this to progress to oneiric-updates ASAP.

[Greg Grossmeier (greg.grossmeier)]

I've been running the one uploaded to -proposed since I saw the email from pitti. Working as expected here; no mass deletions on my 42,000+ message folder.

[Martin Pitt (pitti)]

I have extensively used 6.3.4 from precise for more than a week now without any trouble.

[Michael Hudson-Doyle (mwhudson)]

Working for me too.

[Launchpad Janitor (janitor)]

This bug was fixed in the package offlineimap - 6.3.3-3ubuntu1

---------------
offlineimap (6.3.3-3ubuntu1) oneiric-proposed; urgency=low

  * d/p/update-imaplib2-to-2.24.patch: Cherrypick fix from upstream
    to stop randomly deleting messages that are still on the server.
    (LP: #883993)
 -- Clint Byrum <email address hidden> Tue, 08 Nov 2011 22:59:04 -0800