syncfolder: ERROR attempting to sync flags - No such file or directory

[Paul Jackson]

I just upgraded from Kubuntu 11.10 to 12.04, and as soon as I did, a reliably working cron job using offlineimap started generating errors every run. The version of offlineimap I have from 12.04 is Version: 6.3.4-1

The errors looked like the following:

=======
Account sync xxxxxxxxxxxx:
 ***** Processing account xxxxxxxxxxxx
 Copying folder structure from IMAP to Maildir
 Establishing connection to mmmmmmmm:ppp.
Folder sync [xxxxxxxxxxxx]:
 Syncing fffffffffffff: IMAP -> Maildir
Copy message 14677 from fffffffffffff:
 Copy message 14677 IMAP[fffffffffffff] -> Maildir[fffffffffffff]
Folder sync [xxxxxxxxxxxx]:
 Adding flag S to 20 messages on fffffffffffff
 WARNING: ERROR attempting to sync flags for account xxxxxxxxxxxx:Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 421, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 378, in syncmessagesto_flags
    dstfolder.addmessagesflags(addflaglist[flag], [flag])
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 196, in addmessagesflags
    self.addmessageflags(uid, flags)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 192, in addmessageflags
    self.savemessageflags(uid, newflags)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Maildir.py", line 299, in savemessageflags
    os.path.join(self.getfullname(), newfilename))
OSError: [Errno 2] No such file or directory
=======

I was able to (apparently) fix the problem with the following patch to
/usr/share/pyshared/offlineimap/folder/Maildir.py, however I do not
understand the code well enough to be confident this fix is correct.

=======
--- Maildir.py.old 2012-06-18 03:18:48.786071861 -0500
+++ Maildir.py.new 2012-06-18 03:23:41.979234183 -0500
@@ -292,10 +292,9 @@
         infostr += '2,' + ''.join(flags)
         newname += infostr

- newfilename = os.path.join(dir_prefix, newname)
+ newfilename = os.path.join(self.getfullname(), dir_prefix, newname)
         if (newfilename != oldfilename):
- os.rename(os.path.join(self.getfullname(), oldfilename),
- os.path.join(self.getfullname(), newfilename))
+ os.rename(oldfilename, newfilename)
             self.messagelist[uid]['flags'] = flags
             self.messagelist[uid]['filename'] = newfilename
 =======

[Paul Jackson (pj-usa)]

attach patch, since putting it inline mangles the spacing :)

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "patch offlineimap Maildir savemessageflags rename" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[Paul Jackson (pj-usa)]

The above patch 'patch offlineimap Maildir savemessageflags rename' is wrong I believe.

I have a new, improved explanation for this problem (but I could still easily be very wrong.)

With the above patch, I started getting crashes when savemessageflags() was called from a different stack:

===============
Copy message 14678 from INBOX.folder:
 WARNING: ERROR attempting to copy message 14678 for account aaaaaa:Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 255, in copymessageto
    newuid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Maildir.py", line 267, in savemessage
    self.savemessageflags(uid, flags)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Maildir.py", line 297, in savemessageflags
    os.rename(oldfilename, newfilename)
OSError: [Errno 2] No such file or directory
===============

The problem, in my new improved view, is that the code is not consistent in what is stored in the 'filename' field of messagelist.

Most of the code, including the code in savemessageflags() that I attempted to patch above, is expecting the filename field to have exactly the last two directory components, that being the 'cur' or 'new' prefix, and the messagename.

The root cause of my above reported bug was that one routine had a different view of this. The 'scanfolder()' routine was storing the full pathname, including the value of getfullname(), in the messagelist 'filename' field ... oops.

Please ignore the above patch!

Consider instead, if you would, the following attached patch to scanfolder().

Thanks :).

[Paul Jackson (pj-usa)]

ah - I can do in a single os.path.relpath() call what it took me two splits and a join to do.

A new and improved version of the most recent patch above is attached to this post.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.