Occasionally offlineimap gets stuck and eats up all of my RAM

[Clint Byrum]

I run offlineimap every 2 minutes to sync email from one IMAP account. It is a rather large INBOX with quite a few folders, but it should not be anything offlineimap can't handle. I use the ttyui output to keep it from breaking via '-u ttyui'.

Every few hours, I notice that my email has stalled, and offlineimap is eating 500MB+ of RAM.

ctrl-c on the running offlineimap produces

Terminating NOW (this may take a few seconds)...

However, it never terminates. It does shrink RSS down to what I would call "python interpreter + libs" level (40MB or so). But stracing it at this time shows this:

select(0, NULL, NULL, NULL, {0, 50000}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
rt_sigreturn() = -1 EINTR (Interrupted system call)
write(2, "Terminating NOW (this may take a"..., 49) = 49
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)

These selects go on forever. Pressing anything on stdin doesn't change that.

My offlineimap rc (redacted)

[general]
accounts = SpamapS
maxsyncaccounts = 2
maxconnections = 2
ui = Blinkenlights
#autorefresh = 1

[Account SpamapS]
localrepository = Local
remoterepository = Remote
#autorefresh = 2

[Repository Local]
type = Maildir
localfolders = ~/Mail/SpamapS
folderfilter = lambda folder: folder not in ['save', 'oldin']

[Repository Remote]
type = IMAP
remotehost = my.mail.host
remoteuser = xxxxxx
ssl = yes
remotepass = XXXXXXXXXXXXXXX
folderfilter = lambda folder: folder not in ['Sent','sru']
cert_fingerprint = 4b10f1a45ac47494e7a310b653c480df46c24e1a

#cert_fingerprint = 439b2183a562362178eb01e11b904ace5f147758

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: offlineimap 6.5.4-2
ProcVersionSignature: Ubuntu 3.10.0-6.17-generic 3.10.3
Uname: Linux 3.10.0-6-generic x86_64
NonfreeKernelModules: wl nvidia
ApportVersion: 2.12-0ubuntu2
Architecture: amd64
Date: Tue Aug 6 08:52:56 2013
InstallationDate: Installed on 2010-10-22 (1018 days ago)
InstallationMedia: Xubuntu 10.10 "Maverick Meerkat" - Release amd64 (20101008.1)
MarkForUpload: True
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: offlineimap
UpgradeStatus: Upgraded to saucy on 2013-07-30 (6 days ago)

[Max Bernstein (tekknolagi)]

I have not yet seen offlineimap take oodles of RAM, but I do see it download ~1 message per second with *lots* of selects and futex waits.

[Max Bernstein (tekknolagi)]

Oops, my version is:

cedar% offlineimap --version
7.2.3
cedar% lsb_release -irc
Distributor ID: Ubuntu
Release: 20.04
Codename: focal
cedar%