gnus: Incoming mail does not get saved to file specified in mail-crash-crash-box

[Debian Bug Importer]

Automatically imported from Debian bug report #344946 http://bugs.debian.org/344946

[Debian Bug Importer (debzilla)]

Automatically imported from Debian bug report #344946 http://bugs.debian.org/344946

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Tue, 27 Dec 2005 15:22:32 -0500
From: Ivan Raikov <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: gnus: Incoming mail does not get saved to file specified in mail-crash-crash-box

Package: gnus
Version: 5.10.6-1.NO.20051213-1
Severity: grave

Recently, the following situation occurred: Gnus downloaded mail from
the IMAP server I use, and it deleted it on the server, since this is
the way I have configured it. However, there was something in the mail
messages that caused Gnus and Emacs to take up all available memory in
the system, and eventually the kernel started killing processes,
starting of course with Emacs.

    As a consequence, all of the incoming mail was lost. I could see
the new message headers in my Gnus folders, but the message bodies
were irretrievably lost.

    None of the messages were saved to the file specified in
`mail-source-crash-box' (which is the default
`~/.emacs-mail-crash-box'). Perhaps a better crash-save mechanism
should be implemented. The Gnus-relevant settings from my .emacs file
are:

(setq mail-sources (quote ((imap :server "xxxxx.xxxxx.xxx" :port 993
:user "xxxxxx" :stream ssl :authenticator cram-md5 :predicate "UNSEEN
UNDELETED"))))

(setq gnus-use-bbdb t)
(setq gnus-use-sendmail t)
(setq gnus-article-decode-charset 1)
(setq nnmail-delete-incoming t)
(setq mail-source-delete-incoming t)

(setq nnmail-fancy-expiry-targets (quote (("from" ".*" "nnfolder:Chronicle.%Y.%b"))))
(setq nnmail-split-fancy "")
(setq nnmail-split-methods ... ;; (omitted for brevity)
(setq gnus-message-archive-method (quote (nnfolder "Sent"
(nnfolder-directory "~/Mail/Sent/") (nnfolder-active-file
"~/Mail/Sent/active") (nnfolder-get-new-mail nil)
(nnfolder-inhibit-expiry t))))
(setq gnus-outgoing-message-group (quote ("nnfolder+Sent:Sent")))
(gnus-secondary-select-methods (quote ((nnfolder "mail"))))
(setq gnus-select-method (quote (nntp ...
(setq gnus-signature-limit 4.0)
(setq gnus-signature-separator (quote ("^-- $" "^-- *$" "^---*$" "^====*$")))
(setq gnus-thread-sort-functions (quote (gnus-thread-sort-by-date)))

-- System Information:
Debian Release: testing
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.4.27-2-686-smp
Locale: LANG=en_IE.UTF-8, LC_CTYPE=en_IE.UTF-8 (charmap=UTF-8)

Versions of packages gnus depends on:
ii debconf [debconf-2.0] 1.4.65 Debian configuration management sy
ii emacs21 21.4a-1 The GNU Emacs editor
ii make 3.80-11 The GNU version of the "make" util
ii ucf 2.004 Update Configuration File: preserv

gnus recommends no packages.

-- debconf information:
  gnus/upgrade/pre_5.9.0: false

--
"I'm absolutely sure that there are weapons of mass destruction there
 and the evidence will be forthcoming. We're just getting it just
 now."
 --Colin Powell, Remarks to Reporters, May 4, 2003
   http://www.state.gov/secretary/rm/2003/20166.htm

[Manoj (srivasta)]

tags 344946 unreproducible
severity 344946 important
thanks

Hi,

        Well, I am not sure what happened here: I am attaching the
 mail function that is used to get mail from imap folders.

 Note: The removing of stuff asked in mail-source-delete-incoming
 happens in `mail-source-callback' -- after the mail has been saved in
 a file named after the server, i.e. no old incoming files will be
 deleted, unless you receive new mail.

        Once the mail has bee4n saved, Incoming files are deleted, and
 then the crash box is deleted.

        The code looks fine to me. I am reducing the severity since
 when the OOM reaps, all kinds of strange things happen.

        It would help if you called gnus-bug to report this upstream
 as well, and CC <email address hidden>.

        thanks,

        manoj

,----
| (defun mail-source-fetch-imap (source callback)
| "Fetcher for imap sources."
| (mail-source-bind (imap source)
| (mail-source-run-script
| prescript (format-spec-make ?p password ?t mail-source-crash-box
| ?s server ?P port ?u user)
| prescript-delay)
| (let ((from (format "%s:%s:%s" server user port))
| (found 0)
| (buf (generate-new-buffer " *imap source*"))
| (mail-source-string (format "imap:%s:%s" server mailbox))
| (imap-shell-program (or (list program) imap-shell-program))
| remove)
| (if (and (imap-open server port stream authentication buf)
| (imap-authenticate
| user (or (cdr (assoc from mail-source-password-cache))
| password) buf)
| (imap-mailbox-select mailbox nil buf))
| (let ((coding-system-for-write mail-source-imap-file-coding-system)
| str)
| (with-temp-file mail-source-crash-box
| ;; Avoid converting 8-bit chars from inserted strings to
| ;; multibyte.
| (mm-disable-multibyte)
| ;; remember password
| (with-current-buffer buf
| (when (and imap-password
| (not (assoc from mail-source-password-cache)))
| (push (cons from imap-password) mail-source-password-cache)))
| ;; if predicate is nil, use all uids
| (dolist (uid (imap-search (or predicate "1:*") buf))
| (when (setq str
| (if (imap-capability 'IMAP4rev1 buf)
| (caddar (imap-fetch uid "BODY.PEEK[]"
| 'BODYDETAIL nil buf))
| (imap-fetch uid "RFC822.PEEK" 'RFC822 nil buf)))
| (push uid remove)
| (insert "From imap " (current-time-string) "\n")
| (save-excursion
| (insert str "\n\n"))
| (while (let ((case-fold-search nil))
| (re-search-forward "^From " nil t))
| (replace-match ">From "))
| (goto-char (point-max))))
| (nnheader-ms-strip-cr))
| (incf found ...

[Manoj (srivasta)]

severity 344946 grave
tags 344946 +moreinfo
--
"Yeah, but you're taking the universe out of context."
Manoj Srivastava <email address hidden> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Tue, 27 Dec 2005 16:23:51 -0600
From: Manoj Srivastava <email address hidden>
To: Ivan Raikov <email address hidden>
Cc: <email address hidden>, <email address hidden>
Subject: Re: Bug#344946: gnus: Incoming mail does not get saved to file specified in
 mail-crash-crash-box

tags 344946 unreproducible
severity 344946 important
thanks

Hi,

        Well, I am not sure what happened here: I am attaching the
 mail function that is used to get mail from imap folders.

 Note: The removing of stuff asked in mail-source-delete-incoming
 happens in `mail-source-callback' -- after the mail has been saved in
 a file named after the server, i.e. no old incoming files will be
 deleted, unless you receive new mail.

        Once the mail has bee4n saved, Incoming files are deleted, and
 then the crash box is deleted.

        The code looks fine to me. I am reducing the severity since
 when the OOM reaps, all kinds of strange things happen.

        It would help if you called gnus-bug to report this upstream
 as well, and CC <email address hidden>.

        thanks,

        manoj

,----
| (defun mail-source-fetch-imap (source callback)
| "Fetcher for imap sources."
| (mail-source-bind (imap source)
| (mail-source-run-script
| prescript (format-spec-make ?p password ?t mail-source-crash-box
| ?s server ?P port ?u user)
| prescript-delay)
| (let ((from (format "%s:%s:%s" server user port))
| (found 0)
| (buf (generate-new-buffer " *imap source*"))
| (mail-source-string (format "imap:%s:%s" server mailbox))
| (imap-shell-program (or (list program) imap-shell-program))
| remove)
| (if (and (imap-open server port stream authentication buf)
| (imap-authenticate
| user (or (cdr (assoc from mail-source-password-cache))
| password) buf)
| (imap-mailbox-select mailbox nil buf))
| (let ((coding-system-for-write mail-source-imap-file-coding-system)
| str)
| (with-temp-file mail-source-crash-box
| ;; Avoid converting 8-bit chars from inserted strings to
| ;; multibyte.
| (mm-disable-multibyte)
| ;; remember password
| (with-current-buffer buf
| (when (and imap-password
| (not (assoc from mail-source-password-cache)))
| (push (cons from imap-password) mail-source-password-cache)))
| ;; if predicate is nil, use all uids
| (dolist (uid (imap-search (or predicate "1:*") buf))
| (when (setq str
| (if (imap-capability 'IMAP4rev1 buf)
| (caddar (imap-fetch uid "BODY.PEEK[]"
| 'BODYDETAIL nil buf))
| (imap-fetch uid "RFC822.PEEK" 'RFC822 nil buf)))
| (push uid remove)
| (insert "From imap " (current-time-string) "\n")
| ...

[Debian Bug Importer (debzilla)]

Message-ID: <email address hidden>
Date: Tue, 27 Dec 2005 16:45:54 -0600
From: Manoj Srivastava <email address hidden>
To: <email address hidden>
Subject: As requested by the RM

severity 344946 grave
tags 344946 +moreinfo
--
"Yeah, but you're taking the universe out of context."
Manoj Srivastava <email address hidden> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

[Manoj (srivasta)]

severity 344946 normal
thanks

Hi,

        This is a one time, low memory incident -- and there has been
 no follow up, and no one else seems to have been impacted by
 this. There is code in place to save the mail in the new location
 before deleting it from thje old loc. No idea why that code did not
 execute, and nothing to work with.

        manoj
--
"It's not just a computer -- it's your ass." Cal Keegan
Manoj Srivastava <email address hidden> <http://www.golden-gryphon.com/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

[Matt Zimmerman (mdz)]

Downgraded upstream

[Carthik Sharma (carthik)]

Tagged unreproducible in Debian BTS. Closing.