Lucid fetchmail version way outdated and buggy

Bug #557467 reported by Matthias Andree
40
This bug affects 6 people
Affects Status Importance Assigned to Milestone
fetchmail (Ubuntu)
Won't Fix
High
Unassigned
Lucid
Won't Fix
High
Unassigned

Bug Description

Binary package hint: fetchmail

Note that backporting fixes from newer fetchmail releases to the ancient 6.3.9 (2008) will be increasingly difficult, particularly for server/LTS releases, so I advise that fetchmail be updated before Lucid release. Note that the upstream will not support non-current versions in any way, and support on the mailing lists is only provided for the latest release version -- 6.3.16 fixes several frequently-reported bugs!

Upstream changelog: (not showing regression fixes and translation updates)

# CHANGE
* Call OpenSSL_add_all_algorithms(). This is needed to support non-mandatory
  algorithms in certificates. Sjoerd Simons, to fix Debian Bug #576430.
  OpenSSL 0.9.8* does not load - for instance - the SHA256 digest by default.
  Reported as OpenSSL RT#2224.

# FEATURE
* Fetchmail now supports a bad-header command line or rcfile option that takes
  exactly one argument, accept or reject (default). This specifies how messages
  with bad headers retrieved from the current server are to be treated.

# BUG FIXES
* In the rcfile, recognize "local" as abbreviation for "localdomains", as
  documented. The short form has not ever worked since this feature was added in
  January 1997. Reported by Frédéric Marchal.
* Do not close stdout when using mda and "bsmtp -" at the same time.
* Log operating system errors when BSMTP writes fail.
* Check seteuid() return value and abort running MDA if switch fails.
* Set global flags in a consistent manner. Make --nosoftbounce and
  --nobounce work from command line (these used to work in rcfiles).
  Reported and fix confirmed working by N.J. Mann. (Sunil Shetye)
* Properly import h_errno declarations, even on systems where h_errno isn't a
  macro. (Adds ./configure check, fixes Cygwin dllimport warnings.)

# CHANGES
* "make check" now skips HTML validation if xmllint or XHTML DTD are missing.

# DOCUMENTATION
* Web site and documentation were adjusted to reflect the SVN->Git move.
* The fetchmail manual page is now much clearer on the user id switching
  (seteuid) when using --mda while running as the super user.

# BUG FIXES
* The IMAP client no longer skips messages from several IMAP servers including
  Dovecot if fetchmail's "idle" is in use. Causes were that fetchmail (a)
  ignored some untagged responses when it should not (b) relied on EXISTS
  messages in response to EXPUNGE, which aren't mandated by RFC-3501 (the IMAP
  standard) and aren't sent by Dovecot either.
    Fix by Sunil Shetye (the fix also consolidates IMAP response handling,
  improving overall robustness of the IMAP client), bug report and testing by
  Matt Doran, with further hints from Timo Sirainen.
* The SMTP client now recovers from errors (such as servers dropping the
  connection after errors) when sending an RSET command.
    Fix by Sunil Shetye. Report by James Moe.
* The IMAP client now uses "SEARCH UNSEEN" rather than "SEARCH UNSEEN NOT
  DELETED" again on IMAP2, to fix a regression in fetchmail 6.2.5 reported by
  Will Stringer in June 2004. (Sunil Shetye)
* The IMAP client now uses "SEARCH UNSEEN UNDELETED" on IMAP4 and IMAP4r1
  servers (Sunil Shetye).
* Workaround: The IMAP client now falls back to "FETCH n:m FLAGS" if the server
  does not support "SEARCH". (Sunil Shetye)
* The IMAP client now requests message numbers in batches of 1,000 to avoid
  problems if there are more than 1860 unseen messages. (Sunil Shetye)
    Note that this wasn't security relevant because fetchmail would only read up
  to the maximum buffer size and leave the remainder of the string unread, going
  out of synch afterwards.
* Stricter validation of IMAP responses containing byte or message counts.

# CHANGES
* Only include gssapi.h if we're not including gssapi/gssapi.h, to fix a FreeBSD
  compiler warning about gssapi.h being obsolete.

# DOCUMENTATION
* The README.SSL document was revised for grammar, spelling, and clarity.
  Courtesy of Robert Mullin.

# BUG FIXES
* Fix error reporting for GSSAPI on Heimdal (h5l) Kerberos.
* Look for MD5_Init in libcrypto rather than libssl, fixes Gentoo Kerberos
  builds; fixes upstream parts of Gentoo Bugs #231400 and #185652, and fixes
  BerliOS Bug #16134.
* Report multiline SMTP errors properly, reported by Earl Chew; fixes Debian Bug
  #529899, reported by Akihiro Terasaki.
* Replace control characters in SMTP replies by '?'.
* Fetchmailconf: Fix descriptions for smtpaddress and smtpname options;
  smtpaddress is for RCPT TO, not MAIL FROM. Found by Gerard Seibert.

# SECURITY BUGFIXES
* CVE-2009-2666: SSL NUL prefix impersonation attack through NULs in a
  part of a X.509 certificate's CommonName and subjectAltName fields. These
  fields use opaque strings with a separate length field, so that the NUL
  character isn't a special character inside the certificate. Fetchmail, being
  written in the C language, used to treat these strings as C strings
  nonetheless, so that the domain comparison would end at the first embedded NUL
  character, rather than at the real end of the string.
    Fetchmail will now abort certificate verification as failed if NULs are
  encountered inside either of these fields regardless of their position, and
  drop the connection even if --sslcertck is not used, because NUL is not a
  valid character in legitimate DNS names.
  See fetchmail-SA-2009-01.txt for details, including a minimal patch.

# BUGFIXES
* Remove the spurious message "message delimiter found while scanning headers".
  RFC-5322 syntax states that the delimiter is part of the body, and the body is
  optional.
* Convert all non-printable characters in certificate Subject/Issuer
  Common Name or Subject Alternative Name fields to ANSI-C hex escapes (\xnn,
  where nn are hex digits).
  See the 6.3.12 documentation above for details and a workaround.

# INCOMPATIBLE BUGFIXES AND CHANGES
* Fetchmail no longer drops permanently undelivered messages by default, to
  match historic documentation. It does this by adding a new "softbounce"
  option, see below.
  Fixes Debian Bug#471283, demotes Debian Bug#494418 to wishlist.
* There is a new "softbounce" global option that prevents the deletion of
  messages that have not been forwarded. It defaults to "true" for fetchmail
  6.3.X in order to match historic documentation. This may change its default
  in the next major release.

# BUGFIXES
* Fix misuse of canonical autoconf target as _TARGET when it should have been
  _HOST. Report and patch courtesy of Diego E. "Flameeyes" Pettenò.
  Details: http://blog.flameeyes.eu/2009/01/01/the-canonical-target
* Do not lose PS_MAXFETCH (13) exit status when hitting maxpoll. Reported by
  Michelle Konzack, Debian Bug#508667.
* Do not overlap source and destination fields in snprintf() in interface.c.
  Courtesy of Nico Golde, Debian.
* When a pre- or post-connect command fails, now report the exit status or
  termination signal properly through sys/wait.h macros.
* When acquiring a body, understand NIL ("no such data item"), as returned by
  some MS Exchange versions. Fixes BerliOS Bug #11980 by KB Sriram.
* Make progress tickers (-v/--showdots) consistent, and update documentation
  accordingly ("." for each 1024 octets read, "#" for a header written, and "*"
  for each body line written.)
  The conditions under which these had been printed were inconsistent,
  illogical, and documentation hadn't matched real behaviour for long.
* For NTLM authentication, use dynamically allocated buffers.
  Fixes Debian Bug#449179, reported by Stepan Golosunov.
* Non-delivery notice ("bounce mail") now mentions the original reason again,
  before the address list. This fixes a regression introduced in 6.3.0.
* Several compiler warnings were fixed.
* The minimum recommended SMTP (RFC-5321) timeouts are enforced to leave
  sufficient time for the listener to respond. Some synchronous listeners,
  particularly when used with spam filtering and other policy enforcement
  services, take extended amounts of time to process messages after the sender,
  recipient, or data block and EOM line. This can cause fetchmail to not wait
  long enough for the "250 Ok" and make fetchmail believe the message wasn't
  properly delivered when in fact it was; fetchmail would then retry the
  download next time and never make progress.
  Fixes Berlios Bug #10972, reported by Viktor Binzberger.
* The ESMTP/LMTP client will now apply an application-specific timeout while
  waiting for the EHLO/LHLO response, rather than wait for the server or TCP
  connection timeout.
* Treat 530 errors as temporary, so as not to delete messages on configuration
  errors. Partially taken from Petr Cerny's patch in Novell Bugzilla #246829.
  The 501 part of said patch was not added, as the maintainer is not convinced
  501 is a temporary condition, and softbounce takes care of this anyways.

# CHANGES
* Make the comparison of the SSL fingerprints case insensitive, to
  ease its use. Suggested by Daniel Richard G.
* Proper precedence ordering for the syslog and logfile options. If the logfile
  option is effective (i. e. we're in daemon mode and nodetach isn't used),
  reset the syslog option. If logfile is ineffective (we're not in daemon mode,
  or nodetach is set), syslog takes precedence.
* The sleeping at/awakened at messages appear in logfiles and syslog only if
  verbose mode is enabled. On the console, they will still appear without
  verbose mode. Fixes Debian Bug#282259.
* fetchmail only requests IPv6 addresses via name service if at least one is
  configured on the local host, likewise for IPv4. (AI_ADDRCONFIG flag to
  getaddrinfo()) Extended version of Redhat's patch.
* If the server name contains "yahoo.com", offers the "ID" capability, and we're
  polling via IMAP, send an ID ("guid" "1") transaction first, ignoring its
  result. This appears needed to be able to log into Yahoo's Zimbra servers, but
  there are open issues (such as being only able to download one message and
  server certificate mismatches).

# CHANGES TO CONTRIB
* Fix bashism in contrib/fetchsetup. Fixes Debian Bug#530081.

# DOCUMENTATION
* Some parts of the the manual page were revised for clarity, accuracy, and
  updated recommendations (particularly SSL/TLS) and formatting conventions from
  man-pages(7).
* The README and README.SSL documents were updated.
* A document, README.SSL-SERVER, was added to describe server-side requirements
  for proper SSL and/or TLS service offerings. These are not specific to
  fetchmail.
* Documentation on how to make "NOMAIL" (exit code 1) not treated an error has
  been added to the EXIT CODES section of the manpage and to the FAQ as item C8.
  The suggested solution uses a tiny POSIX shell script fragment.
  Fixes Debian Bug #530749, filed by Reuben Thomas.

# SECURITY AND CRITICAL BUG FIXES:
* CVE-2007-4565: Denial of service: When fetchmail tries to inject a warning
  message it created itself, and the message is refused by the SMTP listener,
  fetchmail dereferences a NULL pointer and crashes. Report & fix by Earl Chew.
  Note while this is theoretically a remote denial of service attack vector,
  fetchmail by default talks SMTP to the localhost, so the overall risk is
  rather low.
  This bug was apparently introduced on 1998-11-27 when the bouncemail facility
  was modularized. The bug then made its appearance in fetchmail release 4.6.8.
  See also fetchmail-SA-2007-02.txt.
* CVE-2008-2711: Denial of service: When fetchmail logs data blobs
  (for instance, a To: header in -v -v verbose mode) in excess of 2048
  bytes, it will crash, because it hands an uninitialized argument
  pointer (not the format string though) to vsnprintf and reads a
  random memory location (it calls va_arg() too often without
  resetting it with va_start()). Based on a patch (BerliOS patch #2492)
  by Petr Uzel, fixes Novell Bug #354291.
  Note 6.3.9-rc1 did not completely fix this issue, so it was redrawn a few
  hours after its release.
  See also fetchmail-SA-2008-01.txt.
* When expunging, mark the right messages as seen to avoid message loss in "keep
  flush" configurations. Workaround for previous versions: "expunge 0".
  Report and patch by Alexander Cherepanov - thanks a lot, Berlios Bug #11797,
  "imap_mark_seen doesn't consider expunged messages".
* SSL fix: close memory leak when SSL connection fails; fetchmail used to forget
  calling SSL_free() on the SSL context, leaking in excess of 500 kB RAM on a
  x86_64 system per failed SSL connection attempt.
  Bug reported and patch provided by Seiichi Ikarashi, Fujitsu.

# BUG FIXES:
* The configure script will additionally check for 'dn_skipname', to fix build
  failures with µClibc. The new check still recognizes the resolver libraries on
  Ubuntu 7.04, openSUSE 10.2, Solaris 8, NetBSD 4.0_BETA2 and FreeBSD 6.2.
  Fixes Gentoo bug #134187.
  NOTE: this is a bit of a hack, since we twist the HAVE_RES_SEARCH result, but
  res_search() and dn_skipname() are only used together and scheduled for
  removal in future versions, so this is probably fine.
* No longer complain about invalid sslproto "" when POP3 CAPA probe fails.
  Fixes Debian Bug#421446 (Holger Leskien), Novell Bug #247233 (Jon Nelson).
  Thanks to Matthias Strauß for a configuration to reproduce the issue.
* Allow .fetchmailrc and .fetchids to be symlinks, as the manpage does not
  document they aren't allowed - fixes Debian Bug #452907 (Roger Leigh).
  TOCTOU race persists.
* fetchmailconf quotes mailbox (folder) names when writing the configuration.
  Fixes BerliOS Bug #13207 (reported + fix suggested by Terry Brown).
* Only print "Deleting fetchids file" if there actually is one.
  Fixes Debian Bug#374514, reported by Dan Jacobson.
* SSL fix: check and report if SSL_set_fd fails.

# CHANGES:
* autoconf 2.60 is now required to build fetchmail; it uses
  AC_USE_SYSTEM_EXTENSIONS to replace AC_AIX, AC_MINIX, and the like.
* Removed dead FETCHMAIL_DEBUG code from fetchmail.h that was disabled by
  default with no switches in configure to enable it. However, the macro would
  have been prone to a symlink attack. Found by Nico Golde.
* Removed dead FORCE_STUFFING code from socket.c that was disabled by default
  with no switches in configure to enable it.
* Include the typedef for int16 in the #ifndef _AIX in smbencrypt.c (Peter
  O'Gorman)
* Correct check for u_int32_t in configure.ac (seems to be typedef'ed in
  namser.h on some platforms.) (Peter O'Gorman)
* In configure.ac change all CPFLAGS to CPPFLAGS, CEFLAGS to CFLAGS and LDEFLAGS
  to LDFLAGS otherwise the results of some tests (additional -L and -I flags) do
  not get used for later tests causing incorrect configure results. Makefile.am
  was also changed to reflect this. (Peter O'Gorman)
* m4/gethostbyname_r.m4 does AC_TRY_COMPILE, which unfortunately can pass even
  if there is no gethostbyname_r. Changed to AC_TRY_LINK. (Peter O'Gorman)
* Revise getnameinfo check to ensure NULL is defined and the result is properly
  evaluated, to avoid bogus results on for instance FreeBSD and redefinitions of
  NI_* at compile time. (Matthias Andree).
* __attribute__ ((unused)) is a gccism, removed from libesmtp/gethostbyname.c.
  (Peter O'Gorman)
* In KAME/getnameinfo.c it's best to use the correct argument to inet_ntoa.
  (Peter O'Gorman)
* In verbose mode, log if --check mode is enabled.
* Add sslcommonname option (rcfile and commandline) as a way to work around
  misconfigured upstream SSL servers that use the wrong certificate name. It
  specifies which CommonName fetchmail expects and logs. (Daniel Richard G.)
* Changed CRLF to LF line endings in contrib/delete-later (reporter: Petr Uzel)
* SSL change: enable all workarounds with SSL_CTX_set_options(ctx,SSL_OP_ALL)
* All translations have been re-enabled, in an attempt to rekindle translator or
  user interest.

# DOCUMENTATION:
* Add fetchmail-SA-2007-02.txt and fetchmail-SA-2008-01.txt.
* Re-add two lines to the manual page that had accidentally become comments
  to nroff. One was part of the --sslproto documentation, and one in the
  "Awakening the background daemon" section.
* The manual page no longer asserts that .fetchids were for exclusive POP3 use,
  since it is planned to use the file with IMAP4 later.
* Add grammar fixes from Dan Jacobson to fetchmail.man. Debian Bug #461642.
* The manual page now mentions that user descriptions need to come before user
  options. Reported by Francensco Pontortì, to fix Debian Bug #467010.
* The manual page no longer hints that multi-user declarations per server were
  only useful in daemon mode running as root, to avoid hinting people to doing
  that.
* Several manual page rcfile examples now include "ssl".
* The manual page hints that option arguments beginning with numbers can be
  enclosed in quotes.
* The manual page now mentions that the --logfile must already exist before
  fetchmail is run.
* The FAQ now recommends (#I9) not to use Google Mail for their disregard to the
  protocols they claim to support.
* Documentation and program output now /consistently/ claim that the rcfile must
  not have more than 0700 (u=rwx,g=,o=) permissions, but fetchmail will still
  silently accept additional g=x permissions for compatibility with previous
  6.2.X and 6.3.X versions.
  Inconsistency (program 0710, manpage 0600) reported by Petr Uzel.
* The --logfile documentation is now clearer about requiring detached daemon
  mode.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: fetchmail 6.3.9~rc2-4ubuntu5
ProcVersionSignature: Ubuntu 2.6.32-19.28-generic-pae 2.6.32.10+drm33.1
Uname: Linux 2.6.32-19-generic-pae i686
Architecture: i386
Date: Wed Apr 7 19:19:19 2010
EcryptfsInUse: Yes
ProcEnviron:
 LANGUAGE=en_GB:en
 LANG=de_DE.utf8
 SHELL=/bin/bash
SourcePackage: fetchmail

CVE References

Revision history for this message
Matthias Andree (matthias-andree) wrote :
description: updated
Changed in fetchmail (Ubuntu):
status: New → Confirmed
Daniel Hahler (blueyed)
Changed in fetchmail (Ubuntu):
importance: Undecided → Wishlist
importance: Wishlist → High
status: Confirmed → Triaged
Revision history for this message
Thierry Carrez (ttx) wrote :

This is not ready for sponsoring... The Ubuntu changes at 6.3.9~rc2-4ubuntu5 need to be merged with Debian's 6.3.16-1, a FeatureFreeze Exception needs to be granted by the release team, then sponsors can upload.

Unsubscribing sponsors, please resubscribe when there is something to review/upload.

Revision history for this message
Matthias Andree (matthias-andree) wrote :

Can someone make this visible in the Lucid release series please? Else it may be missed.

Revision history for this message
Thierry Carrez (ttx) wrote :

We cannot push to Lucid a complete new version with new features now, though we should fix the critical bugs and the security issues. You should open separate bugs about those, since migrating Lucid completely to 6.3.16 won't happen at this point, and we'll need separate bugs for the Lucid SRU process.

Changed in fetchmail (Ubuntu Lucid):
status: Triaged → Won't Fix
Revision history for this message
Matthias Andree (matthias-andree) wrote : Re: [Bug 557467] Re: Lucid fetchmail version way outdated and buggy

Thierry Carrez wrote on 2010-04-28:

> We cannot push to Lucid a complete new version with new features now,
> though we should fix the critical bugs and the security issues. You
> should open separate bugs about those, since migrating Lucid completely
> to 6.3.16 won't happen at this point, and we'll need separate bugs for
> the Lucid SRU process.

I am definitely:

- not going to help Ubuntu in any way to ship an outdated version (it will
only make users re-report fixed bugs upstream over and over again)

- not going to help Ubuntu help you break out the NEWS file into
individual bugs (I am not reponsible for that the release and package
managers missed to look at the bug in time before the freeze, and sync
 from Debian unstable)

- help you

I am not going to clone/file separate bugs for each of the relevant NEWS
entries.

I am not going to help you ship an outdated version.

It's not as though recent versions were particularly new or only happened
on the day when you froze the distro.

with all the fixes I have neither interest nor resources to do so. Do
update.

You're seeingly and deliberately shipping a version that is unsupported
upstream, when a new one was available even before the freeze

1. Newer releases have been available for long enough, both directly and
through Debian, particularly before the freeze.

2. Relevant pointers have been filed to the Launchpad Bugs section soon
enough before the freeze.

3. I am only supporting the latest upstream release.

--
Matthias Andree

Revision history for this message
Matthias Andree (matthias-andree) wrote :

Sorry, the message was inadvertently sent before it was finished.

Thierry Carrez wrote on 2010-04-28:

> We cannot push to Lucid a complete new version with new features now,
> though we should fix the critical bugs and the security issues. You
> should open separate bugs about those, since migrating Lucid completely
> to 6.3.16 won't happen at this point, and we'll need separate bugs for
> the Lucid SRU process.

I am definitely:

- not going to help Ubuntu in any way to ship an outdated version (it will
only make users re-report fixed bugs upstream over and over again)

- not going to help Ubuntu help you break out the NEWS file into
individual bugs (I am not reponsible for that the release and package
managers missed to look at the bug in time before the freeze, and sync
   from Debian unstable)

- not going to help in any efforts that are not going to reduce the
upstream support load.

It's not as though recent versions were particularly new or only happened
on the day when you froze the distro.

You're seeingly and deliberately shipping a version that is unsupported
upstream, when a new one was available even before the freeze.

Not taking an up-to-date bugfixed version was instead justified because
there were local changes.

I am not responsible for that relevant parties in Ubuntu fail to look at
bug reports in time. Also, the issue was mentioned on ubuntu-developers@
and -sponsors@ in time before the freeze.

What I will be doing is if the upstream bug reports go out of hand invoke
GPL §2a and demand that the package also be changed and a known old
version has been shipped for reasons entirely at Ubuntu's descretion and
fault.

--
Matthias Andree

Revision history for this message
Matthias Andree (matthias-andree) wrote :

Note that comments #5 and #6 do NOT YET constitute a formal request as to the GPL clause 2a; however I reserve the right to invoke it in case the support load for these outdated versions on the upstream list get too high.

Background: There is bad experience in another precent, that being leafnode-1.9.19 on Debian/woody at its time. This was a long-lived release, and updated versions that fixed CRITICAL data-corruption bugs, and had been released in 1.9.20, had not made it into Debian/woody although 1.9.22 or 1.9.24 had been released at the time when woody was frozen.

Revision history for this message
Thierry Carrez (ttx) wrote :

Matthias, I understand your concerns and wish we wouldn't be in that situation. The options are now:
1/ Update fetchmail to a completely new version (6.3.16) once the release is out
2/ Fix the critical issues as part of the SRU process (6.3.9+patches)

Given the policy, I expect the SRU team to reject the global change to 6.3.16 and support solution (2), but that's their final call.

Bryce Harrington (bryce)
Changed in fetchmail (Ubuntu):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.