crashes with SystemError: E:Unable to write mmap - msync

[Martin Pitt]

Binary package hint: python-apt

Current live CD ubiquity crashes with

Traceback (most recent call last):
  File "/usr/share/ubiquity/install.py", line 1787, in <module>
    install.run()
  File "/usr/share/ubiquity/install.py", line 411, in run
    self.install_extras()
  File "/usr/share/ubiquity/install.py", line 1599, in install_extras
    self.do_install(self.query_recorded_installed())
  File "/usr/share/ubiquity/install.py", line 1368, in do_install
    cache = Cache()
  File "/usr/lib/python2.5/site-packages/apt/cache.py", line 49, in __init__
    self.open(progress)
  File "/usr/lib/python2.5/site-packages/apt/cache.py", line 62, in open
    self._cache = apt_pkg.GetCache(progress)
SystemError: E:Unable to write mmap - msync (0 Success), E:The package lists or status file could not be parsed or opened.

This is not reported by apport, since it dies due to the same problem:

Traceback (most recent call last):
  File "/usr/share/apport/apport-gtk", line 252, in <module>
    app.run_argv()
  File "/var/lib/python-support/python2.5/apport/ui.py", line 253, in run_argv
    self.run_report_bug()
  File "/var/lib/python-support/python2.5/apport/ui.py", line 237, in run_report_bug
    self.collect_info()
  File "/var/lib/python-support/python2.5/apport/ui.py", line 348, in collect_info
    icthread.exc_raise()
  File "/var/lib/python-support/python2.5/apport/REThread.py", line 37, in run
    self._retval = self.__target(*self.__args, **self.__kwargs)
  File "/var/lib/python-support/python2.5/apport/ui.py", line 37, in thread_collect_info
    report.add_package_info(package)
  File "/var/lib/python-support/python2.5/apport/report.py", line 211, in add_package_info
    packaging.get_version(package),
  File "/var/lib/python-support/python2.5/apport/packaging_impl.py", line 54, in get_version
    return self._cache(package).installedVersion
  File "/var/lib/python-support/python2.5/apport/packaging_impl.py", line 45, in _cache
    self._apt_cache = apt.Cache()
  File "/usr/lib/python2.5/site-packages/apt/cache.py", line 49, in __init__
    self.open(progress)
  File "/usr/lib/python2.5/site-packages/apt/cache.py", line 62, in open
    self._cache = apt_pkg.GetCache(progress)
SystemError: E:Unable to write mmap - msync (2 No such file or directory), E:The package lists or status file could not be parsed or opened.

[Martin Pitt (pitti)]

This happened on the 20070922 amd64 Ubuntu live CD. This can be reproduced with

  sudo /usr/share/apport/apport-gtk -f -p bash

Setting to critical because it completely breaks ubiquity.

Michael, can you please have a look?

[Martin Pitt (pitti)]

Looking further, a simple 'apt-get update' reproduces the problem, too, so I assume the problem is in apt, not the python bindings.

[Martin Pitt (pitti)]

Hm, I wonder whether this is actually a unionfs bug:

Sep 22 11:46:00 ubuntu kernel: [ 93.022707] __open_dir: branch 0 lower_mnt_idx NULL! Set to ffff81002fe73c00
Sep 22 11:46:00 ubuntu kernel: [ 93.022733] __open_dir: opened ffff8100278989c0, autostart
Sep 22 11:46:00 ubuntu kernel: [ 93.022736] __open_dir: branches bstart 0, bend 1
Sep 22 11:46:00 ubuntu kernel: [ 93.022741] __open_dir: 0: lower_dentry ffff8100278984e0, autostart, mntpoint ffff81002fe73c00
Sep 22 11:46:00 ubuntu kernel: [ 93.022746] __open_dir: 1: lower_dentry ffff8100278984e0, autostart, mntpoint ffff81002fe91700
[...]
Sep 22 11:48:33 ubuntu kernel: [ 244.745543] unionfs: error creating directory tree for rename, bindex = 1, err = -30
Sep 22 11:48:33 ubuntu kernel: [ 244.776534] unionfs: error creating directory tree for rename, bindex = 1, err = -30

Attaching complete kern.log for completeness.

[Anish Bhatt (anish7)]

maybe related to https://bugs.launchpad.net/ubuntu/+bug/141577 ?

[Colin Watson (cjwatson)]

This is also a kernel bug. The manual page for msync() says:

RETURN VALUE
       On success, zero is returned. On error, -1 is returned, and errno is set appropriately.

Yet here an strace shows:

  msync(0xb630c000, 8564, MS_SYNC) = 524288

This is AOP_WRITEPAGE_ACTIVATE, which include/linux/fs.h says may be returned by writepage() methods on address space mappings. Like some other parts of the kernel, unionfs returns this from its writepage() method. The call chain appears to go like this: sys_msync -> do_fsync -> filemap_fdatawrite -> __filemap_fdatawrite -> __filemap_fdatawrite_range -> do_writepages -> generic_writepages -> write_cache_pages -> __writeback -> unionfs_writepage. When unionfs_writepage returns AOP_WRITEPAGE_ACTIVATE, write_cache_pages does check for this and unlocks the page, but it doesn't sanitise the return code (I don't know exactly where this ought to happen). As a result, that return code wrongly propagates all the way back up to userspace.

It is possible to work around this in apt by simply checking msync() < 0 rather than msync() != 0; I've tested this and 'apt-get update' now works fine. I'll upload this now, which will be good enough for beta.

[Colin Watson (cjwatson)]

apt (0.7.6ubuntu11) gutsy; urgency=low

  * apt-pkg/contrib/mmap.cc:
    - don't fail if msync() returns > 0 (LP: #144001)

 -- Colin Watson <email address hidden> Sat, 22 Sep 2007 21:39:29 +0100

[Gaspard Leon (gaspard.leon)]

Yep I'm getting this with the Live CD from Sep 21 live CD

[Gaspard Leon (gaspard.leon)]

downloaded the updated package and ran (in root terminal on live CD):
dpkg --install apt_0.7.6ubuntu11_i386.deb

it installed ok and now Synaptic and apt-get work ok

[Joseph Augustin (joseph-augustin)]

With Gutsy live CD from 2007-09-21,
starting adept I got the error:
The APT Database could not be opened! This may be caused by incorrect APT configuration or some similar problem. Try running apt-setup and apt-get update in terminal and see if it helps to resolve the problem.

apt-setup
command not found
apt-get update
Reading package lists... Error!
E: Unable to write mmap - msync (2 No such file or directory)
E: The package lists or status file could not be parsed or opened.

Is it already fixed on newer releases?

[Joseph Augustin (joseph-augustin)]

It took me half an hour to find how to download the package mentioned by Gaspard.
For all people, who are the same stupid that I am ;-)
You can download it manually from the repository http://packages.ubuntu.com/gutsy/base/apt

[Leann Ogasawara (leannogasawara)]

Hi Colin,

I've done some checking and see that a fix for the AOP_WRITEPAGE_ACTIVATE propogating back up issue has been fixed in the mainline kernel tree.

commit e423003028183df54f039dfda8b58c49e78c89d7
Author: Andrew Morton <email address hidden>
Date: Tue Oct 16 23:26:02 2007 -0700

    writeback: don't propagate AOP_WRITEPAGE_ACTIVATE

    This is a writeback-internal marker but we're propagating it all the way back
    to userspace!.

    Cc: <email address hidden>
    Signed-off-by: Andrew Morton <email address hidden>
    Signed-off-by: Linus Torvalds <email address hidden>

I suspect that once the Hardy kernel is released, this fix will already be incorporated. I'll tag as 'hardy-kernel-candidate' just to make sure I review and verify once the Hardy kernel is released. However against the linux-source-2.6.22 package I don't believe this qualifies for a SRU candidate based on the criteria I read from https://wiki.ubuntu.com/StableReleaseUpdates . As a result I'm marking this as "Won't Fix". Thanks!

[Leann Ogasawara (leannogasawara)]

Patch is included in Hardy.