update-manager crashed with SIGSEGV in debListParser::LoadReleaseInfo()

[cliff tinker]

just would not complete the update

ProblemType: Crash
DistroRelease: Ubuntu 12.04
Package: update-manager 1:0.156.8
ProcVersionSignature: Ubuntu 3.2.0-18.29-generic 3.2.9
Uname: Linux 3.2.0-18-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 1.94.1-0ubuntu2
Architecture: amd64
Date: Fri Mar 16 17:46:18 2012
ExecutablePath: /usr/bin/update-manager
GsettingsChanges:
 com.ubuntu.update-manager first-run false
 com.ubuntu.update-manager launch-time 1331920010
 com.ubuntu.update-manager show-details true
 com.ubuntu.update-manager window-height 932
 com.ubuntu.update-manager window-width 731
InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Alpha amd64 (20120201.2)
InterpreterPath: /usr/bin/python2.7
PackageArchitecture: all
ProcCmdline: /usr/bin/python /usr/bin/update-manager
ProcEnviron:
 SHELL=/bin/bash
 LANGUAGE=en_GB:en
 LANG=en_GB.UTF-8
SegvAnalysis:
 Segfault happened at: 0x7f4c9838a019 <_ZN13debListParser15LoadReleaseInfoERN8pkgCache15PkgFileIteratorER6FileFdSs+1561>: mov %eax,0x14(%rbx)
 PC (0x7f4c9838a019) ok
 source "%eax" ok
 destination "0x14(%rbx)" (0x7f4c7bfea5c4) not located in a known VMA region (needed writable region)!
SegvReason: writing unknown VMA
Signal: 11
SourcePackage: update-manager
StacktraceTop:
 debListParser::LoadReleaseInfo(pkgCache::PkgFileIterator&, FileFd&, std::string) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
 debPackagesIndex::Merge(pkgCacheGenerator&, OpProgress*) const () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
 ?? () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
 pkgCacheGenerator::MakeStatusCache(pkgSourceList&, OpProgress*, MMap**, bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
 pkgCacheFile::BuildCaches(OpProgress*, bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
Title: update-manager crashed with SIGSEGV in debListParser::LoadReleaseInfo()
UpgradeStatus: Upgraded to precise on 2012-03-07 (8 days ago)
UserGroups: adm audio avahi avahi-autoipd backup bin bluetooth cdrom clamav colord couchdb crontab daemon dialout dip disk fax floppy fuse games gdm gnats kmem libuuid lightdm list lp lpadmin mail man messagebus mlocate netdev news ntp operator plugdev proxy pulse pulse-access root rtkit sambashare saned sasl scanner shadow src ssh ssl-cert staff sudo sys syslog tape tty users utempter utmp uucp vboxusers video voice whoopsie www-data

[Apport retracing service (apport)]

StacktraceTop:
 debListParser::LoadReleaseInfo (this=0x7fff774659b0, FileI=..., File=<optimized out>, component=<optimized out>) at deb/deblistparser.cc:865
 debPackagesIndex::Merge (this=0x3584970, Gen=..., Prog=<optimized out>) at deb/debindexfile.cc:343
 BuildCache (Gen=..., Progress=0x7fff77466a60, CurrentSize=@0x7fff77466980, TotalSize=93041015, Start=..., End=...) at pkgcachegen.cc:1173
 pkgCacheGenerator::MakeStatusCache (List=<optimized out>, Progress=0x7fff77466a60, OutMap=0x35533c0, AllowMem=8) at pkgcachegen.cc:1364
 pkgCacheFile::BuildCaches (this=0x35533b0, Progress=0x7fff77466a60, WithLock=<optimized out>) at cachefile.cc:83

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Colin Watson (cjwatson)]

The failing line of code is:

  case Origin: FileI->Origin = WriteUniqString(data); break;

I suspect this might be a problem with the complex C++ gadgets that handle writing into apt's cache, but it's hard to tell. I further suspect that this is quite strongly dependent on things like the exact set of archives you have active. Could people affected by this please attach a tarball of /etc/apt/, in particular including sources.list and sources.list.d/*?

[David Kalnischkies (donkult)]

Colin is right, its about remapping. WriteUniqString() can cause the underlying data storage to move. FileI is registered to be rewritten to accommodate the move, but the compiler already has the address calculated so the rewrite of FileI is too late.

The line must be split into
map_ptrloc foobar = WriteUniqString(data);
FileI->Origin = foobar;
to work properly.

You want to backport rev. 2219.1.8 which fixes this line and a few siblings. There might be more as it was pretty hard and equally dull task to allow the storage to move arround (to fix the "mmap ran out of room") so I overlooked quiet a few cases which were uncovered over the year(s). I haven't seen any segfault with trunk for a while so I hope we finally have found everything, which usually means its the right time to introduce new bugs ;)

And for the record: Most of the time you not only need the sources but the exact same files as with a earlier/later version of the index files its likely that the content has changed causing the remap to happen earlier/later (if at all) so that this line works and might fail at another (or works because other places deal correctly with it).

[KALImar Franklin (kalimar999)]

Colin, I hope this helps..See Attachements

On Wed, Apr 3, 2013 at 10:36 AM, Colin Watson <email address hidden>wrote:

> The failing line of code is:
>
> case Origin: FileI->Origin = WriteUniqString(data); break;
>
> I suspect this might be a problem with the complex C++ gadgets that
> handle writing into apt's cache, but it's hard to tell. I further
> suspect that this is quite strongly dependent on things like the exact
> set of archives you have active. Could people affected by this please
> attach a tarball of /etc/apt/, in particular including sources.list and
> sources.list.d/*?
>
> ** Changed in: apt (Ubuntu)
> Status: Confirmed => Triaged
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1145193).
> https://bugs.launchpad.net/bugs/957231
>
> Title:
> update-manager crashed with SIGSEGV in
> debListParser::LoadReleaseInfo()
>
> Status in “apt” package in Ubuntu:
> Triaged
>
> Bug description:
> just would not complete the update
>
> ProblemType: Crash
> DistroRelease: Ubuntu 12.04
> Package: update-manager 1:0.156.8
> ProcVersionSignature: Ubuntu 3.2.0-18.29-generic 3.2.9
> Uname: Linux 3.2.0-18-generic x86_64
> NonfreeKernelModules: fglrx
> ApportVersion: 1.94.1-0ubuntu2
> Architecture: amd64
> Date: Fri Mar 16 17:46:18 2012
> ExecutablePath: /usr/bin/update-manager
> GsettingsChanges:
> com.ubuntu.update-manager first-run false
> com.ubuntu.update-manager launch-time 1331920010
> com.ubuntu.update-manager show-details true
> com.ubuntu.update-manager window-height 932
> com.ubuntu.update-manager window-width 731
> InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Alpha amd64
> (20120201.2)
> InterpreterPath: /usr/bin/python2.7
> PackageArchitecture: all
> ProcCmdline: /usr/bin/python /usr/bin/update-manager
> ProcEnviron:
> SHELL=/bin/bash
> LANGUAGE=en_GB:en
> LANG=en_GB.UTF-8
> SegvAnalysis:
> Segfault happened at: 0x7f4c9838a019
> <_ZN13debListParser15LoadReleaseInfoERN8pkgCache15PkgFileIteratorER6FileFdSs+1561>:
> mov %eax,0x14(%rbx)
> PC (0x7f4c9838a019) ok
> source "%eax" ok
> destination "0x14(%rbx)" (0x7f4c7bfea5c4) not located in a known VMA
> region (needed writable region)!
> SegvReason: writing unknown VMA
> Signal: 11
> SourcePackage: update-manager
> StacktraceTop:
> debListParser::LoadReleaseInfo(pkgCache::PkgFileIterator&, FileFd&,
> std::string) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
> debPackagesIndex::Merge(pkgCacheGenerator&, OpProgress*) const () from
> /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
> ?? () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
> pkgCacheGenerator::MakeStatusCache(pkgSourceList&, OpProgress*, MMap**,
> bool) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
> pkgCacheFile::BuildCaches(OpProgress*, bool) () from
> /usr/lib/x86_64-linux-gnu/libapt-pkg.so.4.12
> Title: update-manager crashed with SIGSEGV in
> debListParser::LoadReleaseInfo()
> UpgradeStatus: Upgraded to precise on 2012-03-07 (8 days ago)
> UserGroups: adm audio avahi avahi-autoipd backup bin bluetooth cdrom
> clamav colord ...

[Brian Murray (brian-murray)]

Looking at the Ubuntu Error Tracker we can see that this is still occurring (https://errors.ubuntu.com/bucket/?id=/usr/bin/update-manager:11:debListParser::LoadReleaseInfo:debPackagesIndex::Merge:BuildCache:pkgCacheGenerator::MakeStatusCache:pkgCacheFile::BuildCaches) with apt version 0.9.7.7ubuntu4 (you can see that in the dependencies of an instance) which seems to have the fix from rev 2219.1.8. Perhaps those are some of the more cases?

[Brian Murray (brian-murray)]

This seems to be a recurrence of bug 854090 which doesn't seem to have actually been fixed.

[Saikrishna Arcot (saiarcot895)]

My sources, as requested.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michael Vogt (mvo)]

This sounds like its worthwhile to SRU the attached apt diff from commit 2b803d4069e1f05d0461fbad004482ff02100812

[Brian Murray (brian-murray)]

This issue is much more prevalent on Saucy than on Precise.

92k crashes reported on Saucy vs
  3k crashes reported on Precise

https://errors.ubuntu.com/bucket/?id=/usr/bin/update-manager%3A11%3AdebListParser%3A%3ALoadReleaseInfo%3AdebPackagesIndex%3A%3AMerge%3ABuildCache%3ApkgCacheGenerator%3A%3AMakeStatusCache%3ApkgCacheFile%3A%3ABuildCaches

[Olivier Bilodeau (plaxx)]

Any update on the SRU for Precise?

[Colin Watson (cjwatson)]

This also caused some Landscape private PPA builds to segfault mysteriously. After tracking it down to this bug, I've uploaded an SRU, currently awaiting review. That should also provide a useful test case.

[Adam Conrad (adconrad)]

Hello cliff, or anyone else affected,

Accepted apt into precise-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/0.8.16~exp12ubuntu10.25 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Colin Watson (cjwatson)]

Unfortunately, despite my best efforts, my reproduction case went away - I think due to changes in the PPA in question. They may actually have enabled -proposed to get them out of the hole they were stuck in, but I can't tell directly. I've done some basic regression testing on the new version of apt (apt-get update, apt-get dist-upgrade, apt-get install debhelper), and it seems to be behaving itself.

[Andreas Hasenack (ahasenack)]

Hm, I didn't enable proposed for that ppa. There was no need, because only i386 builds were affected and we use amd64. I can confirm new i386 builds worked, but with no action on my part.

[Colin Watson (cjwatson)]

Only i386 builds were affected, but that included all Architecture: all builds. Peculiar. My best speculation is that perhaps a change in precise-updates was enough to perturb away the apt bug, but then that allowed the PPA to also change so that now even a combination of precise-updates at the time of the breakage plus the current state of the PPA isn't enough to reproduce the problem. That sort of thing is, unfortunately, quite usual for this kind of subtle apt bug.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apt - 0.8.16~exp12ubuntu10.25

---------------
apt (0.8.16~exp12ubuntu10.25) precise; urgency=medium

  * Backport patches from David Kalnischkies to fix crashes with dynamic
    cache remapping (LP: #957231):
    - Do not dereference the storage for the unique strings as the pointer
      can change at the time of writing the strings, so first store it
      temporary and then save the index in the (possibly new) pointer
      location
    - Handle moved mmap after UniqFindTagWrite call (Closes: #753941)

 -- Colin Watson <email address hidden> Wed, 22 Jul 2015 13:16:37 +0100

[Adam Conrad (adconrad)]

The verification of the Stable Release Update for apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.