apt-index-watcher corrupts the apt cache database

[Txukie]

Hi,

I've been having this problem on and off during the time ive been using Kubuntu edgy (about 3 weeks now), and it's already made me format and reinstall twice (and looking forward for a third time). I dont know what provokes this problem on a first time, because it appears randomly during the night normally, long after any update. I get a segmentation fault from adept_notifier, and from there I cannot use apt anymore. Everytime I use it i get a segmentation fault. For example

$ sudo apt-get update
Ign http://packages.freecontrib.org edgy-plf Release.gpg
Des:1 http://security.ubuntu.com edgy-security Release.gpg [189B]
Ign http://security.ubuntu.com edgy-security/main Translation-es
Ign http://packages.freecontrib.org edgy-plf/free Translation-es
Des:2 http://es.archive.ubuntu.com edgy Release.gpg [189B]
Obj http://es.archive.ubuntu.com edgy/main Translation-es
Ign http://security.ubuntu.com edgy-security/restricted Translation-es
Ign http://security.ubuntu.com edgy-security/universe Translation-es
Ign http://security.ubuntu.com edgy-security/multiverse Translation-es
Obj http://security.ubuntu.com edgy-security Release
Ign http://packages.freecontrib.org edgy-plf/non-free Translation-es
Obj http://packages.freecontrib.org edgy-plf Release
Des:3 http://es.archive.ubuntu.com edgy/restricted Translation-es [14B]
Obj http://es.archive.ubuntu.com edgy/universe Translation-es
Obj http://es.archive.ubuntu.com edgy/multiverse Translation-es
Des:4 http://es.archive.ubuntu.com edgy-updates Release.gpg [189B]
Ign http://es.archive.ubuntu.com edgy-updates/main Translation-es
Ign http://es.archive.ubuntu.com edgy-updates/restricted Translation-es
Ign http://es.archive.ubuntu.com edgy-updates/universe Translation-es
Ign http://es.archive.ubuntu.com edgy-updates/multiverse Translation-es
Des:5 http://es.archive.ubuntu.com edgy-backports Release.gpg [189B]
Ign http://es.archive.ubuntu.com edgy-backports/main Translation-es
Obj http://packages.freecontrib.org edgy-plf/free Packages
Ign http://es.archive.ubuntu.com edgy-backports/restricted Translation-es
Ign http://es.archive.ubuntu.com edgy-backports/universe Translation-es
Ign http://es.archive.ubuntu.com edgy-backports/multiverse Translation-es
Des:6 http://es.archive.ubuntu.com edgy Release [34,7kB]
Obj http://packages.freecontrib.org edgy-plf/non-free Packages
Des:7 http://security.ubuntu.com edgy-security/main Packages [14B]
Obj http://packages.freecontrib.org edgy-plf/free Sources
Obj http://packages.freecontrib.org edgy-plf/non-free Sources
Des:8 http://security.ubuntu.com edgy-security/restricted Packages [14B]
Des:9 http://security.ubuntu.com edgy-security/universe Packages [14B]
Des:10 http://security.ubuntu.com edgy-security/multiverse Packages [14B]
Obj http://es.archive.ubuntu.com edgy-updates Release
Obj http://es.archive.ubuntu.com edgy-backports Release
Des:11 http://security.ubuntu.com edgy-security/main Sources [14B]
Des:12 http://security.ubuntu.com edgy-security/restricted Sources [14B]
Des:13 http://security.ubuntu.com edgy-security/universe Sources [14B]
Des:14 http://security.ubuntu.com edgy-security/multiverse Sources [14B]
Des:15 http://es.archive.ubuntu.com edgy/main Packages [677kB]
Des:16 http://es.archive.ubuntu.com edgy/restricted Packages [4886B]
Des:17 http://es.archive.ubuntu.com edgy/universe Packages [2630kB]
Des:18 http://es.archive.ubuntu.com edgy/multiverse Packages [96,2kB]
Des:19 http://es.archive.ubuntu.com edgy/main Sources [276kB]
Des:20 http://es.archive.ubuntu.com edgy/restricted Sources [1420B]
Des:21 http://es.archive.ubuntu.com edgy/universe Sources [1057kB]
Des:22 http://es.archive.ubuntu.com edgy/multiverse Sources [44,6kB]
Des:23 http://es.archive.ubuntu.com edgy-updates/main Packages [14B]
Des:24 http://es.archive.ubuntu.com edgy-updates/restricted Packages [14B]
Des:25 http://es.archive.ubuntu.com edgy-updates/universe Packages [14B]
Des:26 http://es.archive.ubuntu.com edgy-updates/multiverse Packages [14B]
Des:27 http://es.archive.ubuntu.com edgy-updates/main Sources [14B]
Des:28 http://es.archive.ubuntu.com edgy-updates/restricted Sources [14B]
Des:29 http://es.archive.ubuntu.com edgy-updates/universe Sources [14B]
Des:30 http://es.archive.ubuntu.com edgy-updates/multiverse Sources [14B]
Des:31 http://es.archive.ubuntu.com edgy-backports/main Packages [14B]
Des:32 http://es.archive.ubuntu.com edgy-backports/restricted Packages [14B]
Des:33 http://es.archive.ubuntu.com edgy-backports/universe Packages [14B]
Des:34 http://es.archive.ubuntu.com edgy-backports/multiverse Packages [14B]
Des:35 http://es.archive.ubuntu.com edgy-backports/main Sources [14B]
Des:36 http://es.archive.ubuntu.com edgy-backports/restricted Sources [14B]
Des:37 http://es.archive.ubuntu.com edgy-backports/universe Sources [14B]
Des:38 http://es.archive.ubuntu.com edgy-backports/multiverse Sources [14B]
Descargados 4823kB en 26s (185kB/s)
Fallo de segmentaciónetes... 0%

Or this one
$ sudo apt-get dist-upgrade
Fallo de segmentaciónetes... 0%

"Fallo de segmentación" means segmentation fault in spanish BTW.

If i run adept from a terminal i get the following output

$ adept_manager
X Error: BadDevice, invalid or uninitialized input device 168
  Major opcode: 148
  Minor opcode: 3
  Resource id: 0x0
Failed to open device
X Error: BadDevice, invalid or uninitialized input device 168
  Major opcode: 148
  Minor opcode: 3
  Resource id: 0x0
Failed to open device
kapture::PkgSystem::PkgSystem()
txukie@kubuntu:~$ KCrash: Application 'adept_manager' crashing...

Krash gives me the following debug information

(no debugging symbols found)
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
[New Thread -1232587088 (LWP 11511)]
[KCrash handler]
#6 0xb76a3c33 in strlen () from /lib/tls/i686/cmov/libc.so.6
#7 0xb782c116 in std::string::compare () from /usr/lib/libstdc++.so.6
#8 0xb7f58cf7 in debPackagesIndex::FindInCache ()
   from /usr/lib/libapt-pkg-libc6.4-6.so.3.51
#9 0xbff6278c in ?? ()
#10 0x00000000 in ?? ()

[magilus (magilus)]

Great. Could you please attach

/var/cache/apt/pkgcache.bin
/var/cache/apt/srcpkgcache.bin

to this bug report?

[Txukie (albertodetena)]

I straced apt and seems like the PLF repository is giving some problems. Still it shouldnt segfault because of a corrupt repo but im gonna investigate a bit further. Don't wanna jump to conclusions too soon though

Here are the files

[Txukie (albertodetena)]

Here is the other file

[magilus (magilus)]

Thanks..

I had the same issue and deleting

/var/cache/apt/pkgcache.bin
/var/cache/apt/srcpkgcache.bin

helped. Thanks again for attaching the files since this should make debugging of the problem much easier for the apt devs.

[Txukie (albertodetena)]

Ok thanks a lot deleting those files worked for me. I guess that after i use apt for a while it might give the same error again. I'l be checking on this bug

[Txukie (albertodetena)]

Just to let you know ive had the same problem again. I've backup-ed the files if you need them.

[Michael Vogt (mvo)]

Thanks for your bugreport and your detailed description. Unfortunately apt-index-watch seems to corrupt the apt database sometimes. We are investigating this issue and more data about this certainly welcome.

Thanks,
 Michael

[magilus (magilus)]

Appears to me again. Apport currently generates a crashreport.

[Martin Pitt (pitti)]

This really confuses apport - apport uses python-apt to figure out package information and thus crashes with

#
Program received signal SIGSEGV, Segmentation fault.
#
[Switching to Thread -1210767696 (LWP 14241)]
#
0xb7dc0c33 in strlen () from /lib/tls/i686/cmov/libc.so.6
#
(gdb) backtrace
#
#0 0xb7dc0c33 in strlen () from /lib/tls/i686/cmov/libc.so.6
#
#1 0xb7854186 in std::string::compare () from /usr/lib/libstdc++.so.6
#
#2 0xb7952a27 in debPackagesIndex::FindInCache () from /usr/lib/libapt-pkg-libc6.4-6.so.3.51

[P (p92)]

bug #57393 depends or is a duplicate of this one

[Michael Vogt (mvo)]

The problem is that apt-index-watcher opens the cache every 5s. It sets "ReadOnlyMode" when it opens it, but that does still rebuild /var/cache/apt/*.bin.

[Michael Vogt (mvo)]

I uploaded a new libapt-front_0.3.9ubuntu4 that should fix the issue. It ensures in apt-index-watcher that the main apt mmap file is not touched.

[depp (li-sun)]

is it really necessary to do it every 5s?
now with my laptop, even no programs are running, the cpu fan run once every 5s due to the apt-index-watcher, whicht takes up above 80% cpu when it's awaken.

[depp (li-sun)]

is it safe to stop it and how?
it's quite annoying now, that every 5s the fan starts once.

[depp (li-sun)]

i noticed, apt-index-watcher stops updating indexes after i run adept once, can anyone confirm it?

[omschaub (omschaub)]

running adept worked on my end.. thanks for the quick fix.. I am SO happy not to have this thing pegging my CPU every 5s

[mrvw0169 (bhnguyen)]

Wow, that sure was driving me nuts - I never touch adept or synaptic (only aptitude) and running adept does indeed stop the index watcher from running every 5 secs. Thanks!

Oh, and I had the exact adept corruption problems described when I tried to run adept the first time and removing those db files does fix it (after running kdesu adept_manager to rebuild the db).

[mrvw0169 (bhnguyen)]

Darn, I take it back... apt-index-watch is back, though it's better since it seems to come in bursts (there will be roughly 30 min of "quiet time" and then it's 10-15 min of apt-index-watch using 100% CPU = instant fan on, then dropping back down every 5-10 secs)... Though still very annoying!

[Erik Meitner (e.meitner)]

This following bug seems to be related to the work going on here and the frequent CPU usage spikes cause by apt-index watcher:

https://launchpad.net/distros/ubuntu/+source/libapt-front/+bug/64531

[Michael Vogt (mvo)]

Thanks for your additional comment. I changed the way it works again to be more efficient with the upload of 0.3.9ubuntu5. Please reopen if it still leads to noticable cpu-hooking.

Cheers,
 Michael