apt-listchanges crashes after python upgrade

[Chad Waters]

Binary package hint: apt-listchanges

I'm getting this crash during upgrades. No changelogs are presented. The upgrades themselves remain successful.

Traceback (most recent call last):
  File "/usr/bin/apt-listchanges", line 231, in <module>
    main()
  File "/usr/bin/apt-listchanges", line 118, in main
    if config.save_seen and seen.has_key(srcpackage):
  File "/usr/lib/python2.5/site-packages/PIL/__init__.py", line 256, in has_key

  File "bsddb/dbutils.py", line 62, in DeadlockWrap
bsddb.db.DBRunRecoveryError: (-30975, 'DB_RUNRECOVERY: Fatal error, run database recovery -- /var/lib/apt/listchanges.db: pgin failed for page 2')
Exception bsddb.db.DBRunRecoveryError: DBRunRecoveryError(-30975, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') in ignored

Symptoms started after upgrade to python2.5 (2.5.1-5ubuntu3)

Thanks
-C

[Stefan Fleiter (stefan-fleiter)]

Same here.

[Sergey V. Udaltsov (sergey-udaltsov)]

Same here

[Daniel Nylander (yeager)]

+1

[Daniel Hahler (blueyed)]

JFI: I could workaround this, by uninstalling apt-listchanges and reinstalling it.
It appears like something (in the upgrade process?) really broke the underlying database..

[Erik Meitner (e.meitner)]

Same here.
I purged apt-listchanges and then followed it up by apt-get install but the install would hang on the preconfigure script. I just killed the preconfigure scripts and purged the package.

[Chad Waters (chad)]

I found that deleting /var/lib/apt/listchanges.db will remove the error. That is the file that became corrupt. It will be rebuilt on the next use of apt-listchanges.

[Gediminas Paulauskas (menesis)]

The upgrade which broke the db is probably this one

python2.5 (2.5.1-5ubuntu2) gutsy; urgency=medium

  * Revert back to use db4.5 for the bsddb3 extension. The DB_HASH access
    method databases can lockup the process.

If you upgrade often, you had the bsddb version 4.6 for some time. And as you probably know, berkeley db files are not compatible between versions.

The solution could be for apt-listchanges in postinstall try to open the db, and if it fails, just delete the file.

[Daniel Hahler (blueyed)]

I've patched apt-listchanges.py to catch the anydbm.error exception during loading/lookup.

The attached debdiff should fix this error.

[Daniel Holbach (dholbach)]

Looks good. This should be pushed to Debian too.

Uploading.

[Daniel Hahler (blueyed)]

apt-listchanges (2.74ubuntu3) gutsy; urgency=low

  * apt-listchanges.py: remove save_seen database on failure while
    opening or querying (LP: #139143)

 -- dAniel hAhler <email address hidden> Thu, 27 Sep 2007 23:41:31 +0200

[Malcolm Parsons (malcolm-parsons)]

I'm still getting this error on every upgrade with 2.74ubuntu3.

[Malcolm Parsons (malcolm-parsons)]

Traceback:
 Traceback (most recent call last):
   File "/usr/bin/apt-listchanges", line 238, in <module>
     main()
   File "/usr/bin/apt-listchanges", line 125, in main
     if config.save_seen and seen.has_key(srcpackage):
   File "bsddb/__init__.py", line 256, in has_key
     return _DeadlockWrap(self.db.has_key, key)
   File "bsddb/dbutils.py", line 62, in DeadlockWrap
 DBRunRecoveryError: (-30975, 'DB_RUNRECOVERY: Fatal error, run database recovery -- /var/lib/apt/listchanges.db: pgin failed for page 6')

[Daniel Hahler (blueyed)]

Malcolm, please attach your /var/lib/apt/listchanges.db file, so I can have a look.

[Daniel Hahler (blueyed)]

In the lookup test "foo" was used, which did not trigger the DBRunRecoveryError.

But using "foo%0" does, which the attached patch provides.

There's still the "Exception ..." line printed though. I don't know, why that's the case:
$ sudo apt-listchanges --save_seen=/var/lib/apt/listchanges.db system-config-printer_0.7.75+svn1653-0ubuntu2_all.deb
Recreating database /var/lib/apt/listchanges.db, because it failed to load.
Reading changelogs... Done
Exception bsddb.db.DBRunRecoveryError: DBRunRecoveryError(-30975, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') in ignored

Also a small program like the following is unable to hide the exception altogether:
import anydbm
seen = anydbm.open('/var/lib/apt/listchanges.db', 'c')
try:
 seen.has_key("foo%0")
except:
 print "caught."

[Scott Kitterman (kitterman)]

Unsubscribing UUS since there is no patch to upload right now.

[Daniel Hahler (blueyed)]

Scott, there is a patch. Posted just above your comment.

However, please note, that upstream (Debian) does not want to use this patch (deleting the database in case of failure).

Remember that it was only required because Python's bsddb3 extension has been downgraded in the Gutsy development cycle.

See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=444478#25

[Scott Kitterman (kitterman)]

OK. I read that comment as "patch doesn't actually work".

[Daniel Hahler (blueyed)]

Hmm.. thinking about it, we probably should not fix it by "silently" removing the database: it only affects people following Gutsy development and is considered bad practice upstream.

I've reworked the patch, so that it prints a friendlier error, links to this bug report and temporarily deactivates the save_seen feature.

[Scott Kitterman (kitterman)]

I'm inclined not to upload this. I suspect that people running Gutsy have already worked through this one way or another and it won't affect people who weren't running Gutsy for the bried period before the db jump was reverted.

[Daniel Holbach (dholbach)]

I think it'd help to mention how to repair it.

[Daniel Hahler (blueyed)]

I would have mentioned it, if it would be trivial/easy.

There does not seem exist an easy way to downgrade the database format (which is the reason here really for the "corruption").

The best I've found is http://pybsddb.sourceforge.net/ref/transapp/recovery.html, which refers to db_recover, which exists e.g. in the package db4.5-util (as db4.5_recover). However, it does not seem to work with "single file" databases, only with directories holding a database.

[Daniel Holbach (dholbach)]

Ok, uploaded.

[Daniel Hahler (blueyed)]

Seems to have been rejected: https://edge.launchpad.net/ubuntu/gutsy/+queue?queue_state=4&queue_text=apt-listchanges

[Luke Yelavich (themuso)]

Unsubscribing universe sponsors as there is no new content that needs sponsors attention right now.

[Daniel Hahler (blueyed)]

Fixed for Hardy in 2.75ubuntu1