crash in indexer.py, AttributeError: 'NoneType' object has no attribute 'unlink'

Bug #1652641 reported by Carlo Vanini on 2016-12-26
40
This bug affects 7 people
Affects Status Importance Assigned to Milestone
apt-xapian-index (Ubuntu)
Medium
Unassigned
Xenial
Medium
Brian Murray
Zesty
Medium
Unassigned

Bug Description

[Impact]

When the apt-xapian-index is already up to date, and under certain conditions, update-apt-xapian-index throws an exception.

The index /var/lib/apt-xapian-index is up to date
Exception ignored in: <bound method ServerProgress.__del__ of
<axi.indexer.ServerProgress object at 0x7f04a5ae4c18>>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/axi/indexer.py", line 306, in __del__
AttributeError: 'NoneType' object has no attribute 'unlink'

This happens more frequently when it is run with nice and ionice, which is the case in the cron.weekly/apt-xapian-index cron job.

This is not severe, since the exception is thrown on exit, and depends on the way python3 cleans up the environment. Anyway, this leaves a socket file behind, and creates noise in cron logs (see comment #2).

The patch uses a weakref finalizer in order to remove the socket file.

[Test Case]

* `sudo nice -n 19 ionice -c 3 update-apt-xapian-index --quiet`
* may need to run 4 times or more
* no exception is thrown

[Regression Potential]

* If the os module is unloaded anyway and os.unlink becomes unreachable, we get the same exception again.
* We import the `weakref` module, which was not used before.

[Other Info]

* This description was updated to reflect the changes in the behavior of the patch that was released in Zesty. It still needs sponsoring for Xenial.

André Verwijs (verwijs) wrote :

(k)ubuntu version 17.04
kernel version: 4.9.0-11.12
apt-xapian-index version: 0.47ubuntu11

Andreas Roth (aroth) wrote :

This morning i received this mail from the cron.weekly run:

/etc/cron.weekly/apt-xapian-index:
Exception ignored in: <bound method ServerProgress.__del__ of <axi.indexer.ServerProgress object at 0x7f373e677710>>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/axi/indexer.py", line 306, in __del__
AttributeError: 'NoneType' object has no attribute 'unlink'

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apt-xapian-index (Ubuntu):
status: New → Confirmed
Andreas Roth (aroth) wrote :

Sorry, but forgot to mention that i'm running Ubuntu yakkety on my machine.

tags: added: yakkety
Brian Murray (brian-murray) wrote :

I didn't see any instances of this Traceback at errors.ubuntu.com. It's worth noting that apt-xapian-index is only included in Task: kubuntu-desktop, kubuntu-full, lubuntu-qt-desktop, ubuntukylin-desktop.

Carlo Vanini (silhusk) wrote :

It looks like the same bug reappearing https://bugs.launchpad.net/ubuntu/+source/apt-xapian-index/+bug/1530518/comments/2

Running a-x-i directly doesn't trigger this bug, but the combination of nice and ionice makes it show up much more often. (When the index is already up to date)

Steps to reproduce:
 * `sudo update-apt-xapian-index`
 * Now that the index is updated we trigger the bug by running the following command (which is the same used in the cron job)
 * `sudo nice -n 19 ionice -c 3 update-apt-xapian-index --quiet`
 * may need to run 4 times or more

The intention of the proposed patch is to work around the problem by keeping a reference to the os.unlink function in the ServerProgress instance.
See http://stackoverflow.com/questions/17084260/imported-modules-become-none-when-running-a-function for an explanation of the issue in python.
Also see http://pydev.blogspot.ch/2015/01/creating-safe-cyclic-reference.html for a possible implementation using weakrefs.

The attachment "keep a reference to os.unlink" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Changed in apt-xapian-index (Ubuntu):
importance: Undecided → Medium
tags: added: zesty
Carlo Vanini (silhusk) on 2017-05-05
description: updated
Simon Quigley (tsimonq2) wrote :

Hello Carlo,

Could you please clarify which release this should go to? Yakkety is EOL now.

Also, irt to your patch, DEP-3 headers don't need "##" in front of them, please remove that.

Thank you for your interest in fixing this bug!

(unsubscribing ~ubuntu-sponsors as there's nothing to sponsor)

Carlo Vanini (silhusk) wrote :

Hello Simon,

version 0.47ubuntu13 in zesty-updates is still affected.
version 0.47ubuntu8.3 in xenial-updates is affected too.

Artful beta1 has 0.47ubuntu13 like Zesty does, but I could not reproduce the bug. The reason could be that Python changed from 3.5.3 (in Zesty) to 3.6.2 (in Artful). The way Python unloads modules might have changed, but I couldn't tell for sure from the docs https://docs.python.org/3/whatsnew/3.6.html

Patch updated, as you suggested.

Carlo Vanini (silhusk) wrote :

Sorry for the noise, patch updated again. This time it uses a weakref.finalizer instead of __del__. This is a little a cleaner solution. Moreover the finalizer is guaranteed to be called (at the latest at exit), whereas __del__ is not.

Amr Ibrahim (amribrahim1987) wrote :

Carlo, I subscribed ~ubuntu-sponsors for you to review the debdiff.

Simon Quigley (tsimonq2) wrote :

Apologies for the delay, sponsored to Zesty. Unsubscribing ~ubuntu-sponsors for now, but it would be great if you could create a patch for Xenial and resubscribe when you're ready. ;)

Thanks!

Changed in apt-xapian-index (Ubuntu Xenial):
status: New → Triaged
Changed in apt-xapian-index (Ubuntu Zesty):
status: New → Triaged
Carlo Vanini (silhusk) wrote :

This is the patch for Xenial. Tested in a 16.04.2 VM.

Hello Carlo, or anyone else affected,

Accepted apt-xapian-index into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt-xapian-index/0.47ubuntu13.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt-xapian-index (Ubuntu Zesty):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-zesty
Changed in apt-xapian-index (Ubuntu):
status: Confirmed → Fix Released
Changed in apt-xapian-index (Ubuntu Xenial):
importance: Undecided → Medium
Changed in apt-xapian-index (Ubuntu Zesty):
importance: Undecided → Medium
Carlo Vanini (silhusk) wrote :

It works for me on zesty with apt-xapian-index version 0.47ubuntu13.1 (python3 3.5.3-1)

tags: added: verification-done-zesty
removed: verification-needed-zesty
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt-xapian-index - 0.47ubuntu13.1

---------------
apt-xapian-index (0.47ubuntu13.1) zesty; urgency=medium

  * debian/patches/08_race_no_attribute_unlink.patch: use a finalizer
    instead of __del__ to close and remove the socket.
    When ServerProgress.__del__() is called the global os module might
    already have been set to None by the Python shutdown, causing an
    exception. (LP: #1652641)

 -- Carlo Vanini <email address hidden> Fri, 24 Nov 2017 21:23:12 -0600

Changed in apt-xapian-index (Ubuntu Zesty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for apt-xapian-index has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Jorma Hytonen (jormahytonen) wrote :

Ubuntu 16.04 (GNOME + KDE)
Linux 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
VERSION="16.04.3 LTS (Xenial Xerus)"

Anacron job 'cron.weekly' on ubuntu-t180

Every time I got email message:
/etc/cron.weekly/apt-xapian-index:
Exception ignored in: <bound method ServerProgress.__del__ of <axi.indexer.ServerProgress object at 0x7fdf1ad81b00>>
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/axi/indexer.py", line 306, in __del__
AttributeError: 'NoneType' object has no attribute 'unlink'

Regards,
Jorma Hytonen
StMichel, Finland

Carlo Vanini (silhusk) on 2018-02-01
description: updated
Changed in apt-xapian-index (Ubuntu Xenial):
status: Triaged → In Progress
assignee: nobody → Brian Murray (brian-murray)

Hello Carlo, or anyone else affected,

Accepted apt-xapian-index into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt-xapian-index/0.47ubuntu8.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt-xapian-index (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Carlo Vanini (silhusk) wrote :

Works for me on Xenial with apt-xapian-index version 0.47ubuntu8.4

tags: added: verification-done-xenial
removed: verification-needed-xenial
tags: removed: verification-needed yakkety
tags: added: yakkety
Jorma Hytonen (jormahytonen) wrote :

Ref to #20
Does this mean that I need to update my Ubuntu 16.04 system?
Download and install apt-xapian-index_0.47ubuntu8.4.tar.gz
Or can I wait for a regular update?

Carlo Vanini (silhusk) wrote :

Hi Jorma,
now that the verification is done it will be moved to xenial-updates and it will be part of the regular updates (and then the status will become "Fix Released").

If, instead, you want to have the fix before that, or you want to confirm that the problem is indeed fixed for you too (that's the goal of verification) you can download and install the new version by hand.

Jorma Hytonen (jormahytonen) wrote :

Hi Carlo
Thank you for the information.
I look forward to a regular update.

Thanks for your activity in solving the problem.

Regards,
Jorma

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt-xapian-index - 0.47ubuntu8.4

---------------
apt-xapian-index (0.47ubuntu8.4) xenial; urgency=medium

  * debian/patches/08_race_no_attribute_unlink.patch: use a finalizer
    instead of __del__ to close and remove the socket.
    When ServerProgress.__del__() is called the global os module might
    already have been set to None by the Python shutdown, causing an
    exception. (LP: #1652641)

 -- Carlo Vanini <email address hidden> Fri, 23 Feb 2018 08:44:53 -0800

Changed in apt-xapian-index (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers