command-not-found crashes: unable to open database file (stack trace)

Bug #1824000 reported by Fabien of Hill
500
This bug affects 87 people
Affects Status Importance Assigned to Milestone
command-not-found
Triaged
High
Noah Gorny
snapd
Invalid
Medium
Unassigned
command-not-found (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

$ synclient -i
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.7.3 final 0
Distributor ID: Ubuntu
Description: Ubuntu Disco Dingo (development branch)
Release: 19.04
Codename: disco
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

Tags: patch

Related branches

Revision history for this message
J. Snow (jon.snow) wrote :

Same here. I tried reinstalling all the related packages but the problem persists.

$ some-wrong-command
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.7.3 final 0
Distributor ID: Ubuntu
Description: Ubuntu 19.04
Release: 19.04
Codename: disco
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

Revision history for this message
Illya Kysil (ikysil) wrote :

It looks like permissions are broken after upgrade to 19.04 for the following files:
$ ls -l /var/lib/command-not-found
total 3184
-rw-r----- 1 root root 3252224 Apr 23 23:01 commands.db
-rw-r----- 1 root root 2403 Apr 23 23:01 commands.db.metadata

I've repaired them with
$ sudo chmod ugo+r /var/lib/command-not-found/commands.db*

command-not-found is not crashing after that.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

The files commands.db* are created by snapd. Presumably snapd should make the files readably by everyone.

Revision history for this message
Illya Kysil (ikysil) wrote :

Next day permissions are broken again

$ ls -l /var/lib/command-not-found/
total 3184
-rw-r----- 1 root root 3252224 Apr 24 19:01 commands.db
-rw-r----- 1 root root 2404 Apr 24 19:01 commands.db.metadata

It looks like a scheduled job, maybe snapd update?

Revision history for this message
Melnofil (melnofil) wrote :

$ toto
Désolé, command-not-found s'est arrêté anormalement ! Veuillez remplir un rapport de bogue à :
https://bugs.launchpad.net/command-not-found/+filebug
Veuillez inclure les informations suivantes dans le rapport :

Version de command-not-found : 0.3
Version de python : 3.6.8 final 0
Distributor ID: Ubuntu
Description: Ubuntu 18.10
Release: 18.10
Codename: cosmic
Information sur l'exception :

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file
$ ls -l /var/lib/command-not-found
total 3180
-rw------- 1 root root 3248128 mai 1 00:23 commands.db
-rw------- 1 root root 2421 mai 1 00:23 commands.db.metadata

summary: - unable to open database file stack trace
+ command-not-found crashes: unable to open database file (stack trace)
Revision history for this message
Sergio Callegari (callegar) wrote :

Removing snapd seems to fix the issue. If you don't use any snap package, the workaround for the issue seems to be as simple as

sudo apt purge snapd

works for me, thanks for pointing out the relationship with snap.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

I think both sides must be improved: on snapd side we should ensure the mode of the file is correct. On command-not-found side we should be much more graceful in face of permission errors. They should not be fatal like they are today.

Changed in command-not-found:
status: New → Triaged
importance: Undecided → High
Changed in snapd:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
G.M. (sexxxenator) wrote :

This is the third time this bugs affects me after an apt-update of a XUbuntu 19.04... Please fix the file permission issue in your (post-install) scripts... I'm annoyed having to do this manually after each update!

Revision history for this message
Sergio Callegari (callegar) wrote :

Issue is still present in ubuntu 19.10.

Bug is particularly serious as it troubles and confuses inexperienced users to whom the 'command-not-found' utility is specifically directed, making 'command-not-found' totally counterproductive.

Please consider making command-not-found conflict with snapd until this is solved.

Revision history for this message
Erik Neubauer (eneubauer) wrote :

Workaround from https://bugs.launchpad.net/command-not-found/+bug/1824000/comments/2
confirmed working in (K)Ubuntu 19.10

Revision history for this message
Sergio Callegari (callegar) wrote :

Please check the thread. Fixing the permissions as in comment #2 fixes the issue only temporarily. See comment #4 by the same author of comment #2. Hence, this is no workaround.

Revision history for this message
D Ledford (dledford-work) wrote :

I was having a similar issue after upgrading from 19.10 to 20.04.

My issue turned out to be the same as this bug: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1827759

The default UMASK on my system is 0027 so when /var/lib/command-not-found/*.db.* was updated the permissions were set 0640. (I don't recall modifying /etc/login.defs but that's what's in there. Perhaps it's the new UMASK on 20.04?)

It turns out the CNF db files are updated every time you install a package via apt due to the post installation hook that runs /usr/lib/cnf-update-db located here: /etc/apt/apt.conf.d/50command-not-found.

I install most software via 'sudo apt install' so each time I ran that command the permissions on /var/lib/command-not-found/*.db got set 0640 because of UMASK 0027. I confirmed my sudo sessions had UMASK 0027 with "sudo /bin/bash -c 'umask'".

I've set a default POSIX ACL giving o+r-- on /var/lib/command-not-found and so far that seems to have fixed my issue.

Complete fix for me:
sudo setfacl --mask -m d:o::r-- /var/lib/command-not-found
sudo rm /var/lib/command-not-found/*.db.*
sudo /usr/lib/cnf-update-db --verbose

It would be better, though, if the cnf-update-db script explicitly set the permissions on the generated DB files to 0644 on update. At the moment the script does not appear to do that directly.

Revision history for this message
Jean Monet (jeanmonet) wrote :

Same problem on Ubuntu 20.04 LTS

Manually fixing permissions with:
  sudo chmod ugo+r /var/lib/command-not-found/commands.db*
  # sets chmod to 644

..gets undone by running:
  sudo apt update

Following the 'sudo apt update', permissions are reset:
  ls -l /var/lib/command-not-found
  -rw-r----- 1 root root 3108864 Jun 30 19:31 commands.db
  -rw-r----- 1 root root 3239 Jun 30 19:31 commands.db.metadata

My sudo sessions have UMASK 0027 (checked with "sudo /bin/bash -c 'umask'"), I believe it may be part of my system's security settings.

A more permanent fix that seems to work for me is setting SGID bit on '/var/lib/command-not-found' directory as such:
  sudo chmod 2755 /var/lib/command-not-found/

Database files in that directory now seem to remain chmod 644 following apt updates and installs.

But this feels like a hack - will this issue be more permanently addressed?

Noah Gorny (nononoha)
Changed in command-not-found:
assignee: nobody → Noah Gorny (nononoha)
Revision history for this message
Noah Gorny (nononoha) wrote :

As I suggested in https://code.launchpad.net/~nononoha/command-not-found/command-not-found/+merge/392242, simply chmoding the file after creation solves the problem!

Revision history for this message
Noah Gorny (nononoha) wrote :

Here is the patch!

Revision history for this message
mkurz (matthias.kurz) wrote :

Any chance to accept the patch soon? It's an annoying bug...

Revision history for this message
Ian Johnson (anonymouse67) wrote :

These files are created by snapd with permission 0644, as can be seen here: https://github.com/snapcore/snapd/blob/master/advisor/backend.go#L71

Additionally, I don't see anything in the maintainer scripts for snapd debian package which would change that permission, so I'm led to believe that the explanation given earlier about the umask being set differently causes apt to re-create the permissions on these files is the root cause.

As such, marking snapd as Invalid for this package.

Changed in snapd:
status: Triaged → Invalid
Revision history for this message
PascalC (p92) wrote :

$ lss
Désolé, command-not-found s'est arrêté anormalement ! Veuillez remplir un rapport de bogue à :
https://bugs.launchpad.net/command-not-found/+filebug
Veuillez inclure les informations suivantes dans le rapport :

Version de command-not-found : 0.3
Version de python : 3.8.6 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.10
Release: 20.10
Codename: groovy
Information sur l'exception :

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

Revision history for this message
intel352 (intel352) wrote :

I'm affected by this as well.
I'm curious if Noah's fix actually fixes the issue, as others noted it can reoccur after others manually chmodded the file.

Revision history for this message
PascalC (p92) wrote :

Hello I use Kubuntu 20.10 and fixed this to be able to find a package and it worked.

But I just tried a few minutes ago and the bug is still present.

So this is a very temporary workaround for me !

Revision history for this message
Alejandro Dini (alete89) wrote :

I'm affected as well. Running Ubuntu Mate 20.04.

Revision history for this message
Jan Remes (remes-abbas) wrote :

I'm affected as well.

Running Ubuntu 20.04, fully updated.

First noticed today when 'thefuck' stopped working. Confirmed that other "unknown" commands trigger this.

$ foobar
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.8.5 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

Confirmed that the issue is the permissions, running "chmod" as per comment 2 fixed it.

Revision history for this message
Jan Remes (remes-abbas) wrote :

Using "auditd", I have discovered that the issue is caused by script "/usr/lib/cnf-update-db", which periodically deletes the old database and creates a new file with restricted permissions.

Revision history for this message
David Tomaschik (matir) wrote :

If your umask is 027 rather than 022, the files end up only readable by root:root. I've modified /etc/apt/apt.conf.d/50command-not-found as follows:

# Refresh AppStream cache when APT's cache is updated (i.e. apt update)
APT::Update::Post-Invoke-Success {
    "if /usr/bin/test -w /var/lib/command-not-found/ -a -e /usr/lib/cnf-update-db; then umask 022 && /usr/lib/cnf-update-db > /dev/null; fi";
};

Note the addition of the `umask 022 &&`. This ensures the database update gets the 022 umask, allowing for it to be world-readable.

Revision history for this message
The Pthyister (pthyister) wrote :

This has been present on my 20.04 and 20.10 Ubuntu's, both on amd64 and arm64 builds. It comes and goes, usually I have to 'sudo apt purge command-not-found*' and then reinstall with 'sudo apt-get install command-not-found*'.

Would be kinda great if there was a final solution for this, instead of just having to go thru the workarounds from one year to another. Thanks.

###

Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.8.6 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.10
Release: 20.10
Codename: groovy
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

###

Revision history for this message
rubo77 (rubo77) wrote :

Same issue here after upgrading from 20.10 to 21.04

this fixed the issue:

$ sudo chmod ugo+r /var/lib/command-not-found/commands.db*

Revision history for this message
rubo77 (rubo77) wrote :

sudo chmod ugo+r /var/lib/command-not-found/commands.db*

only fixes the issue temporarily, a fix for longer: see comment #24 or #12 or #13 (i am not sure, which is the best fix)

Revision history for this message
Samuel Wallung (swuo21815) wrote :

Same issue in Ubuntu 21.10. Apparently there is already a patch to resolve this issue. Any chance of accepting that?

Revision history for this message
Brian Murray (brian-murray) wrote :

This could be a duplicate of bug 1953610 which already has a fixed version of the package in -proposed for multiple releases. Could somebody please test the new version of command-not-found an report back? Thanks in advance!

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Revision history for this message
Joris Vleminckx (jvleminc) wrote :

We noticed this error happening after running CIS Hardening, where the umask is changed from 022 to 027 (rule 5.5.4), as stated in #24.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in command-not-found (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Patches