command-not-found crashes: unable to open database file (stack trace)

[Fabien of Hill]

$ synclient -i
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.7.3 final 0
Distributor ID: Ubuntu
Description: Ubuntu Disco Dingo (development branch)
Release: 19.04
Codename: disco
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

[J. Snow (jon.snow)]

Same here. I tried reinstalling all the related packages but the problem persists.

$ some-wrong-command
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.7.3 final 0
Distributor ID: Ubuntu
Description: Ubuntu 19.04
Release: 19.04
Codename: disco
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

[Illya Kysil (ikysil)]

It looks like permissions are broken after upgrade to 19.04 for the following files:
$ ls -l /var/lib/command-not-found
total 3184
-rw-r----- 1 root root 3252224 Apr 23 23:01 commands.db
-rw-r----- 1 root root 2403 Apr 23 23:01 commands.db.metadata

I've repaired them with
$ sudo chmod ugo+r /var/lib/command-not-found/commands.db*

command-not-found is not crashing after that.

[Zygmunt Krynicki (zyga)]

The files commands.db* are created by snapd. Presumably snapd should make the files readably by everyone.

[Illya Kysil (ikysil)]

Next day permissions are broken again

$ ls -l /var/lib/command-not-found/
total 3184
-rw-r----- 1 root root 3252224 Apr 24 19:01 commands.db
-rw-r----- 1 root root 2404 Apr 24 19:01 commands.db.metadata

It looks like a scheduled job, maybe snapd update?

[Melnofil (melnofil)]

$ toto
Désolé, command-not-found s'est arrêté anormalement ! Veuillez remplir un rapport de bogue à :
https://bugs.launchpad.net/command-not-found/+filebug
Veuillez inclure les informations suivantes dans le rapport :

Version de command-not-found : 0.3
Version de python : 3.6.8 final 0
Distributor ID: Ubuntu
Description: Ubuntu 18.10
Release: 18.10
Codename: cosmic
Information sur l'exception :

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 89, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 81, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file
$ ls -l /var/lib/command-not-found
total 3180
-rw------- 1 root root 3248128 mai 1 00:23 commands.db
-rw------- 1 root root 2421 mai 1 00:23 commands.db.metadata

[Sergio Callegari (callegar)]

Removing snapd seems to fix the issue. If you don't use any snap package, the workaround for the issue seems to be as simple as

sudo apt purge snapd

works for me, thanks for pointing out the relationship with snap.

[Zygmunt Krynicki (zyga)]

I think both sides must be improved: on snapd side we should ensure the mode of the file is correct. On command-not-found side we should be much more graceful in face of permission errors. They should not be fatal like they are today.

[G.M. (sexxxenator)]

This is the third time this bugs affects me after an apt-update of a XUbuntu 19.04... Please fix the file permission issue in your (post-install) scripts... I'm annoyed having to do this manually after each update!

[Sergio Callegari (callegar)]

Issue is still present in ubuntu 19.10.

Bug is particularly serious as it troubles and confuses inexperienced users to whom the 'command-not-found' utility is specifically directed, making 'command-not-found' totally counterproductive.

Please consider making command-not-found conflict with snapd until this is solved.

[Erik Neubauer (eneubauer)]

Workaround from https://bugs.launchpad.net/command-not-found/+bug/1824000/comments/2
confirmed working in (K)Ubuntu 19.10

[Sergio Callegari (callegar)]

Please check the thread. Fixing the permissions as in comment #2 fixes the issue only temporarily. See comment #4 by the same author of comment #2. Hence, this is no workaround.

[D Ledford (dledford-work)]

I was having a similar issue after upgrading from 19.10 to 20.04.

My issue turned out to be the same as this bug: https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1827759

The default UMASK on my system is 0027 so when /var/lib/command-not-found/*.db.* was updated the permissions were set 0640. (I don't recall modifying /etc/login.defs but that's what's in there. Perhaps it's the new UMASK on 20.04?)

It turns out the CNF db files are updated every time you install a package via apt due to the post installation hook that runs /usr/lib/cnf-update-db located here: /etc/apt/apt.conf.d/50command-not-found.

I install most software via 'sudo apt install' so each time I ran that command the permissions on /var/lib/command-not-found/*.db got set 0640 because of UMASK 0027. I confirmed my sudo sessions had UMASK 0027 with "sudo /bin/bash -c 'umask'".

I've set a default POSIX ACL giving o+r-- on /var/lib/command-not-found and so far that seems to have fixed my issue.

Complete fix for me:
sudo setfacl --mask -m d:o::r-- /var/lib/command-not-found
sudo rm /var/lib/command-not-found/*.db.*
sudo /usr/lib/cnf-update-db --verbose

It would be better, though, if the cnf-update-db script explicitly set the permissions on the generated DB files to 0644 on update. At the moment the script does not appear to do that directly.

[Jean Monet (jeanmonet)]

Same problem on Ubuntu 20.04 LTS

Manually fixing permissions with:
  sudo chmod ugo+r /var/lib/command-not-found/commands.db*
  # sets chmod to 644

..gets undone by running:
  sudo apt update

Following the 'sudo apt update', permissions are reset:
  ls -l /var/lib/command-not-found
  -rw-r----- 1 root root 3108864 Jun 30 19:31 commands.db
  -rw-r----- 1 root root 3239 Jun 30 19:31 commands.db.metadata

My sudo sessions have UMASK 0027 (checked with "sudo /bin/bash -c 'umask'"), I believe it may be part of my system's security settings.

A more permanent fix that seems to work for me is setting SGID bit on '/var/lib/command-not-found' directory as such:
  sudo chmod 2755 /var/lib/command-not-found/

Database files in that directory now seem to remain chmod 644 following apt updates and installs.

But this feels like a hack - will this issue be more permanently addressed?

[Noah Gorny (nononoha)]

As I suggested in https://code.launchpad.net/~nononoha/command-not-found/command-not-found/+merge/392242, simply chmoding the file after creation solves the problem!

[Noah Gorny (nononoha)]

Here is the patch!

[mkurz (matthias.kurz)]

Any chance to accept the patch soon? It's an annoying bug...

[Ian Johnson (anonymouse67)]

These files are created by snapd with permission 0644, as can be seen here: https://github.com/snapcore/snapd/blob/master/advisor/backend.go#L71

Additionally, I don't see anything in the maintainer scripts for snapd debian package which would change that permission, so I'm led to believe that the explanation given earlier about the umask being set differently causes apt to re-create the permissions on these files is the root cause.

As such, marking snapd as Invalid for this package.

[P (p92)]

$ lss
Désolé, command-not-found s'est arrêté anormalement ! Veuillez remplir un rapport de bogue à :
https://bugs.launchpad.net/command-not-found/+filebug
Veuillez inclure les informations suivantes dans le rapport :

Version de command-not-found : 0.3
Version de python : 3.8.6 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.10
Release: 20.10
Codename: groovy
Information sur l'exception :

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

[intel352 (intel352)]

I'm affected by this as well.
I'm curious if Noah's fix actually fixes the issue, as others noted it can reoccur after others manually chmodded the file.

[P (p92)]

Hello I use Kubuntu 20.10 and fixed this to be able to find a package and it worked.

But I just tried a few minutes ago and the bug is still present.

So this is a very temporary workaround for me !

[Alejandro Dini (alete89)]

I'm affected as well. Running Ubuntu Mate 20.04.

[Jan Remes (remes-abbas)]

I'm affected as well.

Running Ubuntu 20.04, fully updated.

First noticed today when 'thefuck' stopped working. Confirmed that other "unknown" commands trigger this.

$ foobar
Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.8.5 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

Confirmed that the issue is the permissions, running "chmod" as per comment 2 fixed it.

[Jan Remes (remes-abbas)]

Using "auditd", I have discovered that the issue is caused by script "/usr/lib/cnf-update-db", which periodically deletes the old database and creates a new file with restricted permissions.

[David Tomaschik (matir)]

If your umask is 027 rather than 022, the files end up only readable by root:root. I've modified /etc/apt/apt.conf.d/50command-not-found as follows:

# Refresh AppStream cache when APT's cache is updated (i.e. apt update)
APT::Update::Post-Invoke-Success {
    "if /usr/bin/test -w /var/lib/command-not-found/ -a -e /usr/lib/cnf-update-db; then umask 022 && /usr/lib/cnf-update-db > /dev/null; fi";
};

Note the addition of the `umask 022 &&`. This ensures the database update gets the 022 umask, allowing for it to be world-readable.

[The Pthyister (pthyister)]

This has been present on my 20.04 and 20.10 Ubuntu's, both on amd64 and arm64 builds. It comes and goes, usually I have to 'sudo apt purge command-not-found*' and then reinstall with 'sudo apt-get install command-not-found*'.

Would be kinda great if there was a final solution for this, instead of just having to go thru the workarounds from one year to another. Thanks.

###

Sorry, command-not-found has crashed! Please file a bug report at:
https://bugs.launchpad.net/command-not-found/+filebug
Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.8.6 final 0
Distributor ID: Ubuntu
Description: Ubuntu 20.10
Release: 20.10
Codename: groovy
Exception information:

unable to open database file
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/CommandNotFound/util.py", line 23, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 90, in main
    cnf = CommandNotFound.CommandNotFound(options.data_dir)
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 79, in __init__
    self.db = SqliteDatabase(dbpath)
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 12, in __init__
    self.con = sqlite3.connect(filename)
sqlite3.OperationalError: unable to open database file

###

[rubo77 (rubo77)]

Same issue here after upgrading from 20.10 to 21.04

this fixed the issue:

$ sudo chmod ugo+r /var/lib/command-not-found/commands.db*

[rubo77 (rubo77)]

sudo chmod ugo+r /var/lib/command-not-found/commands.db*

only fixes the issue temporarily, a fix for longer: see comment #24 or #12 or #13 (i am not sure, which is the best fix)

[Samuel Wallung (swuo21815)]

Same issue in Ubuntu 21.10. Apparently there is already a patch to resolve this issue. Any chance of accepting that?

[Brian Murray (brian-murray)]

This could be a duplicate of bug 1953610 which already has a fixed version of the package in -proposed for multiple releases. Could somebody please test the new version of command-not-found an report back? Thanks in advance!

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Joris Vleminckx (jvleminc)]

We noticed this error happening after running CIS Hardening, where the umask is changed from 022 to 027 (rule 5.5.4), as stated in #24.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.