[critical] dpkg error while processing - can't install nor upgrade

[Duong Phan]

My current ubuntu version :
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Jammy Jellyfish (development branch)
Release: 22.04
Codename: jammy

Issue :
when `sudo apt update` is ok, all newest package is downloaded
but `sudo apt upgrade` show current error : ]

sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  librygel-renderer-2.6-2 librygel-server-2.6-2 rygel
The following packages will be upgraded:
  bolt code deja-dup enchant-2 fwupd gdm3 gir1.2-gdm-1.0 gir1.2-gst-plugins-base-1.0 gir1.2-nm-1.0 gir1.2-nma-1.0
  gir1.2-upowerglib-1.0 gjs gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-plugins-base
  gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-good gstreamer1.0-pulseaudio gstreamer1.0-x initramfs-tools
  initramfs-tools-bin initramfs-tools-core iputils-ping iputils-tracepath language-pack-en language-pack-gnome-en
  libenchant-2-2 libfwupd2 libfwupdplugin5 libgdm1 libgjs0g libgstreamer-gl1.0-0 libgstreamer-plugins-base1.0-0
  libgstreamer-plugins-good1.0-0 libnl-3-200 libnl-genl-3-200 libnl-route-3-200 libnm0 libnma-common libnma0
  libpkcs11-helper1 libseccomp2 libunistring2 libunistring2:i386 libupower-glib3 media-types nano network-manager
  network-manager-config-connectivity-ubuntu python3-paramiko python3-software-properties simple-scan
  software-properties-common software-properties-gtk upower
56 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
10 not fully installed or removed.
Need to get 0 B/94,6 MB of archives.
After this operation, 1.928 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Extracting templates from packages: 100%
Preconfiguring packages ...
Setting up dpkg (1.21.1ubuntu1) ...
head: error reading 'dpkg': Is a directory
dpkg: error processing package dpkg (--configure):
 installed dpkg package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 dpkg
E: Sub-process /usr/bin/dpkg returned an error code (1)

i have to resintall all of the package in cache with `sudo dpkg -i /var/cache/apt/archives/*.deb
`, but the cache sometimes is not the newest packages
and when i rerun `sudo apt upgrade`, there is still not the newest package

[Julian Andres Klode (juliank)]

It seems you have created a directory where a file is supposed to be.

For support on your system, please use askubuntu, the ubuntu forums, the mailing list, or an IRC channel.

[Duong Phan (sugar3103)]

To clarify this : i didn't make any change to the system.
I just upgraded from ubuntu 20.04 and get that error.
Step to reproduce :
-in Ubuntu version 20.04
 $ sudo apt update
 $ sudo apt upgrade -> (no error)
 $ sudo apt dist-upgrade
 sudo apt autoremove
 sudo apt install update-manager-core
 sudo do-release-upgrade -d
 after upgrade
-in Ubuntu version 22.04
 sudo apt update
 sudo apt upgrade -> show above error
so please verify whether it's a bug from upgrade older version to jammy
thank you,

[Julian Andres Klode (juliank)]

I tried to reproduce the issue in a podman container, but was unable to do so. This makes sense: After do-release-upgrade -d, obviously apt upgrade will not have any pending upgrades.

Marking this as incomplete, as we'd need to see a clean reproducer to get anywhere.

Looking at the postinst, it fails because the dpkg database in /var/lib/dpkg contains an unexpected "dpkg" directory. Please investigate how this directory was created / what it contains, it should not be there.

[Duong Phan (sugar3103)]

i have fix the dpkg problem following by this solution :

https://unix.stackexchange.com/questions/161866/how-to-recreate-var-lib-dpkg-status

sudo cp /var/backups/dpkg.status.0 /var/lib/dpkg/status
sudo cp /var/backups/dpkg.statoverride.0 /var/lib/dpkg/statoverride
sudo cp /var/backups/dpkg.diversions.0 /var/lib/dpkg/diversions
sudo cp /var/backups/dpkg.arch.0 /var/lib/dpkg/arch

sudo dpkg --clear-avail
sudo apt update

so there could be some bug that remove or change the dpkg status file while upgrade from older version of ubuntu. if you couldn't reproduce then perhaps we are not using the same environment, i am using a HP Elitebook 840 G7.

after recreate dpkg status from backup, now i got this error:

sudo apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libc-bin : Depends: libc6 (> 2.35) but 2.34-0ubuntu3 is installed
 libc6-dbg : Depends: libc6 (= 2.35-0ubuntu1) but 2.34-0ubuntu3 is installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

sudo apt --fix-broken install
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Correcting dependencies... failed.
The following packages have unmet dependencies:
 libc-bin : Depends: libc6 (> 2.35) but 2.34-0ubuntu3 is installed
 libc6-dbg : Depends: libc6 (= 2.35-0ubuntu1) but 2.34-0ubuntu3 is installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
E: Unable to correct dependencies

so i think the package manager of jammy not yet upgrade the libc to 2.35, if this is not bug too then i think you can feel free to close the ticket.
thank you

[Launchpad Janitor (janitor)]

[Expired for dpkg (Ubuntu) because there has been no activity for 60 days.]

[Frank Bergemann (frank8812)]

I had the same yesterday for a fresh new installed Ubuntu 22.04.
Maybe it helps to have information, that the file it complained about in /var/lib/dpkg/ was

 dpkg -> /var/lib/dpkg

This was definitely not created by myself, but by the system

[Duong Phan (sugar3103)]

Yes, that's what I have tried to say, even the completely new installation gives that error.
Thank you

[Martin Belanger (uzebis)]

I hit the same issue upgrading from 20.04 to 22.04. This morning there was a pop-up window telling that a new version of Ubuntu (22.04) was available and whether I wanted to upgrade. I clicked on upgrade now. During the install I saw error message for some packages being upgraded including dpkg. What was suggested above (sudo cp /var/backups/dpkg.*) has improved things a bit, but now I'm getting:

sudo apt --fix-broken install
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 2232 not upgraded.
78 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
dpkg: error: duplicate file trigger interest for filename '/usr/lib/gdk-pixbuf-2.0/2.10.0/loaders' and package 'libgdk-pixbuf-2.0-0:i386'
E: Sub-process dpkg --set-selections returned an error code (2)
E: Couldn't clean the currently selected dpkg states

[Seth Arnold (seth-arnold)]

I have a vague memory of seeing directories showing up where files were expected; I can't recall much of this vague memory though. Was anyone using aufs or overlay or overlayfs or shiftfs or other slightly-unusual filesystem modification tricks? Slightly unusual filesystems? Slightly unusual block device layers?

Thanks

[Gene Weber (gene-weber)]

I ran into this same problem when doing a release upgrade on an AWS EC2 instance. I have not been able to fix a broken upgrade, but here is how to do a successful upgrade.

An "ls" on a brand new EC2 instance running 20.04.1 shows this:

$ ls -lat /var/lib/dpkg
total 1752
drwxr-xr-x 2 root root 4096 Sep 1 09:19 updates
drwxr-xr-x 7 root root 4096 Sep 1 09:19 .
-rw-r--r-- 1 root root 706338 Sep 1 09:19 status
-rw-r----- 1 root root 0 Sep 1 09:19 lock
-rw-r--r-- 1 root root 706377 Sep 1 09:19 status-old
drwxr-xr-x 2 root root 143360 Sep 1 09:19 info
drwxr-xr-x 47 root root 4096 Sep 1 09:19 ..
drwxr-xr-x 2 root root 4096 Sep 1 09:19 triggers
-rw-r--r-- 1 root root 135 Sep 1 09:19 statoverride
drwxr-xr-x 2 root root 4096 Sep 1 09:19 alternatives
-rw-r--r-- 1 root root 268 Nov 29 2021 diversions
-rw-r--r-- 1 root root 336 Nov 29 2021 diversions-old
-rw-r--r-- 1 root root 100 Nov 29 2021 statoverride-old
-rw-r----- 1 root root 0 Nov 29 2021 lock-frontend
-rw-r--r-- 1 root root 181566 Nov 29 2021 available
-rw-r--r-- 1 root root 8 Nov 29 2021 cmethopt
drwxr-xr-x 2 root root 4096 Mar 23 2020 parts

Trying to run a "do-release-upgrade" will not work if any 20.04.01 packages need to be upgraded. If "sudo apt upgrade" is run, "ls" now shows this:

$ ls -lat /var/lib/dpkg
total 1752
drwxr-xr-x 7 root root 4096 Sep 1 09:23 .
lrwxrwxrwx 1 root root 13 Sep 1 09:23 dpkg -> /var/lib/dpkg
drwxr-xr-x 2 root root 4096 Sep 1 09:21 updates
-rw-r--r-- 1 root root 707850 Sep 1 09:21 status
drwxr-xr-x 2 root root 143360 Sep 1 09:21 info
-rw-r----- 1 root root 0 Sep 1 09:21 lock
-rw-r----- 1 root root 0 Sep 1 09:21 lock-frontend
-rw-r--r-- 1 root root 707587 Sep 1 09:21 status-old
drwxr-xr-x 47 root root 4096 Sep 1 09:19 ..
drwxr-xr-x 2 root root 4096 Sep 1 09:19 triggers
-rw-r--r-- 1 root root 135 Sep 1 09:19 statoverride
drwxr-xr-x 2 root root 4096 Sep 1 09:19 alternatives
-rw-r--r-- 1 root root 268 Nov 29 2021 diversions
-rw-r--r-- 1 root root 336 Nov 29 2021 diversions-old
-rw-r--r-- 1 root root 100 Nov 29 2021 statoverride-old
-rw-r--r-- 1 root root 181566 Nov 29 2021 available
-rw-r--r-- 1 root root 8 Nov 29 2021 cmethopt
drwxr-xr-x 2 root root 4096 Mar 23 2020 parts

Proceeding with "do-release-upgrade" at this point breaks. But if I "sudo unlink /var/lib/dpkg/dpkg" before doing the "do-release-upgrade", the upgrade to 22.04 is successful.

[Brian Murray (brian-murray)]

@Gene - When you ran "sudo apt upgrade" what packages were upgraded? /var/log/apt/history.log might be helpful here.

[Stefan Schaub (s-schaub)]

Hi, same here.
Steps to reproduce
apt-get udpate
apt-get upgrade
do-release-upgrade

Here are the outputs requested by @brian-murray
https:// Links shortened because they got replaced by safe links (anti malware at our site)

Thanks @gene-weber for the workaround :-)

Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libfwupdplugin1 linux-headers-5.4.0-126 linux-headers-5.4.0-126-generic linux-image-5.4.0-126-generic linux-modules-5.4.0-126-generic
  linux-modules-extra-5.4.0-126-generic
Use 'sudo apt autoremove' to remove them.
Try Ubuntu Pro beta with a free personal subscription on up to 5 machines.
Learn more at https://...
The following packages will be upgraded:
  libksba8 sosreport
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 standard LTS security update
Need to get 400 kB of archives.
After this operation, 65.5 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://... focal-security/main amd64 libksba8 amd64 1.3.5-2ubuntu0.20.04.1 [95.1 kB]
Get:2 https://... focal-updates/main amd64 sosreport amd64 4.4-1ubuntu0.20.04.1 [305 kB] Fetched 400 kB in 0s (981 kB/s) (Reading database ... 146400 files and directories currently installed.) Preparing to unpack .../libksba8_1.3.5-2ubuntu0.20.04.1_amd64.deb ...
Unpacking libksba8:amd64 (1.3.5-2ubuntu0.20.04.1) over (1.3.5-2) ...
Preparing to unpack .../sosreport_4.4-1ubuntu0.20.04.1_amd64.deb ...
Unpacking sosreport (4.4-1ubuntu0.20.04.1) over (4.3-1ubuntu0.20.04.2) ...
Setting up libksba8:amd64 (1.3.5-2ubuntu0.20.04.1) ...
Setting up sosreport (4.4-1ubuntu0.20.04.1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...

 ls -lat /var/lib/dpkg
total 1752
drwxr-xr-x 2 root root 4096 Oct 19 07:46 updates
drwxr-xr-x 7 root root 4096 Oct 19 07:46 .
-rw-r--r-- 1 root root 710525 Oct 19 07:46 status
-rw-r----- 1 root root 0 Oct 19 07:46 lock
-rw-r--r-- 1 root root 710658 Oct 19 07:46 status-old
drwxr-xr-x 2 root root 139264 Oct 19 07:46 info
drwxr-xr-x 2 root root 4096 Oct 19 06:20 triggers
drwxr-xr-x 2 root root 4096 Sep 21 06:09 alternatives
drwxr-xr-x 45 root root 4096 May 9 15:10 ..
lrwxrwxrwx 1 root root 13 Jul 12 2021 dpkg -> /var/lib/dpkg
-rw-r----- 1 root root 0 Jul 9 2021 lock-frontend
-rw-r--r-- 1 root root 268 Mar 16 2021 diversions
-rw-r--r-- 1 root root 340 Mar 16 2021 diversions-old
-rw-r--r-- 1 root root 100 Feb 1 2021 statoverride
-rw-r--r-- 1 root root 181566 Feb 1 2021 available
-rw-r--r-- 1 root root 8 Feb 1 2021 cmethopt
drwxr-xr-x 2 root root 4096 Mar 23 2020 parts

unlink /var/lib/dpkg/dpkg
ls -lat /var/lib/dpkg
total 1752
drwxr-xr-x 7 root root 4096 Oct 19 07:49 .
drwxr-xr-x 2 root root 4096 Oct 19 07:46 updates
-rw-r--r-- 1 root root 710525 Oct 19 07:46 status
-rw-r----- 1 root root 0 Oct 19 07:46 lock
-rw-r--r-- 1 root root 710658 Oct 19 07:46 status-old
drwxr-xr-x 2 root root 139264 Oct 19 07:46 info
drwxr-xr-x 2 root root 4096 Oct 19 0...

[Tinker Dong (higginsd)]

Same bug observed on Azure image Ubuntu 20.04, when trying to upgrade 22.04.

Confirmed workaround as Stefan Schaub (s-schaub) mentioned, execute "sudo unlink /var/lib/dpkg/dpkg" before doing the "do-release-upgrade".

[Chengen Du (chengendu)]

@juliank Could you please assist me in sponsoring the SRU for jammy? Thank you.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "dpkg_1.21.1ubuntu2.2_jammy.patch" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Julian Andres Klode (juliank)]

Sponsored, but please edit the bug description to include the relevant information as per

https://wiki.ubuntu.com/StableReleaseUpdates

[Chengen Du (chengendu)]

[Impact]
Some unexpected files exist in the /var/lib/dpkg folder,
which can cause the dpkg postinst script to throw an exception even though the dpkg package has already been upgraded.

[Fix]
Several similar problems have been reported by different users, but the files that caused the error are not always the same.
For further information, please see the following links:

https://www.reddit.com/r/homelab/comments/vf52d4/i_get_the_following_error_when_i_excute_any_apt/
https://stackoverflow.com/questions/72448922/dpkg-fails-on-ubuntu-server-22-04-lts
https://stackoverflow.com/questions/72581436/configuring-dpkg-fails-with-error-package-dpkg-is-not-configured-yet

The error is caused by the postinst step during installation of the new version.
The "head" command is used to check the file content in the debian/dpkg.post fixup_misplaced_alternatives function, which will throw an error if the file is a directory.
Upstream fixes this issue by the commit 940f86c37f9f (debian: Ignore directories for the alternatives state fixup).
The new approach will skip files whose type is not a regular file.

[Test Plan]
1. Create a symbolic link in the /var/lib/dpkg folder
   ln -s /var/lib/dpkg /var/lib/dpkg/dpkg
2. Install the dpkg package using the dpkg command:
   root@jammy:~/dpkg# dpkg -i dpkg_1.21.1ubuntu2.1_amd64.deb
   (Reading database ... 162774 files and directories currently installed.)
   Preparing to unpack dpkg_1.21.1ubuntu2.1_amd64.deb ...
   Unpacking dpkg (1.21.1ubuntu2.1) over (1.21.1ubuntu2.1) ...
   Setting up dpkg (1.21.1ubuntu2.1) ...
   head: error reading 'dpkg': Is a directory
   dpkg: error processing package dpkg (--install):
    installed dpkg package post-installation script subprocess returned error exit status 1
   Processing triggers for man-db (2.10.2-1) ...
   Errors were encountered while processing:
    dpkg

[Where problems could occur]
The intended purpose of the fixup is not to throw an error if the file is a directory but to move the alternative state files back into the alternative state directory.
The fixup has also been removed by the upstream commit 03abb4932e9c (debian: Remove no longer needed alternatives fixup from postint).
Keeping the unrecognized files untouched will be the same approach as upstream currently handles.
The regression can be considered low because there is no behavior change compared to the current upstream version.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted dpkg (1.21.1ubuntu2.2) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu82.4 (amd64, arm64, armhf, ppc64el, s390x)
libassuan/2.5.5-1build1 (amd64)
software-properties/0.99.22.6 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#dpkg

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chengen Du (chengendu)]

I have tested dpkg 1.21.1ubuntu2.2 for jammy and can confirm that the issue no longer exists.

[Andreas Hasenack (ahasenack)]

Before this fix can proceed, the DEP8 failures mentioned in comment #18 need to be addressed. Either fixed (via retries perhaps), or deemed unrelated in which case a statement should be added explaining them, and perhaps even a hint[1] MP.

1. https://code.launchpad.net/~ubuntu-release/britney/+git/hints-ubuntu

[Chengen Du (chengendu)]

The fix only modified the postinst script, which should only affect the installation / upgrade flow.
The regression seems not related to the fix, so I asked @juliank to retry it.
The result was successful, and the new package operates as expected.

[Julian Andres Klode (juliank)]

There is still a regression in apport that appears to be a bug there but we could force-skiptest dpkg/1.21.1ubuntu2.2 to ignore that after showing it fails in release pocket too, preferably without a migration-reference trigger.

Will try to resolve it next week. Optimally we should fix apport of course.

[Julian Andres Klode (juliank)]

The apport fix has been sitting in the unapproved queue for some time.

[Andreas Hasenack (ahasenack)]

apport was accepted, I retriggered the tests that were blocking dpkg, let's see.

[Andreas Hasenack (ahasenack)]

All tests are green now

[Launchpad Janitor (janitor)]

This bug was fixed in the package dpkg - 1.21.1ubuntu2.2

---------------
dpkg (1.21.1ubuntu2.2) jammy; urgency=medium

  * Cherry-pick 940f86c37f9f from dpkg.git (LP: #1960582)
    [ Guillem Jover ]
    - debian: Ignore directories for the alternatives state fixup
      (Closes: #1001695)

 -- Chengen Du <email address hidden> Sat, 01 Apr 2023 21:43:42 +0800

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for dpkg has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.