dpkg-deb error processing libc6_2.27, libzstd.so.1 is missing

[Ubfan]

Several updates of Ubuntu 18.04 worked, but yesterday, the update failed because of some missing file in libc6_2.27-3ubuntu1_amd64.deb and libc6_2.27-3ubuntu1_i386.deb

While updating Ubuntu 18.04, the update failed with a failure to find libzstd.so.1 in the libc6_2.27--3ubuntu1_i386 and libc6_2.27-3ubuntu1_amd64.deb packages. Oddly, another problem (1027076) was reported on askubuntu for a missing libzstd in package htop_2.1.0-3_amd64.deb. The libzstd should be in its own package, according to apt-file search.

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  apt-config-icons gnome-initial-setup libblockdev-crypto2 libdns-export1100 libdns1100 libisc-export169 libisc169 libsysmetrics1
  libunwind8 libvolume-key1 ubuntu-report
The following packages will be upgraded:
  apparmor apport apport-gtk apt apt-utils bind9-host brltty command-not-found command-not-found-data console-setup
  console-setup-linux cpp-7 dmeventd dmsetup dnsutils dpkg-dev eog example-content fonts-arphic-uming g++-7 gcc-7 gcc-7-base
  gcc-8-base gcc-8-base:i386 gir1.2-freedesktop gir1.2-glib-2.0 gir1.2-goa-1.0 gir1.2-gtk-3.0 gir1.2-ibus-1.0 gir1.2-mutter-2
  gir1.2-udisks-2.0 gjs gnome-calendar gnome-control-center gnome-control-center-data gnome-control-center-faces
  gnome-online-accounts gnome-session-bin gnome-session-common gnome-shell gnome-shell-common gnome-software gnome-software-common
  gnome-software-plugin-snap gnome-startup-applications google-chrome-stable gtk-update-icon-cache gvfs gvfs-backends gvfs-bin
  gvfs-common gvfs-daemons gvfs-fuse gvfs-libs hunspell-de-at-frami hunspell-de-ch-frami hunspell-de-de-frami hunspell-en-gb
  hunspell-en-za hunspell-es hunspell-it hunspell-pt-br hunspell-pt-pt hunspell-ru hyphen-de hyphen-en-gb hyphen-fr hyphen-it
  hyphen-pt-br hyphen-pt-pt ibus ibus-gtk ibus-gtk3 isc-dhcp-client isc-dhcp-common keyboard-configuration language-pack-de
  language-pack-de-base language-pack-en language-pack-en-base language-pack-es language-pack-es-base language-pack-fr
  language-pack-fr-base language-pack-gnome-de language-pack-gnome-de-base language-pack-gnome-en language-pack-gnome-en-base
  language-pack-gnome-es language-pack-gnome-es-base language-pack-gnome-fr language-pack-gnome-fr-base language-pack-gnome-it
  language-pack-gnome-it-base language-pack-gnome-pt language-pack-gnome-pt-base language-pack-gnome-ru
  language-pack-gnome-ru-base language-pack-gnome-zh-hans language-pack-gnome-zh-hans-base language-pack-it language-pack-it-base
  language-pack-pt language-pack-pt-base language-pack-ru language-pack-ru-base language-pack-zh-hans language-pack-zh-hans-base
  language-selector-common language-selector-gnome libapparmor1 libapt-inst2.0 libapt-pkg5.0 libasan4 libatomic1 libbind9-160
  libbrlapi0.6 libc-bin libc-dev-bin libc6 libc6:i386 libc6-dbg libc6-dev libc6-i386 libcc1-0 libcilkrts5 libdebian-installer4
  libdevmapper-event1.02.1 libdevmapper1.02.1 libdpkg-perl libgail-3-0 libgcc-7-dev libgcc1 libgcc1:i386 libgirepository-1.0-1
  libgjs0g libgoa-1.0-0b libgoa-1.0-common libgoa-backend-1.0-1 libgomp1 libgtk-3-0 libgtk-3-bin libgtk-3-common libibus-1.0-5
  libirs160 libisccc160 libisccfg160 libitm1 libkeyutils1 liblsan0 liblvm2app2.2 liblvm2cmd2.02 liblwres160 libmbim-glib4
  libmbim-proxy libmpx2 libmutter-2-0 libnautilus-extension1a libnih1 libperl5.26 libplymouth4 libpoppler-glib8 libpoppler73
  libpython-stdlib libpython2.7 libpython2.7-minimal libpython2.7-stdlib libpython3-stdlib libqpdf21 libqt5core5a libqt5dbus5
  libqt5gui5 libqt5network5 libqt5svg5 libqt5widgets5 libquadmath0 libstdc++-7-dev libstdc++6 libtiff5 libtsan0 libubsan0
  libudisks2-0 light-themes localechooser-data locales lvm2 make multiarch-support mutter mutter-common mythes-en-us mythes-fr
  mythes-it mythes-pt-pt mythes-ru nautilus nautilus-data netplan.io networkd-dispatcher nplan orca perl perl-base
  perl-modules-5.26 plymouth plymouth-label plymouth-theme-ubuntu-logo plymouth-theme-ubuntu-text poppler-utils python
  python-apt-common python-minimal python-tk python2.7 python2.7-minimal python3 python3-apport python3-apt python3-brlapi
  python3-commandnotfound python3-distupgrade python3-minimal python3-problem-report python3-software-properties qpdf
  qt5-gtk-platformtheme qttranslations5-l10n snapd software-properties-common software-properties-gtk strace thermald thunderbird
  thunderbird-gnome-support thunderbird-locale-de thunderbird-locale-en thunderbird-locale-en-gb thunderbird-locale-en-us
  thunderbird-locale-es thunderbird-locale-es-ar thunderbird-locale-es-es thunderbird-locale-fr thunderbird-locale-it
  thunderbird-locale-pt thunderbird-locale-pt-br thunderbird-locale-pt-pt thunderbird-locale-ru thunderbird-locale-zh-cn
  thunderbird-locale-zh-hans thunderbird-locale-zh-hant thunderbird-locale-zh-tw ubiquity ubiquity-frontend-gtk
  ubiquity-ubuntu-artwork ubuntu-artwork ubuntu-desktop ubuntu-docs ubuntu-drivers-common ubuntu-minimal ubuntu-mono
  ubuntu-release-upgrader-core ubuntu-release-upgrader-gtk ubuntu-session ubuntu-settings ubuntu-software ubuntu-standard
  ubuntu-web-launchers udisks2 unattended-upgrades update-notifier update-notifier-common x11-common xbrlapi xfsprogs xorg
  xserver-common xserver-xephyr xserver-xorg xserver-xorg-core xserver-xorg-input-all xserver-xorg-
legacy xserver-xorg-video-all
  xwayland
283 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
Need to get 16.5 MB/303 MB of archives.
After this operation, 94.1 MB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc6 amd64 2.27-3ubuntu1 [2,824 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu bionic/main i386 libc6 i386 2.27-3ubuntu1 [2,551 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc6-i386 amd64 2.27-3ubuntu1 [2,651 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc6-dbg amd64 2.27-3ubuntu1 [5,161 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc-dev-bin amd64 2.27-3ubuntu1 [71.8 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc6-dev amd64 2.27-3ubuntu1 [2,587 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libc-bin amd64 2.27-3ubuntu1 [641 kB]
Fetched 16.5 MB in 5s (3,387 kB/s)
Extracting templates from packages: 100%
Preconfiguring packages ...
setting xserver-xorg-legacy/xwrapper/allowed_users from configuration file
dpkg-deb: error while loading shared libraries: libzstd.so.1: cannot open shared object file: No such file or directory
dpkg: error processing archive /var/cache/apt/archives/libc6_2.27-3ubuntu1_i386.deb (--unpack):
 dpkg-deb --control subprocess returned error exit status 127
dpkg-deb: error while loading shared libraries: libzstd.so.1: cannot open shared object file: No such file or directory
dpkg: error processing archive /var/cache/apt/archives/libc6_2.27-3ubuntu1_amd64.deb (--unpack):
 dpkg-deb --control subprocess returned error exit status 127
Errors were encountered while processing:
 /var/cache/apt/archives/libc6_2.27-3ubuntu1_i386.deb
 /var/cache/apt/archives/libc6_2.27-3ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

I deleted the downloaded packages today from the /var/cache/apt/archives, and tried the dist-upgrade again, with the same failure.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: libc6 2.27-0ubuntu3
ProcVersionSignature: Ubuntu 4.15.0-15.16-generic 4.15.15
Uname: Linux 4.15.0-15-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.9-0ubuntu4
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Sat Apr 21 21:00:12 2018
Dependencies:
 gcc-8-base 8-20180414-1ubuntu1
 libc6 2.27-0ubuntu3
 libgcc1 1:8-20180414-1ubuntu1
SourcePackage: glibc
UpgradeStatus: No upgrade log present (probably fresh install)

[Brian Murray (brian-murray)]

Could you include the output of the two following commands?

dpkg -S libzstd.so.1
ls -l /usr/lib/x86_64-linux-gnu/libzstd.so*

Thanks in advance!

[Ubfan (ubfan1)]

===As requested ===
$ dpkg -S libzstd.so.1
libzstd1:amd64: /usr/lib/x86_64-linux-gnu/libzstd.so.1.3.3
libzstd1:amd64: /usr/lib/x86_64-linux-gnu/libzstd.so.1

$ ls -l /usr/lib/x86_64-linux-gnu/libzstd.so*
ls: cannot access '/usr/lib/x86_64-linux-gnu/libzstd.so*': No such file or directory

===Plus some more things that might be of interest===

Yet the package claims to be installed
$ dpkg -l |grep libzstd
ii libzstd1:amd64 1.3.3+dfsg-2ubuntu1 amd64 fast lossless compression algorithm

Try to purge it
$ sudo apt-get purge libzstd1
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 dictionaries-common : Depends: debconf (>= 1.5.5) but it is not going to be installed or
                                debconf-2.0
                       Depends: libtext-iconv-perl but it is not going to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

Same failure after creating empty libzstd.so files with touch (them removed them).

Nor is the libzstd1 package in the /var/cache/apt/archives directory.

Tried to download the libzstd1 package:
$ sudo apt-get download libzstd1
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libzstd1 amd64 1.3.3+dfsg-2ubuntu1 [189 kB]
Fetched 189 kB in 1s (335 kB/s)
W: Download is performed unsandboxed as root as file '/usr/lib/x86_64-linux-gnu/libzstd1_1.3.3+dfsg-2ubuntu1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)

Nothing showed up in /var/cache/apt/archives.

[Julian Andres Klode (juliank)]

Can you please attach the tarball of /var/log/installer?

[Ubfan (ubfan1)]

Note that libzstd is typically the last library in alphabetical order, and this is the third different package reported (see bug 1766046) with this error.

What exact log are you requesting?

[Ubfan (ubfan1)]

Here's the fail in /var/log/apt/term.log

Log ended: 2018-04-15 15:56:00 (ok this far)

Log started: 2018-04-20 16:51:35
(Reading database ... 95%
(Reading database ... 187309 files and directories currently installed.)
Preparing to unpack .../dpkg_1.19.0.5ubuntu2_amd64.deb ...
Unpacking dpkg (1.19.0.5ubuntu2) over (1.19.0.5ubuntu1) ...
Setting up dpkg (1.19.0.5ubuntu2) ...
dpkg-deb: error while loading shared libraries: libzstd.so.1: cannot open shared object file: No such file or directory
dpkg: error processing archive /var/cache/apt/archives/libc6_2.27-3ubuntu1_i386.deb (--unpack):
 dpkg-deb --control subprocess returned error exit status 127
Log ended: 2018-04-20 16:51:37

[Julian Andres Klode (juliank)]

Nobody cares about that, though. We would like to know what the log files in /var/log/installer/ say, especially the syslog file in there. And you don't have to post that twice, just in this bug or the other would be enough.

[prithviraj sukale (pvsukale)]

[SOLVED]

I had similar problem as the bug report with similar outputs for requested commands. I got around by manually copying libzstd.so.1 libzstd.so.1.3.3 to libraries folder and it WORKED :)

Steps involved:

1 . Download original .deb from

 http://mirrors.kernel.org/ubuntu/pool/main/libz/libzstd/libzstd1_1.3.3+dfsg-2ubuntu1_amd64.deb

2. Extract it downloads.

3. cd into the extracted folder libzstd1_1.3.3+dfsg-2ubuntu1_amd64

4. extract "data" folder

5. cd into Downloads

6. Run following commands

sudo cp libzstd1_1.3.3+dfsg-2ubuntu1_amd64/data/usr/lib/x86_64-linux-gnu/libzstd.so.1 /usr/lib/x86_64-linux-gnu/

 sudo cp libzstd1_1.3.3+dfsg-2ubuntu1_amd64/data/usr/lib/x86_64-linux-gnu/libzstd.so.1.3.3 /usr/lib/x86_64-linux-gnu/

And dpkg works fine now

[Julian Andres Klode (juliank)]

No, as mentioned before in the main bug, please reinstall from scratch. This issue is the result of the installer crashing, in all cases we've seen, when trying to install the bootloader, because an ESP partition is missing. Therefore, the system is in a vastly inconsistent state and needs to be reinstalled cleanly.