apt-get update to latest libidn2.so.0 causes multiple application failures including apt-update

Bug #1843507 reported by Gregor Munro on 2019-09-10
90
This bug affects 17 people
Affects Status Importance Assigned to Milestone
libidn2 (Ubuntu)
Undecided
Ondřej Surý

Bug Description

I just ran apt-update and apt-upgrade and received the following updates
libidn2-0 libnss-systemd libpam-systemd libsystemd0 libudev1 systemd systemd-sysv udev

this update is causing multiple programs to fail with error messages such as:
"symbol _idn2_punycode_decode version IDN2_0.0.0 not defined in file libidn2.so.0 with link time reference"

Programs experiencing this include apt-get upgrade and running pgsql's pg_dump.

System is running:
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.3 LTS
Release: 18.04
Codename: bionic

The version installed according to apt-cache policy libidn2-0 is:
libidn2-0:
  Installed: 2.2.0-1+ubuntu18.04.1+deb.sury.org+1
  Candidate: 2.2.0-1+ubuntu18.04.1+deb.sury.org+1
  Version table:
 *** 2.2.0-1+ubuntu18.04.1+deb.sury.org+1 500
        500 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status
     2.0.4-1.1build2 500
        500 http://mirrors.digitalocean.com/ubuntu bionic/main amd64 Packages

Upgrade command run was as follows:
sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  libidn2-0 libnss-systemd libpam-systemd libsystemd0 libudev1 systemd systemd-sysv udev
8 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 66.4 kB/4,553 kB of archives.
After this operation, 127 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 libidn2-0 amd64 2.2.0-1+ubuntu18.04.1+deb.sury.org+1 [66.4 kB]
Fetched 66.4 kB in 0s (209 kB/s)
(Reading database ... 133453 files and directories currently installed.)
Preparing to unpack .../libnss-systemd_237-3ubuntu10.29_amd64.deb ...
Unpacking libnss-systemd:amd64 (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Preparing to unpack .../libsystemd0_237-3ubuntu10.29_amd64.deb ...
Unpacking libsystemd0:amd64 (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Setting up libsystemd0:amd64 (237-3ubuntu10.29) ...
(Reading database ... 133453 files and directories currently installed.)
Preparing to unpack .../libpam-systemd_237-3ubuntu10.29_amd64.deb ...
Unpacking libpam-systemd:amd64 (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Preparing to unpack .../systemd_237-3ubuntu10.29_amd64.deb ...
Unpacking systemd (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Preparing to unpack .../udev_237-3ubuntu10.29_amd64.deb ...
Unpacking udev (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Preparing to unpack .../libudev1_237-3ubuntu10.29_amd64.deb ...
Unpacking libudev1:amd64 (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Setting up libudev1:amd64 (237-3ubuntu10.29) ...
Setting up systemd (237-3ubuntu10.29) ...
(Reading database ... 133453 files and directories currently installed.)
Preparing to unpack .../systemd-sysv_237-3ubuntu10.29_amd64.deb ...
Unpacking systemd-sysv (237-3ubuntu10.29) over (237-3ubuntu10.28) ...
Preparing to unpack .../libidn2-0_2.2.0-1+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking libidn2-0:amd64 (2.2.0-1+ubuntu18.04.1+deb.sury.org+1) over (2.0.4-1.1build2) ...
Setting up libidn2-0:amd64 (2.2.0-1+ubuntu18.04.1+deb.sury.org+1) ...
Setting up libnss-systemd:amd64 (237-3ubuntu10.29) ...
Setting up systemd-sysv (237-3ubuntu10.29) ...
Setting up udev (237-3ubuntu10.29) ...
update-initramfs: deferring update (trigger activated)
Setting up libpam-systemd:amd64 (237-3ubuntu10.29) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for dbus (1.12.2-1ubuntu1.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for initramfs-tools (0.130ubuntu3.8) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-62-generic

Marcos BL (marcosbl) wrote :

Can confirm here:

````
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
````

Errors from a lot of apps, even apt-get itself:

````
/usr/lib/apt/methods/http: relocation error: /usr/lib/x86_64-linux-gnu/libgnutls.so.30: symbol _idn2_punycode_decode version IDN2_0.0.0 not defined in file libidn2.so.0 with link time reference
E: Method http has died unexpectedly!
````

Marcos BL (marcosbl) wrote :
Download full text (9.6 KiB)

root@tachyon ~ ツ
# apt update && apt upgrade
Des:1 http://ubuntu.mirrors.ovh.net/ubuntu bionic InRelease [242 kB]
Des:2 http://ubuntu.mirrors.ovh.net/ubuntu bionic-updates InRelease [88,7 kB]
Des:3 http://ubuntu.mirrors.ovh.net/ubuntu bionic-backports InRelease [74,6 kB]
Obj:4 http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic InRelease
Obj:5 http://ppa.launchpad.net/malteworld/ppa/ubuntu bionic InRelease
Des:6 http://ppa.launchpad.net/ondrej/php/ubuntu bionic InRelease [20,8 kB]
Des:7 http://security.ubuntu.com/ubuntu bionic-security InRelease [88,7 kB]
Obj:8 http://ppa.launchpad.net/transmissionbt/ppa/ubuntu bionic InRelease
Obj:9 http://ppa.launchpad.net/webupd8team/java/ubuntu bionic InRelease
Ign:10 http://packages.elastic.co/elasticsearch/2.x/debian stable InRelease
Obj:11 http://packages.elastic.co/elasticsearch/2.x/debian stable Release
Des:12 http://repo.percona.com/percona/apt bionic InRelease [15,8 kB]
Des:13 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 Packages [51,7 kB]
Obj:14 http://nginx.org/packages/mainline/ubuntu bionic InRelease
Des:15 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main Translation-en [24,3 kB]
Des:16 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [503 kB]
Des:18 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [170 kB]
Des:19 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [6.712 B]
Des:20 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [2.872 B]
Des:21 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [604 kB]
Descargados 1.893 kB en 2s (957 kB/s)
Leyendo lista de paquetes... Hecho
Leyendo lista de paquetes... Hecho
Creando árbol de dependencias
Leyendo la información de estado... Hecho
Calculando la actualización... Hecho
Los siguientes paquetes se han retenido:
  linux-image-generic
Se actualizarán los siguientes paquetes:
  libidn2-0 libidn2-0-dev libidn2-dev libnss-systemd libpam-systemd libsystemd0 libtomcat8-java libudev1 systemd systemd-sysv tomcat8 tomcat8-common udev
13 actualizados, 0 nuevos se instalarán, 0 para eliminar y 1 no actualizados.
Se necesita descargar 5.545 kB/9.722 kB de archivos.
Se utilizarán 135 kB de espacio de disco adicional después de esta operación.
¿Desea continuar? [S/n]
Des:1 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 libidn2-dev amd64 2.2.0-1+ubuntu18.04.1+deb.sury.org+1 [68,0 kB]
Des:2 http://ppa.launchpad.net/ondrej/php/...

Read more...

Marcos BL (marcosbl) on 2019-09-10
Changed in libidn2 (Ubuntu):
status: New → Confirmed
Blfrg (i3lfrg) wrote :

I experienced the same issue.

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"

uname -sri
Linux 5.0.0-27-generic x86_64

Machine would no longer boot successfully, primarily due to NetworkManager failing.
These are the steps I took to revert until resolved:
Boot in recovery mode, get to command prompt
dhclient <eth>
wget http://mirrors.kernel.org/ubuntu/pool/main/libi/libidn2/libidn2-0_2.0.4-1.1build2_amd64.deb
dpkg -i libidn2-0_2.0.4-1.1build2_amd64.deb
reboot
---
Note: If you have i386 libs you have to revert them as well:
wget http://mirrors.kernel.org/ubuntu/pool/main/libi/libidn2/libidn2-0_2.0.4-1.1build2_i386.deb
dpkg -i libidn2-0_2.0.4-1.1build2_i386.deb

Konstantin Boyandin (7-det-g) wrote :

Confirming. If one tries rebooting the computer/VM, it may result in total failure, due to NetworkManager failing to start.

Temporary workaround as advised on the Net (works for Ubuntu 18.04.3, x86 64bit):

$ wget http://mirrors.kernel.org/ubuntu/pool/main/libi/libidn2/libidn2-0_2.0.4-1.1build2_amd64.deb
$ sudo dpkg -i libidn2-0_2.0.4-1.1build2_amd64.deb
$ sudo apt-mark hold libidn2-0

Aaron (apelly) wrote :

Disconcerting!

But I note that https://bugs.launchpad.net/ubuntu/+source/libidn2/+bug/1843507/comments/4 solves the issue for me. At least temporarily.

Sudo Ranger (sudoranger) wrote :

This is critical. Now, I can't boot my system due to NetworkManager failing to start.

However, this only affects those with "deb http://ppa.launchpad.net/ondrej/php/ubuntu bionic main"

I did a fresh install without ondrej's PHP-FPM 7.3 repository and I no longer have this problem anymore. Please forward this issue over to https://bugs.launchpad.net/~ondrej

libidn2-0:
  Installed: 2.2.0-1+ubuntu18.04.1+deb.sury.org+1
  Candidate: 2.2.0-1+ubuntu18.04.1+deb.sury.org+1
  Version table:
 *** 2.2.0-1+ubuntu18.04.1+deb.sury.org+1 500
        500 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64

Sebastien Bacher (seb128) wrote :

Thank you for your bug report, the issue seems to be with a ppa and not an Ubuntu version though,
http://ppa.launchpad.net/ondrej/php/ubuntu
I've subscribed Ondřej to the bug now, hopefully he can fix this version or remove it

Ondřej Surý (ondrej) wrote :

Tracked here: https://github.com/oerdnj/deb.sury.org/issues/1247, but it also needs to be fixed in Debian and Ubuntu as it was wrong for upstream to drop public symbols and not bump the SOVERSION. I removed the packages from the PPA and I am working on having a fixed package (with restored symbols) in Debian first and then I'll reupload the fixed package to the PPA.

This is a shout-out to every upstream library authors; Never ever remove public symbols from the ABI even if they are "private" and should not be use...

Georgy Vasilev (georgy-vas) wrote :

We experienced the same issue.

We are also using http://ppa.launchpad.net/ondrej/php/ubuntu

The solution from https://bugs.launchpad.net/ubuntu/+source/libidn2/+bug/1843507/comments/4 solved the issue for now.

My1 (my1) wrote :

I have the same issue here, but on kubuntu, and i am pretty sure i have sury in for php.

I also have someone i know with kubuntu with the same basic symptoms (network down after uodates) a few days ago, but yet still researching details. Keeping you guys updated

Ondřej Surý (ondrej) wrote :

@seb128 make sure the next LTS has a libidn2 version with the symbols restored (I just uploaded 2.2.0-2) or SOVERSION bumped (that's for upstream to decide) or the partial upgrades will suffer from the same fate. The repository has been update

Changed in libidn2 (Ubuntu):
status: Confirmed → Fix Released
assignee: nobody → Ondřej Surý (ondrej)
muzzol (muzzol) wrote :

in addition to #4 I had to purge all versions of libidn2-0 including i386 executing SEVERAL TIMES:

    dpkg --purge --force-all libidn2-0

so my full process was like:

wget http://mirrors.kernel.org/ubuntu/pool/main/libi/libidn2/libidn2-0_2.0.4-1.1build2_amd64.deb

dpkg --purge --force-all libidn2-0 && dpkg --purge --force-all libidn2-0 && dpkg --purge --force-all libidn2-0 && dpkg --purge --force-all libidn2-0

dpkg -i libidn2-0_2.0.4-1.1build2_amd64.deb

apt-mark hold libidn2-0

apt --fix-broken install
apt-get update
apt-get upgrade

Baba NK (hnkumar) wrote :

After comment #11, I just ran apt update && apt upgrade then it working fine with 2.2.0-2 version. Now everything is working fine.

rororo (rororo74) wrote :

Just like you I am on Ubuntu 18.04 LTS using repository ppa:ondrej/php

After Ondřej has released a fix, when I apt-get update or apt update && apt upgrade I am still getting the usual /usr/lib/apt/methods/http: relocation error.

Can someone please tell me step by step in simple terms what to do?
Should I follow the procedure in #4 & #12? or is there something else that must be done now that 2.2.0-2 is uploaded?

Help please.

Fabian Röhl (froehl) wrote :
Sebastien Bacher (seb128) wrote :

@Ondřej, thanks for the status update/details/fix! Also probably a note for the futur to be careful before removing a symbol from the .symbols even if those are _prefixed

Ondřej Surý (ondrej) wrote :

@seb123, yep, I was burnt by this so many times that it wouldn't pass over me. I guess I should also send email to Simon...

Marcos BL (marcosbl) wrote :

Thank you VERY MUCH for the fast response @Ondřej, I can confirm it's fixed by just apt update && apt upgrade

muzzol (muzzol) wrote :

so it is already fixed?

which version is new package?

Konstantin Boyandin (7-det-g) wrote :

The 2.2.0-2 looks working now, thanks for the quick fix.

Gregor Munro (sneeble) wrote :

Confirming 2.2.0-2 looks good. Thanks Ondřej Surý.

This temporary workaround worked for me:

$ wget http://mirrors.kernel.org/ubuntu/pool/main/libi/libidn2/libidn2-0_2.0.4-1.1build2_amd64.deb
$ sudo dpkg -i libidn2-0_2.0.4-1.1build2_amd64.deb
$ sudo apt-mark hold libidn2-0

With that said, I want to avoid problems in the future so at what point should I remove the hold on libidn2-0? I host a few websites so anything that messes up PHP brings down all my websites. Should I downgrade back PHP 7.2 (Ubuntu 18 default) and remove the Ondrej ppa?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.