[UBUNTU 22.04] createrepo_c not working under 22.04.3

[bugproxy]

=== Provided by Thorsten 2024-01-16 ==========================================
We are running Ubuntu 22.04.3 on IBM Z.
createrepo_c is not working and responds:
createrepo_c: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
I seems that it is linked against openssl-1.1, but jammy provide openssl-3.0.2.

Here is the output of ldd:
ldd /usr/bin/createrepo_c
 linux-vdso64.so.1 (0x000003ff85afe000)
 libcreaterepo_c.so.0 => /usr/local/lib/libcreaterepo_c.so.0 (0x000003ff85900000)
 libglib-2.0.so.0 => /lib/s390x-linux-gnu/libglib-2.0.so.0 (0x000003ff85780000)
 libmodulemd.so.2 => /usr/local/lib/libmodulemd.so.2 (0x000003ff85700000)
 libgobject-2.0.so.0 => /lib/s390x-linux-gnu/libgobject-2.0.so.0 (0x000003ff85680000)
 libc.so.6 => /lib/s390x-linux-gnu/libc.so.6 (0x000003ff85480000)
 libbz2.so.1.0 => /lib/s390x-linux-gnu/libbz2.so.1.0 (0x000003ff85400000)
 libcurl-gnutls.so.4 => /lib/s390x-linux-gnu/libcurl-gnutls.so.4 (0x000003ff85300000)
 libmagic.so.1 => /lib/s390x-linux-gnu/libmagic.so.1 (0x000003ff85280000)
 libxml2.so.2 => /lib/s390x-linux-gnu/libxml2.so.2 (0x000003ff85080000)
 liblzma.so.5 => /lib/s390x-linux-gnu/liblzma.so.5 (0x000003ff85000000)
 libcrypto.so.1.1 => not found
 librpm.so.8 => not found
 librpmio.so.8 => not found
 libsqlite3.so.0 => /lib/s390x-linux-gnu/libsqlite3.so.0 (0x000003ff84e80000)
 libz.so.1 => /lib/s390x-linux-gnu/libz.so.1 (0x000003ff84e00000)
 libzck.so.1 => /usr/local/lib/libzck.so.1 (0x000003ff84d80000)
 libpcre.so.3 => /lib/s390x-linux-gnu/libpcre.so.3 (0x000003ff84d00000)
 libm.so.6 => /lib/s390x-linux-gnu/libm.so.6 (0x000003ff84c00000)
 librpmio.so.8 => not found
 libyaml-0.so.2 => /lib/s390x-linux-gnu/libyaml-0.so.2 (0x000003ff84b80000)
 libffi.so.8 => /lib/s390x-linux-gnu/libffi.so.8 (0x000003ff84b00000)
 /lib/ld64.so.1 (0x000003ff85a80000)
 libnghttp2.so.14 => /lib/s390x-linux-gnu/libnghttp2.so.14 (0x000003ff84a80000)
 libidn2.so.0 => /lib/s390x-linux-gnu/libidn2.so.0 (0x000003ff84a00000)
 librtmp.so.1 => /lib/s390x-linux-gnu/librtmp.so.1 (0x000003ff84980000)
 libssh.so.4 => /lib/s390x-linux-gnu/libssh.so.4 (0x000003ff84900000)
 libpsl.so.5 => /lib/s390x-linux-gnu/libpsl.so.5 (0x000003ff84880000)
 libnettle.so.8 => /lib/s390x-linux-gnu/libnettle.so.8 (0x000003ff84800000)
 libgnutls.so.30 => /lib/s390x-linux-gnu/libgnutls.so.30 (0x000003ff84600000)
 libgssapi_krb5.so.2 => /lib/s390x-linux-gnu/libgssapi_krb5.so.2 (0x000003ff84580000)
 libldap-2.5.so.0 => /lib/s390x-linux-gnu/libldap-2.5.so.0 (0x000003ff84500000)
 liblber-2.5.so.0 => /lib/s390x-linux-gnu/liblber-2.5.so.0 (0x000003ff84480000)
 libzstd.so.1 => /lib/s390x-linux-gnu/libzstd.so.1 (0x000003ff84380000)
 libbrotlidec.so.1 => /lib/s390x-linux-gnu/libbrotlidec.so.1 (0x000003ff84300000)
 libicuuc.so.70 => /lib/s390x-linux-gnu/libicuuc.so.70 (0x000003ff84080000)
 libcrypto.so.1.1 => not found
 libunistring.so.2 => /lib/s390x-linux-gnu/libunistring.so.2 (0x000003ff83e80000)
 libhogweed.so.6 => /lib/s390x-linux-gnu/libhogweed.so.6 (0x000003ff83e00000)
 libgmp.so.10 => /lib/s390x-linux-gnu/libgmp.so.10 (0x000003ff83d00000)
 libcrypto.so.3 => /lib/s390x-linux-gnu/libcrypto.so.3 (0x000003ff83900000)
 libp11-kit.so.0 => /lib/s390x-linux-gnu/libp11-kit.so.0 (0x000003ff83780000)
 libtasn1.so.6 => /lib/s390x-linux-gnu/libtasn1.so.6 (0x000003ff83700000)
 libkrb5.so.3 => /lib/s390x-linux-gnu/libkrb5.so.3 (0x000003ff83600000)
 libk5crypto.so.3 => /lib/s390x-linux-gnu/libk5crypto.so.3 (0x000003ff83580000)
 libcom_err.so.2 => /lib/s390x-linux-gnu/libcom_err.so.2 (0x000003ff83500000)
 libkrb5support.so.0 => /lib/s390x-linux-gnu/libkrb5support.so.0 (0x000003ff83480000)
 libsasl2.so.2 => /lib/s390x-linux-gnu/libsasl2.so.2 (0x000003ff83400000)
 libbrotlicommon.so.1 => /lib/s390x-linux-gnu/libbrotlicommon.so.1 (0x000003ff83380000)
 libicudata.so.70 => /lib/s390x-linux-gnu/libicudata.so.70 (0x000003ff81700000)
 libstdc++.so.6 => /lib/s390x-linux-gnu/libstdc++.so.6 (0x000003ff81480000)
 libgcc_s.so.1 => /lib/s390x-linux-gnu/libgcc_s.so.1 (0x000003ff81400000)
 libkeyutils.so.1 => /lib/s390x-linux-gnu/libkeyutils.so.1 (0x000003ff81380000)
 libresolv.so.2 => /lib/s390x-linux-gnu/libresolv.so.2 (0x000003ff81300000)

[Frank Heimes (fheimes)]

Hi Thorsten, thanks for reporting this.

But a quick test on one of my systems doesn't let me reproduce this:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

$ ldd /usr/bin/createrepo_c | grep libcrypto
 libcrypto.so.3 => /lib/s390x-linux-gnu/libcrypto.so.3 (0x000003ffa2000000)

or the complete list (btw. without any 'not found'):

$ ldd /usr/bin/createrepo_c
 linux-vdso64.so.1 (0x000003ff8937e000)
 libcreaterepo_c.so.0 => /lib/s390x-linux-gnu/libcreaterepo_c.so.0 (0x000003ff89180000)
 libglib-2.0.so.0 => /lib/s390x-linux-gnu/libglib-2.0.so.0 (0x000003ff89000000)
 libmodulemd.so.2 => /lib/s390x-linux-gnu/libmodulemd.so.2 (0x000003ff88f80000)
 libgobject-2.0.so.0 => /lib/s390x-linux-gnu/libgobject-2.0.so.0 (0x000003ff88f00000)
 libc.so.6 => /lib/s390x-linux-gnu/libc.so.6 (0x000003ff88d00000)
 libbz2.so.1.0 => /lib/s390x-linux-gnu/libbz2.so.1.0 (0x000003ff88c80000)
 libcurl.so.4 => /lib/s390x-linux-gnu/libcurl.so.4 (0x000003ff88b80000)
 libmagic.so.1 => /lib/s390x-linux-gnu/libmagic.so.1 (0x000003ff88b00000)
 libxml2.so.2 => /lib/s390x-linux-gnu/libxml2.so.2 (0x000003ff88900000)
 liblzma.so.5 => /lib/s390x-linux-gnu/liblzma.so.5 (0x000003ff88880000)
 libcrypto.so.3 => /lib/s390x-linux-gnu/libcrypto.so.3 (0x000003ff88480000)
 librpmio.so.9 => /lib/s390x-linux-gnu/librpmio.so.9 (0x000003ff88400000)
 librpm.so.9 => /lib/s390x-linux-gnu/librpm.so.9 (0x000003ff88380000)
 libsqlite3.so.0 => /lib/s390x-linux-gnu/libsqlite3.so.0 (0x000003ff88200000)
 libz.so.1 => /lib/s390x-linux-gnu/libz.so.1 (0x000003ff88180000)
 libzck.so.1 => /lib/s390x-linux-gnu/libzck.so.1 (0x000003ff88100000)
 libdrpm.so.0 => /lib/s390x-linux-gnu/libdrpm.so.0 (0x000003ff88080000)
 libpcre.so.3 => /lib/s390x-linux-gnu/libpcre.so.3 (0x000003ff88000000)
 libm.so.6 => /lib/s390x-linux-gnu/libm.so.6 (0x000003ff87f00000)
 libyaml-0.so.2 => /lib/s390x-linux-gnu/libyaml-0.so.2 (0x000003ff87e80000)
 libffi.so.8 => /lib/s390x-linux-gnu/libffi.so.8 (0x000003ff87e00000)
 /lib/ld64.so.1 (0x000003ff89300000)
 libnghttp2.so.14 => /lib/s390x-linux-gnu/libnghttp2.so.14 (0x000003ff87d80000)
 libidn2.so.0 => /lib/s390x-linux-gnu/libidn2.so.0 (0x000003ff87d00000)
 librtmp.so.1 => /lib/s390x-linux-gnu/librtmp.so.1 (0x000003ff87c80000)
 libssh.so.4 => /lib/s390x-linux-gnu/libssh.so.4 (0x000003ff87c00000)
 libpsl.so.5 => /lib/s390x-linux-gnu/libpsl.so.5 (0x000003ff87b80000)
 libssl.so.3 => /lib/s390x-linux-gnu/libssl.so.3 (0x000003ff87a80000)
 libgssapi_krb5.so.2 => /lib/s390x-linux-gnu/libgssapi_krb5.so.2 (0x000003ff87a00000)
 libldap-2.5.so.0 => /lib/s390x-linux-gnu/libldap-2.5.so.0 (0x000003ff87980000)
 liblber-2.5.so.0 => /lib/s390x-linux-gnu/liblber-2.5.so.0 (0x000003ff87900000)
 libzstd.so.1 => /lib/s390x-linux-gnu/libzstd.so.1 (0x000003ff87800000)
 libbrotlidec.so.1 => /lib/s390x-linux-gnu/libbrotlidec.so.1 (0x000003ff87780000)
 libicuuc.so.70 => /lib/s390x-linux-gnu/libicuuc.so.70 (0x000003ff87500000)
 libgcrypt.so.20 => /lib/s390x-linux-gnu/libgcrypt.so.20 (0x000003ff87400000)
 libpopt.so.0 => /lib/s390x-linux-gnu/libpopt.so.0 (0x000003ff87380000)
 liblua5.3.so.0 => /lib/s390x-linux-gnu/liblua5.3.so.0 (0x00...

[Sudip Mukherjee (sudipmuk)]

The ldd print is showing:
libcreaterepo_c.so.0 => /usr/local/lib/libcreaterepo_c.so.0 (0x000003ff85900000)

So, its trying to use a locally installed "libcreaterepo_c.so" which is not something Ubuntu has provided.
imho, you should verify why you have locally installed libraries and then use libraries shipped by Ubuntu.

[Frank Heimes (fheimes)]

Thanks Sudip,
that reminds me that it can also be of interest to know more details about the installed package:

$ apt-cache policy createrepo-c
createrepo-c:
  Installed: 0.17.0-1build2
  Candidate: 0.17.0-1build2
  Version table:
 *** 0.17.0-1build2 500
        500 http://ports.ubuntu.com/ubuntu-ports jammy/universe s390x Packages
        100 /var/lib/dpkg/status

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2024-01-17 10:46 EDT-------
(In reply to comment #8)
Hello Frank,

thanks for your help so far.
Our system was upgrade from Ubuntu 20.04.5
On our side we also ran the apt-cache policy command and received the following output:
$ apt-cache policy createrepo-c
createrepo-c:
Installed: 0.17.0-1build2
Candidate: 0.17.0-1build2
Version table:
*** 0.17.0-1build2 500
500 http://de.ports.ubuntu.com/ubuntu-ports jammy/universe s390x Packages
100 /var/lib/dpkg/status

Hopefully we can shed some more light into this by comparing the
# sha256sum /usr/bin/createrepo_c
723d924572a1c8e508fe329a2a9a6602ccced0df38ce8edd72ccdd9afe69bbe1 /usr/bin/createrepo_c

since my current perception is that we receive a different package via apt install than the desired one with openssl-3 support.

Thanks,
Christian

[Sudip Mukherjee (sudipmuk)]

Hi Christian,

The library I mentioned as local built in "/usr/local/lib/libcreaterepo_c.so.0" should have been provided by libcreaterepo-c0 as "/usr/lib/s390x-linux-gnu/libcreaterepo_c.so.0.17.0".

Can you please check what is the apt-cache policy libcreaterepo-c0

[Frank Heimes (fheimes)]

One can usually figure out the package that shipped a certain file by:
dpkg -S <full path to file>

(
I btw. have the same hash for the binary: /usr/bin/createrepo_c
$ sha256sum /usr/bin/createrepo_c
723d924572a1c8e508fe329a2a9a6602ccced0df38ce8edd72ccdd9afe69bbe1 /usr/bin/createrepo_c

And btw. looks like 20.04 didn't had a 'createrepo-c' package:
$ rmadison createrepo-c
 createrepo-c | 0.17.0-1build2 | jammy/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 createrepo-c | 0.17.3-2 | lunar/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 createrepo-c | 0.17.3-2 | mantic/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
 createrepo-c | 0.17.3-2 | noble/universe | source, amd64, arm64, armhf, ppc64el, riscv64, s390x
)

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2024-01-18 02:53 EDT-------
(In reply to comment #7)
> The ldd print is showing:
> libcreaterepo_c.so.0 => /usr/local/lib/libcreaterepo_c.so.0
> (0x000003ff85900000)
>
> So, its trying to use a locally installed "libcreaterepo_c.so" which is not
> something Ubuntu has provided.
> imho, you should verify why you have locally installed libraries and then
> use libraries shipped by Ubuntu.

Hi Sudip,

your approach pointed into the correct direction. We found and fixed it.
It was not /usr/local/lib/createrepo_c.so.0, but these two libs:
/usr/local/lib/libmodulemd.so.2.13.0
/usr/local/lib/libzck.so.1
which overruled the ones in /lib/s390x-linux-gnu - and had dependencies to old packages like librpmio.so.8 and libcrypto.so.1.1 - and that caused the problem.
I deleted all 'ancient' libs in /usr/local/lib - and now it works.

Thanks for your help, sorry for the noise, I'm closing this bugzilla silently.

[Frank Heimes (fheimes)]

Glad to read that it's solved.
Closing this LP bug as well.