"rpm -i ldconfig" removes setup

[Paweł Zuzelski]

On some PLD installations installing some packages removes other ones from rpm database. Poldek reinstalls such packages on each upgrade.

It is the simplest test case:
my environment: th-test/i686, rpm-4.5-13.i686 and rpm-4.5-14.i686
one of PLD users (PawelK) has reported similar bug on ti/x86_64: http://<email address hidden>/msg21777.html

--------- install FHS and setup in empty chroot ---------
# rpm --root=/root/test -ivh FHS-2.3-23.i686.rpm setup-2.6.2-2.i686.rpm
# rpm --root=/root/test -qa
FHS-2.3-23.i686
setup-2.6.2-2.i686
--------- now install ldconfig ---------
# rpm --root=/root/test -i ldconfig-2.9-6.i686.rpm
# rpm --root=/root/test -qa
FHS-2.3-23.i686
ldconfig-2.9-6.i686
# file /root/test/sbin/joinpasswd
/root/test/sbin/joinpasswd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped
--------- so setup is NOT registered by rpm database but it has not been uninstalled---------

I have attached console log with full rpm debug. (note "D: removing "setup" from Name index." line)

some observations:
* It happens with different packages. On my laptop it happens with coreutils - it is reinstalled on each upgrade. PawelK reported the same problem with coreutils and login.
* I'm sure that packages using for test was correct. I have downloaded them directly from ftp.
* I have tested it with rpm-4.5-13 and rpm-4.5-14. PawelK tested id with rpm-4.5-12.
* I tried to create directory, rpm --initdb i that directory, then copy this empty chroot and FHS, setup, ldconfig rpms to another machine. On that machine everything was ok. It means that rpm --initdb works corectly.
* I tried to install FHS and setup on my workstation, then copy chroot and ldconfig.rpm on another machine, and then install ldconfig. Still everything is ok. So something is wrong with rpm -ivh ldconfig on my workstation.

[Elan Ruusamäe (glen666)]

i've done CLEAN installation, and seeing similar lost packages behaviour.

i've also observed that these packages get lost that have %post scriptlet.

[Elan Ruusamäe (glen666)]

here's another install log. after which glibc and FHS are missing

[root@rescue /]# rpm -r /pld -q `grep ':.*#' console.log|awk '{print $1}'|cut -d: -f2 `|grep is.not
package glibc is not installed
package FHS is not installed
[root@rescue /]#

[Elan Ruusamäe (glen666)]

so as FHS and glibc got lost in last test, and FHS does not have any scripts, then the %post is not related to package loss. however the other pks had lots of %post -p /sbin/ldconfig and those were preserved

[Elan Ruusamäe (glen666)]

here's another test where
1. glibc-misc was merged to glibc, except the shell scripts
2. FHS does not depend on setup, but vice versa, so it is installed first
3. libcrypt.so.1 was moved to subpackage to cut out dep loop

[Elan Ruusamäe (glen666)]

more packages changed to have install order as:
FHS, ldconfig, setup, ... glibc

[Patryk Zawadzki (patrys)]

What we dicovered in rpm:

lib/psm.c, rpmpsmStage @ 1643:

  hdrid = rpmteHdrid(psm->te);
  psm->mi = rpmtsInitIterator(ts, RPMTAG_SHA1HEADER, hdrid, 0);

This initiates a search for earlier installation of the very same package. In theory it should only find the exact NEVRA in case where reinstall was required.

lib/psm.c, rpmpsmStage @ 1660:

  while ((psm->oh = rpmdbNextIterator(psm->mi)) != NULL) {
    fi->record = rpmdbGetIteratorOffset(psm->mi);
    psm->oh = NULL;
    break;
  }

This consumes the iterator from above and checks if there was a match. It then takes the first match and remembers it in fi->record.

lib/psm.c, rpmpsmStage @ 2075:

  if (fi->record && !(rpmtsFlags(ts) & RPMTRANS_FLAG_APPLYONLY)) {
    rc = rpmpsmNext(psm, PSM_RPMDB_REMOVE);
    if (rc) break;
  }

This checks if fi->record contains any matches and if so, forces their removal from DB, assuming the header will be readded during the same transaction (if a match was found, we are reinstalling the same package, remember?).

Now what is completely weird is that the search in the first step tends to return completely random results. Like glibc header matching ncurses. This causes random bugs where installing one package will silently remove another package from rpmdb.

At first we though it was some sort of SHA1 substring collision but it seems to happen with completely random data.

glen should be able to give precise versions of db4.7 and rpm4.5 we used for testing.

[Elan Ruusamäe (glen666)]

# rpm -q rpm db4.7
rpm-4.5-18.i686
db4.7-4.7.25-5.i686 (official patches up to 4.7.25.3)

[Elan Ruusamäe (glen666)]

how i have 100% reproducer:

1. boot virtulabox 2.2.0 using this cd image: http://rescuecd.pld-linux.org/download/2009-02-21/x86/RCDx86_297.iso
2. download 5.tar from this bug, unpack it to /
3. make edit reinstall.sh to mkfs proper device whatever you have there available
4. run in / sh -x testcase.sh
5. check.sh should display you list of packages that are lost

[Elan Ruusamäe (glen666)]

please test with rpm-4.5-19, the bug seemed to be related with using "-r / --root" option and one patch has been updated.

in any case if you're able to replicate the original bug (with old package), could you please run the rpm install with chroot, i.e without "-r / --root" option and see if that solves in your case too.

[Paweł Zuzelski (pawelz)]

No, it is not related to '-r' option or at least it also happens without '-r'.

It happens on the base system (i.e. not in chroot) on my notebook (th/i686) and on PawelK's server (ti/x86_64 afair).

Unfortunately I'm unable to reproduce it now on my notebook, but I can give you root access to my notebook if it will help you to debug it.

[Elan Ruusamäe (glen666)]

On Monday 08 June 2009 08:09:03 pawelz wrote:
> No, it is not related to '-r' option or at least it also happens without
> '-r'.
>
> It happens on the base system (i.e. not in chroot) on my notebook
> (th/i686) and on PawelK's server (ti/x86_64 afair).
>
> Unfortunately I'm unable to reproduce it now on my notebook, but I can
> give you root access to my notebook if it will help you to debug it.

if you're not able to reproduce, then i have no use for the account.

for me the applied patch solved problem. tested several times.

--
glen

[Paweł Zuzelski (pawelz)]

 have just upgraded rpm to 4.5-21 on my workstation:

[z@davabel ~]$ rpm -qa | grep rpm
rpm-4.5-21.i686
rpm-debuginfo-4.5-16.i686
rpm-php-pearprov-4.5-21.i686
rpm-build-4.5-21.i686
rpm-lib-4.5-21.i686
rpm-build-macros-1.519-1.noarch
rpm-pythonprov-4.5-21.i686
rpm-debuginfo-4.5-14.i686
rpm-build-tools-4.4.9-17.noarch
rpm-whiteout-1.26-1.noarch
rpm-specdump-0.3-6.i686
python-rpm-4.5-21.i686
rpm-perlprov-4.5-21.i686
rpmorphan-1.4-1.noarch
rpm-base-4.5-21.i686
rpm-utils-4.5-21.i686
rpmlist-1.5-1.noarch
rpm-utils-perl-4.5-21.i686
rpm-devel-4.5-21.i686
rpm-static-4.5-21.i686
rpm-javaprov-4.5-21.i686

and after upgrade it "found" many coreutils packages. Most probably these are packages "lost" during upgrades.

[z@davabel ~]$ rpm -qa | grep coreutils
coreutils-7.1-2.i686
coreutils-7.1-1.i686
coreutils-7.1-2.i686
coreutils-7.1-2.i686
coreutils-7.1-2.i686
coreutils-7.1-2.i686
coreutils-7.4-1.i686
coreutils-7.2-1.i686
coreutils-6.12-3.i686

Does it mean that bug is really fixed?

[Paweł Zuzelski (pawelz)]

I've done some tests. It seems the bug is fixed. IMO we can close this bug.

[Adam Golebiowski (adamg)]

just a note - this change broke chroot installation as report in https://bugs.launchpad.net/pld-linux/+bug/395177