rpm: tilde version support à la Debian

[Jeff Johnson]

Tracker

[Jan Engelhardt (jengelh)]

Created an attachment (id=265756)
tilde support for rpm version comparison

[Mls (mls)]

(The patch doesn't make ~rc1 smaller then -foo, but it's easy to fix that.)

The last time this was suggested (Jan 2006) Jeff turned it down pretty harshly.
Nowadays with the rpm split there might be a chance to get it into rpm4. I just need to convince Panu, who seems to be a bit more open to such changes than Jeff.

(Btw, Jeff's argument was that there are too many version compare implementations out there (perl, python, ruby, package managers,...) that need to be changed as well. He sure has a point, but I thing we should do it nevertheless.)

[Jan Engelhardt (jengelh)]

>the patch doesn't make ~rc1 smaller then -foo

The dash may not be used in the version number, so this seems like a non-issue to me.

$ rpmbuild -bb foo.spec
error: line 6: Illegal char '-' in version: Version: 2.6.29-mm1
$ cat foo.spec
Name: foo
Version: 2.6.29-mm1
Release: 0
[...]

[Jan Engelhardt (jengelh)]

re perl, python, ruby -- should not those package managers use rpm for any version and dependency resolution? Like, the "smart" package manager used rpm-python, and if Everything Was Done Right™ there would only be one version comparison algorithm, would not it?

[Mls (mls)]

#3: try '.' instead of '-'.
#4: I wouldn't bet on "everything done right". Some implementations probably don't want to depend on some external module.

[Jan Engelhardt (jengelh)]

Created an attachment (id=266064)
updated patch

[Kkaempf (kkaempf)]

This needs coordination with rpm upstream. So I'd rather have this outside of bugzilla in a place where rpm maintainers can contribute.
( features.opensuse.org ? a wiki page ? )

[Pavol Rusnak (prusnak)]

For the record, this is ticket#56 in upstream tracker : http://rpm.org/ticket/56

[Richard-purdie (richard-purdie)]

It would be desirable to support the ~ notation in version strings for prerelease versions, e.g. 2.6.34~rc1 instead of 2.6.33+2.6.34-rc1.

To make this change, bitbake and opkg need to both support the syntax in their verson comparision functions, we need to upgrade the dpkg/apt tools to a version which supports it and check how rpm handles it.

[Jeff Johnson (n3npq)]

Tracker

[Jeff Johnson (n3npq)]

RFE tracked at https://blueprints.launchpad.net/rpm/+spec/rpm.org-tilde-in-version
bug tracked at https://bugs.launchpad.net/rpm/+bug/633739

[Jan Engelhardt (jengelh)]

You need to enlighten me what launchpad has to do with this.

[Kkaempf (kkaempf)]

(In reply to comment #10)
> You need to enlighten me what launchpad has to do with this.

Comment #9 is about the outside exposure and coordination I asked for in comment #7

Raising priority to get this resolved (either pro or con) for 11.4/Factory

[devzero2000 (pinto-elia)]

Just for information, being somehow linked to this RFE, i think.

There was already been proposed in 2006 at least, if not before, as was useful to standardize the EVR between RPM and DEB.

https://lists.dulug.duke.edu/pipermail/rpm-devel/2006-August/001491.html
"Using dpkg's EVR comparison instead of rpmvercmp"

[Jeff Johnson (n3npq)]

Tracked at https://bugs.launchpad.net/rpm/+bug/633742

[Mark-hatle (mark-hatle)]

To understand how this may or may not workin the scope of RPM. Can you give some examples on how the comparison logic should work with tilde within the version?

[Richard-purdie (richard-purdie)]

For Mark's benefit, the following is an incremental list of versions illustrating what tilde means:

2.6.24~rc1
2.6.24~rc2
2.6.24
2.6.24.1~rc1
2.6.24.1
2.6.25~rc1
2.6.25~rc2
2.6.25
2.6.25.1

Marking as post 1.1 as this isn't likely to happen in the 1.1 timeframe at this point.

[Jeff Johnson (n3npq)]

The sequence you have given is insufficient to determine
what the collation sequence SHOULD be even if tilde-in-version
is included.

rpmvercmp does segmented (i.e. alphas/digits/other are split into classes) comparisons

The comparison issues start to creep-in with (arbitrarily defined) mixed-mode comparisons.
I.e. How should digits be compared to alphas or punctuation when found in
a comparison sequence?

Your sequence is too predictable to capture the problematic mixed-mode
comparison issues. There are historical issues that prevent ASCII
or LC_COLLATE being simply re-used.

The intent of tilde-in-version is very well known: how to implement
mixed-mode comparison in rpm is what isn't obvious.

[Richard-purdie (richard-purdie)]

The general rule is that for version comparison, we tend to follow the debian scheme (http://www.debian.org/doc/debian-policy/ch-controlfields.html). I don't know the rpm version comparison code well enough to know how this will translate there.

What examples would you want to see added to the sequence to 100% clarify the situation?

[Jeff Johnson (n3npq)]

You can plug whatever versioning scheme you
wish, with whatever collate sequence you desire.

But the mixed mode comparison pathologies
need to be pinned down precisely in explicit
test cases before anything can be "supported".

Saying "Just like Debian!" isn't sufficient either,
even though the Debian comparison (or at least my
rewrite of dpkgcmp code, none has ever bothered
to confirm correctness) has been in RPM for years.

The hysterical issues in rpmvercmp are largely
    1) comparison between, say, digit <-> punctualtion is surprising
    the actual comparison is with "" (or missing or empty) and
    the (arbitrarily defined) comparison result isn't what people
    expect (which is ASCII or float based)

    2) digit padding means that 1.0000000002 > 1.1

    3) strcmp (as in glibc) means that case isn't distinguished.

    4) (mandriva and disttag related) many distros wish a "branding"
    identifier like
            foo-1.2-3mdv2011
    Sadly the distro "branding" bigots also wish the explicitly "branded"
    version to be "newer" than the unbranded version, which breaks
    the natural ASCII (or LC_COLLATE but lets ignore encoding issues) collation

   5) the '-' character is forbidden in version strings. and ':' is almost forbidden
   for similar reasons: rpm MUST be able to split "E:V-R" strings like
        Requires: foo = E:V-R
   into a tuple of {E,V-R} on known boundaries (both '-' and ':' are boundaries)

There's also the additional twist(s) of twiddle-in-epoch and twiddle-in-release
that twiddle-in-version will surely lead to RFE's for.

So tests cases to handle missed-class comparisons, and with missing values,
and '~' in other locations, possibly multiple, with regression tests are needed
to be acceptable for inclusion in RPM itself.

Again, all the RPM itself needs is a boolean return from an opaque comparison.
But "RPM sux! because it doesn't do what Debian does!" (and interoperability
in general) isn't enhanced by opaque pluggable comparisons.

[Jeff Johnson (n3npq)]

I would point out that RPM already permits a trailing *
character in {V,R} strings which will match the remainder of the string
(by exiting immediately with the current comparison result)
is already implemented.

In some ways the trailing '*' character accomplishes what tilde-in-version
accomplishes with rather simpler semantics. Just not what is in Debian ...

[Richard-purdie (richard-purdie)]

I haven't said RPM sux. We have the situation that OE started with opkg, added dpkg, then rpm support. opkg has a defined approach to version numbers which is to follow debian unless it doesn't make sense for embedded use.

We can therefore choose to follow the debian approach, or, if I hear good arguments otherwise, we could patch dpkg and do something different. I'd prefer not to but its possible.

The one thing I want to see is that bitbake, opkg, dpkg and rpm have some kind of similar ideas about how to use the versions, at least in the environment we're using. I've seen what happens when they disagree and we're not doing it again ;-).

In summary I agree we need a good set of definitions and a good test set and those need to be created as part of this work. I worry if we're waiting on rpm to write those definitions, we may have a long wait as I see links with discussions back five years ago and little movement :(.

[Richard-purdie (richard-purdie)]

If * can be made to work and we can come up with some stable translation, that is perfectly fine. We already allow "-" in V but have translations for rpm to deal with that...

[Jeff Johnson (n3npq)]

The lack of "forward"motion isn't from me.

SHow me test cases or set a goal and you will have
an implementation within a month.

The general implementation is harder because of
rampant "RPM sux because ..." comments (not from you).

Again, you can implement whatever version comparison
you wish to see in RPM by plugging in a (*rpmvercmp)
vector. There's a similar vector for the higher level triple
of {E,V,R} that can be plugged as well.

And there's a *RE pattern that is used to split on boundaries
that can be re-written to do whatever you chosse.

Meanwhile RPM is expected to be compatible with itself: that
isn't your opkg/bitbake/ipkg/dpkg/... usage case.

And its not RPM, but rather tools that don't use rpm lib interfaces
correctly, that is going to break. I've seen 5-6 tool bugs over
the last few months with Mandriva adding Distepoch/Disttag
(which the *RE parser handles perfectly, including ignoring
if/when the additionl info isn't needed/useful). So tools like
zypper and/or smart and/or yum are what is going to break
because not correctly using rpm lib comparison functions.

Happy to help with whatever you desire in Poky: its just a collation sequence *shrug*.

[Jeff Johnson (n3npq)]

Instead of transforms, there's already an excluded character
list in version comparison. All excluded characters are treated
equivalently (originally, and still in @rpm.org code, all "other"
punctuation characters are treated equivalently: so
    1.2_3 = 1.2|3

See rpmdb/rpmevr.c (where the comparison routines live)
for two quirks:

/* XXX Force digits to beat alphas. See bugzilla #50977. */
/*@unchecked@*/
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPMVERCMP_DIGITS_BEAT_ALPHA) /* old-comparision-behaviour */
static int _invert_digits_alphas_comparison = -1;
#else
static int _invert_digits_alphas_comparison = 1;
#endif

/* XXX Punctuation characters that are not treated as alphas */
/*@unchecked@*/ /*@observer@*/
static const char * _rpmnotalpha = ".:-";

Unless you need to handle "branding" in release with the expectation of "newer",
don't do what Mandriva is doing.

And add any characters you wish to the excluded character _rpmnotalpha list.
The far harder issue (which likely necessitates awkward/complex transforms always)
is tools like zipper not using rpmlib API calls for version comparison.

[Jeff Johnson (n3npq)]

One last related note is the EVRD parsing. This is the split
into a N-tuple (currently up to 4) and twiddle-in-version is
adding another element to the tuple with speshul comparison semantics.

The *RE parsing is quite general and can be used to split dependency
strings on newer boundaries, and can drop fields as well as permute
the precedence of the tuple comparisons, each of which is the (*rpmvercmp)
segmented string comparison applied sequentially to strings in the tuple.

Here is the configuration (from /usr/lib/rpm/macros) with the explicit
usage cases that match:

#==============================================================================
# ---- EVRD... dependency tuple comparison macros.
#

# STEP 1: Match the string and capture regex parts
# 1 2 3 4
# X ":" X "-"X ":"X
%evr_tuple_match ^(?:([^:-]+):)?([^:-]+)(?:-([^:-]+))?(?::([^:-]+))?$

# STEP 2: Assemble <E,V,R,D> tuple from regex parts
# <E,V,R,D> case 1: case 2: case 3: case 4: case 5: case 6:
# X:X-X:X X:X-X X:X X: X-X X
# %evr_tuple_select 1234 123_ 12__ 1___ _23_ _2__
%evr_tuple_select 123_

# STEP 3: Configure the comparison order of the <E,V,R,D> tuple elements
%evr_tuple_order EVR

SO what is in RPM is quite general even if there isn't the conventionalized
and tightly administratively controlled narros usage of twiddle-in-version
as in Debian. I'm sure that whatever Poky wishes to do with RPM version
comparison can be accommodated.

[Michael Shigorin (shigorin)]

> Sadly the distro "branding" bigots also wish the explicitly "branded"
> version to be "newer" than the unbranded version, which breaks
> the natural ASCII (or LC_COLLATE but lets ignore encoding issues) collation
Well in ALT Linux this is a non-issue due to some sanity applied... we have 1.2.3-alt4 and not 1.2.3-4alt.

[Jan Engelhardt (jengelh)]

This has entered rpm. Now zypper needs to understand this as well.

http://rpm.org/gitweb?p=rpm.git;a=commitdiff;h=db28221a4a48f6ee3c804c92314330637c808638

[Mls-i (mls-i)]

Already done on Friday:
https://github.com/openSUSE/libsolv/commit/dda287fc18111054553c6dfd3a0cd7bafe5e6cb9