rsync 3.2.x in Groovy depends on broken libxxhash 0.7.x
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
xxhash (Ubuntu) |
Fix Released
|
Low
|
Unassigned | ||
Groovy |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
**NOTE: This ticket has been moved/altered. Original problem referred to rsync behaviour with libxxhash0. New subject is the changed ABI of libxxhash0.**
Original-Subject: rsync 3.2.x in Groovy depends on broken libxxhash 0.7.x
Actual problem: libxxhash0 in Focal exposes 128-bit xxhash (xxh128) symbols which behave differently on versions of libxxhash0 in Groovy and above.
In xxhash 0.7.x there is this notice:
https:/
> The XXH3 algorithm is still in development.
> The results it produces may still change in future versions.
>
> Results produced by v0.7.x are not comparable with results
> from v0.7.y. However, the API is completely stable, and it
> can safely be used for ephemeral data (local sessions).
>
> Avoid storing values in long-term storage until the
> algorithm is finalized. XXH3's return values will be
> officially finalized upon reaching v0.8.0.
Unfortunately these ended up as usable symbols in xxhash 0.7.x in Debian:
> XXH128@Base 0.7.0
(etc..)
This means that those linking against libxxhash0 would get a binary dependency on 0.7.0. But behaviour of xxh128 differs between 0.7 and 0.8. So if you're linking against xxh128 from 0.8.x, it should binary-depend on >=0.8 and not >=0.7. (And if possible, you should not use xxh128 at all in 0.7.x.)
The following Debian changeset fixes this for newer libxxhash0 builds:
https:/
> debian/
> - XXH128@Base 0.7.0
> + XXH128@Base 0.8.0
But, this is not fully resolved until:
- that Debian change lands in libxxhash0 in Ubuntu;
- packages linking against libxxhash0 (e.g. rsync) are rebuilt against this newer package, so the Depends is bumped to >=0.8;
- xxh128 in libxxhash0 in Focal is clearly marked as ephemeral (or removed altogether; this applies to xxh128 symbols libxxhash0 and an `xxh128sum` binary in the xxhash package)
Cheers,
Walter Doekes
OSSO B.V.
---vvv--- Original ticket below ---vvv---
**Problem**
$ rsync root@focal-
ERROR: .pwd.lock failed verification -- update discarded.
rsync error: some files/attrs were not transferred
(see previous errors)
(code 23) at main.c(1816) [generator=3.2.3]
$ rsync root@focal-
opening connection using: ssh -l root focal-system rsync \
--server --sender \
-e.LsfxCIvu . /etc/.pwd.lock (10 args)
(Client) Protocol versions: remote=31, negotiated=31
Client negotiated checksum: xxh128
...
**Cause**
focal-system# dpkg -l | grep -E 'libxxhash|rsync'
ii libxxhash0:amd64 0.7.3-1 amd64
ii rsync 3.2.3-2ubuntu1 amd64
**Why this affects only us and not more people?**
On Ubuntu/Focal, there is no rsync 3.2.3, only 3.1.3-8. But because we need the lz4 compression support we've fetched a newer rsync (from Groovy).
However: the rsync 3.2.3 depends on libxxhash0 0.7.1+, while in fact it needs 0.8+.
**Details**
On a Ubuntu/Focal system we have installed a rsync 3.2.3 package from Ubuntu/Groovy because we need the lz4 compression support.
focal-system# apt-cache show rsync
Package: rsync
...
Version: 3.2.3-2ubuntu1
Depends: lsb-base, libacl1 (>= 2.2.23), libc6 (>= 2.15),
liblz4-1 (>= 0.0~r130), libpopt0 (>= 1.14), libssl1.1 (>= 1.1.0),
libxxhash0 (>= 0.7.1), libzstd1 (>= 1.3.8), zlib1g (>= 1:1.1.4)
...
Alongside this we had libxxhash0 0.7.3-1 from Focal:
focal-system# apt-cache policy libxxhash0
libxxhash0:
Installed: 0.7.3-1
Candidate: 0.7.3-1
Version table:
*** 0.7.3-1 500
500 http://
100 /var/lib/
According to the dependencies, this should work. But the combination does not, as this quote from the rsync maintainer would tell you:
https:/
> Yeah, Cyan4973 could have told you that the 128-bit xxhash only
> just stabilized in its 0.8.0 release, so anything older than
> that isn't compatible.
**The fix**
As the maintainer points out, version 0.7 is not stable (= broken for our intents and purposes) and thus not fit for use with rsync 3.2.
I would argue that it's a good idea to bump the dependency of rsync 3.2.3 on Groovy to libxxhash0>=0.8
After all, in Groovy there is a libxxhash0 0.8.0-1ubuntu1.
Thanks!
Walter Doekes
OSSO B.V.
(*) possible patch:
$ diff -pu debian/
--- debian/control.orig 2021-07-08 09:56:57.646861644 +0200
+++ debian/control 2021-07-08 09:57:38.499029903 +0200
@@ -8,7 +8,7 @@ Build-Depends: debhelper-compat (= 13),
- libxxhash-dev,
+ libxxhash-dev (>= 0.8),
You misread my comments. It's only the 128-bit hash that depends on 0.8.0. The 0.7 version works fine with rsync, giving it the 64-bit and 32-bit hashes.