rsync 3.1.2-2.1ubuntu1.6 breaks compatibility with unison 2.48.4-1ubuntu1 on Bionic

Bug #2009706 reported by Thomas S
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
rsync (Ubuntu)
New
Undecided
Unassigned
unison (Ubuntu)
New
Undecided
Unassigned

Bug Description

Rsync 3.1.2-2.1ubuntu1.6, when used by Unison using the copyprog option in Unison, produces an error message and fails to sync files. Rsync 3.1.2-2.1ubuntu1 succeeds in syncing files. I believe Rsync 3.1.2-2.1ubuntu1.5 succeeded but I no longer have the .deb file to test with.

Steps to reproduce:

Create a unison profile similar to the one given below, to sync a folder between a local machine and a remote ssh server. Both client and server need the same Ubuntu and Unison versions, and I used unpassphrased SSH keys for authentication. Place a test file in the folder to be synced, then run 'unison profilename'.

Note that I have redacted domains in the below.

Unison profile:

label = Test Profile
root = /root/unison-test
root = ssh://fs2b.our.domain.org.uk//root/unison-test

# run repeatedly and fully automatically
auto = true
batch = true
copyonconflict = true
#repeat = 60

# the copy program must be manually specified in order to sync ACLs
copythreshold=0
copyprog = rsync -a -A -X --rsh=ssh --inplace --compress
copyprogrest = rsync -a -A -X --rsh=ssh --partial --inplace --compress

# general settings
group = true
owner = true
#path = sharedfolder
#path = users
perms = -1
sortbysize = true
times = true

Unison output with rsync 3.1.2-2.1ubuntu1.6
root@fs72a:~# unison fs2b_unisontest -debug all
[startup] Preferences:
ui = graphic
host =
server = false
prefsdocs = false
doc =
version = false
silent = false
dumbtty = false
testserver = false
rest = fs2b_unisontest
showprev = false
selftest = false
confirmmerge = false
retry = 0
repeat =
contactquietly = false
key =
label = Syncs P and S drives between fs72a and fs2a
expert = false
height = 15
auto = true
maxthreads = 0
maxsizethreshold = -1
prefer =
force =
sortnewfirst = false
sortbysize = true
keeptempfilesaftermerge = false
diff = diff -u CURRENT2 CURRENT1
copyonconflict = true
backupdir =
maxbackups = 2
backups = false
backupsuffix =
backupprefix = .bak.$VERSION.
backuploc = central
copymax = 1
copyquoterem = default
copythreshold = 0
copyprogrest = rsync -a -A -X --rsh=ssh --partial --inplace --compress
copyprog = rsync -a -A -X --rsh=ssh --inplace --compress
rsync = true
fastcheck = default
ignorelocks = false
dumparchives = false
showarchive = false
rootsName =
ignorearchives = false
fastercheckUNSAFE = false
fat = false
allHostsAreRunningWindows = false
someHostIsRunningWindows = false
confirmbigdel = true
batch = true
root = ssh://fs2b.our.domain.org.uk//root/unison-test
root = /root/unison-test
killserver = false
halfduplex = false
stream = true
addversionno = false
servercmd =
sshargs =
rshargs =
rshcmd = rsh
sshcmd = ssh
xferbycopying = true
sshversion =
clientHostName = fs72a
ignoreinodenumbers = false
links-aux = true
links = default
times = true
group = true
owner = true
numericids = false
dontchmod = false
perms = -1
watch = true
rsrc-aux = false
rsrc = default
maxerrors = 1
unicodeCS = false
unicodeEnc = false
unicode = default
someHostIsInsensitive = false
ignorecase = default
timers = false
terse = false
logfile = /root/.unison/unison.log
log = true
debugtimes = false
debug = all
addprefsto =
Contacting server...
[remote] Shell connection: ssh (ssh, fs2b.our.domain.org.uk, -e, none, unison, -server)
[globals] Checking path '' for expansions
Connected [//fs2b//root/unison-test -> //fs72a//root/unison-test]
[startup] Roots:
        /root/unison-test
        ssh://fs2b.our.domain.org.uk//root/unison-test
  i.e.
        /root/unison-test
        ssh://fs2b.our.domain.org.uk//root/unison-test
  i.e. (in canonical order)
       /root/unison-test
       //fs2b//root/unison-test

[props] Setting permission mask to 7777 (777777777777777777777 and 7777)
[stasher] initBackupsLocal
[stasher] d = /
[stasher] Prefix and suffix regexps for backup filenames have been updated
[server: stasher] initBackupsLocal
[server: stasher] d = /
[server: stasher] Prefix and suffix regexps for backup filenames have been updated
Looking for changes
[ui] temp: Globals.paths =
[update] Loading archive from /root/.unison/areaa7785bb2204bed0cb7af3360b3c323
[update] Setting archive for //fs72a//root/unison-test
[server: update] Loading archive from /root/.unison/ar8a6c9477fdd76859c429c5bdd5afe2d9
[server: update] Setting archive for //fs2b//root/unison-test
[update] findOnRoot //fs2b//root/unison-test
[update] findOnRoot /root/unison-test
[update] findLocal /root/unison-test ()
[fpcache] opening cache file /root/.unison/fpeaa7785bb2204bed0cb7af3360b3c323 for input
[fpcache] read chunk of 1 files
[fpcache] opening cache file /root/.unison/fpeaa7785bb2204bed0cb7af3360b3c323 for output
[pred] ignore '' = false
[update] buildUpdateRec: /root/unison-test
[pred] immutable '' = false
[pred] ignore 'hello' = false
[update] buildUpdateRec: /root/unison-test/hello
[update] buildUpdate -> New file
[fpcache] cache hit for path hello
[xferhint] insertEntry: fspath=/root/unison-test, path=hello, fp=(8ddd8be4b179a529afa5f2ffae4b9858,)
[update] Setting archive for //fs72a//root/unison-test

  Waiting for changes from server
[server: update] findLocal /root/unison-test ()
[server: fpcache] opening cache file /root/.unison/fp8a6c9477fdd76859c429c5bdd5afe2d9 for input
[server: fpcache] opening cache file /root/.unison/fp8a6c9477fdd76859c429c5bdd5afe2d9 for output
[server: pred] ignore '' = false
[server: update] buildUpdateRec: /root/unison-test
[server: pred] immutable '' = false
[server: update] Setting archive for //fs2b//root/unison-test
Reconciling changes
[recon] reconcileAll
[recon] reconcile: 1 results
[update] Marking 0 paths equal
[pred] forcepartial 'hello' = false
[pred] preferpartial 'hello' = false
new file ----> hello
Propagating updates
UNISON 2.48.4 started propagating changes at 12:53:09.13 on 08 Mar 2023
[BGN] Copying hello from /root/unison-test to //fs2b//root/unison-test
[files] copy /root/unison-test hello ---> //fs2b//root/unison-test hello
[server: pred] follow 'hello' = false
[server: fspath] Os.findWorkingDir(/root/unison-test,hello) = (/root/unison-test,hello)
[update] updateArchive /root/unison-test hello
[pred] ignore 'hello' = false
[files] copyRec hello --> .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp (really to hello)
[abort] Checking line 0
[copy] copyRegFile(/root/unison-test,hello) -> (//fs2b//root/unison-test,hello,/root/unison-test,.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp,modified on 2023-03-08 at 11:59:12 size 13 --rw-r--r-- user=0 group=0)
[abort] Checking line 0
[server: copy] tryCopyMovedFile: -> .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp /(8ddd8be4b179a529afa5f2ffae4b9858,)/
[server: xferhint] lookup: fp = (8ddd8be4b179a529afa5f2ffae4b9858,)
rsync -a -A -X --rsh=ssh --inplace --compress '/root/unison-test/hello' 'fs2b.our.domain.org.uk:'\''/root/unison-test/.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp'\'''
[copy] transferFileUsingExternalCopyprog hello: returned...
rsync: change_dir#3 "/root//'/root/unison-test" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at main.c(712) [Receiver=3.1.2]

Exited with status 3
[abort] Aborting line 0
Failed: External copy program did not create target file (or bad length): .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp
100% 00:00 ETAFailed [hello]: External copy program did not create target file (or bad length): .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp
UNISON 2.48.4 finished propagating changes at 12:53:09.66 on 08 Mar 2023
Saving synchronizer state
[update] Updating archives
[update] Saving archive in /root/.unison/sceaa7785bb2204bed0cb7af3360b3c323
[server: update] Saving archive in /root/.unison/sc8a6c9477fdd76859c429c5bdd5afe2d9
[update] Copying archive /root/.unison/tmeaa7785bb2204bed0cb7af3360b3c323 to /root/.unison/areaa7785bb2204bed0cb7af3360b3c323
[server: update] Copying archive /root/.unison/tm8a6c9477fdd76859c429c5bdd5afe2d9 to /root/.unison/ar8a6c9477fdd76859c429c5bdd5afe2d9
[update] Removing archive /root/.unison/tmeaa7785bb2204bed0cb7af3360b3c323
[server: update] Removing archive /root/.unison/tm8a6c9477fdd76859c429c5bdd5afe2d9
Synchronization incomplete at 12:53:09 (0 items transferred, 0 skipped, 1 failed)
  failed: hello
root@fs72a:~# [server: remote] Connection closed by the client

Unison output with rsync 3.1.2-2.1ubuntu1

root@fs72a:~# unison fs2b_unisontest -debug all
[startup] Preferences:
ui = graphic
host =
server = false
prefsdocs = false
doc =
version = false
silent = false
dumbtty = false
testserver = false
rest = fs2b_unisontest
showprev = false
selftest = false
confirmmerge = false
retry = 0
repeat =
contactquietly = false
key =
label = Syncs P and S drives between fs72a and fs2a
expert = false
height = 15
auto = true
maxthreads = 0
maxsizethreshold = -1
prefer =
force =
sortnewfirst = false
sortbysize = true
keeptempfilesaftermerge = false
diff = diff -u CURRENT2 CURRENT1
copyonconflict = true
backupdir =
maxbackups = 2
backups = false
backupsuffix =
backupprefix = .bak.$VERSION.
backuploc = central
copymax = 1
copyquoterem = default
copythreshold = 0
copyprogrest = rsync -a -A -X --rsh=ssh --partial --inplace --compress
copyprog = rsync -a -A -X --rsh=ssh --inplace --compress
rsync = true
fastcheck = default
ignorelocks = false
dumparchives = false
showarchive = false
rootsName =
ignorearchives = false
fastercheckUNSAFE = false
fat = false
allHostsAreRunningWindows = false
someHostIsRunningWindows = false
confirmbigdel = true
batch = true
root = ssh://fs2b.our.domain.org.uk//root/unison-test
root = /root/unison-test
killserver = false
halfduplex = false
stream = true
addversionno = false
servercmd =
sshargs =
rshargs =
rshcmd = rsh
sshcmd = ssh
xferbycopying = true
sshversion =
clientHostName = fs72a
ignoreinodenumbers = false
links-aux = true
links = default
times = true
group = true
owner = true
numericids = false
dontchmod = false
perms = -1
watch = true
rsrc-aux = false
rsrc = default
maxerrors = 1
unicodeCS = false
unicodeEnc = false
unicode = default
someHostIsInsensitive = false
ignorecase = default
timers = false
terse = false
logfile = /root/.unison/unison.log
log = true
debugtimes = false
debug = all
addprefsto =
Contacting server...
[remote] Shell connection: ssh (ssh, fs2b.our.domain.org.uk, -e, none, unison, -server)
[globals] Checking path '' for expansions
Connected [//fs2b//root/unison-test -> //fs72a//root/unison-test]
[startup] Roots:
        /root/unison-test
        ssh://fs2b.our.domain.org.uk//root/unison-test
  i.e.
        /root/unison-test
        ssh://fs2b.our.domain.org.uk//root/unison-test
  i.e. (in canonical order)
       /root/unison-test
       //fs2b//root/unison-test

[props] Setting permission mask to 7777 (777777777777777777777 and 7777)
[stasher] initBackupsLocal
[stasher] d = /
[stasher] Prefix and suffix regexps for backup filenames have been updated
[server: stasher] initBackupsLocal
[server: stasher] d = /
[server: stasher] Prefix and suffix regexps for backup filenames have been updated
Looking for changes
[ui] temp: Globals.paths =
[update] Loading archive from /root/.unison/areaa7785bb2204bed0cb7af3360b3c323
[update] Setting archive for //fs72a//root/unison-test
[server: update] Loading archive from /root/.unison/ar8a6c9477fdd76859c429c5bdd5afe2d9
[server: update] Setting archive for //fs2b//root/unison-test
[update] findOnRoot //fs2b//root/unison-test
[update] findOnRoot /root/unison-test
[update] findLocal /root/unison-test ()
[fpcache] opening cache file /root/.unison/fpeaa7785bb2204bed0cb7af3360b3c323 for input
[fpcache] read chunk of 1 files
[fpcache] opening cache file /root/.unison/fpeaa7785bb2204bed0cb7af3360b3c323 for output
[pred] ignore '' = false
[update] buildUpdateRec: /root/unison-test
[pred] immutable '' = false
[pred] ignore 'hello' = false
[update] buildUpdateRec: /root/unison-test/hello
[update] buildUpdate -> New file
[fpcache] cache hit for path hello
[xferhint] insertEntry: fspath=/root/unison-test, path=hello, fp=(8ddd8be4b179a529afa5f2ffae4b9858,)
[update] Setting archive for //fs72a//root/unison-test

  Waiting for changes from server
[server: update] findLocal /root/unison-test ()
[server: fpcache] opening cache file /root/.unison/fp8a6c9477fdd76859c429c5bdd5afe2d9 for input
[server: fpcache] opening cache file /root/.unison/fp8a6c9477fdd76859c429c5bdd5afe2d9 for output
[server: pred] ignore '' = false
[server: update] buildUpdateRec: /root/unison-test
[server: pred] immutable '' = false
[server: update] Setting archive for //fs2b//root/unison-test
Reconciling changes
[recon] reconcileAll
[recon] reconcile: 1 results
[update] Marking 0 paths equal
[pred] forcepartial 'hello' = false
[pred] preferpartial 'hello' = false
new file ----> hello
Propagating updates
UNISON 2.48.4 started propagating changes at 12:58:13.42 on 08 Mar 2023
[BGN] Copying hello from /root/unison-test to //fs2b//root/unison-test
[files] copy /root/unison-test hello ---> //fs2b//root/unison-test hello
[server: pred] follow 'hello' = false
[server: fspath] Os.findWorkingDir(/root/unison-test,hello) = (/root/unison-test,hello)
[update] updateArchive /root/unison-test hello
[pred] ignore 'hello' = false
[files] copyRec hello --> .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp (really to hello)
[abort] Checking line 0
[copy] copyRegFile(/root/unison-test,hello) -> (//fs2b//root/unison-test,hello,/root/unison-test,.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp,modified on 2023-03-08 at 11:59:12 size 13 --rw-r--r-- user=0 group=0)
[abort] Checking line 0
[server: copy] tryCopyMovedFile: -> .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp /(8ddd8be4b179a529afa5f2ffae4b9858,)/
[server: xferhint] lookup: fp = (8ddd8be4b179a529afa5f2ffae4b9858,)
rsync -a -A -X --rsh=ssh --inplace --compress '/root/unison-test/hello' 'fs2b.our.domain.org.uk:'\''/root/unison-test/.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp'\'''
[copy] transferFileUsingExternalCopyprog hello: returned...
[server: props] Setting permissions for /root/unison-test/.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp to --rw-r--r-- (644/7777)
[server: xferhint] insertEntry: fspath=/root/unison-test, path=.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp, fp=(8ddd8be4b179a529afa5f2ffae4b9858,)
[files] rename(root=//fs2b//root/unison-test, pathOld=.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp, pathNew=hello)
[server: update] checkNoUpdates /root/unison-test hello
[server: update] buildUpdateRec: /root/unison-test/hello
[server: update] buildUpdate -> Absent and no archive
[server: files] Renaming .unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp to hello in /root/unison-test; root is /root/unison-test
[server: files] rename: moveFirst=false
[server: stasher] backup: /root/unison-test / hello
[server: pred] follow 'hello' = false
[server: fspath] Os.findWorkingDir(/root/unison-test,hello) = (/root/unison-test,hello)
[server: stasher] File hello in /root/unison-test does not exist, so no need to back up
[server: os] rename /root/unison-test/.unison.hello.8a6c9477fdd76859c429c5bdd5afe2d9.unison.tmp to /root/unison-test/hello
[server: files] Contents of /root/unison-test/hello after renaming = 8ddd8be4b179a529afa5f2ffae4b9858
[server: pred] backupcurr 'hello' = false
[server: xferhint] insertEntry: fspath=/root/unison-test, path=hello, fp=(8ddd8be4b179a529afa5f2ffae4b9858,)
[server: update] replaceArchiveLocal /root/unison-test hello
[server: update] Setting archive for //fs2b//root/unison-test
[update] updateArchive /root/unison-test hello
[pred] ignore 'hello' = false
[update] replaceArchiveLocal /root/unison-test hello
[update] Setting archive for //fs72a//root/unison-test
[pred] backupcurr 'hello' = false
[END] Copying hello
UNISON 2.48.4 finished propagating changes at 12:58:14.13 on 08 Mar 2023
Saving synchronizer state
[update] Updating archives
[update] Saving archive in /root/.unison/sceaa7785bb2204bed0cb7af3360b3c323
[server: update] Saving archive in /root/.unison/sc8a6c9477fdd76859c429c5bdd5afe2d9
[update] Copying archive /root/.unison/tmeaa7785bb2204bed0cb7af3360b3c323 to /root/.unison/areaa7785bb2204bed0cb7af3360b3c323
[server: update] Copying archive /root/.unison/tm8a6c9477fdd76859c429c5bdd5afe2d9 to /root/.unison/ar8a6c9477fdd76859c429c5bdd5afe2d9
[update] Removing archive /root/.unison/tmeaa7785bb2204bed0cb7af3360b3c323
[server: update] Removing archive /root/.unison/tm8a6c9477fdd76859c429c5bdd5afe2d9
Synchronization complete at 12:58:14 (1 item transferred, 0 skipped, 0 failed)
root@fs72a:~# [server: remote] Connection closed by the client

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Looks like the rsync security update doesn't like the way unison is handling arguments. Perhaps adding --old-args to the command lines here would help:

copyprog = rsync -a -A -X --rsh=ssh --inplace --compress
copyprogrest = rsync -a -A -X --rsh=ssh --partial --inplace --compress

Revision history for this message
Thomas S (allworlds) wrote :

That appears to work around the problem, yes, changing them to.

copyprog = rsync -a -A -X --rsh=ssh --inplace --compress --old-args
copyprogrest = rsync -a -A -X --rsh=ssh --partial --inplace --compress --old-args

tags: added: regression-update
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.