Synchronization with earlier Ubuntu versions or a recent Debian version fails

[Julian]

Any synchronization I want to do with my server (Debian buster) fails with the following message:

> Fatal error: Server: Fatal error during unmarshaling (input_value: ill-formed message),
> possibly because client and server have been compiled with differentversions of the OCaml compiler.

Indeed, buster uses OCaml 4.05.0, focal uses 4.08.1. eoan used 4.05.0, as did the prior versions, including bionic, which all still synchronize fine. The current Windows version also synchronizes with that version.

Although intentionally using an old software version for compatibility is not a good thing, in this case it might be necessary. The whole point of unison is its interoperability.

[Julian (gnrpinator)]

Note that this is related to #1568459. But this time it can be fixed by using the same unison version but compiling it with a different OCaml.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[kimo (ubuntu-oldfield)]

This is also a problem between Ubuntu LTS versions.
Specifically: I get the same error message when running unison on 20.04 (Focal Fossa) and trying to unison sync with a remote computer running 18.04 (Bionic Bever).

[Jay (jay-7)]

Can confirm this affects me. Upgraded a computer to 20.04, trying to sync with a 18.04 computer using Unison 2.48.4 and receive the same error message. Please compile unison 2.48.4 with the correct version of OCamel!

[Ferry Toth (ftoth)]

Affects me to. Host is Bionic and Desktop is Focal.

I tried installing the Bionic unison package on focal, but then I get other errors:

Uncaught exception Failure("input_value: ill-formed message")
Raised at file "/build/unison-Y9EcQW/unison-2.48.4/lwt/lwt.ml", line 126, characters 16-23
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/generic/lwt_unix_impl.ml", line 102, characters 8-23
Called from file "/build/unison-Y9EcQW/unison-2.48.4/update.ml" (inlined), line 2096, characters 2-69
Called from file "/build/unison-Y9EcQW/unison-2.48.4/uitext.ml", line 700, characters 16-56
Called from file "/build/unison-Y9EcQW/unison-2.48.4/uitext.ml", line 788, characters 6-90
Called from file "/build/unison-Y9EcQW/unison-2.48.4/uitext.ml", line 810, characters 19-66
Called from file "/build/unison-Y9EcQW/unison-2.48.4/uitext.ml", line 870, characters 21-43

This is strange, as unison doesn't depend on ocaml (and I don't have that installed).

So it looks like unison is trying to force me to upgrade our server, but that won't happen until months from now with a planned LTS update.

[Helge Meinhard (helge-meinhard)]

Same here. Upgraded client to Focal, server is Bionic. Same version numbers of unison, but the cryptic, misspelled message about incompatible ocaml versions appears.

It would be nice to get this fixed, or at least hints about how to work around the problem...

[Helge Meinhard (helge-meinhard)]

FWIW, Bionic ships OCamL 4.05, Focal 4.08.

[monteleo (monteleone-c)]

Same problem. Upgraded client to Focal, server is Bionic. Both have unison 2.48 and I have the message:
"Fatal error: Server: Fatal error during unmarshaling (input_value: ill-formed message),
possibly because client and server have been compiled with differentversions of the OCaml compiler."

[Helge Meinhard (helge-meinhard)]

I tried to narrow down the problem by re-building unison with OCamL 4.05 on my Focal (20.04) client (no ocaml deb package installed):

- Downloaded OCamL 4.05.0, ran make and make install
$ which ocamlopt
/usr/local/bin/ocamlopt
$ ocamlopt -v
The OCaml native-code compiler, version 4.05.0
Standard library directory: /usr/local/lib/ocaml

- Downloaded sources of unison 2.48.4, ran make ('make world' or 'make world opt' didn't seem to make any difference)
$ ls -l ./src/unison
-rwxr-xr-x 1 meinhard v5 5038200 May 20 14:50 ./src/unison
$ file ./src/unison
./src/unison: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=cddb0691b6b9721087be3f2f6f4f2397f57f4171, for GNU/Linux 3.2.0, with debug_info, not stripped
$ ldd ./src/unison
 linux-vdso.so.1 (0x00007ffda03f4000)
 libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f97aa540000)
 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f97aa3f1000)
 libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f97aa3eb000)
 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f97aa1f9000)
 /lib64/ld-linux-x86-64.so.2 (0x00007f97aa8b8000)
$ ./src/unison -version
unison version 2.48.4

Yet no luck when syncing to my Bionic server, even though the error message is very different:
$ ./src/unison <target>
[...]
Looking for changes
Uncaught exception Failure("input_value: ill-formed message")
Raised at file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/lwt/lwt.ml", line 126, characters 16-23
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/lwt/generic/lwt_unix_impl.ml", line 102, characters 8-23
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/update.ml" (inlined), line 2096, characters 2-69
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/uitext.ml", line 700, characters 16-56
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/uitext.ml", line 788, characters 6-90
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/uitext.ml", line 810, characters 19-66
Called from file "/data01/meinhard/Scratch/Software/Unison/unison-2.48.4/src/uitext.ml", line 870, characters 21-43

Not having any clue about OCamL, I don't know what causes this error - the source file lwt.ml doesn't look suspicious in any way at the incriminated line.

Any help I can further provide to understand and fix the problem, please let me know. (For the moment I'll keep OCamL 4.05 and unison 2.48-4 sources around...)

[Helge Meinhard (helge-meinhard)]

In fact, further investigation shows that the binary built with the OCamL 4.05 compiler is working fine. The error message quoted in my previous post is obviously caused by a cache file corrupted by previously running the binary from deb (compiled with OCamL 4.08). Deleting the cache file fixes the problem.

I have now verified that just using the binary copied over from Bionic works equally well on Focal.

So the workaround seems to be to uninstall the unison deb package and to copy the /usr/bin/unison-2.48.4 binary from a Bionic system into /usr/local/bin, /opt/bin, $HOME/bin or whatever else you prefer (and possibly to create a symlink called unison).

[Jeff Abrahamson (jeff-purple)]

@helge-meinhard - could you please specify what cache file(s) you removed and whether it was client or server side or both?

Unison creates numerous cache files in $HOME/.unison/ but their specific meaning and content is mostly unclear.

[Helge Meinhard (helge-meinhard)]

I deleted all fp* files with a creation date newer than my unsuccessful attempt to run a focal client against a bionic server.

[Luna (luna-casius)]

Same issue here. Two machines upgraded to Focal. unison version is 2.48.4 on both machines. Yet I get the error message as in the bug description. For recompiling the packages I need some advice how to do that.

[Sébastien Koechlin (seb-ubuntu)]

Instead of recompiling, I downgraded unison and unison-gtk with Bionic Beaver packages. There is no dependency issues.

Warning: you will not have any security update.

https://packages.ubuntu.com/bionic/unison
https://packages.ubuntu.com/bionic/unison-gtk

[Luna (luna-casius)]

Thank you very much for providing the links.

Unfortunately it didn't do the trick. Now the error message is:

Uncaught exception Failure("input_value: ill-formed message")
Raised at file "/build/unison-Y9EcQW/unison-2.48.4/lwt/lwt.ml", line 126, characters 16-23
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/generic/lwt_unix_impl.ml", line 102, characters 8-23
Called from file "/build/unison-Y9EcQW/unison-2.48.4/update.ml" (inlined), line 2096, characters 2-69
.
.
.

[Julian (gnrpinator)]

So I also found a workaround that works for me so far. It is also downgrading to the bionic versions.

First of all, I downloaded the packages from bionic and installed them manually (only unison and unison-all, though) as suggested by Sebastien Koechlin and Helge Meinhard said. Then, I created a file to pin it to that version so it will not get upgraded, i.e., in /etc/apt/preferences.d/unison I have:

    Package: unison
    Pin: version 2.48.4-1ubuntu1
    Pin-Priority: 1000

    Package: unison-all
    Pin: version 2.48+2
    Pin-Priority: 1000

The other problem is that unison does not like to use the old archive files. You can try running unison with the `-ignorearchives` flag. If unison runs then, you have to remove the old archive file.

If you have only one archive, you can just delete your ~/.unison directory and then run unison afterwards without any flags and it should be fine.
If you have more archives and you do not want to rebuild them all, you have to find the proper file.
I did this in a stupid way by copying them out one-by-one (both the arXXX and the fpXXX).
The, I check if unison still reports an error and if so, it was the wrong archive and I copy it back.
If I found the correct archive, I just stop.
Maybe there is somebody who knows a flag which can tell you the archive file directly.

[christos pontikis (pontikis)]

Hello, same problem here.

It seems that @Helge Meinhard workaround WORKS. Thanks!

In my case:
Workstation: Xubuntu Focal 20.04 LTS
Laptop.....: Xubuntu Bionic 18.04 LTS
Server.....: Debian 10 Buster

(1)
I didn't uninstall unison unison-all unison-all-gtk from Focal

(2)
I copied the binaries
/usr/bin/unison-2.48.4 and /usr/bin/unison-2.48.4-gtk
from Bionic to Focal

(3)
I deleted all ~/.unison/fp* (in all machines)

Various tasks between Focal and (Bionic or Buster) are working.

[Romano Giannetti (romano-giannetti)]

@christos pontikis (pontikis) this saved my day, thanks!

As a small improvement: you do not need to remove all cache, just the fp* files that have been created after you started using the new version.

I think that this is the kind of software that would benefit greatly the snap/appimage/flatpack whatever, to have always the same version in all your machines...

[Christof Arn (christof-arn)]

To make it work, I had also to replace unison-gtk, not only unison. So, what finally helped was:

Problemsituation: I had updated laptop A to a newer version of ubuntu. When trying to synchronize with with laptop B (Debian), and therefore did start unison-gtk on lapotop A, I got the error we discuss here.

The solution that worked for me:
1. On Laptop A: downloading from packages.debian.org exactly the unison Version *and* the unison-gtk version, I have in use on Laptop B
2. On Laptop A: Installing both:
sudo dpkg -i unison_2.48.4-1+b1_amd64.deb
sudo dpkg -i unison-gtk_2.48.4-1+b1_amd64.deb
3. On both, Laptop A and Laptop B: Removing in the hidden folder .unison all files, with a filename starting with "fp...", that are created after I had updated laptop A to a newer version of ubuntu.

To make sure, Laptop A wont "upgrade" unison and unison-gtk I created, like mentioned above, a file with filename "unison" in the folder /etc/apt/preferences.d/ wrote this text into my file "unison":

     Package: unison
     Pin: version 2.48.4-1+b1
     Pin-Priority: 1000

     Package: unison-gtk
     Pin: version 2.48.4-1+b1
     Pin-Priority: 1000

Thanks to all here for those posts that helped me to do so!

[Per Turbatus (perturbatus)]

Unison does not support syncing between two different versions. Hence, syncing across different ubuntu releases this problem will necessarily occur unless they contain many unison versions in the future.

Workaround:

1) On both machines delete the unison cache:

rm -r ~/.unison

2) Make sure no version of unison is installed on the machine with the newer Ubuntu release.

3) On the machine with the newer Ubuntu release download the unison .deb file of the older release. For example the .deb file of the unison version of Ubuntu 18.04 can be found here:

https://ubuntu.pkgs.org/18.04/ubuntu-universe-amd64/unison_2.48.4-1ubuntu1_amd64.deb.html

Install this unison version via the graphical interface.

4) On the machine with the newer Ubuntu release freeze the unison version you just installed. Warning you will not get security updates!

sudo apt-mark hold unison

5) Sync both machines using unison.

Have fun!

[Hadmut Danisch (hadmut)]

Sorry, but this advice is terribly wrong.

rm -r ~/.unison

This will not only delete *all* and not just the synchronization states, that cause trouble, it will furthermore delete all config files and includes.

So don't do that.