Unison from Ubuntu 20.04 is incompatible with Unison from Ubuntu 18.04, Windows or Mac OS X

[Marc Wouts]

Unison from Ubuntu 20.04 LTS (version unison=2.48.4-4ubuntu1) works well with a client in exactly the same version, with exactly the same OS.

However, it does not work well with Unison 2.48.4-1ubuntu1 from Ubuntu 18.04, or with Unison on other OSes.

Once the client has determined the files that were updated, when I click on 'Go', the synchronisation fails with a message like this one:
> Fatal error:
> Uncaught exception Failure("input_value: ill-formed message")

Sometimes there is the mention "possibly because client and server have been compiled with different versions of the Ocaml compiler."

I have seen this issues in the following configurations
- 2.48.4-4ubuntu1 on Ubuntu 20.04 LTS (client) and 2.48.4-1ubuntu1 on Ubuntu 18.04 LTS (server) => Fails
- 2.48.? on Mac OS X (client) and 2.48.4-4ubuntu1 on Ubuntu 20.04 LTS (server) => Fails
- 2.48.3 on Windows 7 (client) and 2.48.4-4ubuntu1 on Ubuntu 20.04 LTS (server) => Fails

Note that the Mac OS and Windows clients were working well with unison 2.48.4-1ubuntu1 (Ubuntu 18.04 LTS, server).

Maybe my problem could be solved by reverting unison to the version 2.48.4-1ubuntu1 from Ubuntu 18.04 LTS. Is is possible to do so with `apt install` on Ubuntu 20.04? Thanks!

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: unison 2.48.4-4ubuntu1
ProcVersionSignature: Ubuntu 5.4.0-26.30-generic 5.4.30
Uname: Linux 5.4.0-26-generic x86_64
ApportVersion: 2.20.11-0ubuntu27
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Mon Apr 27 19:18:59 2020
InstallationDate: Installed on 2020-04-16 (11 days ago)
InstallationMedia: Ubuntu 18.04.2 LTS "Bionic Beaver" - Release amd64 (20190210)
SourcePackage: unison
UpgradeStatus: Upgraded to focal on 2020-04-21 (6 days ago)

[Martin Fiedler (keyj-ubuntu)]

I can reproduce the problem here with 2.48.4-4ubuntu1.

When run via the socket protocol against an older Windows build (2.48.3), the Windows client crashes.
When updating the Windows version to 2.48.4, I get the exact same messages as Marc.

[Marc Wouts (mwouts)]

Note that I solved the incompatibility issue with Windows & Mac OS by installing the former Debian version on the Ubuntu 20.04 LTS server:
```
wget http://ftp.fr.debian.org/debian/pool/main/u/unison/unison_2.48.4-1+b1_amd64.deb
sudo dpkg -i unison_2.48.4-1+b1_amd64.deb
```

Running that breaked the compatibility with Unison from the Ubuntu 20.04 LTS client. To solve that, I ran the previous command on the Ubuntu client as well, plus this one:
```
wget http://ftp.fr.debian.org/debian/pool/main/u/unison/unison-gtk_2.48.4-1+b1_amd64.deb
sudo dpkg -i unison-gtk_2.48.4-1+b1_amd64.deb
```

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[niels (opensource21)]

I think it's a good idea to set this packages on hold.

aptitude hold unison unison-gtk

[Michael Wingender (mwingender)]

Workaround installing older version doesn't work for my Windows-Client/Linux-Server combination. I have installed former version on Ubuntu 20.04. When I start the software von my Windows Client (Unison 2.48.4), I a getting the following error on the server:

Unison failed: Uncaught exception Failure("input_value: ill-formed message")
Raised at file "/build/unison-Y9EcQW/unison-2.48.4/lwt/lwt.ml", line 135, characters 6-13
Called from file "list.ml", line 85, characters 12-15
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/lwt.ml", line 31, characters 2-37
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/lwt.ml" (inlined), line 83, characters 17-46
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/generic/lwt_unix_impl.ml", line 55, characters 6-23
Called from file "/build/unison-Y9EcQW/unison-2.48.4/lwt/generic/lwt_unix_impl.ml", line 147, characters 6-40
Called from file "/build/unison-Y9EcQW/unison-2.48.4/remote.ml", line 1465, characters 11-57
Called from file "/build/unison-Y9EcQW/unison-2.48.4/ubase/util.ml", line 170, characters 6-9
Called from file "/build/unison-Y9EcQW/unison-2.48.4/main.ml", line 131, characters 4-9

Before I was running Ubuntu 18.04 on my server and this was working fine with the client version.

[Gisle Hannemyr (gislehannemyr)]

Interim solution posted on AskUbuntu: https://askubuntu.com/q/1234767/305040

[Gary Hasson (gary-oax)]

The accepted work-around is to manually install prior versions of unison and unison-gtk. It seems to me that the solution to this bug is to replace the faulty unison and unison-gtk in the repository with the known good versions (from 19.04 for example). People who have been using the faulty version from Ubuntu 20.04 would be updated to the good version, after which they would get an error message from Unison when they used the good version with a sync that was previously done with the faulty version. The error message simply tells them to delete the sync archive file and then re-sync.

[Odin Hørthe Omdal (velmont)]

After downgrading and trying the downgraded unison you will only need to delete the "fs*" files in .unison on your local machine I found out. So thankfully the sync-state "ar*", was still working well after that.

I didn't actually get a new error though, but the same error. But thankfully this bug thread helped me by saying there was two steps.

[mine260309 (mine260309)]

> Unison failed: Uncaught exception Failure("input_value: ill-formed message")

I met exactly the same issue after "downgrading" the unison to unison_2.48.4-1+b1_amd64.deb on Ubuntu 20.04

And the workaround is indicated by @velmont, that to delete the files in ~/.unison
But I got an extra issue after deleting the files:

Fatal error: Warning: inconsistent state.
The archive file is missing on some hosts.
...
Please delete archive files as appropriate and try again
or invoke Unison with -ignorearchives flag.

So I need to run unison with `-ignorearchives` flag once to recreate the states, then the whole issue is fixed.

[Etienne URBAH (eurbah)]

On Ubuntu 20.04, I successfully downgraded Unison (text only) to the version compiled with OCaml 4.02 from https://www.urs-mueller.ch/de/computer/download/unison-2.48.4-linux-x86_64-text-static.tar.gz

Using the unison-jz snap installing Unison 2.51.2 compiled against OCaml 4.02.3 as described by https://unix.stackexchange.com/questions/583058/unison-and-version-compiler-conflicts/583377#583377 requires that :
- The --devmode option of the snap command is NOT necessary,
- The problem of the Include line in /etc/ssh/ssh_config is solved,
- Binaries of Unison 2.51.2 compiled against OCaml 4.02.3 are also provided for :
  - Mac OS-X, for example at https://www.urs-mueller.ch/de/computer/index.html
  - Windows, for example at https://www.irif.fr/~vouillon/unison/

On the long term, I would prefer that the Unison packages explicitly contain the OCaml version, and the default value of servercmd should contain the name of the server binary compiled with the same OCaml version. For example :

- unison-OCaml-4.02 version 2.48.4 should contain
  /usr/bin/unison-OCaml-4.02 --> /usr/bin/unison-OCaml-4.02-2.48.4

- unison-OCaml-4.02 version 2.51.2 should contain
  /usr/bin/unison-OCaml-4.02 --> /usr/bin/unison-OCaml-4.02-2.51.2

- unison-OCaml-4.06 version 2.48.4 should contain
  /usr/bin/unison-OCaml-4.06 --> /usr/bin/unison-OCaml-4.06-2.48.4

- unison-OCaml-4.06 version 2.51.2 should contain
  /usr/bin/unison-OCaml-4.06 --> /usr/bin/unison-OCaml-4.06-2.51.2

  This would permit, on the server, to install Unison binaries compiled with ALL OCaml versions.

[Benjamin Pierce (bcpierce)]

I’d suggest one slight tweak to the design: To maintain compatibility with people’s current way of doing business, there should be a new command line / preference file switch that defaults to false and that, when true, causes the current OCaml version to be included in the name of the server command.

> On Jun 6, 2020, at 7:50 PM, Etienne URBAH <email address hidden> wrote:
>
> On Ubuntu 20.04, I successfully downgraded Unison (text only) to the
> version compiled with OCaml 4.02 from https://www.urs-
> mueller.ch/de/computer/download/unison-2.48.4-linux-x86_64-text-
> static.tar.gz
>
> Using the unison-jz snap installing Unison 2.51.2 compiled against OCaml 4.02.3 as described by https://unix.stackexchange.com/questions/583058/unison-and-version-compiler-conflicts/583377#583377 requires that :
> - The --devmode option of the snap command is NOT necessary,
> - The problem of the Include line in /etc/ssh/ssh_config is solved,
> - Binaries of Unison 2.51.2 compiled against OCaml 4.02.3 are also provided for :
> - Mac OS-X, for example at https://www.urs-mueller.ch/de/computer/index.html
> - Windows, for example at https://www.irif.fr/~vouillon/unison/
>
> On the long term, I would prefer that the Unison packages explicitly
> contain the OCaml version, and the default value of servercmd should
> contain the name of the server binary compiled with the same OCaml
> version. For example :
>
> - unison-OCaml-4.02 version 2.48.4 should contain
> /usr/bin/unison-OCaml-4.02 --> /usr/bin/unison-OCaml-4.02-2.48.4
>
> - unison-OCaml-4.02 version 2.51.2 should contain
> /usr/bin/unison-OCaml-4.02 --> /usr/bin/unison-OCaml-4.02-2.51.2
>
> - unison-OCaml-4.06 version 2.48.4 should contain
> /usr/bin/unison-OCaml-4.06 --> /usr/bin/unison-OCaml-4.06-2.48.4
>
> - unison-OCaml-4.06 version 2.51.2 should contain
> /usr/bin/unison-OCaml-4.06 --> /usr/bin/unison-OCaml-4.06-2.51.2
>
> This would permit, on the server, to install Unison binaries compiled
> with ALL OCaml versions.
>
> --
> You received this bug notification because you are subscribed to unison
> in Ubuntu.
> https://bugs.launchpad.net/bugs/1875475
>
> Title:
> Unison from Ubuntu 20.04 is incompatible with Unison from Ubuntu
> 18.04, Windows or Mac OS X
>
> Status in unison package in Ubuntu:
> Confirmed
>
> Bug description:
> Unison from Ubuntu 20.04 LTS (version unison=2.48.4-4ubuntu1) works
> well with a client in exactly the same version, with exactly the same
> OS.
>
> However, it does not work well with Unison 2.48.4-1ubuntu1 from Ubuntu
> 18.04, or with Unison on other OSes.
>
> Once the client has determined the files that were updated, when I click on 'Go', the synchronisation fails with a message like this one:
>> Fatal error:
>> Uncaught exception Failure("input_value: ill-formed message")
>
> Sometimes there is the mention "possibly because client and server
> have been compiled with different versions of the Ocaml compiler."
>
> I have seen this issues in the following configurations
> - 2.48.4-4ubuntu1 on Ubuntu 20.04 LTS (client) and 2.48.4-1ubuntu1 on Ubuntu 18.04 LTS (server) => Fails
> - 2.48.? on Mac OS X (client) and 2.48.4-4ubuntu1 o...

[Stephen Warren (srwarren)]

I need to communicate with an Ubuntu 16.04 server, from an Ubuntu 20.04 system. On 20.04, I tried installing the Unison packages from 20.04, 18.04, and 16.04. None of those worked:-( However, before I upgraded that system I was running 18.04 on it, and communicating with the same 16.04 server just fine. I guess there's some issue that stems from a shared libary?

[codywohlers (codywohlers)]

FYI this also happened with 14.04 -> 16.04
https://bugs.launchpad.net/ubuntu/+source/unison/+bug/1568459

[Paul Gear (paulgear)]

This bug affects compatibility with unison-2.48.4-1+b1 on Debian buster also.

[Brian Candler (b-candler)]

The snap package isn't compatible with the version from macOS homebrew either.

In the end, I just went with the binary tarballs available from
https://github.com/bcpierce00/unison/releases

The reason for these incompatibilities is described here:
https://github.com/bcpierce00/unison/#getting-unison

"Beware that Unison uses OCaml's built-in data marshalling, and that this facility is unstable across versions of "ocaml" (the standard implementation of the OCaml language). Additionally, Unison has incompatible changes across minor releases (e.g. 2.48 vs 2.51, but 2.51.2 and 2.51.3 are compatible). Therefore, you must use the same Unison minor version built with the same ocaml version on all systems."

To make compatible packages for 16.04/18.04/20.04 would require either a newer OCaml to build the 16.04/18.04 packages, or an older OCaml to build the 18.04/20.04 packages.

I do think that the incompatible versions should be reflected in the package naming, e.g. "unison" is a meta-package and "unison-2.51.3-4.12.0" is the actual package. Then potentially multiple versions could be installed and /etc/alternatives/unison could point to the chosen one. But unless someone actually builds these different versions, it doesn't really matter.

There are other more modern filesystem synchronization tools available, like Syncthing.

[polarwinkel (polarwinkel)]

As a different workaround you can also get the [Ubuntu20.04-Version](https://ubuntu.pkgs.org/20.04/ubuntu-universe-amd64/unison_2.48.4-4ubuntu1_amd64.deb.html):

`wget http://archive.ubuntu.com/ubuntu/pool/universe/u/unison/unison_2.48.4-4ubuntu1_amd64.deb`

and install that one on your other system (tested on Debian Bullseye):

`sudo dpkg -i unison_2.48.4-4ubuntu1_amd64.deb`