transmission: Fail to build against OpenSSL 3.0

[Simon Chopin]

Hello,

As part of a rebuild against OpenSSL3, this package failed to build on one or
several architectures. You can find the details of the rebuild at

https://people.canonical.com/~schopin/rebuilds/openssl-3.0.0-impish.html

or for the amd64 failed build, directly at

https://launchpad.net/~schopin/+archive/ubuntu/openssl-3.0.0/+build/22099409/+files/buildlog_ubuntu-impish-amd64.transmission_3.00-1ubuntu2.0~ssl3ppa1.1_BUILDING.txt.gz

We're planning to transition to OpenSSL 3.0 for the 22.04 release, and consider
this issue as blocking for this transition.

You can find general migration informations at
https://www.openssl.org/docs/manmaster/man7/migration_guide.html
For your tests, you can build against libssl-dev as found in the PPA
schopin/openssl-3.0.0

This has been fixed upstream, currently on master:
https://github.com/transmission/transmission/pull/1788

[Simon Chopin (schopin)]

OK, so I've prepared a patch for this, but I can't get it to build because the original package fails to build in the current jammy. We're hitting https://bugs.launchpad.net/intltool/+bug/1921816

The patch itself is a bit massive as it imports the new standalone rc4 implementation.

[Simon Chopin (schopin)]

Note that to correctly resolve the build-deps in an OpenSSL 3.0.0 context, you need to use both schopin/openssl-3.0.0 and schopin/foundation-openssl3 in your sources.list

[Simon Chopin (schopin)]

New patch, where I switched the build-system to CMake. This takes care of the original FTBFS, and allowed me to test the OpenSSL3 patch, with success (still needs both PPAs to build)

[Sebastien Bacher (seb128)]

Thanks Simon, the switch of build system is not needed there, the build fix for the new autoconf is simple, I've sent a patch to Debian https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=978911;filename=build_new_autoconf.patch;msg=20

[Simon Chopin (schopin)]

Quoting Sebastien Bacher (2021-11-12 16:42:24)
> Thanks Simon, the switch of build system is not needed there, the build
> fix for the new autoconf is simple, I've sent a patch to Debian
> https://bugs.debian.org/cgi-
> bin/bugreport.cgi?att=1;bug=978911;filename=build_new_autoconf.patch;msg=20

True, but the new RC4 module uses CMake exclusively, and I figure it's
easier to just switch the whole package rather than try to write in the
autotools support for RC4.

It would all be so much easier if upstream would release a new version,
but they don't seem particularly interested:

https://github.com/transmission/transmission/issues/2126

[Launchpad Janitor (janitor)]

This bug was fixed in the package transmission - 3.00-1ubuntu5

---------------
transmission (3.00-1ubuntu5) jammy; urgency=medium

  * debian/patches/openssl3-compat.patch: Compatibility with OpenSSL 3.
    Closes LP: #1946215.

 -- Steve Langasek <email address hidden> Mon, 13 Dec 2021 14:47:31 -0800

[ajs124 (ajs124)]

It seems that the patch that fixed this might have been faulty and introduced a memory leak.

See https://github.com/NixOS/nixpkgs/issues/207047#issuecomment-1360393405 for some details.

Should I open a new issue for this?

[Steve Langasek (vorlon)]

I can confirm that there is a memory leak at the indicated point in the code, though I'm surprised it is noticeable as it only happens when tr_rc4_new() fails.

Would you mind opening a bug report in Debian for this? (Since this was upstreamed and Ubuntu is now in sync with Debian)

[Andrey Kudinov (elhana)]

There is no leak using gentoo patch it seems:
https://gitweb.gentoo.org/repo/gentoo.git/tree/net-p2p/transmission/files/transmission-3.00-openssl-3.patch

[Andreas Hasenack (ahasenack)]

https://bugs.launchpad.net/ubuntu/+source/transmission/+bug/1973084 is the memleak bug report.