SRU: Backport LLVM 18.1.8 to Ubuntu LTS

[Matthias Klose]

SRU: Backport LLVM 18.1.8 to Ubuntu 24.04 LTS

[ Impact ]

 * A newer Clang/LLVM version will be available for older releases as a non-default LLVM selection. This is particularly requested for building Firefox.

 * On newer releases which had an older LLVM 18 release: some general bug fixes and improvements that release note may or may not mention: https://discourse.llvm.org/t/18-1-8-released/79725 (and mostly about PowerPC fixes in 18.1.7).

[ Test Plan ]

 * Run autopkgtests on reverse dependencies.

 * On LTS releases which don't currently have LLVM 18, we can just examine the builtin unittest results, the unit tests don't need to have a 100% pass rate because that's impossible even for the upstream sometimes.

[ Where problems could occur ]

 * LLVM is a very large piece of software with a lot of moving parts. A lot could go wrong even in a minor release: optimizer misoptimizations, Clang behavior changes, library linkage changes, C/C++ UB (undefined behavior) handling changes, on-disk IR format change (maybe compatible/upgradable), change of "internal" C++ API parameters, and change of C API behaviors.

 * In the backport, since we will need to apply extra patches for LLVM to compile in an older LTS, more LLVM breakages may happen unnoticed as we don't really care about unittests provided by LLVM (because a lot of them don't function correctly on non-amd64 and non-arm64 platforms; a lot of LIT tests may require the build system to have the same environment as the upstream CI, which is by itself a moving target.).

 * In Focal backport, a RISC-V patch was added to fix LLVM code generation and internal linker behavior when targeting RISC-V. This is needed because LLVM is by default a cross-compiler that could generate any supported machine code on any supported platform (we also enabled LLVM Orc JIT remoting functionality, which allows LLVM JIT to transform LLVM IR into machine code over the network). This may pose extra risks since our binutils on Focal only supports RISC-V ABI v2.0 while LLVM 18 speaks RISC-V ABI v2.1 (whose linker identity string is incompatible with v2.0, which is quite counter-intuitive). The patch is needed for older binutils to correctly link the RISC-V binaries when the RISC-V targets contains these RISC-V standard CPU extensions: +I (i2p0) +M (i2p0) +A (i2p0) +F (i2p0) +D (i2p0) +Zicsr (i2p0 when using +G with binutils in 20.04, otherwise not understood with i2p1) +Zifencei +Zipause

[ Other Info ]

 * [Noble-only] A test rebuild using these packages was done as described in LP: #2076024.

 * [Noble-only] The llvm-18 testsuite run during the build doesn't show regressions.

 * [Noble-only] The update exposes an underlinking of bpftrace, which should be SRUed at the same time. See LP: #2081848

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Łukasz Zemczak (sil2100)]

Hello Matthias, or anyone else affected,

Accepted llvm-toolchain-18 into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/llvm-toolchain-18/1:18.1.8-9ubuntu1~24.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Daniel Richard G. (skunk)]

I tried building this package in a container environment, and ran into https://bugs.launchpad.net/bugs/2084316 .

(Summary: There is a check for a /proc file that is not needed by the build, but when it is inaccessible, it causes the build not to produce certain libraries and the absence of those files in the output causes the build to fail)

(ETA: To be clear, the reported bug is for the oracular package, but I get the same failure mode with this one for noble-proposed)

[Zixing Liu (liushuyu-011)]

Hi, I would like to propose the LLVM 18 backports for Jammy and Focal.

You can find the backports made here:

Jammy: https://code.launchpad.net/~liushuyu-011/ubuntu/+source/llvm-toolchain-18/+git/llvm-toolchain-18/+ref/ubuntu/jammy-devel

Focal: https://code.launchpad.net/~liushuyu-011/ubuntu/+source/llvm-toolchain-18/+git/llvm-toolchain-18/+ref/ubuntu/focal-devel

PPA builds:

Jammy: https://launchpad.net/~liushuyu-011/+archive/ubuntu/misc/+sourcepub/16614089/+listing-archive-extra

Focal: https://launchpad.net/~liushuyu-011/+archive/ubuntu/misc/+sourcepub/16527070/+listing-archive-extra

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted llvm-toolchain-18 (1:18.1.8-9ubuntu1~24.04) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

bpftrace/0.20.2-1ubuntu4 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#llvm-toolchain-18

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Zixing Liu (liushuyu-011)]

Please find a minor changelog entry fix to the PPA build (https://launchpad.net/~liushuyu-011/+archive/ubuntu/misc/+sourcepub/16527070/+listing-archive-extra) to include the SRU reference.

[Simon Chopin (schopin)]

I'm not going to upload this as I'm half-convinced this shouldn't have been uploaded to -proposed in the first place, due to https://wiki.ubuntu.com/StableReleaseUpdates#Toolchain_Updates (and I'm comforted in this by the stated rationale of Firefox updates, which are typically security-related).

Lukasz, could you chime in and confirm/infirm this reasoning?

Meanwhile, Zixing, your PPA doesn't seem to be security-read (it shouldn't have -proposed and -updates enabled).

From a purely cosmetic PoV, the .2 changelog entry should probably only have the bug reference and the .1 should be kept around, as it was published (in -proposed only, but still), but again, purely cosmetic, not a blocker for me.

[Łukasz Zemczak (sil2100)]

We do periodic toolchain updates like this one here, so essentially it's not wrong to request this. However I do agree with Simon and I'd prefer the llvm packages to be built in a clean, security-only PPA so that we can do a binary copy to -proposed. For both series these will be essentially NEW packages, but I would still prefer them to appear in both -updates and -security, due to their nature (and the nature of packages that could depend on them).

In general this can go through -proposed (for more exposure and testing), but with the idea in mind that this will go to -security. So, as mentioned above, we'd need to have it built in a PPA with only security enabled - and then synced from that.

[Zixing Liu (liushuyu-011)]

Asked for a review and sponsorship from the Security team.