Update dwarves-dfsg in focal to version 1.21 from impish

[Seth Forshee]

[Impact]

BTF is an extremely useful tool for BPF developers. Enabling BTF support in the HWE kernel requires pahole from dwarves-dfsg 1.21 or later, but in focal we have only 1.15. The simplest path for us to get a sufficiently updated version of pahole is to backport dwarves-dfsg from hirsute, which is at version 1.21 with v5.13 compat fixes.

[Test Case]

Building a v5.13 based kernel with CONFIG_DEBUG_INFO_BTF enabled fails with the version of pahole currently in focal, complaining that pahole 1.16 or later is needed. Building with a newer version of pahole is successful.

[Where problems could occur]

Moving the package forward carries significant risks. The biggest risk is clearly new bugs. There may also be interface changes which break existing workflows. I believe the following factors indicate that the level of risk here is acceptable:

 * Currently there are no bug reports open against the package in groovy (there are virtually no bug reports open at all against the package in fact).

 * I have done a review of the changes between 1.15 and 1.21. The bulk of the changes are for the required BTF support, and the vast majority of these are an update of its local copy of libbpf. The balance of the changes are bug fixes and enhancements to pahole, none of which look to change the user interface in a way which will break existing workflows. Note that in later releases libbpf is now packaged stand alone, so this SRU covers libbpf as well.

 * But also the 1.15 dwarves is kind of useless, as it doesn't work with latest kernels for the BPF features.

 * No packages in focal have dependencies or build dependencies on the dwarves package.

[Dimitri John Ledkov (xnox)]

Preparing packages in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4619/+packages

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Łukasz Zemczak (sil2100)]

With my SRU hat on, I'm generally +1 on pulling in such a big new upstream version to focal, but sadly I don't feel too comfortable for us to do that without a bit more complete test case. I don't think I ever saw an SRU for dwarves-dfsg before, so I'm also not sure of what workflows we need to keep in mind, but would it be possible to perform some exploratory testing as part of the verification? It just feels to me that only a test build of the kernel is not enough for such a huge jump - even if the functional changes in the new version is not as big as one could think.

Can someone work out a better test plan for this?

[TuxInvader (tuxinvader)]

Couldn't this be treated like other packages for which multiple versions exist using the "alternatives system". Ie add the newer dwarves package required for new kernels as dwarves1.21 and run update-alternatives afterwards? Anyone out there who uses dwarves for anything other than the kernel would be able to keep the current focal version. dwarves1.21 would only be required by the kernel source package, and users could find it themselves if they want to build from kernel.org?

[DanglingPointer (ferncasado)]

Need this fixed as we're jumping through hoops to get newer kernels into 20.04!

[Robie Basak (racb)]

I understand that this type of update is needed to support the HWE kernels in Focal, and to that end I think it's in principle acceptable to SRU this under the HWE kernel exception we have already.

However, now that we seem to have a plan to revitalise the backports pocket, I wonder if that might be a more suitable target for this update, on the basis of the regression risk for existing users. The biggest different is that users have to opt in to receiving the backport, but perhaps that isn't a big hurdle for the developer use case for this package?

To be clear, it's not my intention to close off the path to -updates here, if you really want that; I'm just presenting -backports as an alternative option which might be a safer approach if that works for everyone.

To consider -updates we need an answer to Lukasz's question in comment 7, so I'm marking the task as Incomplete accordingly.

[Dimitri John Ledkov (xnox)]

backports is no good, as it needs to be kernel's build-dep. and most kernels are built in -security only pocket.

also, nothing else really uses these, apart from kernel. And kernels have stop using these, because they are too old in focal.

[Dimitri John Ledkov (xnox)]

groovy is EOL hence marking groovy task wont-fix.

[Steve Langasek (vorlon)]

> also, nothing else really uses these, apart from kernel.

It's not true that nothing else uses libbpf besides the kernel, and you're calling for a new upstream release of libbpf with an 11kloc debdiff. I don't think this falls under the HWE exception, and a better test case is needed before this can be accepted for hirsute.

(For focal, libbpf is a new package and regression-testing is therefore not a problem.)

[Dimitri John Ledkov (xnox)]

libbpf is meant to work with the bpf that kernel provides. In focal, newer versions of bpf toolchain are provided, because llvm is regularly backported. In focal, libbpf used to be vendored inside dwarves-dfsg and it is desired as a new package to i.e. build linux bpf tools, and because focal receives hwe kernel backports from impish.

From kernel point of view, I don't need new libbpf in hirsute, because hirsute does not receive hwe kernels.

Can I update libbpf in focal, without having it in hirsute?

Would it help to vendor newer libbpf inside src:dwarves-dfsg for the SRU purposes, as it shipped in focal? Such that libbpf newer than hirsute does not appear in focal?

[Dimitri John Ledkov (xnox)]

Uploaded new dwarves-dfsg SRUs that use embeded libbpf (which in turn is updated to the same source as used in impish).

This makes dwarves-dfsg SRU stand-alone, without introducing or upgrading the system-wide libbpf.

[Steve Langasek (vorlon)]

Hello Seth, or anyone else affected,

Accepted dwarves-dfsg into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dwarves-dfsg/1.21-0ubuntu1~21.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Seth, or anyone else affected,

Accepted dwarves-dfsg into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dwarves-dfsg/1.21-0ubuntu1~20.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[TuxInvader (tuxinvader)]

Hello,

I can confirm that with the dwarves package in proposed (1.21-0ubuntu1~20.04) I can successfully build the Ubuntu mainline kernel from source (tested version 5.13.19) using git commit from: https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.13.19/

When using the version of dwarves currently in focal (1.15-2) the build fails with:

  AR drivers/gpu/built-in.a
  AR drivers/built-in.a
  GEN .version
  CHK include/generated/compile.h
  LD vmlinux.o
  MODPOST vmlinux.symvers
  MODINFO modules.builtin.modinfo
  GEN modules.builtin
BTF: .tmp_vmlinux.btf: pahole version v1.15 is too old, need at least v1.16
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
make[2]: *** [/home/source/Makefile:1216: vmlinux] Error 1
make[2]: Leaving directory '/home/source/debian/build/build-generic'
make[1]: *** [Makefile:220: __sub-make] Error 2
make[1]: Leaving directory '/home/source'
make: *** [debian/rules.d/2-binary-arch.mk:53: /home/source/debian/stamps/stamp-build-generic] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2

[Dimitri John Ledkov (xnox)]

Using 1.21-0ubuntu1~20.04 and 1.21-0ubuntu1~21.04 it was possible to create BTF enabled kernels on all architectures.

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for dwarves-dfsg has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dwarves-dfsg - 1.21-0ubuntu1~20.04

---------------
dwarves-dfsg (1.21-0ubuntu1~20.04) focal; urgency=medium

  * Backport new dwarves-dfsg to stable series without upgrading libbpf
    system-wide (LP: #1912811):
    + Update lib/bpf to libbpf_0.4.0-1ubuntu1.
    + Disable lto, since it is disabled in libbpf.
    + Build with embedded libbpf.
    + Drop libbpf-dev build dependency.
    + Lower debhelper-compat to 12.

 -- Dimitri John Ledkov <email address hidden> Mon, 20 Sep 2021 13:10:02 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package dwarves-dfsg - 1.21-0ubuntu1~21.04

---------------
dwarves-dfsg (1.21-0ubuntu1~21.04) hirsute; urgency=medium

  * Backport new dwarves-dfsg to stable series without upgrading libbpf
    system-wide (LP: #1912811):
    + Update lib/bpf to libbpf_0.4.0-1ubuntu1.
    + Disable lto, since it is disabled in libbpf.
    + Build with embedded libbpf.
    + Drop libbpf-dev build dependency.

 -- Dimitri John Ledkov <email address hidden> Mon, 20 Sep 2021 13:10:02 +0100

[Chris Halse Rogers (raof)]

Hello Seth, or anyone else affected,

Accepted dwarves-dfsg into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dwarves-dfsg/1.21-0ubuntu1~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Dimitri John Ledkov (xnox)]

Built linux-hwe-5.4 in bionic with BTF enabled and proposed enabled:

Build used dwarves (= 1.21-0ubuntu1~18.04)

$ zgrep BTF buildlog_ubuntu-bionic-amd64.linux-hwe-5.4_5.4.0-90.101~18.04.1_BUILDING.txt.gz
  BTF .btf.vmlinux.bin.o
  BTF .btf.vmlinux.bin.o

BTF turned on correctly.

See builds in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4702

[Launchpad Janitor (janitor)]

This bug was fixed in the package dwarves-dfsg - 1.21-0ubuntu1~18.04

---------------
dwarves-dfsg (1.21-0ubuntu1~18.04) bionic; urgency=medium

  * Backport new dwarves-dfsg to stable series without upgrading libbpf
    system-wide (LP: #1912811):
    + Update lib/bpf to libbpf_0.4.0-1ubuntu1.
    + Disable lto, since it is disabled in libbpf.
    + Build with embedded libbpf.
    + Drop libbpf-dev build dependency.
    + Lower debhelper-compat to 11.
    + Cherrypick patch from 1.22 that fixes FTBFS with old elfutils

dwarves-dfsg (1.21-0ubuntu1) impish; urgency=medium

  * New upstream release. Remaining changes:
    - pahole: cherry-pick removal of the ftrace filter and its related
    functions to prevent build errors with linux 5.13.
    - build with libbpf 0.4, as will be required in the next release.

dwarves-dfsg (1.20-1ubuntu1) impish; urgency=medium

  * pahole: remove the ftrace filter and its related functions to prevent
    build errors with linux 5.13 (LP: #1928244)

dwarves-dfsg (1.20-1) unstable; urgency=medium

  * New upstream release.
    Changes since 1.19:

    BTF encoder:
      - Improve ELF error reporting using elf_errmsg(elf_errno()).
      - Improve objcopy error handling.
      - Fix handling of 'restrict' qualifier, that was being treated
        as a 'const'.
      - Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF
        objects with more than 65534 sections, for instance, which
        happens with kernels built with 'KCFLAGS="-ffunction-sections
        -fdata-sections", Other cases may include when using FG-ASLR, LTO.
      - Cope with functions without a name, as seen sometimes when
        building kernel images with some versions of clang, when a
        SEGFAULT was taking place.
      - Fix BTF variable generation for kernel modules, not skipping
        variables at offset zero.
      - Fix address size to match what is in the ELF file being
        processed, to fix using a 64-bit pahole binary to generate BTF
        for a 32-bit vmlinux image.
      - Use kernel module ftrace addresses when finding which functions
        to encode, which increases the number of functions encoded.
    DWARF loader:
      - Support DW_AT_data_bit_offset
      - DW_FORM_implicit_const in attr_numeric() and attr_offset()
      - Support DW_TAG_GNU_call_site, its the standardized rename of
        the previously supported DW_TAG_GNU_call_site.
    build:
        - Fix compilation on 32-bit architectures.

  * Refresh patches.
  * Remove ctfdwdiff and README.cross so to reuse upstream tarball as-is.
  * Update Standards-Version to 4.5.1
  * Update debhelper compatibility to 13:
    - install missing binaries: btfdiff and fullcircle
  * Add upstream metadata file
  * gbp: set upstream tarball compression to xz
  * Add lintian override for orig-tarball-missing-upstream-signature,
    see #882694.

dwarves-dfsg (1.19-1) unstable; urgency=high

  * New upstream release. Closes: #978691.
    Changes since 1.18:
    - Support split BTF, where a main BTF file, vmlinux, can be used
      to find types and then a kernel module, for instance, can have
      just what is unique to it
    - Update libbpf to get the spli...

[Christian Ehrhardt  (paelzer)]

FYI potential regression in release due to that at bug 1951438