[MIR] bpfcc

Bug #2052813 reported by Mate Kukri
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
bpfcc (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

[Availability]
- The package bpfcc is already in Ubuntu universe.
- The package bpfcc build for the architectures it is designed to work on.
- It currently builds and works for architectures: any
- Link to package https://launchpad.net/ubuntu/+source/bpfcc

[Rationale]
- The package bpfcc is required in Ubuntu main as a runtime dependency of
  bpftrace.
- There is no other/better way to solve this that is already in main or
  should go universe->main instead of this.
- The package bpftrace is required in Ubuntu main as part of the Noble Numbat
  realease, and hence should be promoted to main before NN feature freeze.

[Security]
- No CVEs/security issues in this software in the past
- no `suid` or `sgid` binaries
- Binaries *-bpfcc in sbin are no problem because they are part of the
  expected functionality of the package
- Package does not install services, timers or recurring jobs
- Security has been kept in mind and common isolation/risk-mitigation
  patterns are in place utilizing the following features:
  the package is a debugging tool, and cannot be fully isolated.
- Packages does not open privileged ports (ports < 1024).
- Package does not expose any external endpoints
- Packages does not contain extensions to security-sensitive software
  (filters, scanners, plugins, UI skins, ...)

[Quality assurance - function/usage]
- The package works well right after install

[Quality assurance - maintenance]
- The package is maintained well in Debian/Ubuntu/Upstream and does
  not have too many, long-term & critical, open bugs
  - Ubuntu https://bugs.launchpad.net/ubuntu/+source/bpfcc/+bug
  - Debian https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=bpfcc
  - Upstream's bug tracker, e.g., GitHub Issues
    https://github.com/iovisor/bcc/issues
- The package has important open bugs, listing them:
   https://bugs.launchpad.net/ubuntu/+source/bpfcc/+bug/1969626
- The package does not deal with exotic hardware we cannot support

[Quality assurance - testing]
- The package does not run a test at build time
  (Potential issue?)
- The package does not run an autopkgtest
  (Potential issue?)
- The package does have not failing autopkgtests right now

[Quality assurance - packaging]
- debian/watch is present and works
- debian/control defines a correct Maintainer field
- This package does not yield massive lintian Warnings, Errors
- Please link to a recent build log of the package
  https://launchpadlibrarian.net/703777424/buildlog_ubuntu-noble-amd64.bpfcc_0.29.1+ds-1ubuntu2_BUILDING.txt.gz
- Please attach the full output you have got from `lintian --pedantic` as an
  extra post to this bug:
  ```
  E: bpfcc changes: bad-distribution-in-changes-file noble
  W: bpfcc source: no-nmu-in-changelog [debian/changelog:1]
  W: bpfcc source: source-nmu-has-incorrect-version-number 0.29.1+ds-1ubuntu2 [debian/changelog:1]
  P: bpfcc source: trailing-whitespace [debian/changelog:41]
  ```
- Lintian overrides are present, they disable warnings about the lack of manpages
  (Potential issue?)
- This package does not rely on obsolete or about to be demoted packages.
- This package has no python2 or GTK2 dependencies
- The package will not be installed by default
- Packaging and build is easy, link to debian/rules
  https://git.launchpad.net/ubuntu/+source/bpfcc/tree/debian/rules?h=ubuntu/noble

[UI standards]
- Application is not end-user facing (does not need translation)

[Dependencies]
- No further depends or recommends dependencies that are not yet in main

[Standards compliance]
- This package correctly follows FHS and Debian Policy

[Maintenance/Owner]
- The owning team will be Foundations and I have their acknowledgement for
  that commitment
- The future owning team is not yet subscribed, but will subscribe to
  the package before promotion
- This does not use static builds
- This does not use vendored code
- This package is not rust based
- The package successfully built during the most recent test rebuild

[Background information]
- The Package description explains the package well
- Upstream Name is bcc
- Link to upstream project bcc: https://github.com/iovisor/bcc

Tags: sec-3897

CVE References

Revision history for this message
Mate Kukri (mkukri) wrote :

This has some potential issues but I am opening a bug about it to have somewhere to discuss.

Changed in bpfcc (Ubuntu):
assignee: nobody → Christian Ehrhardt  (paelzer)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Download full text (9.5 KiB)

Review for Source Package: bpfcc

[Summary]
MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

I'm unsure if this needs a security review, I'll be back with an update
on that once I've consulted security.

List of specific binary packages to be promoted to main: libbpfcc

Specific binary packages built, but NOT to be promoted to main: python3-bpfcc,
bpfcc-tools, libbpf-tools, bpfcc-introspection, bpfcc-lua
Those are only excluded for not beeing needed in the dependency chain right now.
They are ok to be promoted later as needed and especially the collection
of tools can be quite handy for users to start with bpf.

Notes:
- #3 I'll need to check with the security team if they think this needs
  their review as I'm unsure about that detail.

Required TODOs:
- #1 Please get Doko or Samir to confirm that it is ok in their
  POV to also promote libclang-cpp17 as part of this.
- #5 Please have a look if some of the tests could run at build time (chance is
  low due to the kernel dependency.
- #6 Please do wrap the provided tests/* into an autopkgtest against a system
  and kernel and dependencies of the same release. (more below)
- #7 There is no versioned library package nor symbols tracking. It isn't meant
  to be stable as shown on soname 0.x, but adding this to main means we might
  patch things along the way and some certainty if we by accident break API/ABI
  is a nice feature. Please have a look at adding that, symbols tracking for
  sure and considering proper soname versioned package for the lib at lower
  priority and in sync with Debian later on.
  If you believe symbols tracking isn't the way, have a look at
  abi-compliance-check or abigail or anything else that will solve the same.

Recommended TODOs:
- #2 while none is super critical, the list as of today is still of reasonable
  size. This would be your chance to triage all of the open bugs
  so that you can from now on consider it fully under control with ongoing
  triage when it is added to main.
  So please consider doing a full triage (and fix those worth) run of
  https://bugs.launchpad.net/ubuntu/+source/bpfcc/+bugs

- #4 The package should get a team bug subscriber before being promoted

[Rationale, Duplication and Ownership]
- There is no other package in main providing the same functionality.
  This is not in conclict with libbpf, it is using libbpf to provide
  the next layer of value on top.
  It is somewhat related to gcc-bpf, but libbpfcc is what the ecosystem
  seems to rely on more.
  Further there is overlap with perf-tools-unstable which inspired some
  tools now found here, but perf-tools-unstable is not in main either and
  less maintained these days, hence chosing bpfcc is the right choice AFICS.

- A team is committed to own long term maintenance of this package.
  => Foundations

- The rationale given in the report seems valid and useful for Ubuntu
  BPF use cases are useful in testing and analysis and slowly replace older
  tools by being more efficient or able to do things not possible without BPF.
  This completes the suite of programs and packages aiming for this to b...

Read more...

Changed in bpfcc (Ubuntu):
status: New → Incomplete
assignee: Christian Ehrhardt  (paelzer) → Mate Kukri (mkukri)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

We discussed the need or lack of need for security review here.

[16:38] <sarnold> I'm not sure either; on the one hand, administrative privilege is required to run these, so there's a thin barrier at best
[16:38] <sarnold> most of the security layer happens in the kernel
[16:39] <sarnold> I believe that this package itself is very little risk to the security team, but the kernel portion might -- so, I'm inclined to say that this doesn't need security team review
[16:39] <eslerm> a quick review might remove some footguns
[16:40] <cpaelzer> eslerm: is there a good way to express "we should have a quick check but not a full review"
[16:41] <cpaelzer> how about you volunteer for that "quick but not full" check
[16:41] <cpaelzer> then the solution is that I'll assign you
[16:41] <cpaelzer> actually it is back with mkukri so I'd subscribe you
[16:41] <eslerm> a short audit might find something useful to report upstream, it might just be bugs, if the security context cannot be made worse by bugs
[16:42] <eslerm> I can do that
[16:42] <cpaelzer> thank you

Revision history for this message
Steve Langasek (vorlon) wrote :

llvm-toolchain-17 is in main and there's nothing special about its libraries for MIR. This is an ordinary binary promotion.

Mark Esler (eslerm)
tags: added: sec-3897
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

> llvm-toolchain-17 is in main and there's nothing special about its libraries for MIR. This is an ordinary binary promotion.

Perfectly fine with me, thanks for confirming.
One less detail to think about.

Revision history for this message
Mate Kukri (mkukri) wrote :

#2 i had a look through the bug tracker, but some of the bugs are rather difficult to triage given that they reference years old Ubuntu releases

#5 some probably could, but I don't have the understanding needed of the BPF ecosystem to pull this off

#6 see attached patch for this, unfortunately some of these tests do still fail currently

#7 unfortunately this seems rather difficult to do as libbpfcc seem to export a large number of c++ symbols

Changed in bpfcc (Ubuntu):
status: Incomplete → Opinion
status: Opinion → In Progress
assignee: Mate Kukri (mkukri) → nobody
Revision history for this message
Seth Arnold (seth-arnold) wrote :

> Specific binary packages built, but NOT to be promoted to main: python3-bpfcc,
> bpfcc-tools, [...]

I would have thought that getting these tools would have been the entire point of this MIR. There's an immense amount of value built in them, and without the tools we've got the framework but no pre-built way to consume it. These pre-built tools are 99% of why I want this package promoted.

(A similar question was raised for the bpftrace package -- we might as well just stop reviewing these packages if we remove the actual tools from the packages.)

What's the rationale for not promoting the tools?

Thanks

Revision history for this message
Mate Kukri (mkukri) wrote :

I believe we do want to promote bpfcc-tools as well, at least that binary name was mentioned before, but this package is also needed as a dependency for bpftrace too.

Revision history for this message
Mark Esler (eslerm) wrote (last edit ):

Promoting bpfcc-tools and bpftrace is driving promotion of bpfcc based on FO147.

Also, bpftrace's /usr/sbin/*.bt files re-implement bpfcc-tools with bpftrace.

Assigning to Security for MIR, with root-use scope kept in mind. Only code for libbpfcc and bpfcc-tools will be reviewed.

Changed in bpfcc (Ubuntu):
assignee: nobody → Ubuntu Security Team (ubuntu-security)
Revision history for this message
Mark Esler (eslerm) wrote :

Máté, could you please see if the rational can be broadened for FO147?

I suspect that libbpf-tools is also important.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 2052813] Re: [MIR] bpfcc

On Wed, Feb 28, 2024 at 5:00 AM Mark Esler <email address hidden> wrote:
>
> Máté, could you please see if the rationale can be broadened for FO147?
> I suspect that libbpf-tools is also important.

As far as I can see it is even the same functionality and the same
code, just built differently to work with less dependencies.
So a review of quality and maintainability of the source should cover
both anyway (with the difference being in the build instructions).

Revision history for this message
Mark Esler (eslerm) wrote :

Some of the bpf tools do not work on mantic.

e.g. `/usr/sbin/tcptop-bpfcc` from `bpfcc-tools` does not work, but `/usr/sbin/tcptop` from `libbpfcc` does (on mantic)

Kernel configs and pahole version used to build mantic's kernel should be okay https://github.com/iovisor/bcc/tree/master/libbpf-tools ?

Revision history for this message
Mate Kukri (mkukri) wrote :

This should bring some working autopkgtests to this package by disabling a set of failing tests.

Revision history for this message
Mate Kukri (mkukri) wrote :

Added DEP3 header

Revision history for this message
Mark Esler (eslerm) wrote :
Download full text (3.3 KiB)

I reviewed bpfcc 0.29.1+ds-1ubuntu2 as checked into noble. This shouldn't be
considered a full audit but rather a quick gauge of maintainability.

- CVE History
  - no CVEs tracked in UCT, initially
  - searching for "bcc" CVEs finds false-positives
- Build-Depends
  - nothing concerning
- pre/post inst/rm scripts
  - typical dh_python3 for python3-bpfcc
- init scripts
  - none
- systemd units
  - none
- dbus services
  - none
- setuid binaries
  - none
- binaries in PATH
  - numerous. +220.
- sudo fragments
  - none
- polkit files
  - none
- udev rules
  - none
- unit tests / autopkgtests
  - some added
- cron jobs
  - none
- Build logs
  - hardening-no-pie is not a concern in this case
  - manual page warnings
  - W: libbpfcc: package-name-doesnt-match-sonames libbcc-bpf0 libbcc0

- Processes spawned
  - popen use looks okay
  - system("clear") is fine
  - memleak.c uses fork, etc
- Memory management
  - extremely heavy use
  - in context, I am not concerned with occult practices in this package
- File IO
  - heavy use
- Logging
  - extremely heavy use
- Environment variable usage
  - none
- Use of privileged functions
  - Security's MIR tooling finds many false-positives
  - vmlinux headers are fine
- Use of cryptography / random number sources etc
  - none
  - vminux*.h sets certificate configs
- Use of temp files
  - tmp race conditions possibly allow unauthenticated users to control unpacked kernel headers
    - Resolved quickly by upstream! CVE-2024-2314
    - see related issue in bpftrace MIR (LP#2052809)
- Use of networking
  - heavy use
- Use of WebKit
  - none
- Use of PolicyKit
  - none

- Any significant cppcheck and Covreity results
  - bugs found (memory leaks etc), but not concerned about these being vulnerabilities in context
  - parsing untrusted data (e.g., network traffic) could possibly lead to exploitation
  - coverity.txt attached
- Any significant shellcheck results
  - not concerning
- Any significant bandit results
  - none
  - subprocess calls cannot be controlled without root access
- Any significant govulncheck results
  - none
- Any significant Semgrep results
  - none
  - complaints about system() and strtok excused in context

There is 986,872 loc. Security's review is limited.

As with bpftrace, these are admin tools which require root access. It is unlikely that most bugs in bpfcc would cause a loss of security and become a vulnerability; root already has control. Parsing untrusted data with a root process can lead to trouble. This review expects that developers will want to use these tools and that system administrators will make wise choices.

Some binaries do not work out of box. This needs testings. e.g., /usr/sbin/tcptop-bpfcc from bpfcc-tools does not work, but /usr/sbin/tcptop from libbpfcc does.

Binaries from bpfcc-tools, libbpfcc, and bpftrace have redundant functions. Please consider which binaries should be made default. In particular, most bpftrace binaries are merely examples.

The bcc snap is published by Canonical and should be updated. See ./snap/README.md

Upstream was extraordinarily quick at addressing a potential security issue which was reported to them \o/
 - CVE-2024-2314

Security t...

Read more...

Changed in bpfcc (Ubuntu):
assignee: Ubuntu Security Team (ubuntu-security) → nobody
Revision history for this message
Lukas Märdian (slyon) wrote :

Security review OK (comment #15). I subscribed ~foundations-bugs.

Issue #1 confirmed in comment #4 (and now upgraded to LLVM-18 by doko)
Issue #5 probably not possible due to kernel dependency
Issue #6 autopkgtests sponsored: https://launchpad.net/ubuntu/+source/bpfcc/0.29.1+ds-1ubuntu6

Issue #7 still open, consider using something like c++filt, https://wiki.debian.org/UsingSymbolsFiles#C.2B-.2B-_libraries

Changed in bpfcc (Ubuntu):
status: In Progress → Incomplete
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thank you for adding all the QA and clarifying more.

When you change dependencies to pull this in (I see Ubuntu-meta already did change).

This is trying to get the value to the users, but not clutter where possible.
I think we should depend on libbpf-tools instead of the current bpfcc-tools.

WDYT? Could we do that change before doing promotions?

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I'll not mark it "in progress" yet waiting for the "which -tools" to use.
But it would be ready once you settled.

Revision history for this message
Robie Basak (racb) wrote :

The main user story being implemented here is the one specified by Brendan Gregg in his literature, including his book "Systems Performance, 2nd Edition". In that book he specifies "Linux crisis tool packages" [Table 4.1], which specifies bpfcc-tools and a bunch of binaries provided by that.

Looking at the difference between libbpf-tools and bpfcc-tools, the former is only a subset and doesn't include all the tools specified in the book. For example, the book covers trace and argdist in section 15.1.6. These have -bpfcc versions provided by bpfcc-tools, but I don't see any equivalents in libbpf-tools.

The intention is that we install by default what the book covers. Therefore, I'd like to request (at least initially) that we have bpfcc-tools in main. If that's not possible then libbpf-tools will be better than nothing, but will add to the caveats in this entire effort.

Please could you consider the MIR request to be for the bpfcc-tools binary package?

Revision history for this message
Robie Basak (racb) wrote :

> [Rationale]
> - The package bpfcc is required in Ubuntu main as a runtime dependency of
> bpftrace.

Correction: this is true, but we'd also like bpfcc-tools in its own right since these tools are recommended by experts for direct use.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thank you for the check and explanation.

While that is sad for size (80m vs 20m) it makes sense for the purpose.
And this isn't in minimal so it seems sort of ok to me.

Trying to unblock things this seems ready now.

Changed in bpfcc (Ubuntu):
status: Incomplete → In Progress
status: In Progress → Fix Committed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Dependency is shown in mismatches, setting to Fix Committed.

I thought in the current situation we'd need to promote all (-release, -updates, -proposed), but on bpftrace only -release was moved. Let me check on that before that might interfere with the current archive activities.

If anyone else comes by knowing for sure, this can be promoted (src:bpfcc bin:libbpfcc bin:python3-bpfcc bin:bpfcc-tools).

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Download full text (4.8 KiB)

Clarified, moving -release and -proposed as -updates will be deleted anyway

Also libbpfcc-dev is an auto-include and has no weird dependencies that make us need to exclude it.

Override component to main
bpfcc 0.29.1+ds-1ubuntu4 in noble: universe/misc -> main
Override [y|N]? y
1 publication overridden.

Override component to main
bpfcc 0.29.1+ds-1ubuntu6 in noble: universe/misc -> main
Override [y|N]? y
1 publication overridden.

Override component to main
libbpfcc 0.29.1+ds-1ubuntu4 in noble arm64: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu4 in noble armhf: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu4 in noble ppc64el: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu4 in noble riscv64: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu4 in noble s390x: universe/misc/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble amd64: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble arm64: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble armhf: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble i386: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble ppc64el: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble riscv64: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu4 in noble s390x: universe/python/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble amd64: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble arm64: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble armhf: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble i386: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble ppc64el: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble riscv64: universe/misc/optional/100% -> main
bpfcc-tools 0.29.1+ds-1ubuntu4 in noble s390x: universe/misc/optional/100% -> main
libbpfcc-dev 0.29.1+ds-1ubuntu4 in noble arm64: universe/libdevel/optional/100% -> main
libbpfcc-dev 0.29.1+ds-1ubuntu4 in noble armhf: universe/libdevel/optional/100% -> main
libbpfcc-dev 0.29.1+ds-1ubuntu4 in noble ppc64el: universe/libdevel/optional/100% -> main
libbpfcc-dev 0.29.1+ds-1ubuntu4 in noble riscv64: universe/libdevel/optional/100% -> main
libbpfcc-dev 0.29.1+ds-1ubuntu4 in noble s390x: universe/libdevel/optional/100% -> main
Override [y|N]? y
24 publications overridden.

Override component to main
libbpfcc 0.29.1+ds-1ubuntu6 in noble amd64: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu6 in noble arm64: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu6 in noble armhf: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu6 in noble ppc64el: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu6 in noble riscv64: universe/misc/optional/100% -> main
libbpfcc 0.29.1+ds-1ubuntu6 in noble s390x: universe/misc/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu6 in noble amd64: universe/python/optional/100% -> main
python3-bpfcc 0.29.1+ds-1ubuntu6 in noble arm64:...

Read more...

Changed in bpfcc (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.