[MIR] liburing

[Andreas Hasenack]

[Availability]
liburing is in universe in groovy at version 0.6-3 without Ubuntu Delta at the moment.
It builds for the Ubuntu architectures amd64, arm64, armhf, ppc64el, riscv64, s390x.

[Rationale]
liburing can be used for advanced asynchronous IO in qemu (>=5),
samba (>=4.12.x) and probably more down the road.
- https://lwn.net/Articles/776703/
- https://unixism.net/loti/
- https://github.com/axboe/liburing/

Since groovy is the first step towards 22.04 I think it would be great to
enable liburing now and see how things behave and if/how they are further
adopted.

[Security]

There was a CVE of the kernel side of the interface
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-19241
It is already handled and fixed in all Ubuntu releases:
https://people.canonical.com/~ubuntu-security/cve/2019/CVE-2019-19241.html

So far nothing else came up, but generally I/O interfaces are a good place
to exploit so there is an elevated risk I guess.

[Quality assurance]

The package has build time tests that are not yet working, so it ignores the
return value for now, but runs them to gather data. Mostly it seems permission
or kernel config errors.

It also has autopkgtests but those also miss permissions.

Note: I have forwarded an MP to Debian about the root permission at
build/test time.

Further all seems ok:
- No debconf questions.
- No long-term outstanding bugs.
- The package is maintained well in Debian/Ubuntu (sync, no open bugs)
- The package does not deal with exotic hardware (just very recent kernels)
- The package uses a debian/watch file
- not using python(2)
- symbols tracking is in place
- lintian --pedantic is rather happy

[UI standards]

this has no end-user UI, so no translations seem needed.

[Dependencies]

No other dependencies than libc6. This really is just a path to the kernel
and does not need many other components.

[Standards compliance]
- The package meets the FHS and Debian Policy standards.
- d/rules and d/control as small and well written

[Maintenance]

The Server team will subscribe for the package for maintenance

[Background]
quote https://unixism.net/loti/
"""
io_uring is a powerful new way to do asynchronous I/O programming under Linux.
Doing away with various limitations of previous generation I/O subsystems,
io_uring holds immense promise. For more details on what io_uring brings to
the table, please see the chapter What is io_uring?.
"""

[Andreas Hasenack (ahasenack)]

More context:
<andreas> brauner: hi, kernel uring, exciting?
<andreas> brauner: I'm wondering if it's too early to consider it for an MIR
<andreas> samba 4.12.x can use it
* trudd ([e-mail]) has joined
<brauner> andreas: a lot of people want it i'm sure. especially db people. but it is a lot of code and relatively new. it should be enabled by default anyway, no?
<andreas> brauner: what do you mean enabled by default? Where?
<andreas> in the kernel?
<brauner> andreas: i.e. it's a new feature that defaults to =y in the kernel
<brauner> andreas: yes
<andreas> ah, sure
<andreas> I was asking about the userspace library
<brauner> andreas: oh ok
<andreas> but yeah, also about the feature in general
<andreas> agreed with "it's new"
<brauner> andreas: so if you have the kernel stuff enabled you can likely enable the userspace stuff too
<brauner> andreas: the problem really is the kernel side default
<andreas> brauner: right, but it's in universe
<andreas> the userspace bit
<brauner> andreas: one thing to consider is that io_uring offloads unprivileged user workloads on async kernel threads
<brauner> andreas: and that's pretty scary
<brauner> andreas: it has seen some naste cves already
<andreas> cves in the kernel?
<brauner> andreas: yes
<andreas> interesting
<andreas> mind if I paste this conversation in the MIR bug I'm preparing?
<brauner> andreas: an obvious problem is that kernel threads run with kernel creds usually and io_uring needs to override them with the creator's cred (of the io_uring instance)
<brauner> andreas: and they forgot that at one point so ...
<brauner> andreas: that was the first cve
<brauner> andreas: no, go ahead
<andreas> it's my understanding this shared space is the big benefit of uring
<brauner> andreas: there's more to it than that but yes, it means you don't have a lot of context switches
<andreas> no data to copy between kernel and user space
<andreas> right
<brauner> andreas: you register work, kernel does it, notifies you when done. data is shared in mmaped buffers basically

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-19241

[Christian Ehrhardt  (paelzer)]

FYI: qemu >=5.0 also could make use of it:
From https://wiki.qemu.org/ChangeLog/5.0
"The file-posix driver can now use the io_uring interface of Linux with aio=io_uring"

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Ehrhardt  (paelzer)]

It seems that qemu even picked it up by default without liburing-dev being explicitly pulled in.
I guess we really want this, so while the tests are running let me prep the MIR for this.

[Christian Ehrhardt  (paelzer)]

Set the right state for the MIR bug to enter review.

[Christian Ehrhardt  (paelzer)]

[Summary]
MIR Team Ack, but please work on the tests to get working.
It will need a security review, but gladly is a very very small codebase.

TODOs:
- please continue to work on the tests to get them running mid term
- I'll reach out in the MIR team meeting since this was almost a full
  self review. In case we decide we need another full review we will do so.

[Duplication]
There is no other package in main providing the same functionality.

[Dependencies]
OK:
- no other Dependencies to MIR due to this
- no -dev/-debug/-doc packages that need exclusion

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not open a port
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)

Problems:
- does parse data formats

It doesn't really do much other than wrapping a complex kernel interface.
But I/O always is somewhat security sensitive as people could manipulate the
underlying data and/or devices in some way.
Therefore I'd ask for a security review - gladly it is just 1680 lines of code
and about half of it is a copy of a few kernel headers. So this should really
be a quick reivew.

[Common blockers]
- does not FTBFS currently
- does have a test suite that runs at build time
- does have a test suite that runs as autopkgtest
- The package has a team bug subscriber
- no translation present, but none needed for this case (user visible)?
- not a python package, no extra constraints to consider int hat regard
- no new python2 dependency

Problems:
- test suite does not fail the build upon error.

[Packaging red flags]
OK:
- Ubuntu does not carry a delta
- symbols tracking is in place
- d/watch is present and looks ok
- Upstream update history is (rather new, good so far)
- Debian/Ubuntu update history is (rather new, good so far)
- the current release is packaged
- promoting this does not seem to cause issues for MOTUs that so far
  maintained the package
- no massive Lintian warnings
- d/rules is rather clean
- not using Built-Using
- Does not have Built-Using

[Upstream red flags]
OK:
- no Errors/warnings during the build
- no incautious use of malloc/sprintf (other than the tests just one)
- no use of sudo, gksu, pkexec, or LD_LIBRARY_PATH
- no use of user nobody
- no use of setuid
- no important open bugs (crashers, etc) in Debian or Ubuntu
- no dependency on webkit, qtwebkit, seed or libgoa-*
- no embedded source copies
- not part of the UI for extra checks

[Christian Ehrhardt  (paelzer)]

I did the full formal part of the review, it is ok and now is on security to take a look.

I'll myself work on checking if the testcases can be made working (at least better than atm).

[Christian Ehrhardt  (paelzer)]

FYI: Tests are fixed and reported to Debian https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964077

[Alex Murray (alexmurray)]

I reviewed liburing 0.6-3 as checked into groovy. This shouldn't be
considered a full audit but rather a quick gauge of maintainability.

liburing is a C library to help setup and remove io_uring instances, used
to perform efficient asynchronous communication between userspace and the
kernel.

- No CVE History
- No security relevant Build-Depends
  - debhelper-compat, procps
- No pre/post inst/rm scripts
- No init scripts
- No systemd units
- No dbus services
- No setuid binaries
- No binaries in PATH
- No sudo fragments
- No polkit files
- No udev rules
- No cron jobs
- Extensive unit tests / autopkgtests
  - Package has more than 10x amount of test code than actual library code
  - Runs both the unit test suite and example code as autopkgtests
  - Autopkgtests currently fail and are ignored...
  - Currently unit test failures are ignored with the following entry in
    debian/rules - this should be resolved before this package is promoted
    to main() as otherwise regressions could easily be introduced with no
    warning
      # XXX: The tests do not pass yet on Linux 5.5, and they are not being
      # skipped either.
      override_dh_auto_test:
          dh_auto_test -- runtests || true

- Build logs:
  - Build logs show unit test failures
  - No significant lintian failures

- No processes spawned
- No dynamic memory management (except for at probe)
- No file IO
- No logging
- No environment variable usage
- No use of privileged functions
- No use of cryptography / random number sources etc
- No use of temp files
- No use of networking
- No use of WebKit
- No use of PolicyKit

- No significant cppcheck results
  - only some warnings of memory leaks in the test code
- No significant Coverity results
- No significant shellcheck results
- No significant bandit results

Whilst still a very new library, liburing appears to be well written and
relatively defensive, and given its minimal scope, does not appear to
provide any real attack surface or similar. It is well tested, but the
current tests are not well maintained in Ubuntu.

Security team ACK for promoting liburing to main, conditional on ensuring
the autopkgtests are fixed so that they are in a known state and can be
used for regression testing etc on future updates.

[Andreas Hasenack (ahasenack)]

I'll pull into the ubuntu package the patches @paelzer sent to debian

[Christian Ehrhardt  (paelzer)]

I'll upload the test fixes I submitted to Debian after retesting them for Ubuntu Groovy

[Christian Ehrhardt  (paelzer)]

Tests work in a local qemu autopkgtest, infra tests are odd since the (so far) never worked they are skipped on bileto: https://bileto.ubuntu.com/excuses/4129/groovy.html

Therefore an upload to groovy should be as good or better and will set the new baseline.

[Launchpad Janitor (janitor)]

This bug was fixed in the package liburing - 0.6-3ubuntu1

---------------
liburing (0.6-3ubuntu1) groovy; urgency=medium

  * Fix build and autopkgtest self tests (LP: #1878006).

 -- Christian Ehrhardt <email address hidden> Wed, 01 Jul 2020 11:09:38 +0200

[Christian Ehrhardt  (paelzer)]

The tests are complete, MIR and Security Team ack are present.
The change pulling this into main is present in the groovy and it shows up as component mismatch.

We just added the Team subscription so this is ready to be promoted now.

Subscribing ubuntu-archive to do so.

[Sebastien Bacher (seb128)]

Override component to main
liburing 0.6-3ubuntu1 in groovy: universe/misc -> main
liburing-dev 0.6-3ubuntu1 in groovy amd64: universe/libdevel/optional/100% -> main
liburing-dev 0.6-3ubuntu1 in groovy arm64: universe/libdevel/optional/100% -> main
liburing-dev 0.6-3ubuntu1 in groovy armhf: universe/libdevel/optional/100% -> main
liburing-dev 0.6-3ubuntu1 in groovy ppc64el: universe/libdevel/optional/100% -> main
liburing-dev 0.6-3ubuntu1 in groovy riscv64: universe/libdevel/optional/100% -> main
liburing-dev 0.6-3ubuntu1 in groovy s390x: universe/libdevel/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy amd64: universe/libs/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy arm64: universe/libs/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy armhf: universe/libs/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy ppc64el: universe/libs/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy riscv64: universe/libs/optional/100% -> main
liburing1 0.6-3ubuntu1 in groovy s390x: universe/libs/optional/100% -> main