[MIR] s390-tools Rust dependencies (vendored)

[Lukas Märdian]

[Availability]
The package s390-tools is already in Ubuntu main, and is re-reviewed due to signinficant changes in the package (new Rust code-base, including vendored dependencies).
The package s390-tools builds for the architectures it is designed to work on.
It currently builds and works for architectures: s390x, and to a much more limited extent, amd64, arm64 and ppc64el
Link to package https://launchpad.net/ubuntu/+source/s390-tools

[Rationale]
- The package s390-tools is required in Ubuntu main for hardware enablement on s390x machines
- The package s390-tools will not generally be useful for a large part of
  our user base, but is important/helpful still because it's necessary for the proper operation
  of IBM Z mainframe.
- There is no other/better way to solve this that is already in main or
  should go universe->main instead of this.

- The package s390-tools is required in Ubuntu main no later than Mantic Beta freeze

[Security]
- No CVEs/security issues in this software in the past (CVE-2021-25316 doesn't apply)

- no `suid` or `sgid` binaries
- There are a lot of binaries in /sbin, which is expected as they are used for machine administration.
- Package does install services, timers or recurring jobs
  * cpacfstatsd -> system statistics
  * cpi.service -> used to provide system data to the hypervisor
  * cpuplugd.service -> CPU hotplug
  * dumpconf.service -> Configures dumps on panics
  * iucvtty-login@.service, ttyrun-getty@.service -> TTY handling
  * mon_fsstatd.service, mon_procd.service -> monitoring

Vendored dependencies security history:

- https://github.com/rustsec/advisory-db/blob/main/crates/once_cell/RUSTSEC-2019-0017.md
- https://github.com/rustsec/advisory-db/tree/main/crates/openssl
  -> Note that while the vendored crate is affected by RUSTSEC-2023-0044 the
     relevant function is never called by the compiled binary, either directly or
     indirectly.
- https://github.com/rustsec/advisory-db/blob/main/crates/serde_yaml/RUSTSEC-2018-0005.md
- https://github.com/rustsec/advisory-db/blob/main/crates/socket2/RUSTSEC-2020-0079.md

There doesn't seem to be any specific security features attached to those services.

In addition, there are several udev rules shipped with the software, to deal with s390-specific hardware.

- Packages does not contain extensions to security-sensitive software
  (filters, scanners, plugins, UI skins, ...)

[Quality assurance - function/usage]
- The package works well right after install

[Quality assurance - maintenance]
- The package is maintained well in Ubuntu/Upstream and does
  not have too many, long-term & critical, open bugs
  - Ubuntu https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug
    -> mostly feature requests
  - Upstream's bug tracker: https://github.com/ibm-s390-linux/s390-tools/issues
  Note that we've completely diverged from Debian, so their package isn't relevant to this MIR.
  Upstream is heavily involved with the Ubuntu packaging, often providing us with verifications for SRUs
  and tests of potential packages.
- The package does deal with exotic hardware, the Canonical Partners Engineering team has access
  to the relevant machines to be able to test, fix and verify bugs.

[Quality assurance - testing]
- The package does not run a test at build time because no test suite is
  provided upstream. Things recently changed a bit with the new Rust code
  having a few tests, but I'm reluctant to enable them as the vendored
  dependency tree would more than double in size (compressed!)
- The package does not run an autopkgtest.

- The package can not be well tested at build or autopkgtest time
  because the majority of tools inside of the package need either:
  - a special hardware level (for example z14 for secure boot, z15 for secure execution aka confidential computing) and/or
  - a native (LPAR) installation (for lowest level hardware access) and/or
  - special configuration settings (in the LPAR activation profile, for exampel for counters) and/or
  - specially assigned hardware cards (like crypto, RoCE, NVMe, or other hardware) and/or
  - hardware cards setup in a special way (for example in case of crypto with a master key set) and/or
  - run the hardware management console (hmc) in different modes (PR/SM vs DPM, but there is no simple way to switch between modes)To make up for that:
It's contractually agreed with our partner that the partner runs (and is in charge of) the testing on hardware that we do not have at Canonical (that is btw. also the case for SRUs) and that we (actually Solutions QA) do (does) a manual test around GA (that incl. s390-tools, but also manual and autoinstallations, which again make use of various s390-tools components) for every Ubuntu release, where the result is added to an overall test spreadsheet for that particular Ubuntu release for s390x.
The corresponding S-QA doc is: https://docs.google.com/document/d/1ixvRDgEHNjZwOujYJ9hmfbQte9A05ffOTrKi_PK82cs
- It is not possible to leave s390-tools in universe, since a lot of it's content (like bootloader, tools to activate hardware - just to name a few) are mandatory at install time and are required even for a base and minimal installation.

[Quality assurance - packaging]
- debian/watch is present and works
- debian/control defines a correct Maintainer field

- Recent build logs
https://launchpadlibrarian.net/682423862/buildlog_ubuntu-mantic-s390x.s390-tools_2.29.0-0ubuntu1_BUILDING.txt.gz

There is the usual issue of noisy Rust warnings in the dependencies.

- Lintian output is attached. It doesn't look very good, probably due to the
  fact that since the package basically only fully build on s390x we rarely
  produce binary packages on development machines, which is where Lintian runs
  would usually scream at us.

- Lintian overrides are present, but ok because they're about Ubuntu-specific
  source fields.

- This package does not rely on obsolete or about to be demoted packages.
- This package has no python2 or GTK2 dependencies

- The package will be installed by default on s390x, but does not ask debconf
  questions higher than medium

- Packaging and build is fairly easy, link to debian/rules:
https://git.launchpad.net/ubuntu/+source/s390-tools/tree/debian/rules

There's a little bit of complexity due to the signing requirements, the fact
that is mostly builds on s390x, and also due to the Rust integration, but it's
still mostly straightforward.

[UI standards]
- Application is not end-user facing (does not need translation)

[Dependencies]
- No further depends or recommends dependencies that are not yet in main

[Standards compliance]
- This package correctly follows FHS and Debian Policy

[Maintenance/Owner]
- Foundations team is already subscribed to the package. Note that most of the
  day-to-day work is done by Frank Heimes

- This does not use static builds using static archive from other packages.

- The Foundations team is aware of the implications of vendored code and (as
  alerted by the security team) commits to provide updates and backports
  to the security team for any affected vendored code for the lifetime
  of the release (including ESM).

- This package uses vendored rust code tracked in the Vendored-Sources-Rust field
  in the package, refreshing that code is outlined in debian/README.source (not yet uploaded, see attached debdiff)

- This package is rust based and vendors all non language-runtime dependencies.
  To be noted, upstream has defined a policy regarding which Rust dependencies
  are acceptable, whic hseems fairly sensible and should reduce the inevitable growth
  of that dep tree:

  https://github.com/ibm-s390-linux/s390-tools/tree/master/rust#what-third-party-crates-can-be-used-for-s390-tools

- The package has been built in the archive more recently than the last
  test rebuild

Feature request: bug #2030316
Original s390-tools MIR: bug #1521984

[Lukas Märdian (slyon)]

Some of the new (vendored) Dependencies (Rust crates) are listed here:

https://github.com/ibm-s390-linux/s390-tools/tree/v2.29.0/rust#what-third-party-crates-can-be-used-for-s390-tools

https://github.com/ibm-s390-linux/s390-tools/blob/v2.29.0/rust/pv/Cargo.toml

https://github.com/ibm-s390-linux/s390-tools/blob/v2.29.0/rust/pvsecret/Cargo.toml (https://github.com/ibm-s390-linux/s390-tools/blob/v2.29.0/rust/pvsecret/Cargo.lock)

https://github.com/ibm-s390-linux/s390-tools/blob/v2.29.0/rust/utils/Cargo.toml (no dependencies)

[Lukas Märdian (slyon)]

Apparently, some of those high-level vendored dependencies went through a MIR+security process already, as part of the rustc MIR (LP: #1957932):

$ for dep in anyhow byteorder cfg-if clap curl libc log openssl serde serde_yaml thiserror zerocopy; do grep Vendor /var/lib/apt/lists/*_main_source_Sources | egrep -q "$dep" && echo "FOUND $dep" || echo "MISSING $dep"; done;
FOUND anyhow
MISSING byteorder
FOUND cfg-if
FOUND clap
MISSING curl
FOUND libc
FOUND log
MISSING openssl
FOUND serde
MISSING serde_yaml
FOUND thiserror
MISSING zerocopy

[Simon Chopin (schopin)]

Attached is the debdiff for the upcoming upload (I'm waiting 'til after the beta comes out). It is code related to vendoring, as well as documentation for updating the vendored dependencies, so I figured it'd be better to have this available for the review in any case.

[Ioanna Alifieraki (joalif)]

Review for Source Package: s390-tools

[Summary]

The package and in particular the addition of the rust part has a couple of
problems but both seem to be workedaround.
The first is the lack of any test suite, however the partner and solutions QA
have been engaged to help with testging and therefore we are good on that front.

Secondly, the package vendors code, but Founfations team are already aware and
have agreed to provide updates and backports of security fixes for any affected
vendored code for the lifetime of the release (including ESM).

MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

This does need a security review, so I'll assign ubuntu-security

List of specific binary packages to be promoted to main: libekmfweb-dev, libekmfweb1, libkmipclient-dev,
libkmipclient1, s390-tools-chreipl-fcp-mpath, s390-tools-cpuplugd, s390-tools-data, s390-tools-osasnmpd,
s390-tools-statd, s390-tools-zkey, s390-tools
Specific binary packages built, but NOT to be promoted to main: <None>

Notes:

- The package is already in main and have a team subscriber.

Recommended TODOs:
1. Please double check lintian output and confirm nothing is critical.

[Duplication]
The package s390-tools is already in Ubuntu main, and is re-reviewed due to signinficant changes in the package (new Rust code-base, including vendored dependencies).

[Dependencies]
OK:
- no other Dependencies to MIR due to this
 - s390-tools checked with `check-mir`
 - all dependencies can be found in `seeded-in-ubuntu` (already in main)
 - none of the (potentially auto-generated) dependencies (Depends
   and Recommends) that are present after build are not in main
- no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems: None

[Embedded sources and static linking]
OK:
- no static linking
- does not have unexpected Built-Using entries
- Rust package that has all dependencies vendored. It does neither
  have *Built-Using (after build). Nor does the build log indicate
  built-in sources that are missed to be reported as Built-Using.
- rust package using dh_cargo (dh ... --buildsystem cargo)
- Includes vendored code, the package has documented how to refresh this
  code at https://launchpadlibrarian.net/688249928/s390-tools.debdiff
  This is only a debdiff, but when the uplaod is done the process can be found
  in the package at debian/README.source /

Problems: None

[Security]
OK:
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.
- does not expose any external endpoint (port/socket/... or similar)
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- this makes appropriate (for its exposure) use of established risk
   mitigation features (dropping permissions, using temporary environme...

[Mark Esler (eslerm)]

I reviewed s300-tools 2.28.0-0ubuntu2 and 2.29.0-0ubuntu2 as released in
Mantic's development. This shouldn't be considered a full audit, but rather
a quick gauge of maintainability.

s390-tools already exists in main. This review only considers novel Rust
code which has not been included in main before.

Most Rust additions are bindings between s390-tools and other tools, such
as rust-openssl, rust-libc, and rust-curl. Vendored packages are not being
reviewed. rustc/cargo package contains ~700 vendored packages. s390-tools
defensively limits the number of vendored packages ++1

Note, that the s390x deb packages is ~4x larger than debs for other archs,
as expected. Security's uaudit tooling is designed for amd64. Due to
urgency, only amd64 was compiled and s390x debs between Lunar and Mantic
were compared for this review.

Please also note that Rust tools are not included in s390-tools-signed.

The Security team thanks Simon Choplin for his work to define Rust code in
main: https://wiki.ubuntu.com/RustCodeInMain

s390-tools are: Tools for use with the s390 Linux kernel and device
drivers

- CVE History:
  - only CVE in history only affects SUSE
  - see vendored security history above
  - GitHub Issue and PR tracker are maintained
  - changelog since 2.26.0 (lunar main) not concerning
- Build-Depends?
  - see new ./rust-vendor/
- pre/post inst/rm scripts?
  - d/s390-tools-statd.postinst loads systemd module
    - already existed in s390-tools lunar main
    - see systemd below
  - d/s390-tools-zkey.postinst adds zkeyadm group, and sets folders/files
    to 0770 root:zkeyadm
    - already existed in s390-tools lunar main
  - d/s390-tools.postinst no longer exists in mantic
    - d/debian/s390-tools.postinst.s390x does
  - many .install scripts
    - s390-tools-data.install is new
      - adds genprotimg bins
      - in lunar, genprotimg was in d/not-installed
      - not Rust related, future C review needed
      - https://github.com/ibm-s390-linux/s390-tools/tree/master/genprotimg
- init scripts?
  - none
- systemd units?
  - already existed in s390-tools lunar main
    - no re-review
    - see mon_tools and d/modules-load.d/s390-tools.conf
- dbus services?
  - none
- setuid binaries?
  - none
- binaries in PATH?
  - amd64 binaries:
    - all are root:root owned
    - /usr/bin/genprotimg
    - /usr/bin/pvattest
    - /usr/bin/pvextract-hdr
  - s390x binaries not reviewed
- sudo fragments?
  - none
- polkit files?
  - none
- udev rules?
  - none
- unit tests / autopkgtests?
  - see MIR review notes on tests
  - some new Rust code contains tests
- cron jobs?
  - none
- Build logs:
  - see MIR review about noisy logs
  - overall, mantic's build logs are much better than lunar's

- note: only novel, non-vendored, Rust code is being considered for MIR
  - see header
  - related ./debian/ code is not a concern
  - diff -qr lunar/s390-tools-2.26.0/ mantic/s390-tools-2.29.0/
  - Only ./rust/ directory requires review!
- Processes spawned?
  - none detected
  - only std::process::ExitCode
- Memory management?
  - unsafe std::ptr::write_volatile used to zero secrets, ++1
  - defensive memory use
- File IO?
  - std::fs use
    - see ./rust/pv/src/uvde...

[Christian Ehrhardt  (paelzer)]

Thank you all, thereby the re-review is completed (in a rush, but completed).
The recent updates of 2.29.0-0ubuntu2 are great as they will help future servicing and document how to update single crates.

Being a "re-"review the source and binaries never left main.
There is no component mismatch holding it back right now.

So this does not need to be promoted as a package in the MIR process usually would be.

The pure component status they have in mantic and mantic-proposed shows this:
$ rmadison -u ubuntu -s mantic-proposed,mantic -a s390x $(for p in $(grep '^Pack' debian/control | cut -d' ' -f 2); do echo $p; done | xargs)
 s390-tools | 2.28.0-0ubuntu2 | mantic | s390x
 s390-tools | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 s390-tools-cpuplugd | 2.28.0-0ubuntu2 | mantic/universe | s390x
 s390-tools-cpuplugd | 2.29.0-0ubuntu2 | mantic-proposed/universe | s390x
 s390-tools-statd | 2.28.0-0ubuntu2 | mantic/universe | s390x
 s390-tools-statd | 2.29.0-0ubuntu2 | mantic-proposed/universe | s390x
 s390-tools-osasnmpd | 2.28.0-0ubuntu2 | mantic/universe | s390x
 s390-tools-osasnmpd | 2.29.0-0ubuntu2 | mantic-proposed/universe | s390x
 s390-tools-zkey | 2.28.0-0ubuntu2 | mantic | s390x
 s390-tools-zkey | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 libekmfweb1 | 2.28.0-0ubuntu2 | mantic | s390x
 libekmfweb1 | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 libekmfweb-dev | 2.28.0-0ubuntu2 | mantic | s390x
 libekmfweb-dev | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 libkmipclient1 | 2.28.0-0ubuntu2 | mantic | s390x
 libkmipclient1 | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 libkmipclient-dev | 2.28.0-0ubuntu2 | mantic | s390x
 libkmipclient-dev | 2.29.0-0ubuntu2 | mantic-proposed | s390x
 s390-tools-chreipl-fcp-mpath | 2.28.0-0ubuntu2 | mantic/universe | s390x
 s390-tools-chreipl-fcp-mpath | 2.29.0-0ubuntu2 | mantic-proposed/universe | s390x
 s390-tools-data | 2.28.0-0ubuntu2 | mantic | all
 s390-tools-data | 2.29.0-0ubuntu2 | mantic-proposed | all

So this is not a classic promotion to main, but what is it?

Excuses reminds us that the only thing holding it back is the block-proposed tag here, which we can remove now that the process is completed.

P.S. Given that we are in final freeze you might (I'm not sure) still need the release team to move it from -proposed to -release.

[Simon Chopin (schopin)]

❯ rmadison -s mantic s390-tools
 s390-tools | 2.29.0-0ubuntu2 | mantic | source, amd64, arm64, ppc64el, s390x

Marking this as Fix Released since the version with the Rust code hit the release pocket.

[Frank Heimes (fheimes)]

Also a big thanks from my side to all people involved - from Partner Engineering !