[MIR] rpi-eeprom; raspberrypi-userland

[Dave Jones]

= rpi-eeprom =

[Availability]
The package is in proposed, pending a correction to Architecture to permit it to migrate to multiverse (LP: #1884748).

[Rationale]
The package is required for updating the boot EEPROM on the Raspberry Pi 4.

[Security]
I am not aware of any open CVEs against the tools in rpi-eeprom.

[Quality assurance]
The package is extensively used upstream on Raspbian, and is obviously actively maintained there. There is no meaningful test suite included in the package, but the contents of the package are regularly exercised in image testing (and boot EEPROM testing).

[UI standards]
Manual pages are included for both utilties included in the package (rpi-eeprom-config and rpi-eeprom-update), but localization is missing from both utilities at present. However, most users will never use these utilities directly. Rather, they are typically launched by a systemd service on boot which automatically applies new versions of the boot EEPROM.

[Dependencies]
The package depends on binutils, python3, and pciutils, all of which are already in main. It also depends on linux-firmware-raspi2 and libraspberrypi-bin which are the subject of other MIRs (LP: #1867813, LP: #1895133).

[Standards compliance]
The package installs its scripts under /usr/bin.

[Maintenance]
The package is maintained by the Ubuntu Foundations team.

[Background information]
As this is a dependency for keeping the boot EEPROM on the Raspberry Pi 4 up to date, the intention is to install this by default in all pi-related images going forward.

---

= raspberrypi-userland =

[Availability]
The package is already in universe.

[Rationale]
The package is depended upon by the new raspi-common seed, for inclusion in all pi related images. The reason for its inclusion in the seed is that the libraspberrypi-bin package provides the vcgencmd and dtoverlay utilities which are both required by rpi-eeprom (the subject of a separate MIR, LP: #1895137) for updating the boot EEPROM on the Raspberry Pi 4.

The libraspberrypi0 package is a dependency of libraspberrypi-bin and both are built from the raspberrypi-userland source package.

[Security]
I am not aware of any open CVEs against the tools in libraspberrypi-bin or the libraries in libraspberrypi0.

It may be worth noting that the -bin package installs a udev rule (in /lib/udev/10-local-rpi.rules) permitting members of the "video" group access to /dev/vchiq, which is required for all the VC related utilities (including vcgencmd, raspivid, and raspistill) to be operated without root privileges.

[Quality assurance]
The package is extensively used upstream on Raspbian, and is obviously actively maintained there. There is no meaningful test suite included in the package, but the contents of the package are regularly exercised in image testing (and boot EEPROM testing).

[UI standards]
I've added manual pages for all the utilities I'm able to, but localization is missing from all utilities at present. However, most users will never use these utilities directly (bar, perhaps, the raspivid and raspistill utilities for the camera module). Instead the most common scenario is that the utilities will be used (invisibly) by other scripts (e.g. rpi-eeprom-update) for maintenance purposes like manipulating the boot EEPROM.

[Dependencies]
As noted above, libraspberrypi-bin depends on libraspberrypi0. It also depends on device-tree-compiler and libc6, both of which are already in main. libraspberrypi0 in turn merely depends on libc6.

[Standards compliance]
The package installs its binaries under /usr/bin, and its libraries under /usr/lib. Upstream does not version their API, so the libraries are unversioned.

[Maintenance]
The package is maintained by the Ubuntu Foundations team.

[Background information]
As noted above, the package is a dependency of the recently added raspi-common seed (https://lists.ubuntu.com/archives/ubuntu-release/2020-September/005086.html). As it is a dependency for keeping the boot EEPROM on the Raspberry Pi 4 up to date, the intention is to install this by default in all pi-related images going forward.

[Christian Ehrhardt  (paelzer)]

FYI: Fused the thematically and use-case wise raspberry-userland MIR with the bug here.

[Christian Ehrhardt  (paelzer)]

[Summary]
This one is hard to decide as it has binary blobs in the form of
the RPI firmware. Usually for normal package that would be a denial
reason, but other microcode delivering packages work the same way.
I'll need to discuss and co-ack with other MIR Team members and
also need security to have a word.

For now call it "ok enough to get security review", but there are
enough todos listed for you to start right away - for that I'll also
mark it incomplete. Set it back to new once the mentioned TODOs are
done for re-evaluation.

This does need a security review, so I'll assign ubuntu-security

List specific binary packages to be promoted to main: rpi-eeprom

Info:
In this cases this isn't from Debian but from Raspbian and the upstream
is https://github.com/raspberrypi/rpi-eeprom

Required TODOs:
- Get Foundations-bug subscribed to the package(s)
- Even not being used there avoid packages to totally fail on install
  and breaking apt thereby. Please get it to be gracefully-unusable there.
  Raspbian only publishes for Pi, we do have more arm* models to support.
  On non Pi arm* HW this is on install:
    Unpacking rpi-eeprom (7.8-0ubuntu2) ...
    Setting up linux-firmware-raspi2 (1.20200601+arm64-0ubuntu3) ...
    Error: missing /boot/firmware, did you forget to mount it?
    dpkg: error processing package linux-firmware-raspi2 (--configure):
     installed linux-firmware-raspi2 package post-installation script
     subprocess returned error exit status 1
    Setting up binutils-common:arm64 (2.35-3ubuntu1) ...
    Setting up libctf-nobfd0:arm64 (2.35-3ubuntu1) ...
    Setting up libraspberrypi0 (0~20200520+git2fe4ca3-0ubuntu2) ...
    dpkg: dependency problems prevent configuration of rpi-eeprom:
     rpi-eeprom depends on linux-firmware-raspi2 (>= 1.20190819); however:
      Package linux-firmware-raspi2 is not configured yet.
    dpkg: error processing package rpi-eeprom (--configure):
- The Diff from 7.5-1 to our 7.8-0 can't be explained by the changelog.
  It seems we have much more than just the new version.
  Could you please ensure that the changelog clearly indicates what our Delta
  over Raspbian is?
- You refer to an empty VCS, having the changes commit-by-commit in this or
  another one (depending where you push) woud be great. Please fix the VCS
  entry to point at such a valid repo.

Recommended TODOs:
- Please investigate if it makes sense to be arch:all since it depends on
  arm only packages:
  rpi-eeprom : Depends: libraspberrypi-bin but it is not installable
        Depends: linux-firmware-raspi2 (>= 1.20190819) but it is not installable
- Please clarify the Focal situation, the same version is stuck in proposed
  there as well.
- Any chance to add some tests verifying the functional integrity of the package
  to run at build or autopkgtest time?
- Please consider updating to a newer version before you SRU things to >=Focal

[Duplication]
OK:
We don't have this package or a similar function. Also The Internet is full
of people asking for it - so it is important nad not otherwise available.
There is no other package in main providing the same functionality.

[Dependencies]
OK:
- no other Dependencies to MIR due to this
  ...

[Dave Jones (waveform)]

> This one is hard to decide as it has binary blobs in the form of
> the RPI firmware. Usually for normal package that would be a denial
> reason, but other microcode delivering packages work the same way.

Just a quick note on this one, in case it changes anything: as rpi-eeprom is currently intended to migrate to multiverse I assumed it would be MIR'd to restricted rather than main (as you note: it primarily consists of the binary blobs for the second stage bootloader).

In contrast, libraspberrypi{-bin,0} contains no binary blobs and should indeed be targetted to main.

> - Get Foundations-bug subscribed to the package(s)

Now done for both rpi-eeprom and raspberrypi-userland

> - Even not being used there avoid packages to totally fail on install
> and breaking apt thereby. Please get it to be gracefully-unusable there.
> Raspbian only publishes for Pi, we do have more arm* models to support.
> On non Pi arm* HW this is on install:
[snip]

Just to clarify, is this suggesting it should install cleanly on non-pi arm hardware, but *then* refuse to work (with some appropriate error message) or should it refuse to install at all e.g. at dependency resolution time. I'd love to implement the latter but I've no idea how (is there such a thing as a package that's only available to pi images?). The former is rather more complex as it means fixing how linux-firmware-raspi2 installs its boot firmware (which is something that's been sat on my TODO list for yonks, but it means some rather invasive flash-kernel changes where we're already carrying a huge delta).

> - The Diff from 7.5-1 to our 7.8-0 can't be explained by the changelog.
> It seems we have much more than just the new version.
> Could you please ensure that the changelog clearly indicates what our Delta
> over Raspbian is?

Will do (as you've probably guessed this all started out with 7.5 and I must've bungled the migration of the changelog while dealing with 7.7 then 7.8 (3 days later!).

> - You refer to an empty VCS, having the changes commit-by-commit in this or
> another one (depending where you push) woud be great. Please fix the VCS
> entry to point at such a valid repo.

Ah, I was under the misapprehension that the launchpad repo would be populated by the git-ubuntu import (in other words the Vcs-Browser entry was "pre-emptive"). As I'm not an uploader for this (or any) package, would it be more correct to just remove that for now?

> - Please investigate if it makes sense to be arch:all since it depends on
> arm only packages:

Yup, that's definitely an issue - in fact that's the reason rpi-eeprom's stuck in proposed because the dependency (libraspberrypi-bin) is armhf/arm64 only. I've already fixed that in an upload for 8.0 to my PPA (https://launchpad.net/~waveform/+archive/ubuntu/eeprom), but of course that's now been superseded by 9.0!

> - Please clarify the Focal situation, the same version is stuck in proposed
> there as well.

rpi-eeprom on Focal is currently awaiting the SRU of libraspberrypi{-bin,0} to Focal (LP: #1883111), but as mentioned above there's also the Arch: all issue. Still the intention is indeed to have this in the current LTS and Groovy (pos...

[Christian Ehrhardt  (paelzer)]

> Just to clarify, is this suggesting it should install cleanly on non-pi
> arm hardware, but *then* refuse to work (with some appropriate error
> message) or should it refuse to install at all e.g. at dependency
> resolution time. I'd love to implement the latter but I've no idea how
> (is there such a thing as a package that's only available to pi
> images?). The former is rather more complex as it means fixing how
> linux-firmware-raspi2 installs its boot firmware (which is something
> that's been sat on my TODO list for yonks, but it means some rather
> invasive flash-kernel changes where we're already carrying a huge
> delta).

Either way works, but I'm not aware of any differentiation beyond
architecture that you can use.
And since arm64/armhf can be so many different things I'd ask to get
something into e.g. postinst that detects if it runs on the right HW
and if it doesn't skips the rest.
That way the binaries are available for evaluation but have no function.
If there are other active bits/hooks, then yes you'd want to disable
them as well in that case.

We've had similar cases where a package makes no sense in a container context.
But making it fail gracefully avoided people getting stuck with broken
packages which can be a big problem especially for less experienced
users.

[Christian Ehrhardt  (paelzer)]

> > - You refer to an empty VCS, having the changes commit-by-commit in this or
> > another one (depending where you push) woud be great. Please fix the VCS
> > entry to point at such a valid repo.
>
> Ah, I was under the misapprehension that the launchpad repo would be
> populated by the git-ubuntu import (in other words the Vcs-Browser entry
> was "pre-emptive"). As I'm not an uploader for this (or any) package,
> would it be more correct to just remove that for now?

Well, you'll maintain it somewhere atm - point there.

It will be imported once it is in main (or could be whitelisted now) -
then you could leave the current VCS entry.
The problem is that in any case we will miss all the history as it
never was in Debian to import from.
Gladly you know racb to talk about options from here.

[Christian Ehrhardt  (paelzer)]

> > - Please consider updating to a newer version before you SRU things to
> >=Focal
>
> Is it best at this point to fix the existing 7.8 upload, or reject that
> and fix all this in a new 9.0 upload? Happy to do whichever is easier
> from the MIR/security team's perspective.

IMHO cancel what is there, get it right in Groovy or later and then
SRU the correct thing.
The usual reason is this, right now nothing is in Focal, so you can
(under constraints) backport anything.
But if you have 7.8+fixes there first and then plan to update to 9.0
this update would have to follow SRU rules.
Which means you'd need to retain any behavior of the former version
making it much harder for you.

TL;DR: "Get it right first, SRU once afterwards" usually is the right approach

[Launchpad Janitor (janitor)]

This bug was fixed in the package rpi-eeprom - 9.0-1ubuntu1

---------------
rpi-eeprom (9.0-1ubuntu1) groovy; urgency=medium

  * Ubuntu port (LP: #1884748), including changes for MIR (LP: #1895137)
  * Added d/watch
  * Added d/p/use_python3_no_env.patch to remove env from rpi-eeprom-config
  * Removed redundant postinst and prerm scripts
  * Adjusted d/control to remove redundant rpi-eeprom-images definitions,
    correct Architecture to armhf and arm64 only, and the bootloader
    definition to the Ubuntu package name (linux-firmware-raspi2)
  * Fixed paths for Ubuntu in d/defaults/rpi-eeprom-update
  * Updated d/s/lintian-overrides

rpi-eeprom (9.0-1) buster; urgency=medium

  [ Tim Gover ]
  * pieeprom-2020-09-03: Promote latest stable to default/critical
  * Update releases.md
  * Update README.md
  * Archive 2020-08-31 and 2020-09-02 beta releases.

rpi-eeprom (8.0-1) buster; urgency=medium

  [ Tim Gover ]
  * Promote pieeprom-2020-09-03 to STABLE
  * Update releases.md

rpi-eeprom (7.14-1) buster; urgency=medium

  [ Kyle J. McKay ]
  * rpi-eeprom-update: avoid any possible od accidents

  [ Andrew Scheller ]
  * Add BETA label to release notes

  [ Tim Gover ]
  * pieeprom-2020-09-03.bin - Fix filename

rpi-eeprom (7.13-1) buster; urgency=medium

  [ Tim Gover ]
  * Use od instead of hexdump to simplify package dependencies
  * pieeprom-2020-09-02: Simplify green activity LED behavior

  [ MichaIng ]
  * Remove rpi-eeprom-images from dependencies

rpi-eeprom (7.12-1) buster; urgency=medium

  [ Tim Brooks ]
  * Fix #211 - hexdump error on older Pi models

rpi-eeprom (7.11-1) buster; urgency=medium

  [ Tim Gover ]
  * pieeprom-2020-08-31 - Disable self update mode from SD cards
  * Update link to latest stable release
  * Update releases.md
  * Update README.md

  [ andrum99 ]
  * Update README.md

  [ Hristo Venev ]
  * rpi-eeprom-update: Upstream kernel fix

  [ Serge Schneider ]
  * Use Python 3

rpi-eeprom (7.10-1) buster; urgency=medium

  [ Andrew Scheller ]
  * Typos

  [ Tim Gover ]
  * Promote beta 2020-07-31 to stable

rpi-eeprom (7.9-1) buster; urgency=medium

  [ Tim Gover ]
  * Update releases.md
  * rpi-eeprom-update: Set file permissions on the EEPROM update files
  * Rename .config.yml to config.yml
  * pieeprom-2020-07-31.bin - Standardize USB port power off across Pi4 models

rpi-eeprom (7.8-1) buster; urgency=medium

  [ Tim Gover ]
  * pieeprom-2020-07-16.bin - Promote to STABLE
  * recovery.bin - Update beta/stable after rebase

 -- Dave Jones <email address hidden> Thu, 24 Sep 2020 13:46:31 +0100

[Dimitri John Ledkov (xnox)]

@christian updated rpi-eeprom is now in groovy, and it is now "done right".

[Christian Ehrhardt  (paelzer)]

Thanks Dave and Dimitri!
Former issues:
- subscription - done
- install in other places is now working fine
  The service fails (split to bug 1898160), but that isn't too bad as at least
  no half-installed packages are happening anymore.
- updated to the most recent version
- version depends on libraspberrypi-bin fixed

- Still no tests, but ok I guess there isn't much you can test for an eeprom
  package at build time ?! :-/

MIR Team Ack for rpi-eeprom - is is already in component mismatch, so settign Fix Committed.

@Doko:
Since overall you need all or nothing - what about the raspberrypi-userland review?

[Christian Ehrhardt  (paelzer)]

Arr I forgot, we still wait for Securtity on rpi-eeprom.
But ok, the MIR Ack you have - it can go to Fix Committed as soon as they are happy as well.

[Seth Arnold (seth-arnold)]

I reviewed rpi-eeprom version 9.0-1ubuntu1 as checked into groovy. This
isn't a full security audit but a very quick gauge of maintainability.

Because this is an architecture-specific review, the usual tooling doesn't
work for this case. This is a slight problem for maintenance, because the
security team may not have sufficient hardware for update preparation or
update testing. We will probably need to rely upon other teams to provide
testing facilities.

The firmwares themselves are blackboxes similar to intel-microcode or
amd64-microcode. The only recourse we have in the face of regressions is
reverting updates and the same will hold with this package as well.

I'm also concerned about the update frequency given in the
debian/changelog file or releases.md file. The security team does not
have the bandwidth to track these updates ourselves. It is unlikely
each one has security relevance, but if Canonical gets too far behind
on these packages, the Ubuntu security team may have trouble importing
new versions.

So, I would like some assurances that another team will be SRUing new
versions of the package for all supported releases on an appropriate
time-scale to manage the risks associated with high-rate-of-change
projects.

Security team ACK for promoting rpi-eeprom to restricted CONDITIONAL
on another team stating that they will provide testing resources and
periodically import updated versions as appropriate.

Thanks

[Dave Jones (waveform)]

Thanks Seth,

> Security team ACK for promoting rpi-eeprom to restricted CONDITIONAL
> on another team stating that they will provide testing resources and
> periodically import updated versions as appropriate.

Foundations team will handle regular importing of updated versions, and SRUing to all supported releases. Devices certification team will handle provision of testing resources and periodic testing of all supported releases.

[Matthias Klose (doko)]

raspberrypi-userland:

things which have to be fixed:

 - the debian/copyright is incomplete. an fgrep -ri copyright
   shows missing copyright holders, although these might
  not be extensive. Please also check for possible missing
  copyright holders which are not covered by the simple
  check above.

- host_applications/linux/apps/hello_pi/hello_video/README
  not mentioned in d/copyright, and no license given. If the license
  is not clear, it should be removed from the source package.

- having an upstream not caring about ABI stability is one thing,
  however that's not what Ubuntu requires for main inclusion.
  Especially for this, symbols files are required.

 - libraspberrypi-bin.lintian-overrides says:
  "The container binaries are undocumented and don't support -h
  so I've no idea what they do".
  How can we support these if we don't know what these do, and
  why they are included?

- did you check for conflicting files in /usr/bin and /usr/include?
  some names look pretty generic

things which should be fixed:

- the library packages should be marked as M-A: foreign

- there are a few (test?) binaries in /usr/bin, polluting the
  name space, which don't even have man pages. Wouldn't it
  be more appropriate to move those into some package
  specific libdir?

[Christian Ehrhardt  (paelzer)]

FYI in MIR meeting of today:
[16:44] <cpaelzer> waveform: I wanted to know if this is incomplete as in "ack I'll do it at some point" or as in "we'll give u then"
[16:44] <waveform> cpaelzer, that's basically done - I've just got to update the bug

[Dave Jones (waveform)]

The userland packages in the following PPA should address all points raised by doko in the review in comment #13:

https://launchpad.net/~waveform/+archive/ubuntu/userland

I'm happy to attach a debdiff if that's preferable, but given the scale of the changes I'm not sure that'd be easier to review. Not mentioned in the changelog is that file conflicts have now been checked and verified as not applicable.

[Brian Murray (brian-murray)]

Hello Dave, or anyone else affected,

Accepted linux-firmware-raspi2 into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/linux-firmware-raspi2/3-0ubuntu1~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Comment #16 was inappropriately added due to the following in the linux-firmware-raspi2 changelog:

* Replaced postinst with flash-kernel trigger (relates to LP: #1895137)

So please ignore it, sorry for the noise!

[Christian Ehrhardt  (paelzer)]

We have assigned doko to re-review, but he is too often lost in too important bugs (e.g. today in https://bugs.launchpad.net/ubuntu/+source/fakeroot/+bug/1915250). So I want to help here by doing the re-review.

The builds of https://bugs.launchpad.net/ubuntu/+source/rpi-eeprom/+bug/1895137/comments/15 are in hirsute by now.
 raspberrypi-userland | 0~20200520+git2fe4ca3-0ubuntu3 | hirsute/universe | source

Now we need to check if all concerns were adressed:
per CL they are:
  3 * d/copyright audit
  4 * Added d/p/no-specific-libgps.patch to use whatever libgps the system
  5 provides
  6 * Excluded the various /usr/bin/containers_* test binaries from
  7 libraspberrypi-bin
  8 * Added d/librasperrypi0.symbols file
  9 * Fixed pkgconfig paths in d/p/fix-multiarch-dir.patch

I checked the changes between this and the former versions
- symbols are really present now for libraspberrypi0
  - this was a bit ugly (one pkg, many libs - no versioned names, ..)
  - but eventully the .symbols made sense, covered all
  - also I see why we need the dev-pkg-without-shlib-symlink and shlib-without-versioned-soname overrides unless we want to mess with the common filenames
- copyright was massively updated
- containers* is removed after dh_install
- test binaries are moved
- There are a few more fixes, none look concerning

This really looks mostly good by now.

Remaining Questions:
- multi-arch: foreign was requested, same it became - reason?

I've pinged Dave to clarify ...

[Christian Ehrhardt  (paelzer)]

Ok clarified, and good - thanks Dave!

All former concerns are resolved AFAICS.
Therefore since all else was ready let use mark this ready for promotion (and after all this also is one of the cases that we kind of "already support but want to make official").

[Christian Ehrhardt  (paelzer)]

Set the states to be ready for the AAs per [1].
This should resolve soon'ish.

[1]: https://wiki.ubuntu.com/MainInclusionProcess?action=show&redirect=MIRTeam#Process_states

[Matthias Klose (doko)]

Override component to main
raspberrypi-userland 0~20200520+git2fe4ca3-0ubuntu3 in hirsute: universe/libs -> main
libraspberrypi-bin 0~20200520+git2fe4ca3-0ubuntu3 in hirsute arm64: universe/misc/optional/100% -> main
libraspberrypi-bin 0~20200520+git2fe4ca3-0ubuntu3 in hirsute armhf: universe/misc/optional/100% -> main
libraspberrypi-dev 0~20200520+git2fe4ca3-0ubuntu3 in hirsute arm64: universe/libdevel/optional/100% -> main
libraspberrypi-dev 0~20200520+git2fe4ca3-0ubuntu3 in hirsute armhf: universe/libdevel/optional/100% -> main
libraspberrypi0 0~20200520+git2fe4ca3-0ubuntu3 in hirsute arm64: universe/libs/optional/100% -> main
libraspberrypi0 0~20200520+git2fe4ca3-0ubuntu3 in hirsute armhf: universe/libs/optional/100% -> main
7 publications overridden.

[Matthias Klose (doko)]

Override component to main
rpi-eeprom 11.3-1ubuntu1 in hirsute: multiverse/misc -> main
rpi-eeprom 11.3-1ubuntu1 in hirsute arm64: multiverse/misc/optional/100% -> main
rpi-eeprom 11.3-1ubuntu1 in hirsute armhf: multiverse/misc/optional/100% -> main
3 publications overridden.