[MIR] python-boto3 as a dependency of simplestreams

[Jeremy Bícha]

[Availability]
The package python-boto3 is already in Ubuntu universe.
The package python-boto3 build for the architectures it is designed to work on.
They build amd64 only (but binary is arch-all)
Link to package: https://launchpad.net/ubuntu/+source/python-boto3

[Rationale]
Simplestreams itself is in main for quite a while since bug 1220427.
The package python-boto3 is required in Ubuntu main for simplestreams
The package python-boto3 will not generally be useful for a large part of
our user base, but is important/helpful still because it is required by simplestreams
Additionally new use-cases enabled by this are demoting python-boto to universe because
the only reverse depency in main is simplestreams and debian is going to drop support
for python-boto, see LP: 2052437
The package python-boto3 is a new runtime dependency of package simplestreams that
we already support
python-boto is not compatible with python3.12, the only python supported in noble, thus
to commit to a long term support of simplestreams, it's better to depend on dependencies
that have upstream support

The package python-boto3 is required in Ubuntu main no later than noble is released
due to being required by a new upload of simplestreams in noble-proposed to make it
work on python3.12 and to be able to drop python-boto from noble archives.

python-s3transfer is a binary dependency of python-boto3
python-botocore is a binary dependency of python-s3transfer

[Security]
- No CVEs/security issues in this software in the past:
(0)https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python-boto3
(0)https://ubuntu.com/security/cves?q=&package=python-boto3&priority=&version=&status=
(0)https://security-tracker.debian.org/tracker/source-package/python-boto3

No `suid` or `sgid` binaries
No executables in `/sbin` and `/usr/sbin`
Package does not install services, timers or recurring jobs
Packages does not open privileged ports (ports < 1024).
Package does not expose any external endpoints
Packages does not contain extensions to security-sensitive software
(filters, scanners, plugins, UI skins, ...)

[Quality assurance - function/usage]
The package works well right after install. It's a python library.

[Quality assurance - maintenance]
- The package is maintained well in Debian/Ubuntu/Upstream and does
  not have too many, long-term & critical, open bugs
  - Ubuntu (2)https://bugs.launchpad.net/ubuntu/+source/python-boto3/+bug
  - Debian (0)https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=python-boto3
  - Upstream's bug tracker (192)https://github.com/boto/boto3/issues
    Looks normal for the age and impact of these libraries
- The package does not deal with exotic hardware we cannot support

[Quality assurance - testing]
- The package does not run a test at build time because it is not configured to do so,
the upstream source code contains unit tests
- The package does not run an autopkgtest because they are not configured to do so.

[Quality assurance - packaging]
- debian/watch is present and works
- debian/control defines a correct Maintainer field
- Lintian does not run as part of the build
  https://launchpadlibrarian.net/715514166/buildlog_ubuntu-noble-amd64.python-boto3_1.34.46+dfsg-1_BUILDING.txt.gz
- Lintian output attached
- Lintian overrides are present, but ok because they are justified:

# This is a false positive, likely an occurrence of #1019980
python-boto3 source: source-is-missing [docs/source/_templates/page.html]

- This package does not rely on obsolete or about to be demoted packages.
- This package has no python2 or GTK2 dependencies
- The package will not be installed by default
- Packaging and build is easy, link to debian/rules:
  https://git.launchpad.net/ubuntu/+source/python-boto3/tree/debian/rules

[UI standards]
- Application is not end-user facing (does not need translation)

[Dependencies]
- There are further dependencies that are not yet in main, MIR for them
  is at:
  - python-s3trasnfer: https://bugs.launchpad.net/ubuntu/+source/python-s3transfer/+bug/2061750
  - python-botocore: https://bugs.launchpad.net/ubuntu/+source/python-botocore/+bug/2061751

[Standards compliance]
- This package correctly follows FHS and Debian Policy

[Maintenance/Owner]
- The owning team will be Ubuntu Server and I have their acknowledgement for
  that commitment
- The future owning team is already subscribed to the package
- This does not use static builds
- This does not use vendored code
- This package is not rust based

[Background information]
The Package description explains the package well
Upstream Name is boto3
Link to upstream project https://github.com/boto/boto3

[Alberto Contreras (aciba)]

I am going to start working on the MIR template.

[Utkarsh Gupta (utkarsh)]

Whilst looking at the package with Alberto, we found that python3-s3transfer, one of the boto3's runtime dependencies, is in Universe, too. So this needs to be MIR'd alongside python3-boto as well. Adding python-s3transfer as a task to this same bug itself. Hopefully that should be sufficient here.

[Alberto Contreras (aciba)]

python-botocore is a dependency of python-s3transfer thus it has to be MIRed too per the same reasoning as https://bugs.launchpad.net/ubuntu/+source/python-boto3/+bug/2061217/comments/2

[Alberto Contreras (aciba)]

$ lintian --pedantic python-boto3_1.34.46+dfsg-1.dsc
P: python-boto3 source: package-uses-old-debhelper-compat-version 12
P: python-boto3 source: silent-on-rules-requiring-root [debian/control]
P: python-boto3 source: trailing-whitespace [debian/changelog:80]

[Alberto Contreras (aciba)]

I am working on adding build / autopkg test in https://code.launchpad.net/~aciba/ubuntu/+source/python-boto3/+git/python-boto3/+merge/464411

[Mark Esler (eslerm)]

Hello, the MIR process says any MIRs assigned to the security team after
the Beta Freeze deadline need to be discussed with the Director of
Security Engineering:

    For a MIR to be considered for a release, it must be assigned to the
    Security team (by the MIR team) before Beta Freeze. This does not
    guarantee that a security review can be completed by Final Release.
    Ask the director of Security for exceptions.

https://github.com/canonical/ubuntu-mir?tab=readme-ov-file#security-reviews

Please find a few minutes on Alex Burrage's calendar and schedule
a meeting.

Thanks

[Christian Ehrhardt  (paelzer)]

Review for Source Package: python-boto3

[Summary]
MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

List of specific binary packages to be promoted to main: python3-boto3

Specific binary packages built, but NOT to be promoted to main: <none>

This does need a security review, so I'll assign ubuntu-security
But it is also a bit of a special case. On one hand tests are only added right
now while I'm writing this and on the other a security review is required.
But given the context, the former outdated and unmaintained code we have in
python-boto does not have any of that.

But look at these three aspects on their own:
1. testing: In regard to testing we would not lose anything, and we would very
   soon get the testing added improving the situation.
2. security: The old code this replaces is the ancestor of the new code and
   that new code is better maintained, better split and more modern. So it
   should be better even before a review (which we should have)
3. maintenance: The old code is discontinued, we can not expect upstream support
   on issues with the old - so maintenance for bugs as well as security should
   be much better with the new stack.

Under these conditions I'd propose we allow it to be promoted now, will still
enqueue a security review and CPC/Alberto as well as Server will work on
anything found along that (if any).

Required TODOs:
- #1 please add testing as discussed. I've spoken with Alberto and he is
     already on extending the package to have proper tests.

[Rationale, Duplication and Ownership]
There is another package in main providing the same functionality: python-boto.
But that alternative is outdated and discontinued and this request is about
replacing the unmaintainable (and incompatible with the python in noble) old
version with the new one.
=> OK

A team is committed to own long term maintenance of this package.
=> Server, team-subscription already added

The rationale given in the report seems valid and useful for Ubuntu,
for simplestreams as reported and for other users of the library.

[Dependencies]
OK:
- no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems:
- other Dependencies to MIR due to this, cases are already open and linked here

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- not a rust package, no extra constraints to consider in that regard

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not expose any external endpoint (port/socket/... or similar)
- does not process arbitrary web content
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- does not deal with cryptography (en-/decryption, c...

[Christian Ehrhardt  (paelzer)]

To update everyone on what makes this case special, we discussed this today in the MIR meeting [1].

While everyone would have appreciated if all of this would just have happened 2 months ago that is not where we are right now and this isn't complaining - in fact we thank Alberto and CPC for picking it up at all. But due to that we are in a special situation where not-accepting it seems worse (due to the old python-boto it replaces being discontinued and outdated and incompatible) than accepting this with the requirements that we can make happen in the next few hours (see the review above).

Due to that, while no one was happy or super satisfied - we voted unanimously to allow this to be promoted in noble to demote python-boto in the same action.

The security review that we require is not expected to fit in the remaining time before the release and much less before RC images. Therefore it will be scheduled as if there would be no special urgency (usually completing in 2-4 weeks). Thereby it will not be a special urgent "change all other priority to complete it before the release" case which also answers @marks comment #6. No need to bring one more hard to deal with issue to Alex in this case.

Waiting with a state-change for the uploads with the tests to be in proposed and then planning a last check with the release team to not interfere with any RC builds (clearing -proposed is always good, but we want to be sure).

[1]: https://ubottu.com/meetingology/logs/ubuntu-meeting/2024/ubuntu-meeting.2024-04-16-14.32.moin.txt

[Christian Ehrhardt  (paelzer)]

Full stack is ready now

Override component to main
python-boto3 1.34.46+dfsg-1ubuntu1 in noble: universe/misc -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble amd64: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble arm64: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble armhf: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble i386: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble ppc64el: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble riscv64: universe/python/optional/100% -> main
python3-boto3 1.34.46+dfsg-1ubuntu1 in noble s390x: universe/python/optional/100% -> main
Override [y|N]? y
8 publications overridden.