proposed patch: remove dependency on python3-requests

[Shivaram Lingamneni]

I'm attaching a patch (based on 15ab343d7c887ca0) which removes apport's dependency on python3-requests.

It's similar to a change that was merged into software-properties recently:

https://code.launchpad.net/~slingamn/software-properties/+git/software-properties/+merge/389182

Thanks for your time!

[Shivaram Lingamneni (slingamn)]

I'm attaching a new version of the patch, rebased on the latest ubuntu/devel and incorporating review feedback received from software-properties.

[Brian Murray (brian-murray)]

Could you please explain the rational behind wanting to remove the dependency on python3-requests?

Additionally, I've some concerns about bundling this code "http_unixdomain.py" in multiple packages. Why should we do this?

[Shivaram Lingamneni (slingamn)]

Requests is a "kitchen-sink" library that's really more suitable for webapps and suchlike. It bundles its own certificate store (python3-certifi), which was necessary back when Python didn't verify SSL/TLS certificates by default, but is no longer needed on Linux. (In fact, updates to python3-certifi currently seem to be lagging behind updates to ca-certificates.) It also includes a character encoding detection library (python3-chardet), which is unnecessary in a systems context like software-properties or apport. (The dependencies on python3-idna and python3-urllib are similar.)

So there are concrete performance costs to pulling all of this code in; on my system, importing `requests` consumes an additional 10 MB of memory relative to just importing `json` and `http.client`. (It also takes an extra 100 milliseconds, which might be a secondary concern.)

ddstreet also had concerns about the size of the bundled shim code. In this MR, he refactored it so that the only bundled code is `UHTTPConnection`, which is just a few lines:

https://code.launchpad.net/~ddstreet/software-properties/+git/software-properties/+merge/396926

I could do something similar for apport.

Thanks for your time!

[Shivaram Lingamneni (slingamn)]

Speeding up apport's import time in particular would probably help system performance because of apport_python_hook (right now, unhandled exceptions cause an import of apport, which imports requests, which imports chardet and all the rest).

[Shivaram Lingamneni (slingamn)]

Any thoughts? Thanks for your time.

[Brian Murray (brian-murray)]

Thanks for the detailed information. I'm currently rather busy with the 20.04.2 release but plan to look at this later this week.

[Brian Murray (brian-murray)]

I'd like to see the bundled shim code reduced in size as was done with software-properties. Is this something you could do? Thanks again for bringing this up.

[Shivaram Lingamneni (slingamn)]

Sorry about the delay; I'm attaching a version that closely resembles the final version of the software-properties code.

[Shivaram Lingamneni (slingamn)]

Testing this code in isolation, it seems to work as expected:

>>> pprint.pprint(find_snap('core'))
{'channel': 'latest/stable',
 'confinement': 'strict',
 'contact': 'mailto:<email address hidden>',
 'description': 'The core runtime environment for snapd',
 'developer': 'canonical',
 'devmode': False,
 'id': '99T7MUlRhtI3U0QFgl5mXXESAiSwt776',
 'ignore-validation': False,
 'install-date': '2021-02-08T23:39:19.890249908-05:00',
 'installed-size': 103129088,
 'jailmode': False,
 'mounted-from': '/var/lib/snapd/snaps/core_10823.snap',
 'name': 'core',
 'private': False,
 'publisher': {'display-name': 'Canonical',
               'id': 'canonical',
               'username': 'canonical',
               'validation': 'verified'},
 'revision': '10823',
 'status': 'active',
 'summary': 'snapd runtime environment',
 'title': 'core',
 'tracking-channel': 'latest/stable',
 'type': 'os',
 'version': '16-2.48.2.1'}

[Shivaram Lingamneni (slingamn)]

Thoughts? I was hoping to get this in before the freeze.

[Brian Murray (brian-murray)]

I've uploaded this for hirsute, thanks again for looking into this and providing a patch.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "norequests.diff" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Brian Murray (brian-murray)]

I neglected to add the bug number when uploading the fix.

apport (2.20.11-0ubuntu58) hirsute; urgency=medium

  [ Shivaram Lingamneni ]
  * Remove dependency on python3-requests thereby reducing memory usage of
    apport.