python-acme will break on November 1st

[Brad Warren]

[Impact]

This bug affects the python-acme package in all released versions of Ubuntu, with the exception of Eoan Ermine which uses a newer version of python-acme.

The major change in the package is the backporting of fixes to allow the python-acme package to continue to work with Let’s Encrypt’s “ACMEv2” endpoint, which is their RFC 8555 compliant endpoint for issuing and renewing TLS certificates, after service changes are made on November 1st. See https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380 for more details about this change.

The primary concern here is that users of the library, most commonly users of the certbot package, will no longer be able to obtain new certificates and existing certificates issued via certbot will no longer be able to renew, resulting in broken TLS configurations for many users and sites hosted on Ubuntu where certbot is used to request and renew TLS certificates.

For further reference, see https://wiki.ubuntu.com/StableReleaseUpdates/Certbot

[Major Changes]

There are no backwards incompatible API changes being introduced by the backported changes to this library, as such interoperability with existing packages should not be impacted. All changes being introduced are either new features or fixes to ensure the library's behaviour remains compatible with the ACME protocol, which was only finalised in March of this year. These changes are required to maintain compatibility with the ACMEv2 servers operated by LetsEncrypt per the Impact section.

Key changes being introduced by this backport:

The changelog entries for the update from 0.31.0-1 to 0.31.0-2 are:

* The acme module uses now a POST-as-GET request to retrieve the registration from an ACMEv2 server.
* The acme module now avoids sending the keyAuthorization field in the JWS payload when responding to a challenge as the field is not included in the current ACME protocol. To ease the migration path for ACME CA servers, Certbot and its acme module will first try the request without the keyAuthorization field but will temporarily retry the request with the field included if a malformed error is received.
* The Content-Type in the POST-as-GET request to retrieve a certificate was corrected from "application/pkix-cert" to "application/jose+json".

In addition to those changes, the relevant changelog entries when updating from 0.23.0 are:

* Added support for initiating (but not solving end-to-end) TLS-ALPN-01 challenges with the acme module.
* Added External Account Binding support.
* Use the ACMEv2 newNonce endpoint when a new nonce is needed, and newNonce is available in the directory.
* Warn when using deprecated acme.challenges.TLSSNI01
* When using acme.client.ClientV2 (or acme.client.BackwardsCompatibleClientV2 with an ACME server that supports a newer version of the ACME protocol), an acme.errors.ConflictError will be raised if you try to create an ACME account with a key that has already been used. Previously, a JSON parsing error was raised in this scenario when using the library with Let's Encrypt's ACMEv2 endpoint.
* You can now call query_registration without having to first call new_account on acme.client.ClientV2 objects.
* Support for the ready status type was added to acme. Without this change, Certbot and acme users will begin encountering errors when using Let's Encrypt's ACMEv2 API starting on June 19th for the staging environment and July 5th for production. See https://community.letsencrypt.org/t/acmev2-order-ready-status/62866 for more information.
* acme now supports specifying the source address to bind to when sending outgoing connections.
* acme now parses the wildcard field included in authorisations so it can be used by users of the library.

[Test Plan]

See https://wiki.ubuntu.com/StableReleaseUpdates/Certbot#SRU_Verification_Process

[Regression Potential]

Upstream performs extensive testing before release, giving us a high degree of confidence in the general case. There problems are most likely to manifest in Ubuntu-specific integrations, such as in relation to the versions of dependencies available and other packaging-specific matters.

As opposed to upgrading to the newer version of python-acme (0.36.0-1) from Eoan Ermine, and advantage of SRU'ing the 0.31.0-2 version to Xenial, Bionic, Cosmic and Disco, is that there are no breaking API changes between python-acme 0.31.0-2 and the version of python-acme currently in the repositories. Therfore, SRU'ing 0.31.0-2 carries the least risk of regression while enabling the library to function correctly after November 1st.

The regression potential of backporting 0.36.0-1 and associated newer dependencies would be higher, as more packages would need to be backported and the risk of introducing breaking API changes to dependant applications would therefore be increased.

[Robie Basak (racb)]

Thank you Brad for caring for this package in Ubuntu.

Please could you set Fix Released against the package task if Eoan at 0.36.0-1 is already fixed?

For the others, I think your plan sounds fine in principle.

[James Hebden (ec0)]

Robie, Brad -

I've made adjustments to the buster package (0.31.0-2) and pushed the changes to my Salsa guest account as discussed: https://salsa.debian.org/ec0-guest/acme/commits/ubuntu

I have tested this branch against Xenial, Bionic, Cosmic and Disco and the resulting package builds and installs alongside the existing distribution packages, and attempting to do a registration indeed speaks to the v2 ACME endpoint using the regular distro certbot command.

I have also run the package test suite on each distribution without issue. I have attached also a debdiff.

[Robie Basak (racb)]

Hi James,

Thank you for preparing and testing this!

Please could you follow https://wiki.ubuntu.com/StableReleaseUpdates#Procedure, update the bug description with the required SRU information, and provide individual diffs (or branches) for each of the Ubuntu releases to be updated? I need to be able to trivially generate the source packages for each of the required uploads, and they should include the debian/changelog changes including the correct packaging version strings, the run of "update-maintainer", etc.

Since you've based your upload on 0.31.0-2, Eoan will also need to be updated to include the change in -2 also to avoid user regression when users upgrade to Eoan after release. Please therefore also provide a branch or diff for the Eoan update, or (if you think it's better) base your SRU uploads on something already in Eoan.

[Robie Basak (racb)]

> Since you've based your upload on 0.31.0-2, Eoan will also need to be updated to include the change in -2 also to avoid user regression when users upgrade to Eoan after release.

Oh, this might already be the case, since Eoan has 0.36.0-1. If so, please just confirm.

[James Hebden (ec0)]

Ah, yes - I meant to mention that. I believe we should also target Eoan with this, as the -2 included additional fixes not present in -1. I will test the -2 packet on Eoan and generate a diff against Eoan as well.

[Brad Warren (bradmwarren)]

I trust your judgement here, but for what it's worth, from the standpoint of our code upstream, 0.36.0 should contain all the changes made in 0.31.0-2.

[James Hebden (ec0)]

Ah, yes - I meant to mention that. With Eoan having 0.36.0-1, this update is not needed on Eoan. And backporting 0.36.0-1 is a bigger job as it incurs other dependencies being updated.

I have added the following branches to the git repository, rather than generating individual debdiffs. There are no changes to build 0.31.0-2 between Xenial, Bionic, Cosmic and Disco.

I have also updated the description. Running update-maintainer resulted in no changes, so I might be missing something there. If you take a look at the branches and feel that there needs to be further changes for each branch, please let me know what is required and I'll gladly get the changes made.

[James Hebden (ec0)]

Per discussion on IRC, I have added a changelog entry per-branch (per-release - Xenial, Bionic, Cosmic and Dingo) adding a per-release version change (e.g. +ubuntu18.04.3 in the Bionic branch) noting the change to the packaging files. Please let me know if any further changes or information is required.

[Robie Basak (racb)]

For reference since I keep losing them, James' branches are here: https://salsa.debian.org/ec0-guest/acme

[Robie Basak (racb)]

Hi James,

Thank you for the latest updates. The code functionally looks fine (though I'm relying entirely on your testing) but there was quite a bit of tweaking necessary process-wise. I thought it would be easiest to suggest the changes in a Salsa Merge Request that I'll file shortly. For Disco, I've provided one commit per change, with an individual explanation in each commit. Please could you examine the commits to check you're happy with them, ask any questions you need, make any adjustments you think required and then pull them into your branch? Then, please could you apply matching changes to the other stable branches? The Salsa MR will hopefully be a good place/process to ask questions in context if needed, but here is also fine if you prefer.

On the documention side of the process, the bug description adjustments generally look good.

What you've written under Major Changes is good, but I would call this really the justification for the SRU and thus it's really the sort of (useful!) explanation that's intended to go under the Impact section.

For Major Changes what we need is a summary of the changes that will be experienced by the user receiving the SRU. So that's not just the specific changes we're trying to land but in our case it is also everything else that will come bundled with the backport. This will be different and be a bigger list the older the release to which we're backporting. For example, for Disco users, we need to summarise the changes from 0.31.0-1 to 0.31.0-2 only. For Bionic users, we need to summarise the changes all the way from 0.22.2-1ubuntu0.1 to 0.31.0-2. I don't mind how you want to lay this information out, but the information should be present.

I suggest you move what you've written currently under Major Changes to the Impact section, and then fill out Major Changes based on upstream release notes and understanding. Could Brad perhaps help with this?

I will update the template on the exception process page to try and make this clearer for next time.

Here's a summary of what I think are the next steps:

1. Examine the Salsa MR once I file it, go through that with me if needed, with the goal of landing into your branch the review changes I have suggested.

2. Apply equivalent changes to your other branches.

3. Adjust the bug description as I requested above.

Once that's done, I will provide an SRU team +1, and I will ask a colleague to review for a sponsorship +1, and then we can get the package updates into the proposed pockets ready for wider testing.

[Robie Basak (racb)]

Cosmic is now EOL, so there is no need to work on the Cosmic update any longer.

[Robie Basak (racb)]

I filed the Salsa MR here, which contains my review change suggestions: https://salsa.debian.org/ec0-guest/acme/merge_requests/1/commits

[Brad Warren (bradmwarren)]

> fill out Major Changes based on upstream release notes and understanding. Could Brad perhaps help with this?

Happy to help here.

There are no backwards incompatible API changes being made. All changes are either new features or fixes to keep the library's behavior compatible with the ACME protocol
which was only finalized in March of this year.

This is likely more detail than you want, but as a starting point, here are the relevant entries from our changelog at https://github.com/certbot/certbot/blob/master/CHANGELOG.md.

The changelog entries for the update from 0.31.0-1 to 0.31.0-2 are:

* The acme module uses now a POST-as-GET request to retrieve the registration from an ACMEv2 server.
* The acme module avoids sending the keyAuthorization field in the JWS payload when responding to a challenge as the field is not included in the current ACME protocol. To ease the migration path for ACME CA servers, Certbot and its acme module will first try the request without the keyAuthorization field but will temporarily retry the request with the field included if a malformed error is received. This fallback will be removed in version 0.34.0.
* The Content-Type in the POST-as-GET request to retrieve a certificate was corrected from "application/pkix-cert" to "application/jose+json".

In addition to those changes, the relevant changelog entries when updating from 0.23.0 are:

* Added support for initiating (but not solving end-to-end) TLS-ALPN-01 challenges with the acme module.
* Added External Account Binding support.
* Use the ACMEv2 newNonce endpoint when a new nonce is needed, and newNonce is available in the directory.
* Warn when using deprecated acme.challenges.TLSSNI01
* When using acme.client.ClientV2 (or acme.client.BackwardsCompatibleClientV2 with an ACME server that supports a newer version of the ACME protocol), an acme.errors.ConflictError will be raised if you try to create an ACME account with a key that has already been used. Previously, a JSON parsing error was raised in this scenario when using the library with Let's Encrypt's ACMEv2 endpoint.
* You can now call query_registration without having to first call new_account on acme.client.ClientV2 objects.
* Support for the ready status type was added to acme. Without this change, Certbot and acme users will begin encountering errors when using Let's Encrypt's ACMEv2 API starting on June 19th for the staging environment and July 5th for production. See https://community.letsencrypt.org/t/acmev2-order-ready-status/62866 for more information.
* acme now supports specifying the source address to bind to when sending outgoing connections.
* acme now requires and uses pytest when running tests with setuptools with python setup.py test.
* acme now parses the wildcard field included in authorizations so it can be used by users of the library.

Please let me know if there's anything else I can do to help!

[James Hebden (ec0)]

Thanks Robie, Brad, for both providing such useful replies and information.

Robie, I have updated the three branches (for Xenial, Bionic and Disco) to incorporate your suggested control and changelog changes. Thanks for the extra clarity there. I've also updated the description based on the changelog summary provided by Brad, and the shuffling of information you suggested. I've also made some light edits to clarify further the intention and potential impact of the changes to other packages.

Let me know if further changes or information are required.

[Robie Basak (racb)]

Thank you for working on the updates.

Your branches all look good to me:

Xenial 49f3173
Bionic 51e1c86
Disco 333564b

A couple of minor observations:

The Xenial version string of 0.31.0-2~ubuntu16.04.6 is presumably a typo and we can change to 0.31.0-2~ubuntu16.04.1 when uploading.

The merge commit in your Disco branch looks a bit screwed up, but the end result is correct so is fine for the purposes of this SRU.

I will now pass to a colleague for a second uploader review and sponsorship.

[Andreas Hasenack (ahasenack)]

Due to bug #1837673, we will also want to update the certbot packages. The ones available in disco, 0.31.0, seem like a good candidate, since they match the 0.31.0 version for python-acme we want here.

[Andreas Hasenack (ahasenack)]

I'm doing the second review and will handle uploading/sponsoring.

[Andreas Hasenack (ahasenack)]

This isn't building on xenial, due to unsatisfied build-depends:
The following packages have unmet dependencies:
 builddeps:./ : Depends: python-cryptography (>= 1.3.4) but 1.2.3-1ubuntu0.2 is to be installed
                Depends: python-idna (>= 2.5) but 2.0-3 is to be installed
                Depends: python3-cryptography (>= 1.3.4) but 1.2.3-1ubuntu0.2 is to be installed
                Depends: python3-idna (>= 2.5) but 2.0-3 is to be installed

xenial has:
python-cryptography{,3} 1.2.3-1ubuntu0.2
python-idna{,3} 2.0-3

@ec0 how did you build and test this updated package on xenial? Did you have something installed from outside of the ubuntu archive, or did you backport those dependencies as well?

[Brad Warren (bradmwarren)]

I'm not sure where those requirements are coming from in the .deb packages, but neither of those dependencies should need to be updated. We test with both cryptography 1.2.3 and idna 2.0 for the purpose of keeping compatibility with the packages in Xenial.

[Andreas Hasenack (ahasenack)]

@ec0 sorry if this was discussed before, but I don't see it here in the bug. Why did you downgrade the debhelper version in the disco branch?

The current package in disco, 0.31.0-1, is using debhelper 11, but your commit ca2913310288b2bb72976cce29b383d52e233ac0 changes that to 10, and build-depends on debhelper >= 9~ even. Maybe you just wanted to have the source package the same as in xenial, which is at debhelper 9 level?

[Andreas Hasenack (ahasenack)]

Same thing for bionic, which also has debhelper 11 out of the box, and the existing bionic package (0.22.2-1ubuntu0.1) is using that level.

[Andreas Hasenack (ahasenack)]

PPA with test packages: https://launchpad.net/~ahasenack/+archive/ubuntu/october-certbot-sru/

I reverted the debhelper changes mentioned above in these ppa builds

[Brad Warren (bradmwarren)]

To summarize some conversation that happened in the #certbot-dev IRC channel on Freenode to make sure everyone sees it, the python-cryptography and python-idna requirements come from python-acme's dependency on requests "security" extras which we declare at https://github.com/certbot/certbot/blob/v0.31.0/acme/setup.py#L20. These are extra security features which python-requests has but to use them, you need additional dependencies.

To express this, our Debian package maintainer lifted requests' dependencies out of the python-requests package and into the python-acme package, but the requirements for the version of python-requests in Debian and Ubuntu Xenial are different. The requirements for requests "security" extras for the version in Xenial are https://github.com/psf/requests/blob/v2.9.1/setup.py#L72.

To express this, I think for both python and python3:

* The cryptography requirement should be relaxed to >=1.2.3 to match the requirement from python-acme: https://github.com/certbot/certbot/blob/v0.31.0/acme/setup.py#L12
* The ndg-httpsclient and pyasn1 dependency should be added back.
* The idna dependency should be removed.

[Brad Warren (bradmwarren)]

I also think it's worth noting that the version of python-acme currently in Xenial has this same requirement on requests "security" extras: https://github.com/certbot/certbot/blob/v0.22.2/acme/setup.py#L21

[James Hebden (ec0)]

Hi Andreas,

Follow on from our IRC conversation, I've revisited and fixed the Xenial branch. Thank you for such a thorough review, and for catching those mistakes. What we're seeing here are simply mistakes on my behalf, when working to move my patches to Salsa - the debdiffs were generated at this point also. So I appreciate your work to validate and test the provided branches and diffs. I've addressed the Xenial case but also your feedback on the Bionic and Disco branches. The intention here was indeed to try and keep the branches similar to reduce backport-burden, however I can see the value in more tightly specifying the debhelper version also, so I've updated that and fixed the consistency mistakes around version (compat vs control vs changelog).

Hi Brad,

Thanks for jumping in again and adding more context around the upstream testing that has occurred around older versions. I had alluded to this backwards compatibility testing in my IRC conversation with Andreas, so the additional detail is very helpful.

Now, to the changes I have made, and the testing I have no done and re-done:
- Xenial: changelog has been updated to reflect the changes to the python dependency versions
- Xenial: requirement for python(3)-cryptography packages have been lowered to 1.2.3
- Xenial: requirement for python(3)-idna packages have been lowered to 2.0
- Xenial: debhelper version has been made consistent between changelog, control and compat (v9)
- Bionic: debhelper version has been returned to the original 11 in control, compat and changelog
- Disco: debhelper version has been returned to the original 11 in control, compat and changelog

And, to the testing I have performed on the updated branches:
- Xenial: Full clean rebuild of the package from the current ubuntu-xenial branch [0] of my salsa repo, this was successful with the changes above, and the test suite runs successfully.
- Xenial: I took the built package and updated one of my servers running Xenial which uses Certbot to managed TLS certificates. I was able to, using the distro packages (no PPA) upgrade successfully to the backported package, and execute a renewal against the test ACME endpoint. This successfully renewed and the debug log shows the v2 endpoint being used successfully.
- Bionic: Full clean rebuild of the package from the ubuntu-bionic branch of my salsa repo [1], and the test suite ran successfully
- Disco: Full clean rebuild of the package from the ubuntu-disco branch of my salsa repo [2], and the test suite ran successfully

Please let me know if any further changes or testing is required. I believe I have addressed the feedback raised so far and have tested each change, but if I've missed anything, let me know.

[0] https://salsa.debian.org/ec0-guest/acme/tree/ubuntu-xenial
[1] https://salsa.debian.org/ec0-guest/acme/tree/ubuntu-bionic
[2] https://salsa.debian.org/ec0-guest/acme/tree/ubuntu-disco

[James Hebden (ec0)]

Also Brad -

Regarding:
* The ndg-httpsclient and pyasn1 dependency should be added back.
* The idna dependency should be removed.

Let me know if this is required to maintain functionality, I'd like to keep the diff for the SRU as simple as possible. If this is a change that is desirable to better reflect the ongoing direction of the project, I'm happy to address in the PPA.

[Andreas Hasenack (ahasenack)]

Hi James,

thanks for working on this.

I think you made a mistake in your last commit to the xenial branch: https://salsa.debian.org/ec0-guest/acme/commit/8346b60b7550b26c8ae0d57a17558fe7c9f64c69

It says one thing, but the actual change has much more. In fact, it's touching the upstream source code. The package shouldn't even build like this, given the source won't match the orig tarball.

[Brad Warren (bradmwarren)]

> * The ndg-httpsclient and pyasn1 dependency should be added back.

I would recommend having the python(3)-ndg-httpsclient and python(3)-pyasn1 dependency. It looks like the python(3)-ndg-httpsclient dependency is already there in https://salsa.debian.org/ec0-guest/acme/blob/8346b60b7550b26c8ae0d57a17558fe7c9f64c69/debian/control.

The need for these dependencies and difference from Debian come from the requests "security" extra that I previously described. It looks like one of our dependencies in Xenial (python(3)-cryptography) already depends on python(3)-pyasn1 so there's not currently a problem, but I think it's worth adding one ourselves to avoid problems in the future in case this changes.

James, to give you a bit more detail, I think we could pretty easily make changes to our upstream code to avoid these issues since there doesn't seem to be a good way to indirectly require some of requests' optional dependencies in the .deb packages. I'm happy to talk to you about this more in the future. If I remember correctly, these dependencies are only actually used in very old versions of Python, but we claimed an unconditional dependency on them for simplicity/consistency. The reason I am recommending adding the python(3)-pyasn1 dependency in spite of this is because we declare a dependency on "requests[security]" in setup.py, setuptools will cause the program to crash in some scenarios if this dependency isn't satisfied. We could alternatively change setup.py here, but this to me seems like a more aggressive and risker change than just explicitly specifying a dependency which already transitively exists.

> * The idna dependency should be removed.

I am not sure if Ubuntu's packaging policy has anything to say about this, but this dependency is not actually needed by python(3)-acme in Xenial. This is a dependency of requests' "security" extras in version of requests in Debian, but not the version of requests in Xenial.

With that said, one of our other dependencies (also python(3)-cryptography) depends on python(3)-idna so it is going to currently get installed regardless, but if this would change in the future, a dependency on the package from python(3)-acme would cause it to be installed unnecessarily.

[James Hebden (ec0)]

@Andrea - right, the acme folder had been accidentally checked in to the ubuntu-xenial branch. The build process touches those files and they should not be checked in.

I have updated the branch and removed those checked in files, the only changes should be to the debian directory compared to upstream, per the other branches.

[James Hebden (ec0)]

Sorry, typo. *Andreas

[Andreas Hasenack (ahasenack)]

Ok, this ppa is ready for testing:

https://launchpad.net/~ahasenack/+archive/ubuntu/october-certbot-sru/

Or:

sudo add-apt-repository ppa:ahasenack/october-certbot-sru

I was able to verify today that the existing xenial client indeed stopped working (we are in the first brown-out day):
Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
An unexpected error occurred:
The client lacks sufficient authorization :: Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details.
Please see the logfiles in /var/log/letsencrypt for more details.

I then upgraded to the packages from the ppa, re-ran "certbot run", and then it used the acmev2 endpoint and worked.

For the actual sru verification I'll just have the verifier check the url that is used, as the v1 endpoint will probably be active by then again (unless this update misses the deadline).

[Brad Warren (bradmwarren)]

I tested the packages in the PPA on Ubuntu 16.04, 18.04, and 19.04 using the steps described at https://wiki.ubuntu.com/StableReleaseUpdates/Certbot#SRU_Verification_Process.

When testing against Let's Encrypt's ACMEv2 server (which is the default if you use the Certbot package from the PPA or you can include "--server https://acme-v02.api.letsencrypt.org/directory" on Certbot's command line if testing with the non-updated Certbot package) and starting with a clean /var/log/letsencrypt directory, you can use a command like:

grep 'GET' /var/log/letsencrypt/* | grep -v '/directory'

If this command produces any output, something has gone wrong.

To run https://wiki.ubuntu.com/StableReleaseUpdates/Certbot/TestScript, you first have to install all Certbot packages and set the environment variable CERTBOT_PREINSTALLED=1 otherwise the script will try to install from proposed-updates.

I also had to introduce an environment variable into the script to pin the version of boulder (Let's Encrypt's ACME server software) used for testing. Their most recent version has dropped support for features that are still included in Certbot and tested in the current script. The value I used here is BOULDERBRANCH="release-2019-03-11" which I believe is the most recent tag that works.

The output of dpkg-query at the end of the script about tested packages was:

On 19.04:

certbot 0.31.0-1
letsencrypt 0.31.0-1
python3-acme 0.31.0-2~ubuntu19.04.1~ppa4
python3-certbot 0.31.0-1
python3-certbot-apache 0.31.0-1
python3-certbot-nginx 0.31.0-1
python3-josepy 1.1.0-2

On 18.04:

certbot 0.27.0-1~ubuntu18.04.1~ppa3
letsencrypt 0.27.0-1~ubuntu18.04.1~ppa3
python3-acme 0.31.0-2~ubuntu18.04.1~ppa3
python3-certbot 0.27.0-1~ubuntu18.04.1~ppa3
python3-certbot-apache 0.23.0-1
python3-certbot-nginx 0.23.0-1
python3-josepy 1.1.0-1

On 16.04:

certbot 0.27.0-1~ubuntu16.04.1~ppa3
letsencrypt 0.27.0-1~ubuntu16.04.1~ppa3
python-acme 0.31.0-2~ubuntu16.04.1~ppa2
python-certbot 0.27.0-1~ubuntu16.04.1~ppa3
python-certbot-apache 0.23.0-1~ubuntu16.04.1
python-josepy 1.1.0-1~ubuntu16.04.1
python-letsencrypt 0.7.0-0ubuntu0.16.04.1
python-letsencrypt-apache 0.7.0-0ubuntu0.16.04.1

[Andreas Hasenack (ahasenack)]

> Thank you for working on the updates.
>
> Your branches all look good to me:
>
> Xenial 49f3173
> Bionic 51e1c86
> Disco 333564b

I cloned ec0's salsa repo into https://salsa.debian.org/andreas-guest/acme, and branched from ec0's ubuntu-<release> branches into my own ubuntu-<release>-andreas-review branches.

[Robie Basak (racb)]

Hello Brad, or anyone else affected,

Accepted python-acme into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-acme/0.31.0-2~ubuntu19.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Hello Brad, or anyone else affected,

Accepted python-acme into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-acme/0.31.0-2~ubuntu18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Hello Brad, or anyone else affected,

Accepted python-acme into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-acme/0.31.0-2~ubuntu16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brad Warren (bradmwarren)]

I tested this finding no problems using the same approach described in https://bugs.launchpad.net/ubuntu/+source/python-acme/+bug/1836823/comments/32.

The output of dpkg-query about the relevant installed packages was:

Xenial:
certbot 0.27.0-1~ubuntu16.04.1
letsencrypt 0.27.0-1~ubuntu16.04.1
python-acme 0.31.0-2~ubuntu16.04.1
python-certbot 0.27.0-1~ubuntu16.04.1
python-certbot-apache 0.23.0-1~ubuntu16.04.1
python-josepy 1.1.0-1~ubuntu16.04.1
python-letsencrypt 0.7.0-0ubuntu0.16.04.1
python-letsencrypt-apache 0.7.0-0ubuntu0.16.04.1

Bionic:
certbot 0.27.0-1~ubuntu18.04.1
letsencrypt 0.27.0-1~ubuntu18.04.1
python3-acme 0.31.0-2~ubuntu18.04.1
python3-certbot 0.27.0-1~ubuntu18.04.1
python3-certbot-apache 0.23.0-1
python3-certbot-nginx 0.23.0-1
python3-josepy 1.1.0-1

Disco:
certbot 0.31.0-1
letsencrypt 0.31.0-1
python3-acme 0.31.0-2~ubuntu19.04.1
python3-certbot 0.31.0-1
python3-certbot-apache 0.31.0-1
python3-certbot-nginx 0.31.0-1
python3-josepy 1.1.0-2

[Andreas Hasenack (ahasenack)]

In addition to the verification done by Brad, in bug https://bugs.launchpad.net/ubuntu/+source/python-certbot/+bug/1837673/comments/15 I did my own test run using these same packages, and it passed as well.

xenial verification succeeded

[Andreas Hasenack (ahasenack)]

In addition to the verification done by Brad, in bug https://bugs.launchpad.net/ubuntu/+source/python-certbot/+bug/1837673/comments/16 I did my own bionic test run using these same packages, and it passed as well.

bionic verification succeeded

[Andreas Hasenack (ahasenack)]

Disco verification

In addition to the verification done by Brad, I also went through a manual interactive run of requesting a certificate, fake renewing it, and revoking it.

Using python3-acme from proposed:
 *** 0.31.0-2~ubuntu19.04.1 500
        500 http://us.archive.ubuntu.com/ubuntu disco-proposed/universe amd64 Packages

root@disco-acme-sru-1836823:~# certbot run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
...
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for certbot-test2.justgohome.co.uk
Enabled Apache rewrite module
Waiting for verification...
Cleaning up challenges
Created an SSL vhost at /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-available/000-default-le-ssl.conf
Enabling available site: /etc/apache2/sites-available/000-default-le-ssl.conf
...
Congratulations! You have successfully enabled
https://certbot-test2.justgohome.co.uk

Fake renewal:
root@disco-acme-sru-1836823:~# certbot --dry-run renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/certbot-test2.justgohome.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate ...

[Brad Warren (bradmwarren)]

We also uploaded very similar packages to the SRU here to our PPA [1] that has tens/hundreds of thousands of users on Friday and received no bug reports.

[1] https://launchpad.net/~certbot/+archive/ubuntu/certbot

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-acme - 0.31.0-2~ubuntu19.04.1

---------------
python-acme (0.31.0-2~ubuntu19.04.1) disco; urgency=medium

  [ James Hebden ]
  * Backport packaging to build on Ubuntu Disco (LP: #1836823)

  [ Andreas Hasenack ]
  * d/p/series: drop unused -p1

python-acme (0.31.0-2) unstable; urgency=medium

  * Backport POST-as-GET support (Closes: #928452)

 -- James Hebden <email address hidden> Sat, 07 Sep 2019 16:15:04 +1000

[Robie Basak (racb)]

The verification of the Stable Release Update for python-acme has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-acme - 0.31.0-2~ubuntu18.04.1

---------------
python-acme (0.31.0-2~ubuntu18.04.1) bionic; urgency=medium

  * Backport packaging to build on Ubuntu Bionic (LP: #1836823)

python-acme (0.31.0-2) unstable; urgency=medium

  * Backport POST-as-GET support (Closes: #928452)

python-acme (0.31.0-1) unstable; urgency=medium

  * Bump dependency on josepy to >= 1.1.0
  * Add Breaks on python-acme against certbot << 0.20
  * New upstream version 0.31.0
  * Add dep on python-idna required by security extra.
  * Bump S-V; no changes needed.

python-acme (0.28.0-1) unstable; urgency=medium

  * New upstream version 0.28.0

python-acme (0.27.0-1) unstable; urgency=medium

  * New upstream release.
  * Bump S-V; no changes needed.

python-acme (0.26.0-1) unstable; urgency=medium

  * New upstream version 0.26.0
  * Bump S-V; add Rules-Require-Root: no

python-acme (0.25.1-1) unstable; urgency=medium

  * New upstream version 0.25.1

python-acme (0.25.0-1) unstable; urgency=medium

  * New upstream version 0.25.0
  * Add new dependency on requests-toolbelt
  * Drop unnecessary X-Python-Version fields
  * Add pytest as build-time dep only.

python-acme (0.24.0-2) unstable; urgency=medium

  * Update team email address. (Closes: #895863)

python-acme (0.24.0-1) unstable; urgency=medium

  * New upstream release.
  * Bump S-V; no changes needed.

 -- James Hebden <email address hidden> Sat, 07 Sep 2019 16:15:04 +1000

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-acme - 0.31.0-2~ubuntu16.04.1

---------------
python-acme (0.31.0-2~ubuntu16.04.1) xenial; urgency=medium

  [ James Hebden ]
  * Backport packaging to build on Ubuntu Xenial (LP: #1836823)
    - Drop debhelper compat level to 9
    - Drop Rules-Requires-Root control file field

  [ Andreas Hasenack ]
  * d/control: adjust dependencies, see comment #23 in LP #1836823 for
    details:
    - relax python-cryptography requirement to version 1.2.3
    - add back pyasn1 dependency
    - drop idna dependency

python-acme (0.31.0-2) unstable; urgency=medium

  * Backport POST-as-GET support (Closes: #928452)

python-acme (0.31.0-1) unstable; urgency=medium

  * Bump dependency on josepy to >= 1.1.0
  * Add Breaks on python-acme against certbot << 0.20
  * New upstream version 0.31.0
  * Add dep on python-idna required by security extra.
  * Bump S-V; no changes needed.

python-acme (0.28.0-1) unstable; urgency=medium

  * New upstream version 0.28.0

python-acme (0.27.0-1) unstable; urgency=medium

  * New upstream release.
  * Bump S-V; no changes needed.

python-acme (0.26.0-1) unstable; urgency=medium

  * New upstream version 0.26.0
  * Bump S-V; add Rules-Require-Root: no

python-acme (0.25.1-1) unstable; urgency=medium

  * New upstream version 0.25.1

python-acme (0.25.0-1) unstable; urgency=medium

  * New upstream version 0.25.0
  * Add new dependency on requests-toolbelt
  * Drop unnecessary X-Python-Version fields
  * Add pytest as build-time dep only.

python-acme (0.24.0-2) unstable; urgency=medium

  * Update team email address. (Closes: #895863)

python-acme (0.24.0-1) unstable; urgency=medium

  * New upstream release.
  * Bump S-V; no changes needed.

 -- James Hebden <email address hidden> Sat, 07 Sep 2019 16:15:04 +1000