python-acme to start crashing on June 19th

[Brad Warren]

[Impact]

Without this fix, on June 19, the library will start to fail when using Let's Encrypt's new ACMEv2 endpoint. We should avoid breaking this for users.

[Test Case]

On June 19, try to use Let's Encrypt's new ACMEv2 endpoint; it will error out, as described in https://community.letsencrypt.org/t/acmev2-order-ready-status/62866

[Regression Potential]

If the endpoint changes again, this will need another update, but the only potential regression I see is server-side, which needs patches on our end to adjust (like in this case).

[Original Bug Description]

I am the upstream maintainer of python-acme. This bug only affects python-acme in Ubuntu 18.04.

Starting on June 19th, this library will start failing when used with Let's Encrypt's new ACMEv2 endpoint. This is because the library does not recognize the changes described in https://community.letsencrypt.org/t/acmev2-order-ready-status/62866 and will error out when it sees them.

To fix this, python-acme either needs to be upgraded to 0.25.1 (which came out two days ago) or the one line patch that originally landed upstream at https://github.com/certbot/certbot/commit/5940ee92ab5c9a9f05f7067974f6e15c9fa3205a applied. I think the latter is the safer option.

Please let me know what I can do to help get this resolved.

[Simon Quigley (tsimonq2)]

Uploaded to Bionic.

Assuming this gets accepted into bionic-proposed, I would argue that this needs to be released early, so it can be released to users before error messages start.

Let me know what I can do to help.

Thanks.

[Robie Basak (racb)]

Thanks Brad and Simon for your help.

> I would argue that this needs to be released early...

Agreed, given that this is confirmed by upstream.

> [Test Case]

> On June 19, try to use Let's Encrypt's new ACMEv2 endpoint...

Any chance of a test plan that doesn't involve waiting until it's broken, so that we can confirm it won't break and thus release it before it breaks? :)

I am happy to trust Brad or another upstream person's judgment on this given that the patch is trivial. Please check that the built package still appears to work, do whatever you technically think you need to ensure that this bug is fixed, and explain what you did and confirm the Ubuntu package version you tested, and that should be fine. No need to use the full major version bump exception document's test plan or anything since this is a regular SRU and not an exceptional major version bump one.

[Robie Basak (racb)]

Hello Brad, or anyone else affected,

Accepted python-acme into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-acme/0.22.2-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brad Warren (bradmwarren)]

I agree that this should be released early if possible.

The diff at https://launchpadlibrarian.net/374731991/python-acme_0.22.2-1_0.22.2-1ubuntu0.1.diff.gz looks good and I tested this successfully using a modified version of https://wiki.ubuntu.com/StableReleaseUpdates/Certbot/TestScript. The acme package I used during testing was:

python3-acme 0.22.2-1ubuntu0.1

Let me know if there's anything else I can do to help.

(If anyone's curious, the script I used for testing is at https://pastebin.com/DvQb3mT6. It uses some code we normally use for python-acme development and after running all tests, the script errors out due to not hitting the minimum code coverage percentage. This is OK and only happens because our tests aren't configured to monitor coverage on files located outside of our git repo. If you modify the script to stop installing python3-acme from proposed, the script fails much earlier due to the unpatched version of python3-acme not recognizing the new status type.)

[Brad Warren (bradmwarren)]

What still needs to be done to get this released?

Let's Encrypt's change went live yesterday and this bug can be easily reproduced. Probably the easiest way to do this is to set up an Ubuntu 18.04 server that is reachable on port 80 with nothing currently listening to the port and a public domain pointing to the server. After that, install certbot and run:

certbot certonly --agree-tos --domain <your domain> --register-unsafely-without-email --staging --standalone
certbot renew --force-renewal

Certbot should fail with output including:

Attempting to renew cert (<your domain>) from /etc/letsencrypt/renewal/<your domain>.conf produced an unexpected error: Deserialization error: Could not decode 'status' ('ready'): Deserialization error: Status not recognized. Skipping.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-acme - 0.22.2-1ubuntu0.1

---------------
python-acme (0.22.2-1ubuntu0.1) bionic; urgency=medium

  * Add ready status type to be compatible with the new Let's Encrypt ACMEv2
    endpoint (LP: #1777205).

 -- Simon Quigley <email address hidden> Fri, 15 Jun 2018 21:05:49 -0500

[Robie Basak (racb)]

Simon tells me this is already fixed in Cosmic.

[Brad Warren (bradmwarren)]

> Simon tells me this is already fixed in Cosmic.

That is correct. The only affected versions of python-acme are in the range [0.22.0 - 0.25.0).

Thanks for helping us resolve this quickly!

[Robie Basak (racb)]

@Mathew

No, that's not right. We discovered problems in the packages in proposed and are working on a set of fixes at the moment.

[Robie Basak (racb)]

Oh I'm sorry, wrong bug.