Comment 0 for bug 1640978

Revision history for this message
Peter Eckersley (pde-lists) wrote : letsencrypt 0.4.1 contains numerous bugs fixed upstream

This bug contains a list of known major and other issues fixed between upstream letsencrypt 0.4.1 and the latest version, certbot 0.9.3 (the project has also been renamed to avoid confusion between the python client software and the Let's Encrypt CA service).

[Impact]

MAJOR BUGS FIXED

https://github.com/certbot/certbot/issues/2750
letsencrypt < 0.5.0 was not compatible with future configuration files, so users who run certbot-auto then downgrade to the Xenial packages will encounter errors.

https://github.com/certbot/certbot/issues/2709
Failure to remember choices of authenticator plugins for renewal operation. This would essentially make "letsencrypt renew" useless on Xenial. Numerous less severe automated renewal-related bugs fixed in subsequent releases:
https://github.com/certbot/certbot/issues?utf8=%E2%9C%93&q=is%3Aissue%20milestone%3A0.5.0%20is%3Aclosed%20label%3Arenewal%20
https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.7.0+is%3Aclosed+label%3Arenewal
https://github.com/certbot/certbot/issues?utf8=%E2%9C%93&q=is%3Aissue%20milestone%3A0.6.0%20is%3Aclosed%20label%3Arenewal%20
https://github.com/certbot/certbot/issues?utf8=%E2%9C%93&q=is%3Aissue%20milestone%3A0.8.1%20is%3Aclosed%20label%3Arenewal%20
https://github.com/certbot/certbot/issues?utf8=%E2%9C%93&q=is%3Aissue%20milestone%3A0.9.0%20is%3Aclosed%20label%3Arenewal%20

https://github.com/certbot/certbot/issues/2613
Failure to handle IPv6 Virtual hosts in Apache configurations

https://github.com/certbot/certbot/issues/2320
Erroneous behaviour with Apache configs that have multiple vhosts in a single file (these are still not supported for cert installation in 0.9.3, but at least produce clear error messages)

https://github.com/certbot/certbot/issues/2768
Incompatibility with the specified version of the ACME protocol, preventing the Let's Encrypt serverside code from following it correctly

https://github.com/certbot/certbot/issues/2731
Failure to parse Plesk's apache config files

https://github.com/certbot/certbot/issues/1243
Apache plugin errors out when transformations to a configuration turn out to be a no-op.

https://github.com/certbot/certbot/issues/3210
Incorrect handling of RewriteCond directives when trying to avoid Apache inifinite redirect loops

https://github.com/certbot/certbot/issues/1833
Problems running Apache renewal in cron due to cron's default PATH

UX: fail to re-ask for email address if the first one seems invalid:
https://github.com/certbot/certbot/issues/2675

UX: when re-running is a NOOP (due to renewal not being needed yet), print an explanation:
https://github.com/certbot/certbot/issues/1918

OTHER BUGS FIXED

Reduce the risk of incorrect or corrupt state in case of control-C interrupts:
https://github.com/certbot/certbot/issues/3219

Failure to correctly parse certain rewrite directives in Apache configs:
https://github.com/certbot/certbot/issues/2735

Failure to correctly enable HTTP -> HTTPS redirects in some Apache configs:
https://github.com/certbot/certbot/issues/3003

Failure to provide a sensible error if the user requests a Unicode domain:
(support for those is being added in 0.10.0)
https://github.com/certbot/certbot/issues/2661

Directory deletion permission errors are fatal when using the webroot plugin for non-root users (but shouldn't be):
https://github.com/certbot/certbot/issues/2678

UX: provide helpful guidance for people who want to run Certbot as a non-root user:
https://github.com/certbot/certbot/issues/2306

SIGNIFICANT NEW FEATURES WARRANTING AN SRU:

Support --quiet / -q

https://github.com/certbot/certbot/issues/2512

User interface for requesting certificates for multiple domain names with the
webroot plugin:
https://github.com/certbot/certbot/issues/1393

Support for DNS based authentication:
https://github.com/certbot/certbot/issues/1826

[Test Case]

All or almost all of the pull requests for the bugs above include unit test coverage.
Some also include integration or compatibility test coverage.

[Regression Potential]

The Certbot team has viewed breakage of existing workflows (especially ones that may be automated) as a serious issue, has strived to avoid them, and has treated workflow changes as regressions where it has occurred.

We have the following test suites in place for Certbot:

* Nosetest unit tests with coverage for each module between 97% and 100%; *test.py in the relevant tree.
* Integration tests that run Certbot against the current copy of Let's Encrypt's serverside boulder codebase. These require docker and are a little more involved to run. See tests/boulder_integration.sh for instructions.
* "Compatibility tests" that run the Apache and Nginx plugins against corpora of configuration files for those webservers; these live in certbot-compatibility-test/
* Test farm tests, which we use to check that our releases run correctly on a wide range of platforms. These spin up Amazon EC2 instances for numerous OSes and run various tests on them. They live in tests/letstest

We recommend that Ubuntu run the first of these test suites during build (but we believe the Debian packages already do that).

All of these tests mitigate the risk of regressions in our releases; nonetheless, some regressions do slip past. Because many of our users auto-update, these tend to be reported and fixed quickly in point releases. For instance, regressions in 0.9.0 were fixed in 0.9.1, 0.9.2 and 0.9.3. Certbot 0.9.3 has been used to issue hundreds of thousands of Certs in the field, so we are fairly confident that no further significant regressions exist in it, and that release is likely to be safe as a Xenial SRU.

At least two changes in functionality between 0.4.1 and 0.9.3 do bear specific consideration for Xenial though:

Debian has added a "certbot renew" twice-daily cron job to their packages between 0.4.1 and 0.9.3; we believe this is low regression risk (having secondary renewal mechanisms in place is a NOOP) but Xenial packages may want to increase the debconf verbosity to get consent for this from Xenial users who are upgrading?

We had a custom log rotation scheme (rotate logs after every run), we now act like a more typical daemon, so packages need to be rotating our logs:
https://github.com/certbot/certbot/issues/3382

[Other Info]

RAOF has offered to sponsor 0.9.3 into Xenial.