certbot renew fails

Bug #1907420 reported by Nigel Hathaway
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-certbot (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

2020-12-09 07:51:51,669:DEBUG:certbot.main:certbot version: 0.40.0
2020-12-09 07:51:51,670:DEBUG:certbot.main:Arguments: []
2020-12-09 07:51:51,671:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2020-12-09 07:51:51,699:DEBUG:certbot.log:Root logging level set at 20
2020-12-09 07:51:51,700:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2020-12-09 07:51:51,724:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0xb3677e20> and installer <certbot.cli._Default object at 0xb3677e20>
2020-12-09 07:51:51,756:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2021-01-07 22:00:58 UTC.
2020-12-09 07:51:51,757:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2020-12-09 07:51:51,757:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2020-12-09 07:51:51,758:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0xb3677ec8>
Prep: True
2020-12-09 07:51:51,759:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0xb3677ec8> and installer None
2020-12-09 07:51:51,759:INFO:certbot.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2020-12-09 07:51:51,767:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0xb3677d18>)>), contact=('mailto:<email address hidden>',), agreement='https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf', status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v01.api.letsencrypt.org/acme/reg/8799437', new_authzr_uri='https://acme-v01.api.letsencrypt.org/acme/new-authz', terms_of_service='https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf'), 5e38bd2d89ed306c4b7a454f91b7537c, Meta(creation_dt=datetime.datetime(2017, 1, 26, 18, 1, 57, tzinfo=<UTC>), creation_host='XXX.co.uk'))>
2020-12-09 07:51:51,769:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2020-12-09 07:51:51,772:WARNING:certbot.renewal:Attempting to renew cert (XXX.co.uk) from /etc/letsencrypt/renewal/XXX.co.uk.conf produced an unexpected error: Failed to parse: https://acme-v02.api.letsencrypt.org/directory. Skipping.
2020-12-09 07:51:51,778:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 379, in prepare_url
    scheme, auth, host, port, path, query, fragment = parse_url(url)
  File "/usr/lib/python3/dist-packages/urllib3/util/url.py", line 392, in parse_url
    return six.raise_from(LocationParseError(source_url), None)
  File "<string>", line 3, in raise_from
urllib3.exceptions.LocationParseError: Failed to parse: https://acme-v02.api.letsencrypt.org/directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 462, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1206, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 614, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 261, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 46, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 808, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1138, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1088, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 452, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 313, in prepare
    self.prepare_url(url, params)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 381, in prepare_url
    raise InvalidURL(*e.args)
requests.exceptions.InvalidURL: Failed to parse: https://acme-v02.api.letsencrypt.org/directory

2020-12-09 07:51:51,779:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2020-12-09 07:51:51,780:ERROR:certbot.renewal: /etc/letsencrypt/live/XXXX.co.uk/fullchain.pem (failure)
2020-12-09 07:51:51,780:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.40.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1382, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1287, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 486, in handle_renewal_request
    raise errors.Error("{0} renew failure(s), {1} parse failure(s)".format(
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

[XXX replaces domain name above. Package version is 0.40.0-1ubuntu0.1]

https://acme-v02.api.letsencrypt.org/directory

returns

{
  "f7Q7byd2ihQ": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: certbot 0.40.0-1ubuntu0.1
Uname: Linux 5.4.77-217 armv7l
ApportVersion: 2.20.11-0ubuntu27.13
Architecture: armhf
CasperMD5CheckResult: skip
Date: Wed Dec 9 07:54:39 2020
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: python-certbot
UpgradeStatus: Upgraded to focal on 2020-11-15 (23 days ago)
mtime.conffile..etc.cron.d.certbot: 2020-11-16T13:39:17.711853

Revision history for this message
Nigel Hathaway (nhathaway) wrote :
Revision history for this message
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

One other person reported this issue in Debian, but nobody managed to reproduce it. Details, and request for further details from an affected user at: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=951696

Revision history for this message
Nigel Hathaway (nhathaway) wrote :

In this function:

_normalize_host(host, scheme):

return six.ensure_str(
   b".".join([_idna_encode(label) for label in host.split(".")])
)

The parameter passed is b'acme-v02.api.letsencrypt.org', but six.ensure_str() throws an exception with that parameter.

In six.ensure_str, I imagine this is what throws the exception

    if not isinstance(s, (text_type, binary_type)):
       raise TypeError("not expecting type '%s'" % type(s))

Revision history for this message
Nigel Hathaway (nhathaway) wrote :

Problem found. the package 'six' was installed both by pip3 and by apt. The pip3 version in /usr/local/lib/python3.8/dist-packages/six.py was at version 1.11.0, whereas the Ubuntu version is at 1.14.0

Removing the old version fixed the problem.

Changed in python-certbot (Ubuntu):
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.