upgrade-charm fails with a Traceback

[Brian Murray]

2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 command: /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/bin/python /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-mpuep57n/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- setuptools wheel
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 cwd: None
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Complete output (39 lines):
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Looking in links: wheelhouse
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Processing ./wheelhouse/setuptools-65.5.0.tar.gz
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Getting requirements to build wheel: started
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Getting requirements to build wheel: finished with status 'error'
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 ERROR: Command errored out with exit status 1:
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 command: /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/bin/python /tmp/tmplhr1uvlg get_requires_for_build_wheel /tmp/tmpleq4lk0a
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 cwd: /tmp/pip-install-yuyavbbo/setuptools
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Complete output (29 lines):
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Traceback (most recent call last):
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/tmplhr1uvlg", line 280, in <module>
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 main()
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/tmplhr1uvlg", line 263, in main
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 json_out['return_val'] = hook(**hook_input['kwargs'])
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/tmplhr1uvlg", line 114, in get_requires_for_build_wheel
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 return hook(config_settings)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 return self._get_build_requires(config_settings, requirements=['wheel'])
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/build_meta.py", line 320, in _get_build_requires
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 self.run_setup()
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/build_meta.py", line 335, in run_setup
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 exec(code, locals())
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "<string>", line 87, in <module>
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/__init__.py", line 87, in setup
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 return distutils.core.setup(**attrs)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/_distutils/core.py", line 147, in setup
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 _setup_distribution = dist = klass(attrs)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/dist.py", line 476, in __init__
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 _Distribution.__init__(
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/_distutils/dist.py", line 282, in __init__
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 self.finalize_options()
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/dist.py", line 899, in finalize_options
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 for ep in sorted(loaded, key=by_order):
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/dist.py", line 898, in <lambda>
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 loaded = map(lambda e: e.load(), filtered)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/tmp/pip-install-yuyavbbo/setuptools/setuptools/_vendor/importlib_metadata/__init__.py", line 196, in load
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 return functools.reduce(getattr, attrs, module)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 ----------------------------------------
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 ERROR: Command errored out with exit status 1: /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/bin/python /tmp/tmplhr1uvlg get_requires_for_build_wheel /tmp/tmpleq4lk0a Check the logs for full command output.
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 ERROR: Command errored out with exit status 1: /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/bin/python /var/lib/juju/agents/unit-autocert-haproxy-10/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-mpuep57n/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- setuptools wheel Check the logs for full command output.
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 Traceback (most recent call last):
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/var/lib/juju/agents/unit-autocert-haproxy-10/charm/hooks/upgrade-charm", line 8, in <module>
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 basic.bootstrap_charm_deps()
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/var/lib/juju/agents/unit-autocert-haproxy-10/charm/lib/charms/layer/basic.py", line 202, in bootstrap_charm_deps
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 _update_if_newer(pip, pre_install_pkgs)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/var/lib/juju/agents/unit-autocert-haproxy-10/charm/lib/charms/layer/basic.py", line 366, in _update_if_newer
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 check_call([pip, 'install', '-U', '--no-index', '-f', 'wheelhouse',
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 raise CalledProcessError(retcode, cmd)
2022-11-29 22:22:16 WARNING unit.autocert-haproxy/10.upgrade-charm logger.go:60 subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-autocert-haproxy-10/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status 1.
2022-11-29 22:22:16 ERROR juju.worker.uniter.operation runhook.go:140 hook "upgrade-charm" (via explicit, bespoke hook script) failed: exit status 1

[Brian Murray (brian-murray)]

I'm using rev 34 of the autocert charm on Ubuntu 20.04 LTS. As I understand it that is the latest version of the charm:

(mojo-stg-proposed-migration)stg-proposed-migration@wendigo:~$ juju info autocert --series focal --arch amd64
name: autocert
charm-id: 0CBXXKWorqtrHmlv4PVcE4Yqpo86onFI
summary: Automated X.509 certificate/key renewal service client
publisher: Autocert Charmers
supports: jammy, focal, bionic, xenial, trusty
subordinate: true
store-url: https://charmhub.io/autocert
description: |
  This charm installs and configures the Autocert X.509 certificate/key renewal
  service client, and automatically restarts relevant services when new
  certificates and/or keys are installed.
relations:
  provides:
    nrpe-external-master: nrpe-external-master
  requires:
    juju-info: juju-info
channels: |
  latest/stable: 8a21f91 2022-10-24 (34) 6MB
  latest/candidate: ↑
  latest/beta: ↑
  latest/edge: d0ae3d7 2022-10-20 (18) 6MB

[Haw Loeung (hloeung)]

As per MM, this is due to a bug that's been fixed in Juju 2.9.36 (LP:1993758) where it selects the charm built for the wrong series. You'll want to upgrade to the charm with the right series, in your case, r34 per the `juju info` output provided.

It can also be seen that a unit is stuck and not upgrading, running `juju resolved --no-retry` once and wait until you see:

| juju.worker.uniter.charm bundles.go:78 downloading ch:amd64/focal/autocert-34 from API server

You may need to re-run the `resolved --no-retry` a second time.