An Apt source cannot use the $KEY_FILE replacement variable

[thingy]

If you define an apt source which has a '[signed-by=$KEY_FILE]' definition then the installer crashes.

Documentation: https://github.com/canonical/cloud-init/blob/main/doc/examples/cloud-config-apt.txt#L323-L324

Example:

---snip---
sources:
  ansible-ppa.list:
    source: "deb [signed-by=$KEY_FILE] https://ppa.launchpadcontent.net/ansible/ansible/ubuntu $RELEASE main"
    key: |
      -----BEGIN PGP PUBLIC KEY BLOCK-----
      Comment: Hostname:
      Version: Hockeypuck 2.1.0-152-ga266fd3

      xsFNBFOXbTABEADKLcY3MYZyHIiCEu8cQ+0UzxZolTYZf8xZ06/d8xzUW/UBPTLV
      x/40IzWJEZU87GCYVXUhdYXECrFQKQOOEWQswBxOt26/g0nIrU0edZ9cCZ9o9+ZO
      sKDcScWNxWeclLr2+YHdmX9eph+2p+zIPmhNJaaWV6/bSo1fBi7ganZFRAc13zO8
      jtQsYyIeoAFwjt1vbk9CifdFhEchRVbwvZYhZDWfBZjhZ15UOunjo47gEkpK0PsW
      HY5N3/c75pGMwMTCjHMKno6KFzhdKLaz/QgbNnpy1aMkA/LITeU6Pgg6iqofrSY3
      Fx73MCAb8EEytvSV/65wztKzexHjGYxhm5ygoET5tPyXciX0+XzojkZAHzUEeT5z
      4kgL6OLyn+JQAyRDNnOJEkYmvwQveZuUjsYUkgc5DJdg6w7lTgQHWObOENpXls3B
      DnphA4DJyc0PQODCxdSZo4ZXMLn5lE/0qvAJ6g/wntY/ee/vRKwy9iDMOWQvihTI
      Y/L70/TnE4qZdaNbOJEvW59LT6GBTrcU1MX2fS5hU/mQa2CNixSVZG98rba3tfNa
      LDDNqivkcxtWryEWd0giBzzUS+MhtxhHm77YgxNVyyFn8bXb25/W+Jq+VvWWx2KD
      3ZwVD37X8wIBRLVozoH0jHVW7jKTnf2z+D6FCM+pVlm8zV1upXbHbdaRqwARAQAB
      zR9MYXVuY2hwYWQgUFBBIGZvciBBbnNpYmxlLCBJbmMuwsF4BBMBAgAiBQJTl20w
      AhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRCTxKP9e7nDZwzdD/9yFq+3
      ptLyWP1JN3RXsvTIJLhhjOqMMJCOocV0G3slHeUtdIDmvd2RaS8PeFNSd9QFbUzk
      a7pBXec3a7kgEVorty2/+cZSpEVHT26oAbJnoH7E1YGii8bbnk9LqOCF/nLpasEq
      PKqtVzXz+vGB7G0ox8qmRm6JynrMeLBftYemxJ8e4fii6APJxE6FPz/AvgcN3BtB
      guseFwcga6lGcmp0JLofGhTbejfS9dW9bnCFJBtRfzUVd+Cb8aYuzV5zgCJgqOBJ
      Hf0L6xDwn8UuzRfi0MWzEskuC/KxUJqVHMuTaLdrSeP1/czompetbsSHTfDklf/p
      4EbSM9VqZFiKr17we0LdigAEv2JnL2Qj0oN3eRNU0sDJ3kZvh5qhnteNgSRM/el0
      Sly+34CRZzX25vd+pOTgwK6VFx3wvWfqn3pVNT9ASNalhogFXuBY2ukYJaiUTPsS
      R4+KHiD3eIhWyayBGP+GRvGb5dVC29k4CjLvQM9lISmZrrxGmMGoV9S1dh7siZYE
      CaVW0nKI95d6bBHaKH1g5HJ7NEsVTwf8LRY/FFpLsEPVw4HNVqqqwhuJnSW70WQ3
      blh0RIX/+z9hAla+M0kix7r0lS89ZBdTgwPuiCrkPRpoxi06ah/Q62uP3ZN4+dew
      1sBZdaC9kaKdOOWTqArVnaPObQgUUJFhY9wUsA==
      =uKv4
      -----END PGP PUBLIC KEY BLOCK-----
---snip---

Results in the following error:

---snip---
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: finish: cmd-apt-config: FAIL: curtin command apt-config
Jul 9 20:59:18 ubuntu-server curtin_event.2034.1[2316]: finish: cmd-apt-config: FAIL: curtin command apt-config
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: Traceback (most recent call last):
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/commands/main.py", line 202, in main
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: ret = args.func(args)
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/commands/apt_config.py", line 663, in apt_command
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: handle_apt(apt_cfg, target)
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/commands/apt_config.py", line 103, in handle_apt
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: add_apt_sources(cfg['sources'], target,
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/commands/apt_config.py", line 453, in add_apt_sources
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: source = util.render_string(source, template_params)
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/util.py", line 1203, in render_string
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: return basic_template_render(content, params)
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/util.py", line 1193, in basic_template_render
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: return BASIC_MATCHER.sub(replacer, content)
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: File "/snap/subiquity/3359/lib/python3.8/site-packages/curtin/util.py", line 1191, in replacer
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: return str(selected_params[key])
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: KeyError: 'KEY_FILE'
Jul 9 20:59:18 ubuntu-server subiquity_log.2034[2316]: 'KEY_FILE'
Jul 9 20:59:18 ubuntu-server systemd[1]: run-u20.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Jul 9 20:59:18 ubuntu-server systemd[1]: run-u20.service: Failed with result 'exit-code'.
---snip---

The reason I was trying to make use of '[signed-by=$KEY_FILE]' is that apt-key is deprecated and I wanted to avoid getting repeated warnings when using apt-get update.

[Chris Hillery (ceejatec)]

I fought with this for a solid day too. It turns out that the subiquity autoinstall config file is NOT a cloud-init file and will not be processed by cloud-init, so the documentation you referenced is not the correct place to look.

The autoinstall config file is documented here: https://ubuntu.com/server/docs/install/autoinstall-reference and unfortunately the magic of $KEY_FILE is not implemented in subiquity.

This is breathtakingly confusing, since the autoinstall config is very close to a subset of cloud-init. It's even more confusing because subiquity uses cloud-init to find its config file, and there's even a module called "cc_ubuntu_autoinstall" in cloud-init, which as far as I can tell does almost nothing and is mostly there so that cloud-init doesn't choke on the top-level "autoinstall:" key in the file. And it's even MORE confusing because the autoinstall documentation I linked above doesn't even mention the required top-level "autoinstall:" field - that's only documented in an off-hand comment as part of the introductory page (https://ubuntu.com/server/docs/install/autoinstall) - so as documented it looks like it IS the same as cloud-init. Sigh.

I was unable to find any way to do this "right" with autoinstall. There's no hook in the process that lets you create the key files in an appropriate location such that the apt: block will be able to use them. I ultimately gave up and used the documented arrangement, which uses the deprecated approach of loading the keys globally. This ticket could be re-construed as an enhancement request to make subiquity support the same feature that cloud-init already does.

FYI, you can actually do two different things to get the cloud-init implementation of apt: to consume your config, but unfortunately neither of them work in the context of an autoinstall. First, you can put the apt: and packages: keys at the top-level of the config file (siblings to autoinstall:). Secondly, you can put a user-data: key under autoinstall: and then put any cloud-init keys under that. However, through experimentation, I can say that those end up being executed at wrong times during the autoinstall process, so neither works. But it might be useful for some other tasks.