fetch.configure_sources parses gpg keys incorrectly

[Pen Gale]

The configure_sources routine, in charmhelpers/fetch/__init__.py, exhibits surprising behavior that makes writing valid config.yaml and bundle.yaml files more difficult.

After the function retrieves a list of sources and keys from the config object, it calls yaml.safe_load on the values that it has retrieved. These values were parsed as yaml when the config.yaml was initially read, so they effectively get parsed twice.

Most of the time, this does not cause problems, as operators use a one item or space separated list of simple strings when setting install_sources and install_keys. These lists survive the double parsing.

Things get weird when somebody does something more complicated, like embedding a GPG key in the config yaml. Since the yaml gets parsed twice, you need to populate install_keys with a string that contains a yaml formatted list. So the resultant yaml looks like this:

    install_keys: |
        - |
            -----BEGIN PGP PUBLIC KEY BLOCK-----
            ...

Note that the yaml list is escaped by a pipe. This is unintuitive and surprising, leading to confusion when creating config yaml files in the field.

The comments suggest that configure_sources behaves this way to work around a bug that the maintainers ran into in 2014. I suspect that there is a better way of addressing the bug, which leads to more intuitive, less surprising config.yaml and bundle.yaml files. Thus this ticket :-)

[Stuart Bishop (stub)]

Alas, Juju does not provide richer data structures than simple strings, integers and boolean (most likely deliberately, to simplify CLI setting of the fields, although there are open bugs on adding this feature).

I've never come across a more usable approach than the trick of using the YAML | string escape mechanism to embed richer data structures. I'm flagging this as a duplicate of Bug #1320080, as unless someone can come up with a more usable approach we are stuck with this awful API (both in charm-helpers and the apt layer).