sensitive metadata and jinja templates

[Peter Surda]

The documentation doesn't explain well how to use sanitized metadata (that will show up in instance-data-sensitive.json rather than instance-data.json) with jinja templates inside user-data. As far as I can see, it doesn't work. The source code mentions two magic keys that are sanitized: "merged_cfg" and "security-credentials". Defining variables with these names inside meta-data correctly sanitizes them and only puts them inside files only readable by root, however then they don't work inside user-data as jinja templates (as "{{ds.meta_data.security-credentials}}", for example), they are instead replaced by CI_MISSING_JINJA_VAR. Using differently named variables makes the template work, but they aren't sanitized in the logs/runtime files.

In what way, if any, this is supposed to work? Should I instead just chmod the relevant log/runtime files through an entry in bootcmd?

[James Falcon (falcojr)]

Jinja templates do not work with instance-data-sensitive.json. They only worked in instance-data.json, as explained in https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data

[Peter Surda (surda)]

Thank you for your response. I still find both the design as well as documentation confusing. For example, the documentation says:

> Because user-data and vendor-data can contain passwords both of these files are readonly for root as well.

Ok so what's the rationale for the different handling of the sensitive variables in meta-data then?

> Any instance-data-sensitive.json variables are surfaced as dot-delimited jinja template variables because cloud-config modules are run as ‘root’ user."

I have no idea what that means. Does it explain or contradict the behaviour I'm observing?

I found by trial that the only file storing the custom meta data variables which is readable by non-root is instance-data.json, so now I chmod it at the top of in bootcmd and that solves the technical aspect of the issue for me.

I don't per se mind the behaviour, I just find the documentation confusing. If there was at least an example of how to deal with situations like mine, that would be a huge improvement.

[James Falcon (falcojr)]

Yes, I see what you mean. The documentation is confusing. The concept of sensitive or redacted metadata was added to accommodate the "cloud-init query" command. On a launched instance, you can run something like:

cloud-init query merged_cfg

As root, you'll see the entire config, as non-root you'll see redacted as we don't want to expose sensitive data to non-root users. This is what your second quote refers to.

Since cloud-init is run as root, I think it was an oversight that the template rendering wasn't extended to include the sensitive data when this change was made. I'll put up a PR to allow sensitive data in a jinja templated config and hopefully clarify the documentation some.

[James Falcon (falcojr)]

See https://github.com/canonical/cloud-init/pull/917

[James Falcon (falcojr)]

This bug is believed to be fixed in cloud-init in version 21.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3881