Unable to specify embedded newline in nova_vendor_data_overrides

[Bob Gibson]

I'm trying to configure static vendor data in nova-api-metadata so we can automatically register instances with Ubuntu Pro on boot. The process works when I manually create `/etc/nova/vendor_data.json` with the following content as specified in the cloud-init documentation: https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data:

```
# cat /etc/nova/vendor_data.json
{"cloud-init": "#cloud-config\nubuntu_pro:\n token: <REDACTED>"}
```

However, I'm unable to create the same content using the `nova_vendor_data_overrides` variable.

I can get close if I specify the string value in single-quotes:

```
nova_vendor_data_overrides:
  cloud-init: '#cloud-config\nubuntu_pro:\n token: <REDACTED>'
```

but the rendered JSON includes an extraneous backslash character for each newline:

```
# cat /etc/nova/vendor_data.json
{"cloud-init": "#cloud-config\\nubuntu_pro:\\n token: <REDACTED>"}
```

i.e. '\\n` instead of '\n'

Perhaps this is just the natural consequence of converting YAML strings containing embedded newlines to JSON. If so, I'm hoping that someone has found an alternate way to achieve this and can offer a working example.

Cheers,
/rjg

[Dmitriy Rabotyagov (noonedeadpunk)]

I think it's a valid bug.

Logic which leads to the result is part of config_template module:
https://opendev.org/openstack/ansible-config_template/src/branch/master/plugins/action/config_template.py#L613-L619

So far I can't tell why it's done this way and what a good usecase it's covering.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ansible-config_template/+/924217

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924218

[Dmitriy Rabotyagov (noonedeadpunk)]

With these patches you should be able to define variable as following and get the aimed result:

nova_vendor_data_overrides:
  cloud-init: "#cloud-config\nubuntu_pro:\n token: <REDACTED>"

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ansible-config_template/+/924217
Committed: https://opendev.org/openstack/ansible-config_template/commit/13b52a62cd35751b26be520a72214c2d10e8c471
Submitter: "Zuul (22348)"
Branch: master

commit 13b52a62cd35751b26be520a72214c2d10e8c471
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Jul 16 13:26:14 2024 +0200

    Respect yml_multilines for JSON configs

    At the moment there is no good way to have new lines ("\n") in JSON
    files and not to convert the result to a list.
    In the meanwhile this can be totally expected/required thing to do,
    for example to define cloud-init vendor data [1]

    [1] https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data

    Related-Bug: #2073171
    Change-Id: I00d74a26357a8458e38d9eb816a3f3eceeae1e0d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924218
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/1b6740f3f897062228ccc8b767208e0d41530f1a
Submitter: "Zuul (22348)"
Branch: master

commit 1b6740f3f897062228ccc8b767208e0d41530f1a
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Jul 16 13:30:13 2024 +0200

    Allow to supply multiline overrides to vendor_data

    According to the documentation, it is expected to have a multiline data
    inside vendor_data.json [1]

    [1] https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data

    Depends-On: https://review.opendev.org/c/openstack/ansible-config_template/+/924217
    Closes-Bug: #2073171
    Change-Id: Ifc1239e4ef768e94c44d8d07df7a0b93c73638f9

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924280

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924281

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924282

[Bob Gibson (rjg-at-work)]

I've applied the patches and confirmed that they fix my use case. Thanks very much for the fix and the quick response!

Cheers,
/rjg

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924280
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/f174c87f08ea132a2be6d9988c98140c01dbd6d2
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit f174c87f08ea132a2be6d9988c98140c01dbd6d2
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Jul 16 13:30:13 2024 +0200

    Allow to supply multiline overrides to vendor_data

    According to the documentation, it is expected to have a multiline data
    inside vendor_data.json [1]

    [1] https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data

    Depends-On: https://review.opendev.org/c/openstack/ansible-config_template/+/924217
    Closes-Bug: #2073171
    Change-Id: Ifc1239e4ef768e94c44d8d07df7a0b93c73638f9
    (cherry picked from commit 1b6740f3f897062228ccc8b767208e0d41530f1a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924281
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/449b6e869fdb3001410eb97fd96bf680ac2e976b
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 449b6e869fdb3001410eb97fd96bf680ac2e976b
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Jul 16 13:30:13 2024 +0200

    Allow to supply multiline overrides to vendor_data

    According to the documentation, it is expected to have a multiline data
    inside vendor_data.json [1]

    [1] https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data

    Depends-On: https://review.opendev.org/c/openstack/ansible-config_template/+/924217
    Closes-Bug: #2073171
    Change-Id: Ifc1239e4ef768e94c44d8d07df7a0b93c73638f9
    (cherry picked from commit 1b6740f3f897062228ccc8b767208e0d41530f1a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/924282
Committed: https://opendev.org/openstack/openstack-ansible-os_nova/commit/51b80aaa8b2f6821c7af55e614e624580f8099a0
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 51b80aaa8b2f6821c7af55e614e624580f8099a0
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Jul 16 13:30:13 2024 +0200

    Allow to supply multiline overrides to vendor_data

    According to the documentation, it is expected to have a multiline data
    inside vendor_data.json [1]

    [1] https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data

    Depends-On: https://review.opendev.org/c/openstack/ansible-config_template/+/924217
    Closes-Bug: #2073171
    Change-Id: Ifc1239e4ef768e94c44d8d07df7a0b93c73638f9
    (cherry picked from commit 1b6740f3f897062228ccc8b767208e0d41530f1a)