Deploying with node specific overrrides is error prone

[John Fulton]

Using Node Specific Overrides to pass a different list of devices to a Ceph deployment [1] is error prone. The example in the doc embeds JSON in YAML. It's easy to make a simple mistake in the hand pasted JSON but not realize it until the deployment fails nearly an hour later.

It would be better if TripleO provided a simple utility to create a JSON environment file with NodeDataLookup set for all disks (except the root disk) found in introspection data. It could converts the output of `openstack baremetal introspection data save <node>` into a Heat environment file o be passed to `openstack overcloud deploy`.

[1] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/node_specific_hieradata.html

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/696401

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: master
Review: https://review.opendev.org/696401
Reason: Clearing the gate now, see https://bugs.launchpad.net/tripleo/+bug/1856864
Do not restore the patch yet, I'll take care of it when the gate is back online.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/696401
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=fb8009458c949c3815256d126a483d47dae90b18
Submitter: Zuul
Branch: master

commit fb8009458c949c3815256d126a483d47dae90b18
Author: John Fulton <email address hidden>
Date: Wed Nov 27 14:46:27 2019 -0500

    Provide utility to generate NodeDataLookup from Ironic

    Add small python program to tools directory for deployers
    which generates a THT environment file containing valid
    JSON to be used with NodeDataLookup. Uses ironic generated
    data as input.

    Tools like this end up getting written in the field after
    deployers experience failures. Better to provide them an
    official tool up front to solve this problem so they do
    not have a bad experience.

    Change-Id: I513e6564d18c7fe1c1e5dd63f164f3b5042cacf4
    Closes-bug: #1854214

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/700035

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/700036

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/700037

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/700038

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/700035
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=c9dd7ebcf12f8ff3131ed6858724b4d99d40ea17
Submitter: Zuul
Branch: stable/train

commit c9dd7ebcf12f8ff3131ed6858724b4d99d40ea17
Author: John Fulton <email address hidden>
Date: Wed Nov 27 14:46:27 2019 -0500

    Provide utility to generate NodeDataLookup from Ironic

    Add small python program to tools directory for deployers
    which generates a THT environment file containing valid
    JSON to be used with NodeDataLookup. Uses ironic generated
    data as input.

    Tools like this end up getting written in the field after
    deployers experience failures. Better to provide them an
    official tool up front to solve this problem so they do
    not have a bad experience.

    Change-Id: I513e6564d18c7fe1c1e5dd63f164f3b5042cacf4
    Closes-bug: #1854214
    (cherry picked from commit fb8009458c949c3815256d126a483d47dae90b18)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/700036
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=66645d5ff610ed0179c5a840f873698f1dbf6a24
Submitter: Zuul
Branch: stable/stein

commit 66645d5ff610ed0179c5a840f873698f1dbf6a24
Author: John Fulton <email address hidden>
Date: Wed Nov 27 14:46:27 2019 -0500

    Provide utility to generate NodeDataLookup from Ironic

    Add small python program to tools directory for deployers
    which generates a THT environment file containing valid
    JSON to be used with NodeDataLookup. Uses ironic generated
    data as input.

    Tools like this end up getting written in the field after
    deployers experience failures. Better to provide them an
    official tool up front to solve this problem so they do
    not have a bad experience.

    Change-Id: I513e6564d18c7fe1c1e5dd63f164f3b5042cacf4
    Closes-bug: #1854214
    (cherry picked from commit fb8009458c949c3815256d126a483d47dae90b18)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/700038
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=16393095485b37602e63e89a57da1edc176ec6ea
Submitter: Zuul
Branch: stable/queens

commit 16393095485b37602e63e89a57da1edc176ec6ea
Author: John Fulton <email address hidden>
Date: Wed Nov 27 14:46:27 2019 -0500

    Provide utility to generate NodeDataLookup from Ironic

    Add small python program to tools directory for deployers
    which generates a THT environment file containing valid
    JSON to be used with NodeDataLookup. Uses ironic generated
    data as input.

    Tools like this end up getting written in the field after
    deployers experience failures. Better to provide them an
    official tool up front to solve this problem so they do
    not have a bad experience.

    Change-Id: I513e6564d18c7fe1c1e5dd63f164f3b5042cacf4
    Closes-bug: #1854214
    (cherry picked from commit fb8009458c949c3815256d126a483d47dae90b18)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/700037
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=55b0e371b0a71b3b63c4d6f57ee47198d178e292
Submitter: Zuul
Branch: stable/rocky

commit 55b0e371b0a71b3b63c4d6f57ee47198d178e292
Author: John Fulton <email address hidden>
Date: Wed Nov 27 14:46:27 2019 -0500

    Provide utility to generate NodeDataLookup from Ironic

    Add small python program to tools directory for deployers
    which generates a THT environment file containing valid
    JSON to be used with NodeDataLookup. Uses ironic generated
    data as input.

    Tools like this end up getting written in the field after
    deployers experience failures. Better to provide them an
    official tool up front to solve this problem so they do
    not have a bad experience.

    Change-Id: I513e6564d18c7fe1c1e5dd63f164f3b5042cacf4
    Closes-bug: #1854214
    (cherry picked from commit fb8009458c949c3815256d126a483d47dae90b18)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 12.1.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/708715

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708715
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a67981d1e8117c74c20adf3bc657eaf81cb8971a
Submitter: Zuul
Branch: master

commit a67981d1e8117c74c20adf3bc657eaf81cb8971a
Author: Francesco Pantano <email address hidden>
Date: Wed Feb 19 18:57:43 2020 +0100

    NodeDataLookup utility should rely on python env

    This change ensures that if there are several versions
    of Python installed, the interpreter used is the first
    one on the environment's $PATH.

    Change-Id: I48eec3d22e5a2a018554c6864efd52e4ab3c87ce
    Related-Bug: #1854214

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/708978

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/708979

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/708980

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/708981

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708978
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=eb31a0fa5380f52d803a33e58706578792e6b0e9
Submitter: Zuul
Branch: stable/queens

commit eb31a0fa5380f52d803a33e58706578792e6b0e9
Author: Francesco Pantano <email address hidden>
Date: Wed Feb 19 18:57:43 2020 +0100

    NodeDataLookup utility should rely on python env

    This change ensures that if there are several versions
    of Python installed, the interpreter used is the first
    one on the environment's $PATH.

    Change-Id: I48eec3d22e5a2a018554c6864efd52e4ab3c87ce
    Related-Bug: #1854214
    (cherry picked from commit a67981d1e8117c74c20adf3bc657eaf81cb8971a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708980
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=e22d9fb073a7253ca452fa345d5709b3301e4d64
Submitter: Zuul
Branch: stable/stein

commit e22d9fb073a7253ca452fa345d5709b3301e4d64
Author: Francesco Pantano <email address hidden>
Date: Wed Feb 19 18:57:43 2020 +0100

    NodeDataLookup utility should rely on python env

    This change ensures that if there are several versions
    of Python installed, the interpreter used is the first
    one on the environment's $PATH.

    Change-Id: I48eec3d22e5a2a018554c6864efd52e4ab3c87ce
    Related-Bug: #1854214
    (cherry picked from commit a67981d1e8117c74c20adf3bc657eaf81cb8971a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708981
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=e2a5fef9769cc6f326ebc3eb0662e47cf64bbee5
Submitter: Zuul
Branch: stable/rocky

commit e2a5fef9769cc6f326ebc3eb0662e47cf64bbee5
Author: Francesco Pantano <email address hidden>
Date: Wed Feb 19 18:57:43 2020 +0100

    NodeDataLookup utility should rely on python env

    This change ensures that if there are several versions
    of Python installed, the interpreter used is the first
    one on the environment's $PATH.

    Change-Id: I48eec3d22e5a2a018554c6864efd52e4ab3c87ce
    Related-Bug: #1854214
    (cherry picked from commit a67981d1e8117c74c20adf3bc657eaf81cb8971a)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates rocky-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates stein-eol release.