TLS everywhere deployments fail when using many composable networks

[Harald Jensås]

In Edge/spine-and-leaf use case with multiple composable networks deployment fail with " Invalid input for field/attribute compact_service_HTTP"

overcloud.ControllerDc.1.ControllerDc:
  resource_type: OS::TripleO::ControllerDcServer
  physical_resource_id:
  status: CREATE_FAILED
  status_reason: |
    BadRequest: resources.ControllerDc: Invalid input for field/attribute compact_service_HTTP. Value: ["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]. u'["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]' is too long (HTTP 400) (Request-ID: req-2ff28ca1-6a5b-4208-94dc-8d94fc2e286a)
overcloud.ControllerDc.0.ControllerDc:
  resource_type: OS::TripleO::ControllerDcServer
  physical_resource_id:
  status: CREATE_FAILED
  status_reason: |
    BadRequest: resources.ControllerDc: Invalid input for field/attribute compact_service_HTTP. Value: ["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]. u'["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]' is too long (HTTP 400) (Request-ID: req-cd8dad4f-f19f-4595-a64a-626ec5595de5)
overcloud.ControllerDc.2.ControllerDc:
  resource_type: OS::TripleO::ControllerDcServer
  physical_resource_id:
  status: CREATE_FAILED
  status_reason: |
    BadRequest: resources.ControllerDc: Invalid input for field/attribute compact_service_HTTP. Value: ["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]. u'["ctlplane", "storage", "storageedge1", "storageedge2", "storageedge3", "storageedge4", "storagemgmt", "storagemgmtedge1", "storagemgmtedge2", "storagemgmtedge3", "storagemgmtedge4", "internalapi", "internalapiedge1", "internalapiedge2", "internalapiedge3", "internalapiedge4", "tenantedge1", "tenantedge2", "tenantedge3", "tenantedge4", "external", "management"]' is too long (HTTP 400) (Request-ID: req-f02b9826-2d37-4a33-b4de-75c3cd3840b4)
END return value: 0
Heat Stack create failed.
Heat Stack create failed.
END return value: 1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/646005

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/646005
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=d5ecc1f6518d14f1ae6fefebd73533f11762bedf
Submitter: Zuul
Branch: master

commit d5ecc1f6518d14f1ae6fefebd73533f11762bedf
Author: Harald Jensås <email address hidden>
Date: Sat Mar 23 14:50:27 2019 +0100

    Make krb-service-principal metadata per-Role

    Not all roles are connected to all networks, there is no
    need to create metadata for networks not associated with
    the role.

    In edge/spine-and-leaf deployments the total number of
    composable networks used can be high. Passing all the
    networks we quickly go beyond the nova metadata fields
    size limit (each field cannot exceed 256 bytes).

    Also update tools/check-up-to-date.sh script to use the
    simple yaml-diff.py instead of diff. The env generator
    code will sort data, while jinja rendered environments
    are not sorted, thus need to diff the data in yaml not
    the text.

    Closes-Bug: #1821377
    Change-Id: I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.5.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/656797

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/661917

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/661917
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=560f8853222378b4516e5ef022865479d8ddd3a9
Submitter: Zuul
Branch: stable/queens

commit 560f8853222378b4516e5ef022865479d8ddd3a9
Author: Harald Jensås <email address hidden>
Date: Sat Mar 23 14:50:27 2019 +0100

    Make krb-service-principal metadata per-Role

    Not all roles are connected to all networks, there is no
    need to create metadata for networks not associated with
    the role.

    In edge/spine-and-leaf deployments the total number of
    composable networks used can be high. Passing all the
    networks we quickly go beyond the nova metadata fields
    size limit (each field cannot exceed 256 bytes).

    Also update tools/check-up-to-date.sh script to use the
    simple yaml-diff.py instead of diff. The env generator
    code will sort data, while jinja rendered environments
    are not sorted, thus need to diff the data in yaml not
    the text.

    Conflicts:
        environments/ssl/enable-internal-tls.j2.yaml
        extraconfig/nova_metadata/krb-service-principals/role.role.j2.yaml

    Closes-Bug: #1821377
    Change-Id: I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b
    (cherry picked from commit d5ecc1f6518d14f1ae6fefebd73533f11762bedf)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/656797
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=218582c4cc47d35181ce34a2b27f899fc1316be3
Submitter: Zuul
Branch: stable/rocky

commit 218582c4cc47d35181ce34a2b27f899fc1316be3
Author: Harald Jensås <email address hidden>
Date: Sat Mar 23 14:50:27 2019 +0100

    Make krb-service-principal metadata per-Role

    Not all roles are connected to all networks, there is no
    need to create metadata for networks not associated with
    the role.

    In edge/spine-and-leaf deployments the total number of
    composable networks used can be high. Passing all the
    networks we quickly go beyond the nova metadata fields
    size limit (each field cannot exceed 256 bytes).

    Also update tools/check-up-to-date.sh script to use the
    simple yaml-diff.py instead of diff. The env generator
    code will sort data, while jinja rendered environments
    are not sorted, thus need to diff the data in yaml not
    the text.

    Conflicts:
        environments/ssl/enable-internal-tls.j2.yaml

    Closes-Bug: #1821377
    Change-Id: I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b
    (cherry picked from commit d5ecc1f6518d14f1ae6fefebd73533f11762bedf)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/664883

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/664885

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/664883
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=fb5f69dcb2f84797d75da8b2a75692e06ed873f0
Submitter: Zuul
Branch: stable/rocky

commit fb5f69dcb2f84797d75da8b2a75692e06ed873f0
Author: Harald Jensås <email address hidden>
Date: Wed Jun 12 15:04:06 2019 +0200

    [FUP] Backport - krb-service-principal metadata per-Role

    The rocky and queens backport of change[1] missed updating a
    deprecated environment file that is no longer present since
    stein.

    This change updates the deprecated environment file to be a
    j2 file with the jinja code to render the per-role metadata.

    [1] I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b

    Related-Bug: #1821377
    Change-Id: Ie7c3c448e8d0f53a5347a19b6d0b99a0c6097114

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.4.0 release.

[Harald Jensås (harald-jensas)]

Re-opening, the fix implemented does not solve the problem.

The compact_services_HTTP still have all the networks except the tenant network wich is filterd by name[1].

outputs:
- description: actual metadata entries that will be passed to the server.
  output_key: metadata
  output_value:
    compact_service_HTTP:
    - ctlplane
    - storage
    - storageedge1
    - storageedge2
    - storageedge3
    - storageedge4
    - storagemgmt
    - storagemgmtedge1
    - storagemgmtedge2
    - storagemgmtedge3
    - storagemgmtedge4
    - internalapi
    - internalapiedge1
    - internalapiedge2
    - internalapiedge3
    - internalapiedge4
    - tenantedge1
    - tenantedge2
    - tenantedge3
    - tenantedge4
    - external
    - management
    compact_service_haproxy:
    - ctlplane
    - storage
    - storagemgmt
    - internalapi
    compact_service_libvirt-vnc:
    - internalapi
    compact_service_mysql:
    - internalapi
    compact_service_neutron:
    - internalapi
    compact_service_novnc-proxy:
    - internalapi
    compact_service_rabbitmq:
    - internalapi
    managed_service_haproxyctlplane: haproxy/overcloud.ctlplane.redhat.local
    managed_service_haproxyexternal: haproxy/overcloud.redhat.local
    managed_service_haproxyinternal_api: haproxy/overcloud.internalapi.redhat.local
    managed_service_haproxystorage: haproxy/overcloud.storage.redhat.local
    managed_service_haproxystorage_mgmt: haproxy/overcloud.storagemgmt.redhat.local
    managed_service_mysqlinternal_api: mysql/overcloud.internalapi.redhat.local
    managed_service_redisinternal_api: redis/overcloud.internalapi.redhat.local

[1] https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/apache/apache-baremetal-puppet.j2.yaml#L57-L69

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/668407

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668407
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=578bcb2ffad32c6a39d68b5dc360504e95972ffa
Submitter: Zuul
Branch: master

commit 578bcb2ffad32c6a39d68b5dc360504e95972ffa
Author: Harald Jensås <email address hidden>
Date: Mon Jul 1 12:05:39 2019 +0200

    Per-Role krb-service-principal for CompactServices

    Filter krb-service-principals for the CompactServices
    based on the networks associated with the role.

    Filtering for the IndividualServices was added in previous
    fix https://review.openstack.org/646005, which did'nt
    fully fix the bug.

    Closes-Bug: #1821377
    Change-Id: Id54477ca5581e1f5fe8a09c3bc60a238d114dbb2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/668797

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/668798

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/668800

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668797
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=223ddba9137a3c9129fc33593db086518bf75a78
Submitter: Zuul
Branch: stable/stein

commit 223ddba9137a3c9129fc33593db086518bf75a78
Author: Harald Jensås <email address hidden>
Date: Mon Jul 1 12:05:39 2019 +0200

    Per-Role krb-service-principal for CompactServices

    Filter krb-service-principals for the CompactServices
    based on the networks associated with the role.

    Filtering for the IndividualServices was added in previous
    fix https://review.openstack.org/646005, which did'nt
    fully fix the bug.

    Closes-Bug: #1821377
    Change-Id: Id54477ca5581e1f5fe8a09c3bc60a238d114dbb2
    (cherry picked from commit 578bcb2ffad32c6a39d68b5dc360504e95972ffa)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668798
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f72d576f67a0751289eb3d2aac53933c89b71e90
Submitter: Zuul
Branch: stable/rocky

commit f72d576f67a0751289eb3d2aac53933c89b71e90
Author: Harald Jensås <email address hidden>
Date: Mon Jul 1 12:05:39 2019 +0200

    Per-Role krb-service-principal for CompactServices

    Filter krb-service-principals for the CompactServices
    based on the networks associated with the role.

    Filtering for the IndividualServices was added in previous
    fix https://review.openstack.org/646005, which did'nt
    fully fix the bug.

    Closes-Bug: #1821377
    Change-Id: Id54477ca5581e1f5fe8a09c3bc60a238d114dbb2
    (cherry picked from commit 578bcb2ffad32c6a39d68b5dc360504e95972ffa)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668800
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=43cf4c13aad798e0438e534f23386175ce21775e
Submitter: Zuul
Branch: stable/queens

commit 43cf4c13aad798e0438e534f23386175ce21775e
Author: Harald Jensås <email address hidden>
Date: Mon Jul 1 12:05:39 2019 +0200

    Per-Role krb-service-principal for CompactServices

    Filter krb-service-principals for the CompactServices
    based on the networks associated with the role.

    Filtering for the IndividualServices was added in previous
    fix https://review.openstack.org/646005, which did'nt
    fully fix the bug.

    Closes-Bug: #1821377
    Change-Id: Id54477ca5581e1f5fe8a09c3bc60a238d114dbb2
    (cherry picked from commit 578bcb2ffad32c6a39d68b5dc360504e95972ffa)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.1.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/664885
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=ff679e51fcab17a32f42f1f81e888669d7f8d1b8
Submitter: Zuul
Branch: stable/queens

commit ff679e51fcab17a32f42f1f81e888669d7f8d1b8
Author: Harald Jensås <email address hidden>
Date: Wed Jun 12 15:04:06 2019 +0200

    [FUP] Backport - krb-service-principal metadata per-Role

    The rocky and queens backport of change[1] missed updating a
    deprecated environment file that is no longer present since
    stein.

    This change updates the deprecated environment file to be a
    j2 file with the jinja code to render the per-role metadata.

    [1] I5ae3bc845b0a6ad6986d44b14ff4b0737a9b033b

    Related-Bug: #1821377
    Change-Id: Ie7c3c448e8d0f53a5347a19b6d0b99a0c6097114
    (cherry picked from commit fb5f69dcb2f84797d75da8b2a75692e06ed873f0)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.6.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.4.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.4.1 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/697497

[OpenStack Infra (hudson-openstack)]

Change abandoned by Harald Jensås (<email address hidden>) on branch: master
Review: https://review.opendev.org/697497
Reason: Wrong bug ID.