Cell deployment fail ping_test_gateway_ips

Bug #1973866 reported by Harald Jensås
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Harald Jensås

Bug Description

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=2087231

When using `ManageNetworks: false` the gateway IPs are '' (empty string) instead of `null` when not set. This happens because the default[2] is '' and in the `ManageNetwork: false` case the `gateway_ip` in `network_attributes_map` is the value of the parameter. In the `ManageNetwork: true` case the value is from the Heat resource which is a `null` value when no gateway is set.

The YAQL expression[1] filters `null` values, but the '' (empty string) value is included in the list. The ping gateway test end up trying to run "ping $args $empty_string" which of course fails.

2022-05-17 13:34:43.787038 | 525400f9-7a8a-d346-8ffd-000000000b9f | FATAL | Check all networks Gateway availability | cell1-compute-0 | error={"ansible_loop_var": "gateway_ip", "changed": false, "cmd": ["ping", "-w", "10", "-c", "1"], "delta": "0:00:00.004407", "end": "2022-05-17 09:34:43.657259", "gateway_ip": "", "msg": "non-zero return code", "rc": 2, "start": "2022-05-17 09:34:43.652852", "stderr": "Usage: ping [-aAbBdDfhLnOqrRUvV64] [-c count] [-i interval] [-I interface]\n [-m mark] [-M pmtudisc_option] [-l preload] [-p pattern] [-Q tos]\n [-s packetsize] [-S sndbuf] [-t ttl] [-T timestamp_option]\n [-w deadline] [-W timeout] [hop1 ...] destination\nUsage: ping -6 [-aAbBdDfhLnOqrRUvV] [-c count] [-i interval] [-I interface]\n [-l preload] [-m mark] [-M pmtudisc_option]\n [-N nodeinfo_option] [-p pattern] [-Q tclass] [-s packetsize]\n [-S sndbuf] [-t ttl] [-T timestamp_option] [-w deadline]\n [-W timeout] destination", "stderr_lines": ["Usage: ping [-aAbBdDfhLnOqrRUvV64] [-c count] [-i interval] [-I interface]", " [-m mark] [-M pmtudisc_option] [-l preload] [-p pattern] [-Q tos]", " [-s packetsize] [-S sndbuf] [-t ttl] [-T timestamp_option]", " [-w deadline] [-W timeout] [hop1 ...] destination", "Usage: ping -6 [-aAbBdDfhLnOqrRUvV] [-c count] [-i interval] [-I interface]", " [-l preload] [-m mark] [-M pmtudisc_option]", " [-N nodeinfo_option] [-p pattern] [-Q tclass] [-s packetsize]", " [-S sndbuf] [-t ttl] [-T timestamp_option] [-w deadline]", " [-W timeout] destination"], "stdout": "", "stdout_lines": []}

For the overcloud deployment:

/var/lib/mistral/overcloud/global_vars.yaml contains:
ping_test_gateway_ips:
  BlockStorage: []
  CephStorage: []
  Compute: []
  Controller:
  - 10.0.0.1
  ObjectStorage: []

For the Cell deployment:

/var/lib/mistral/cell1/global_vars.yaml contains:

ping_test_gateway_ips:
  CellController:
  - ''
  - ''
  - ''
  - ''
  - 10.0.0.1
  Compute:
  - ''
  - ''
  - ''

[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/overcloud.j2.yaml#L1244
[2] https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/network/network.j2#L61-L68
[3] https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/network/network.j2#L322

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/842274
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/7f6a4b43856246995b8fa26b833155559f95e2ca
Submitter: "Zuul (22348)"
Branch: master

commit 7f6a4b43856246995b8fa26b833155559f95e2ca
Author: Harald Jensås <email address hidden>
Date: Wed May 18 09:22:33 2022 +0200

    Filter empty string in PingTestGatewayIPsMap yagl

    When using `ManageNetworks: false` the gateway IPs are
    empty string instead of `null` when not set. The YAQL
    expression filters `null` values, but the empty string
    value is included in the list. The ping gateway test end
    up trying to run "ping $args $empty_string" which fails.

    This change improves the yaql expression to also filter
    strings with 0 lenght.

    Closes-Bug: #1973866
    Change-Id: I7d8712223d077ab8e25239b891bd03a1324e01a8

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/842934

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/842934
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/ba7ef6b6ce6619e0c03f724abee2f41b9ca00ecf
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit ba7ef6b6ce6619e0c03f724abee2f41b9ca00ecf
Author: Harald Jensås <email address hidden>
Date: Wed May 18 09:22:33 2022 +0200

    Filter empty string in PingTestGatewayIPsMap yagl

    When using `ManageNetworks: false` the gateway IPs are
    empty string instead of `null` when not set. The YAQL
    expression filters `null` values, but the empty string
    value is included in the list. The ping gateway test end
    up trying to run "ping $args $empty_string" which fails.

    This change improves the yaql expression to also filter
    strings with 0 lenght.

    Closes-Bug: #1973866
    Change-Id: I7d8712223d077ab8e25239b891bd03a1324e01a8
    (cherry picked from commit 7f6a4b43856246995b8fa26b833155559f95e2ca)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/846552

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/846552
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/778ca8da06876f05c2565da578c2f1a4d4dce1ce
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 778ca8da06876f05c2565da578c2f1a4d4dce1ce
Author: Harald Jensås <email address hidden>
Date: Wed May 18 09:22:33 2022 +0200

    Filter empty string in PingTestGatewayIPsMap yagl

    When using `ManageNetworks: false` the gateway IPs are
    empty string instead of `null` when not set. The YAQL
    expression filters `null` values, but the empty string
    value is included in the list. The ping gateway test end
    up trying to run "ping $args $empty_string" which fails.

    This change improves the yaql expression to also filter
    strings with 0 lenght.

    Closes-Bug: #1973866
    Change-Id: I7d8712223d077ab8e25239b891bd03a1324e01a8
    (cherry picked from commit 7f6a4b43856246995b8fa26b833155559f95e2ca)
    (cherry picked from commit ba7ef6b6ce6619e0c03f724abee2f41b9ca00ecf)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/846555
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/d3749deb33b87b6f61eb08c64e56ba24f38feacc
Submitter: "Zuul (22348)"
Branch: stable/train

commit d3749deb33b87b6f61eb08c64e56ba24f38feacc
Author: Harald Jensås <email address hidden>
Date: Wed May 18 09:22:33 2022 +0200

    Filter empty string in PingTestGatewayIPsMap yagl

    When using `ManageNetworks: false` the gateway IPs are
    empty string instead of `null` when not set. The YAQL
    expression filters `null` values, but the empty string
    value is included in the list. The ping gateway test end
    up trying to run "ping $args $empty_string" which fails.

    This change improves the yaql expression to also filter
    strings with 0 lenght.

    Closes-Bug: #1973866
    Change-Id: I7d8712223d077ab8e25239b891bd03a1324e01a8
    (cherry picked from commit 7f6a4b43856246995b8fa26b833155559f95e2ca)
    (cherry picked from commit ba7ef6b6ce6619e0c03f724abee2f41b9ca00ecf)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates victoria-eol

This issue was fixed in the openstack/tripleo-heat-templates victoria-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 17.0.0

This issue was fixed in the openstack/tripleo-heat-templates 17.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates train-eol

This issue was fixed in the openstack/tripleo-heat-templates train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.