openstack overcloud ceph deploy --network-data could better handle an invalid network_data_v2.yaml file

[John Fulton]

The error [1] was seen after running the following command with [2] as network_data_v2.yaml:

    openstack overcloud ceph deploy \
              --ceph-spec ceph_spec.yaml \
              --skip-user-create \
              -y -o deployed_ceph.yaml \
              --network-data network_data_v2.yaml \
              --container-namespace quay.io/ceph \
              --container-image daemon \
              --container-tag v6.0.6-stable-6.0-pacific-centos-8-x86_64 \
              --stack overcloud

[1]
~~~
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 186, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/local/lib/python3.6/site-packages/tripleoclient/v2/overcloud_ceph.py", line 323, in take_action
    parsed_args.cluster_network_name)
  File "/usr/local/lib/python3.6/site-packages/tripleoclient/utils.py", line 3273, in get_ceph_networks
    subnet = _get_subnet(net, ip_subnet)
  File "/usr/local/lib/python3.6/site-packages/tripleoclient/utils.py", line 3251, in _get_subnet
    return ','.join(ip_subnets)
TypeError: sequence item 0: expected str instance, NoneType found
sequence item 0: expected str instance, NoneType found
~~~

[2]
~~~
- name: Storage
  name_lower: storage
  subnets:
    storage_subnet:
      enable_dhcp: true
      ipv6_address_mode: dhcpv6-stateful
      ipv6_allocation_pools:
      - end: fd00:fd00:fd00:3000:aaaa:aaaa:ffff:fffe
        start: fd00:fd00:fd00:3000::10
      ipv6_ra_mode: dhcpv6-stateful
      ipv6_subnet: fd00:fd00:fd00:3000::/64
      physical_network: storage
      vlan: 30
  vip: true
- name: StorageMgmt
  name_lower: storage_mgmt
  subnets:
    storage_mgmt_subnet:
      enable_dhcp: true
      ipv6_address_mode: dhcpv6-stateful
      ipv6_allocation_pools:
      - end: fd00:fd00:fd00:4000:aaaa:aaaa:ffff:fffe
        start: fd00:fd00:fd00:4000::10
      ipv6_ra_mode: dhcpv6-stateful
      ipv6_subnet: fd00:fd00:fd00:4000::/64
      physical_network: storage_mgmt
      vlan: 40
  vip: true
- name: InternalApi
  name_lower: internal_api
  subnets:
    internal_api_subnet:
      enable_dhcp: true
      ipv6_address_mode: dhcpv6-stateful
      ipv6_allocation_pools:
      - end: fd00:fd00:fd00:2000:aaaa:aaaa:ffff:fffe
        start: fd00:fd00:fd00:2000::10
      ipv6_ra_mode: dhcpv6-stateful
      ipv6_subnet: fd00:fd00:fd00:2000::/64
      physical_network: internal_api
      vlan: 20
  vip: true
- name: Tenant
  name_lower: tenant
  subnets:
    tenant_subnet:
      allocation_pools:
      - end: 172.17.2.149
        start: 172.17.2.10
      ip_subnet: 172.17.2.0/24
      physical_network: tenant
      vlan: 50
- name: External
  name_lower: external
  vip: true
  subnets:
    external_subnet:
      enable_dhcp: true
      gateway_ipv6: 2620:52:0:13b8::fe
      ipv6_address_mode: dhcpv6-stateful
      ipv6_allocation_pools:
      - end: 2620:52:0:13b8:5054:ff:fe3e:aa
        start: 2620:52:0:13b8:5054:ff:fe3e:1
      ipv6_ra_mode: dhcpv6-stateful
      ipv6_subnet: 2620:52:0:13b8::/64
      physical_network: external
      vlan: 10
- name: Management
  name_lower: management
  subnets:
    management_subnet:
      enable_dhcp: true
      gateway_ipv6: fd00:fd00:fd00:6000::1
      ipv6_address_mode: dhcpv6-stateful
      ipv6_allocation_pools:
      - end: fd00:fd00:fd00:6000:ffff:ffff:ffff:fffe
        start: fd00:fd00:fd00:6000::10
      ipv6_ra_mode: dhcpv6-stateful
      ipv6_subnet: fd00:fd00:fd00:6000::/64
      physical_network: management
      vlan: 60
~~~

[John Fulton (jfulton-org)]

This is not a bug. Invalid input was provided.

The network_data_v2.yaml is missing the 'ipv6' key which must be provided when using IPv6. As per Harald Jensås in a review of the code in question [1]

"""
When creating networks/subnets as IPv6 the 'ipv6' key for the network must be set to true, see:
https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/ansible_plugins/modules/tripleo_composable_network.py#L437

i.e you can determine which version to use from the network-data provided. If ipv6 != true you pick the IPv4 keys, else pick the ipv6 keys. See: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/ansible_plugins/modules/tripleo_composable_network.py#L270-L305
"""

I updated the network_data_v2.yaml file to make it valid and confirmed I no longer get the error message which was reported.

[1] https://review.opendev.org/c/openstack/python-tripleoclient/+/817131/6/tripleoclient/v2/overcloud_ceph.py#103

[John Fulton (jfulton-org)]

Though the input was not valid the program could display a more helpful error message and not stop running because of a type error. I will use this bug to track returning a more helpful error message, but it's still correct to return an error message. To avoid this issue you must pass valid input.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/823582

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/823582
Committed: https://opendev.org/openstack/python-tripleoclient/commit/79fa6fbe1cd73c55a2b56dcb81ace2c2c01ce75b
Submitter: "Zuul (22348)"
Branch: master

commit 79fa6fbe1cd73c55a2b56dcb81ace2c2c01ce75b
Author: John Fulton <email address hidden>
Date: Wed Jan 5 22:11:21 2022 +0000

    Handle missing ipv6 boolean in network_data.yaml for Ceph

    It's not valid input to exclude the ipv6 boolean when
    using IPv6, but when this happens the client fails with
    a type error. Handle this situation better and return
    a better error message.

    Closes-Bug: 1956517
    Change-Id: Id99346ac0fc336b306550e71da0f9b746ff83ef1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/823845

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/823845
Committed: https://opendev.org/openstack/python-tripleoclient/commit/c41459a61d8c0154f0724e78c1bdc53fbf6c518a
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit c41459a61d8c0154f0724e78c1bdc53fbf6c518a
Author: John Fulton <email address hidden>
Date: Wed Jan 5 22:11:21 2022 +0000

    Handle missing ipv6 boolean in network_data.yaml for Ceph

    It's not valid input to exclude the ipv6 boolean when
    using IPv6, but when this happens the client fails with
    a type error. Handle this situation better and return
    a better error message.

    Closes-Bug: 1956517
    Change-Id: Id99346ac0fc336b306550e71da0f9b746ff83ef1
    (cherry picked from commit 79fa6fbe1cd73c55a2b56dcb81ace2c2c01ce75b)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 18.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 16.5.0 release.