Deployed Ceph fails when using custom network name

Bug #1950178 reported by John Fulton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
John Fulton

Bug Description

Deployed Ceph fails when using custom network name

During deployment of Ceph the following error I seen:

2021-11-08 15:15:30.838746 | 244201c6-c8f6-fff8-c2cd-00000000001c | TASK | Ensure storage networks are in the inventory
2021-11-08 15:15:30.906178 | 244201c6-c8f6-fff8-c2cd-00000000001c | FATAL | Ensure storage networks are in the inventory | oc0-ceph-0 | error={
    "changed": false,
    "msg": "Both Storage and StorageMgmt should be in ['StorageCloud0', 'StorageMgmtCloud0', 'ctlplane'] but are not. If desired, override the network name variables, e.g. -e storage_network_name=ctlplane -e storage_mgmt_network_name=ctlplane\n"
}

In roles_data.yaml my ceph nodes had the following entry:

  networks:
    Storage:
      subnet: storage_cloud_0_subnet
    StorageMgmt:
      subnet: storage_mgmt_cloud_0_subnet

Which differs from the default:

  networks:
    Storage:
      subnet: storage_subnet
    StorageMgmt:
      subnet: storage_mgmt_subnet

In the config-download directory my tripleo-ansible-inventory has the following:

  vars:
    networks_all:
    - StorageCloud0
    - StorageMgmtCloud0
    networks_lower:
      StorageCloud0: storage_cloud_0
      StorageMgmtCloud0: storage_mgmt_cloud_0
      ctlplane: ctlplane
    role_networks:
    - StorageCloud0
    - ctlplane
    - StorageMgmtCloud0

Later in the inventory each host has its own variable map:

CephAll:
  hosts:
    ...
    oc0-ceph-2:
      ...
      storage_cloud_0_hostname: oc0-ceph-2.storagecloud0.mydomain.tld
      storage_cloud_0_ip: 172.16.11.96
      storage_mgmt_cloud_0_hostname: oc0-ceph-2.storagemgmtcloud0.mydomain.tld
      storage_mgmt_cloud_0_ip: 172.16.12.134

As it's valid for a user to customize these parameters and the cusotmized
roles_data.yaml is passed when the command is run with --roles-data. The
playbook should be updated handle this and use the customized network name.

https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/playbooks/cli-deployed-ceph.yaml#L117-L141

description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/817131

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (master)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-docs (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-docs/+/818822

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (master)

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/817131
Committed: https://opendev.org/openstack/python-tripleoclient/commit/149099c19e2a18dffdd55a7aba4961c8416e1206
Submitter: "Zuul (22348)"
Branch: master

commit 149099c19e2a18dffdd55a7aba4961c8416e1206
Author: John Fulton <email address hidden>
Date: Tue Nov 9 03:07:13 2021 +0000

    Support custom network names with deployed ceph

    The cli-deployed-ceph playbook has pre_tasks to define the Ceph
    public_network, cluster_network and their names but those tasks
    do not support custom network names as per the related bug.

    Add a get_ceph_networks utility which extracts the public_network,
    cluster_network and their names from the network_data file and
    then pass those values directly to the cli-deployed-ceph playbook
    via extra_vars.

    Related-Bug: #1950178
    Change-Id: If96e1d34d9a918d6e99366590b05c8219dc05e1c

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/818740

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/819161

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/818740
Committed: https://opendev.org/openstack/python-tripleoclient/commit/ac828a2cf3ca9a293d88d4c788d10400ea09954a
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit ac828a2cf3ca9a293d88d4c788d10400ea09954a
Author: John Fulton <email address hidden>
Date: Tue Nov 9 03:07:13 2021 +0000

    Support custom network names with deployed ceph

    The cli-deployed-ceph playbook has pre_tasks to define the Ceph
    public_network, cluster_network and their names but those tasks
    do not support custom network names as per the related bug.

    Add a get_ceph_networks utility which extracts the public_network,
    cluster_network and their names from the network_data file and
    then pass those values directly to the cli-deployed-ceph playbook
    via extra_vars.

    Related-Bug: #1950178
    Change-Id: If96e1d34d9a918d6e99366590b05c8219dc05e1c
    (cherry picked from commit 149099c19e2a18dffdd55a7aba4961c8416e1206)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (master)

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/819161
Committed: https://opendev.org/openstack/python-tripleoclient/commit/7d3c412ee5a391b838e659de3922aa89e2a71a1d
Submitter: "Zuul (22348)"
Branch: master

commit 7d3c412ee5a391b838e659de3922aa89e2a71a1d
Author: John Fulton <email address hidden>
Date: Wed Nov 24 19:49:15 2021 +0000

    Set Ceph ms_bind options for IPv6

    If IPv6 is being used set ms_bind_ipv4 false
    and ms_bind_ipv6 true in the Ceph configuration.

    Without this change it is always necessary to use
    the more complicated --config option to deploy IPv6
    even if IPv4 is not being used. Instead the --config
    option can be reserved for more complex scenarios like
    experimental dual stack IPv4 and IPv6.

    Follow up patch to If96e1d34d9a918d6e99366590b05c8219dc05e1c.

    Related-Bug: #1950178
    Change-Id: I86ad9631c62ae2abac7e01825ea2266a9def0b83

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/817132
Committed: https://opendev.org/openstack/tripleo-ansible/commit/2e34dacab87a80165136c34c89f129de422f1e79
Submitter: "Zuul (22348)"
Branch: master

commit 2e34dacab87a80165136c34c89f129de422f1e79
Author: John Fulton <email address hidden>
Date: Tue Nov 9 03:09:39 2021 +0000

    Require Ceph network parameters to be passed to deployed_ceph playbook

    Remove logic to determine public_network, cluster_network and their
    names and require them instead to be passed by python-tripleoclient
    as per the depends-on patch.

    Also add support for client --config option and set ms_bind options.

    Closes-Bug: #1950178
    Depends-On: If96e1d34d9a918d6e99366590b05c8219dc05e1c
    Change-Id: I55ed63330e19d22902378791aefb8229690800dc

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-docs (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-docs/+/818822
Committed: https://opendev.org/openstack/tripleo-docs/commit/f6ee007c38992adbe8a809456d3d86e792b5dc1d
Submitter: "Zuul (22348)"
Branch: master

commit f6ee007c38992adbe8a809456d3d86e792b5dc1d
Author: John Fulton <email address hidden>
Date: Mon Nov 22 13:56:49 2021 -0500

    Document deployed ceph --network-data options

    Also, fix broken links to Ceph documentation.

    Related-Bug: #1950178
    Depends-On: I86ad9631c62ae2abac7e01825ea2266a9def0b83
    Change-Id: Iffb800d52a62625d1170ec949dd67427fed08cce

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/python-tripleoclient/+/819589

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-ansible/+/819670

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-ansible/+/819670
Committed: https://opendev.org/openstack/tripleo-ansible/commit/774001ede39c78e77a16a55ba17227bbf7844bb8
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 774001ede39c78e77a16a55ba17227bbf7844bb8
Author: John Fulton <email address hidden>
Date: Tue Nov 9 03:09:39 2021 +0000

    Require Ceph network parameters to be passed to deployed_ceph playbook

    Remove logic to determine public_network, cluster_network and their
    names and require them instead to be passed by python-tripleoclient
    as per the depends-on patch.

    Also add support for client --config option and set ms_bind options.

    Closes-Bug: #1950178
    Depends-On: If96e1d34d9a918d6e99366590b05c8219dc05e1c
    Change-Id: I55ed63330e19d22902378791aefb8229690800dc
    (cherry picked from commit 2e34dacab87a80165136c34c89f129de422f1e79)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/python-tripleoclient/+/819589
Committed: https://opendev.org/openstack/python-tripleoclient/commit/1de4a4e3bf1d9c337948ccddb938fed850664edf
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 1de4a4e3bf1d9c337948ccddb938fed850664edf
Author: John Fulton <email address hidden>
Date: Wed Nov 24 19:49:15 2021 +0000

    Set Ceph ms_bind options for IPv6

    If IPv6 is being used set ms_bind_ipv4 false
    and ms_bind_ipv6 true in the Ceph configuration.

    Without this change it is always necessary to use
    the more complicated --config option to deploy IPv6
    even if IPv4 is not being used. Instead the --config
    option can be reserved for more complex scenarios like
    experimental dual stack IPv4 and IPv6.

    Follow up patch to If96e1d34d9a918d6e99366590b05c8219dc05e1c.

    Related-Bug: #1950178
    Change-Id: I86ad9631c62ae2abac7e01825ea2266a9def0b83
    (cherry picked from commit 7d3c412ee5a391b838e659de3922aa89e2a71a1d)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ansible 4.2.0

This issue was fixed in the openstack/tripleo-ansible 4.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.