Create the RGW Daemon spec definition | The error was: ImportError: cannot import name 'ceph_spec'

[swogat pradhan]

I am deploying openstack (wallaby) overcloud using deployed ceph.
and i am facing one error stating The error was: ImportError: cannot import name 'ceph_spec'

The error at
/home/stack/overcloud-deploy/overcloud/config-download/overcloud/cephadm/cephadm_command.log

2022-05-13 14:17:50,923 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.922956 | 48d539a1-1679-730f-272a-0000000000ca | TASK | Get certificate from remote node if needed
2022-05-13 14:17:50,987 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.986916 | 48d539a1-1679-730f-272a-0000000000ca | SKIPPED | Get certificate from remote node if needed | overcloud-controller-0
2022-05-13 14:17:51,022 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.022353 | 48d539a1-1679-730f-272a-0000000000cb | TASK | Create the RGW Daemon spec definition
2022-05-13 14:17:51,668 p=636835 u=stack n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: cannot import name 'ceph_spec'
2022-05-13 14:17:51,669 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.669288 | 48d539a1-1679-730f-272a-0000000000cb | FATAL | Create the RGW Daemon spec definition | overcloud-controller-0 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_ceph_mkspec_payload_iwqyfs7g/ansible_ceph_mkspec_payload.zip/ansible/modules/ceph_mkspec.py\", line 24, in <module>\nImportError: cannot import name 'ceph_spec'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | PLAY RECAP *********************************************************************
2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | overcloud-controller-0 : ok=22 changed=1 unreachable=0 failed=1 skipped=27 rescued=0 ignored=0
2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684007 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684240 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 50 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684514 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:45.563925 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684678 | UUID | Info | Host | Task Name | Run Time
2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684849 | 48d539a1-1679-730f-272a-0000000000b7 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create pool(s) | 17.08s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685012 | 48d539a1-1679-730f-272a-0000000000a6 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : insert new default crush rule into daemon to prevent restart | 3.57s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685177 | 48d539a1-1679-730f-272a-0000000000a3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : create configured crush rules | 3.28s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685340 | 48d539a1-1679-730f-272a-00000000002b | SUMMARY | overcloud-controller-0 | generate tripleo_cephadm_assimilate_conf on bootstrap node | 2.56s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685518 | 48d539a1-1679-730f-272a-0000000000a4 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : get id for new default crush rule | 1.39s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685682 | 48d539a1-1679-730f-272a-00000000001b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : List Ceph daemon instances on this host | 1.24s
2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685869 | 48d539a1-1679-730f-272a-000000000076 | SUMMARY | overcloud-controller-0 | Assimilate configuration from tripleo_cephadm_assimilate_conf | 1.04s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686032 | 48d539a1-1679-730f-272a-000000000197 | SUMMARY | overcloud-controller-0 | Gathering Facts | 0.95s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686195 | 48d539a1-1679-730f-272a-000000000026 | SUMMARY | overcloud-controller-0 | push tripleo_cephadm_spec to bootstrap node if spec file exsits | 0.88s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686354 | 48d539a1-1679-730f-272a-0000000000ba | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set trash interval | 0.74s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686536 | 48d539a1-1679-730f-272a-000000000023 | SUMMARY | overcloud-controller-0 | Ensure tripleo_cephadm_config_home (e.g. /etc/ceph) exists | 0.72s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686703 | 48d539a1-1679-730f-272a-000000000019 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Stat cephadm file | 0.72s
2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686863 | 48d539a1-1679-730f-272a-0000000000cb | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create the RGW Daemon spec definition | 0.65s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687025 | 48d539a1-1679-730f-272a-0000000001f3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.54s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687194 | 48d539a1-1679-730f-272a-000000000024 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Ensure specs directory exists | 0.39s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687359 | 48d539a1-1679-730f-272a-0000000000a5 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : set_fact info_ceph_default_crush_rule_yaml, ceph_osd_pool_default_crush_rule_name | 0.35s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687546 | 48d539a1-1679-730f-272a-00000000005b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Run cephadm bootstrap | 0.23s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687715 | 48d539a1-1679-730f-272a-0000000000c9 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Collect the host and build the resulting host list | 0.21s
2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687889 | 48d539a1-1679-730f-272a-000000000020 | SUMMARY | overcloud-controller-0 | Set random tripleo_cephadm_fsid if no running ceph containers were found | 0.21s
2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688054 | 48d539a1-1679-730f-272a-000000000220 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.20s
2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688209 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688435 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688611 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~
2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688772 | The following node(s) had failures: overcloud-controller-0
2022-05-13 14:17:51,689 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688943 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

my deploy.sh file:

#!/bin/bash
THT=/usr/share/openstack-tripleo-heat-templates/
CNF=/home/stack/
openstack overcloud deploy --templates $THT \
-r $CNF/templates/roles_data.yaml \
-n $CNF/workplace/custom_network_data.yaml \
-e ~/containers-prepare-parameter.yaml \
-e $CNF/templates/node-info.yaml \
-e $CNF/templates/scheduler-hints.yaml \
-e $CNF/workplace/networks-deployed-environment.yaml \
-e $CNF/workplace/vip-deployed-environment.yaml \
-e $CNF/workplace/overcloud-baremetal-deployed.yaml \
-e $CNF/workplace/custom-net-bond-with-vlans.yaml \
-e $CNF/workplace/clouddomain.yaml \
-e $CNF/workplace/deplyed_ceph.yaml \
-e $THT/environments/cephadm/cephadm.yaml \
-e $CNF/workplace/crush_rule_override.yaml \
-e $THT/environments/cinder-backup.yaml \
-e $THT/environments/enable-legacy-telemetry.yaml \
-e $THT/environments/services/octavia.yaml \
--ntp-server xx.XX.XX.xx