Activity log for bug #1973267

Date Who What changed Old value New value Message
2022-05-13 06:58:43 swogat pradhan bug added bug
2022-05-13 06:58:43 swogat pradhan attachment added attached full log https://bugs.launchpad.net/bugs/1973267/+attachment/5589402/+files/cephadm_command.log
2022-05-13 07:05:57 swogat pradhan description I am deploying openstack (wallaby) overcloud using deployed ceph. and i am facing one error stating The error was: ImportError: cannot import name 'ceph_spec' The error at /home/stack/overcloud-deploy/overcloud/config-download/overcloud/cephadm/cephadm_command.log 2022-05-13 14:17:50,923 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.922956 | 48d539a1-1679-730f-272a-0000000000ca | TASK | Get certificate from remote node if needed 2022-05-13 14:17:50,987 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.986916 | 48d539a1-1679-730f-272a-0000000000ca | SKIPPED | Get certificate from remote node if needed | overcloud-controller-0 2022-05-13 14:17:51,022 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.022353 | 48d539a1-1679-730f-272a-0000000000cb | TASK | Create the RGW Daemon spec definition 2022-05-13 14:17:51,668 p=636835 u=stack n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: cannot import name 'ceph_spec' 2022-05-13 14:17:51,669 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.669288 | 48d539a1-1679-730f-272a-0000000000cb | FATAL | Create the RGW Daemon spec definition | overcloud-controller-0 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_ceph_mkspec_payload_iwqyfs7g/ansible_ceph_mkspec_payload.zip/ansible/modules/ceph_mkspec.py\", line 24, in <module>\nImportError: cannot import name 'ceph_spec'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} 2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | PLAY RECAP ********************************************************************* 2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | overcloud-controller-0 : ok=22 changed=1 unreachable=0 failed=1 skipped=27 rescued=0 ignored=0 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684007 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684240 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 50 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684514 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:45.563925 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684678 | UUID | Info | Host | Task Name | Run Time 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684849 | 48d539a1-1679-730f-272a-0000000000b7 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create pool(s) | 17.08s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685012 | 48d539a1-1679-730f-272a-0000000000a6 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : insert new default crush rule into daemon to prevent restart | 3.57s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685177 | 48d539a1-1679-730f-272a-0000000000a3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : create configured crush rules | 3.28s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685340 | 48d539a1-1679-730f-272a-00000000002b | SUMMARY | overcloud-controller-0 | generate tripleo_cephadm_assimilate_conf on bootstrap node | 2.56s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685518 | 48d539a1-1679-730f-272a-0000000000a4 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : get id for new default crush rule | 1.39s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685682 | 48d539a1-1679-730f-272a-00000000001b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : List Ceph daemon instances on this host | 1.24s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685869 | 48d539a1-1679-730f-272a-000000000076 | SUMMARY | overcloud-controller-0 | Assimilate configuration from tripleo_cephadm_assimilate_conf | 1.04s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686032 | 48d539a1-1679-730f-272a-000000000197 | SUMMARY | overcloud-controller-0 | Gathering Facts | 0.95s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686195 | 48d539a1-1679-730f-272a-000000000026 | SUMMARY | overcloud-controller-0 | push tripleo_cephadm_spec to bootstrap node if spec file exsits | 0.88s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686354 | 48d539a1-1679-730f-272a-0000000000ba | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set trash interval | 0.74s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686536 | 48d539a1-1679-730f-272a-000000000023 | SUMMARY | overcloud-controller-0 | Ensure tripleo_cephadm_config_home (e.g. /etc/ceph) exists | 0.72s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686703 | 48d539a1-1679-730f-272a-000000000019 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Stat cephadm file | 0.72s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686863 | 48d539a1-1679-730f-272a-0000000000cb | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create the RGW Daemon spec definition | 0.65s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687025 | 48d539a1-1679-730f-272a-0000000001f3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.54s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687194 | 48d539a1-1679-730f-272a-000000000024 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Ensure specs directory exists | 0.39s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687359 | 48d539a1-1679-730f-272a-0000000000a5 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : set_fact info_ceph_default_crush_rule_yaml, ceph_osd_pool_default_crush_rule_name | 0.35s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687546 | 48d539a1-1679-730f-272a-00000000005b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Run cephadm bootstrap | 0.23s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687715 | 48d539a1-1679-730f-272a-0000000000c9 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Collect the host and build the resulting host list | 0.21s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687889 | 48d539a1-1679-730f-272a-000000000020 | SUMMARY | overcloud-controller-0 | Set random tripleo_cephadm_fsid if no running ceph containers were found | 0.21s 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688054 | 48d539a1-1679-730f-272a-000000000220 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.20s 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688209 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688435 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688611 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688772 | The following node(s) had failures: overcloud-controller-0 2022-05-13 14:17:51,689 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688943 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I am deploying openstack (wallaby) overcloud using deployed ceph. and i am facing one error stating The error was: ImportError: cannot import name 'ceph_spec' The error at /home/stack/overcloud-deploy/overcloud/config-download/overcloud/cephadm/cephadm_command.log 2022-05-13 14:17:50,923 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.922956 | 48d539a1-1679-730f-272a-0000000000ca | TASK | Get certificate from remote node if needed 2022-05-13 14:17:50,987 p=636835 u=stack n=ansible | 2022-05-13 14:17:50.986916 | 48d539a1-1679-730f-272a-0000000000ca | SKIPPED | Get certificate from remote node if needed | overcloud-controller-0 2022-05-13 14:17:51,022 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.022353 | 48d539a1-1679-730f-272a-0000000000cb | TASK | Create the RGW Daemon spec definition 2022-05-13 14:17:51,668 p=636835 u=stack n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: cannot import name 'ceph_spec' 2022-05-13 14:17:51,669 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.669288 | 48d539a1-1679-730f-272a-0000000000cb | FATAL | Create the RGW Daemon spec definition | overcloud-controller-0 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_ceph_mkspec_payload_iwqyfs7g/ansible_ceph_mkspec_payload.zip/ansible/modules/ceph_mkspec.py\", line 24, in <module>\nImportError: cannot import name 'ceph_spec'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} 2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | PLAY RECAP ********************************************************************* 2022-05-13 14:17:51,683 p=636835 u=stack n=ansible | overcloud-controller-0 : ok=22 changed=1 unreachable=0 failed=1 skipped=27 rescued=0 ignored=0 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684007 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684240 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 50 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684514 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:45.563925 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684678 | UUID | Info | Host | Task Name | Run Time 2022-05-13 14:17:51,684 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.684849 | 48d539a1-1679-730f-272a-0000000000b7 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create pool(s) | 17.08s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685012 | 48d539a1-1679-730f-272a-0000000000a6 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : insert new default crush rule into daemon to prevent restart | 3.57s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685177 | 48d539a1-1679-730f-272a-0000000000a3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : create configured crush rules | 3.28s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685340 | 48d539a1-1679-730f-272a-00000000002b | SUMMARY | overcloud-controller-0 | generate tripleo_cephadm_assimilate_conf on bootstrap node | 2.56s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685518 | 48d539a1-1679-730f-272a-0000000000a4 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : get id for new default crush rule | 1.39s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685682 | 48d539a1-1679-730f-272a-00000000001b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : List Ceph daemon instances on this host | 1.24s 2022-05-13 14:17:51,685 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.685869 | 48d539a1-1679-730f-272a-000000000076 | SUMMARY | overcloud-controller-0 | Assimilate configuration from tripleo_cephadm_assimilate_conf | 1.04s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686032 | 48d539a1-1679-730f-272a-000000000197 | SUMMARY | overcloud-controller-0 | Gathering Facts | 0.95s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686195 | 48d539a1-1679-730f-272a-000000000026 | SUMMARY | overcloud-controller-0 | push tripleo_cephadm_spec to bootstrap node if spec file exsits | 0.88s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686354 | 48d539a1-1679-730f-272a-0000000000ba | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set trash interval | 0.74s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686536 | 48d539a1-1679-730f-272a-000000000023 | SUMMARY | overcloud-controller-0 | Ensure tripleo_cephadm_config_home (e.g. /etc/ceph) exists | 0.72s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686703 | 48d539a1-1679-730f-272a-000000000019 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Stat cephadm file | 0.72s 2022-05-13 14:17:51,686 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.686863 | 48d539a1-1679-730f-272a-0000000000cb | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Create the RGW Daemon spec definition | 0.65s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687025 | 48d539a1-1679-730f-272a-0000000001f3 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.54s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687194 | 48d539a1-1679-730f-272a-000000000024 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Ensure specs directory exists | 0.39s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687359 | 48d539a1-1679-730f-272a-0000000000a5 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : set_fact info_ceph_default_crush_rule_yaml, ceph_osd_pool_default_crush_rule_name | 0.35s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687546 | 48d539a1-1679-730f-272a-00000000005b | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Run cephadm bootstrap | 0.23s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687715 | 48d539a1-1679-730f-272a-0000000000c9 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Collect the host and build the resulting host list | 0.21s 2022-05-13 14:17:51,687 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.687889 | 48d539a1-1679-730f-272a-000000000020 | SUMMARY | overcloud-controller-0 | Set random tripleo_cephadm_fsid if no running ceph containers were found | 0.21s 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688054 | 48d539a1-1679-730f-272a-000000000220 | SUMMARY | overcloud-controller-0 | tripleo_cephadm : Set ceph CLI | 0.20s 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688209 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688435 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688611 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~ 2022-05-13 14:17:51,688 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688772 | The following node(s) had failures: overcloud-controller-0 2022-05-13 14:17:51,689 p=636835 u=stack n=ansible | 2022-05-13 14:17:51.688943 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ my deploy.sh file: #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml \ -e $CNF/workplace/clouddomain.yaml \ -e $CNF/workplace/deplyed_ceph.yaml \ -e $THT/environments/cephadm/cephadm.yaml \ -e $CNF/workplace/crush_rule_override.yaml \ -e $THT/environments/cinder-backup.yaml \ -e $THT/environments/enable-legacy-telemetry.yaml \ -e $THT/environments/services/octavia.yaml \ --ntp-server xx.XX.XX.xx