while deploy cloudkitty problem with horizon

[ruskraina]

*Bug Report**

What happened:
I cannot login to horizon. Docker: horizon and ceilometer_central is unhealthy.

[root@control-1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c7c566232363 quay.io/openstack.kolla/centos-source-cloudkitty-processor:master "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) cloudkitty_processor
ecee6a4ec129 quay.io/openstack.kolla/centos-source-cloudkitty-api:master "dumb-init --single-…" 26 minutes ago Up 26 minutes (healthy) cloudkitty_api
ce67e70ae2ca quay.io/openstack.kolla/centos-source-ceilometer-central:master "dumb-init --single-…" 29 minutes ago Up 29 minutes (unhealthy) ceilometer_central
7a48bcf9e7b4 quay.io/openstack.kolla/centos-source-ceilometer-notification:master "dumb-init --single-…" 29 minutes ago Up 29 minutes (healthy) ceilometer_notification
bd96399083a0 quay.io/openstack.kolla/centos-source-gnocchi-metricd:master "dumb-init --single-…" 31 minutes ago Up 31 minutes (healthy) gnocchi_metricd
3404c518e585 quay.io/openstack.kolla/centos-source-gnocchi-api:master "dumb-init --single-…" 32 minutes ago Up 32 minutes (healthy) gnocchi_api
25b5825162a4 quay.io/openstack.kolla/centos-source-horizon:master "dumb-init --single-…" 34 minutes ago Up 34 minutes (unhealthy) horizon

Fri Mar 04 08:51:56.245953 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] During handling of the above exception, another exception occurred:
[Fri Mar 04 08:51:56.245957 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822]
[Fri Mar 04 08:51:56.245964 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] Traceback (most recent call last):
[Fri Mar 04 08:51:56.246070 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
[Fri Mar 04 08:51:56.246075 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] response = get_response(request)
[Fri Mar 04 08:51:56.246082 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/utils/deprecation.py", line 117, in __call__
[Fri Mar 04 08:51:56.246087 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] response = response or self.get_response(request)
[Fri Mar 04 08:51:56.246095 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 49, in inner
[Fri Mar 04 08:51:56.246099 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] response = response_for_exception(request, exc)
[Fri Mar 04 08:51:56.246107 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
[Fri Mar 04 08:51:56.246112 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
[Fri Mar 04 08:51:56.246119 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 152, in handle_uncaught_exception
[Fri Mar 04 08:51:56.246126 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] callback = resolver.resolve_error_handler(500)
[Fri Mar 04 08:51:56.246134 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/urls/resolvers.py", line 615, in resolve_error_handler
[Fri Mar 04 08:51:56.246139 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] callback = getattr(self.urlconf_module, 'handler%s' % view_type, None)
[Fri Mar 04 08:51:56.246146 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/utils/functional.py", line 48, in __get__
[Fri Mar 04 08:51:56.246151 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] res = instance.__dict__[self.name] = self.func(instance)
[Fri Mar 04 08:51:56.246158 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/urls/resolvers.py", line 595, in urlconf_module
[Fri Mar 04 08:51:56.246163 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] return import_module(self.urlconf_name)
[Fri Mar 04 08:51:56.246170 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
[Fri Mar 04 08:51:56.246175 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] return _bootstrap._gcd_import(name[level:], package, level)
[Fri Mar 04 08:51:56.246182 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 994, in _gcd_import
[Fri Mar 04 08:51:56.246189 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 971, in _find_and_load
[Fri Mar 04 08:51:56.246197 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
[Fri Mar 04 08:51:56.246204 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
[Fri Mar 04 08:51:56.246212 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap_external>", line 678, in exec_module
[Fri Mar 04 08:51:56.246219 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
[Fri Mar 04 08:51:56.246227 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/openstack_dashboard/urls.py", line 42, in <module>
[Fri Mar 04 08:51:56.246232 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] url(r'', horizon.base._wrapped_include(horizon.urls)),
[Fri Mar 04 08:51:56.246239 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/horizon/base.py", line 101, in _wrapped_include
[Fri Mar 04 08:51:56.246244 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] return include((pattern_list, app_namespace), namespace=instance_namespace)
[Fri Mar 04 08:51:56.246252 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/urls/conf.py", line 33, in include
[Fri Mar 04 08:51:56.246256 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] if isinstance(urlconf_module, str):
[Fri Mar 04 08:51:56.246264 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/utils/functional.py", line 246, in inner
[Fri Mar 04 08:51:56.246268 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] self._setup()
[Fri Mar 04 08:51:56.246277 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/django/utils/functional.py", line 382, in _setup
[Fri Mar 04 08:51:56.246282 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] self._wrapped = self._setupfunc()
[Fri Mar 04 08:51:56.246289 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/horizon/base.py", line 841, in url_patterns
[Fri Mar 04 08:51:56.246294 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] return self._urls()[0]
[Fri Mar 04 08:51:56.246301 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/horizon/base.py", line 866, in _urls
[Fri Mar 04 08:51:56.246306 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] import_module('%s.%s' % (package, mod_name))
[Fri Mar 04 08:51:56.246313 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
[Fri Mar 04 08:51:56.246318 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] return _bootstrap._gcd_import(name[level:], package, level)
[Fri Mar 04 08:51:56.246325 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 994, in _gcd_import
[Fri Mar 04 08:51:56.246340 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 971, in _find_and_load
[Fri Mar 04 08:51:56.246348 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
[Fri Mar 04 08:51:56.246355 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
[Fri Mar 04 08:51:56.246363 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap_external>", line 678, in exec_module
[Fri Mar 04 08:51:56.246370 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
[Fri Mar 04 08:51:56.246378 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkittydashboard/overrides.py", line 19, in <module>
[Fri Mar 04 08:51:56.246394 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] from openstack_dashboard.dashboards.project.instances.workflows \\
[Fri Mar 04 08:51:56.246406 2022] [wsgi:error] [pid 64:tid 139712451557120] [remote 10.1.17.51:34822] ImportError: cannot import name 'create_instance'

What you expected to happen:
I install cloudkitty

How to reproduce it (minimal and precise):
I use 7 virtual machines on vmware vsphere environment.

**Environment**:
* OS (e.g. from /etc/os-release):
NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"

/etc/redhat-release
CentOS Linux release 8.4.2105

* Kernel (e.g. `uname -a`): Linux control-1 4.18.0-358.el8.x86_64 #1 SMP Mon Jan 10 13:11:20 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
* Docker version if applicable (e.g. `docker version`): 20.10.12
* Kolla-Ansible version (e.g. `git head or tag or stable branch` or pip package version if using release):
13.1.0.dev134

* Docker image Install type (source/binary): source
* Docker image distribution: centos
* Are you using official images from Docker Hub or self built? Docker Hub
* If self built - Kolla version and environment used to build:
* Share your inventory file, globals.yml and other configuration files if relevant

kolla_base_distro: "centos"
kolla_install_type: "source"
openstack_release: "master"
kolla_internal_vip_address: "10.1.17.50"
network_interface: "ens192"
tunnel_interface: "ens256"
neutron_external_interface: "ens224"
neutron_plugin_agent: "openvswitch"
enable_openstack_core: "yes"
enable_haproxy: "yes"
enable_ceilometer: "yes"
enable_cinder: "yes"
enable_cinder_backend_nfs: "yes"
enable_cloudkitty: "yes"
enable_elasticsearch: "{{ 'yes' if enable_central_logging | bool or enable_osprofiler | bool or enable_skydive | bool or enable_monasca | bool or (enable_cloudkitty | bool and cloudkitty_storage_backend == 'elasticsearch') else 'no' }}"
enable_fluentd: "no"
enable_gnocchi: "yes"
enable_horizon: "{{ enable_openstack_core | bool }}"
enable_horizon_cloudkitty: "{{ enable_cloudkitty | bool }}"
cloudkitty_collector_backend: "gnocchi"
cloudkitty_storage_backend: "elasticsearch"
nova_compute_virt_type: "qemu"

[Stefan Steinert (steinstef88)]

For me this also happens when I enable trove and I can reproduce the behavior with cloudkitty as in the initial post.

[syuan wu (syuan)]

I found on this website (https://storyboard.openstack.org/#!/story/2009924) that title is "cloudkitty-dashboard still uses horizon create_instance.SetInstanceDetailsAction class code which is dropped in horizon")
and the code is fixed and merged.
but when I set enable_cloudkitty: "yes" and redeploy with kolla-ansible, it seems has not been updated.

So my current workaround is Manually modify the code

1. Enter horizon docker
# docker exec -it horizon bash

2. According to the above website to fix the code of the two file
# vi /var/lib/kolla/venv/lib/python3.8/site-packages/cloudkittydashboard/enabled/_11_project_rating_panel.py
=> use hash(#) to comment the UPDATE_HORIZON_CONFIG line
# vi /var/lib/kolla/venv/lib/python3.8/site-packages/cloudkittydashboard/overrides.py
=> use hash(#) to comment the all lines of this file