Horizon bootstrap fails when murano enabled due to missing MySQL-Python

[Ahmed Ezzat Douban]

TASK [horizon : Copying over existing policy file] **************************************************************************

TASK [horizon : Check horizon container] ************************************************************************************
changed: [controller]

TASK [horizon : include_tasks] **********************************************************************************************
skipping: [controller]

TASK [horizon : include_tasks] **********************************************************************************************
included: /var/virtualenv/share/kolla-ansible/ansible/roles/horizon/tasks/bootstrap.yml for controller

TASK [horizon : Creating Horizon database] **********************************************************************************
ok: [controller -> controller]

TASK [horizon : Creating Horizon database user and setting permissions] *****************************************************
ok: [controller -> controller]

TASK [horizon : include_tasks] **********************************************************************************************
included: /var/virtualenv/share/kolla-ansible/ansible/roles/horizon/tasks/bootstrap_service.yml for controller

TASK [horizon : Running Horizon bootstrap container] ************************************************************************
fatal: [controller -> controller]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 1", "rc": 1, "stderr": "+ sudo -E kolla_set_configs\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:Kolla config strategy set to: COPY_ALWAYS\nINFO:__main__:Copying service configuration files\nINFO:__main__:Copying /var/lib/kolla/config_files/horizon.conf to /etc/httpd/conf.d/horizon.conf\nINFO:__main__:Setting permission for /etc/httpd/conf.d/horizon.conf\nINFO:__main__:Deleting /etc/openstack-dashboard/local_settings\nINFO:__main__:Copying /var/lib/kolla/config_files/local_settings to /etc/openstack-dashboard/local_settings\nINFO:__main__:Setting permission for /etc/openstack-dashboard/local_settings\nINFO:__main__:Copying /var/lib/kolla/config_files/custom_local_settings to /etc/openstack-dashboard/custom_local_settings\nINFO:__main__:Setting permission for /etc/openstack-dashboard/custom_local_settings\nINFO:__main__:Writing out command to execute\n++ cat /run_command\n+ CMD='/usr/sbin/httpd -DFOREGROUND'\n+ ARGS=\n+ [[ ! -n '' ]]\n+ . kolla_extend_start\n++ set -o errexit\n++ FORCE_GENERATE=\n++ HASH_PATH=/var/lib/kolla/.settings.md5sum.txt\n++ [[ binary == \\b\\i\\n\\a\\r\\y ]]\n++ [[ centos =~ ubuntu ]]\n++ [[ binary == \\b\\i\\n\\a\\r\\y ]]\n++ SITE_PACKAGES=/usr/lib/python2.7/site-packages\n++ [[ -f /var/lib/kolla/venv/bin/python ]]\n++ MANAGE_PY='/usr/bin/python2.7 /usr/bin/manage.py'\n++ [[ binary == \\s\\o\\u\\r\\c\\e ]]\n++ [[ -f /etc/openstack-dashboard/custom_local_settings ]]\n++ CUSTOM_SETTINGS_FILE=/usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py\n++ [[ binary == \\b\\i\\n\\a\\r\\y ]]\n++ [[ centos =~ ubuntu ]]\n++ [[ ! -L /usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py ]]\n++ ln -s /etc/openstack-dashboard/custom_local_settings /usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py\n++ [[ -n 0 ]]\n++ /usr/bin/python2.7 /usr/bin/manage.py migrate --noinput\nTraceback (most recent call last):\n File \"/usr/bin/manage.py\", line 23, in <module>\n execute_from_command_line(sys.argv)\n File \"/usr/lib/python2.7/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line\n utility.execute()\n File \"/usr/lib/python2.7/site-packages/django/core/management/__init__.py\", line 338, in execute\n django.setup()\n File \"/usr/lib/python2.7/site-packages/django/__init__.py\", line 27, in setup\n apps.populate(settings.INSTALLED_APPS)\n File \"/usr/lib/python2.7/site-packages/django/apps/registry.py\", line 108, in populate\n app_config.import_models()\n File \"/usr/lib/python2.7/site-packages/django/apps/config.py\", line 202, in import_models\n self.models_module = import_module(models_module_name)\n File \"/usr/lib64/python2.7/importlib/__init__.py\", line 37, in import_module\n __import__(name)\n File \"/usr/lib/python2.7/site-packages/django/contrib/auth/models.py\", line 4, in <module>\n from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager\n File \"/usr/lib/python2.7/site-packages/django/contrib/auth/base_user.py\", line 52, in <module>\n class AbstractBaseUser(models.Model):\n File \"/usr/lib/python2.7/site-packages/django/db/models/base.py\", line 124, in __new__\n new_class.add_to_class('_meta', Options(meta, app_label))\n File \"/usr/lib/python2.7/site-packages/django/db/models/base.py\", line 325, in add_to_class\n value.contribute_to_class(cls, name)\n File \"/usr/lib/python2.7/site-packages/django/db/models/options.py\", line 214, in contribute_to_class\n self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())\n File \"/usr/lib/python2.7/site-packages/django/db/__init__.py\", line 33, in __getattr__\n return getattr(connections[DEFAULT_DB_ALIAS], item)\n File \"/usr/lib/python2.7/site-packages/django/db/utils.py\", line 211, in __getitem__\n backend = load_backend(db['ENGINE'])\n File \"/usr/lib/python2.7/site-packages/django/db/utils.py\", line 115, in load_backend\n return import_module('%s.base' % backend_name)\n File \"/usr/lib64/python2.7/importlib/__init__.py\", line 37, in import_module\n __import__(name)\n File \"/usr/lib/python2.7/site-packages/django/db/backends/mysql/base.py\", line 30, in <module>\n 'Did you install mysqlclient or MySQL-python?' % e\ndjango.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb.\nDid you install mysqlclient or MySQL-python?\n", "stderr_lines": ["+ sudo -E kolla_set_configs", "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json", "INFO:__main__:Validating config file", "INFO:__main__:Kolla config strategy set to: COPY_ALWAYS", "INFO:__main__:Copying service configuration files", "INFO:__main__:Copying /var/lib/kolla/config_files/horizon.conf to /etc/httpd/conf.d/horizon.conf", "INFO:__main__:Setting permission for /etc/httpd/conf.d/horizon.conf", "INFO:__main__:Deleting /etc/openstack-dashboard/local_settings", "INFO:__main__:Copying /var/lib/kolla/config_files/local_settings to /etc/openstack-dashboard/local_settings", "INFO:__main__:Setting permission for /etc/openstack-dashboard/local_settings", "INFO:__main__:Copying /var/lib/kolla/config_files/custom_local_settings to /etc/openstack-dashboard/custom_local_settings", "INFO:__main__:Setting permission for /etc/openstack-dashboard/custom_local_settings", "INFO:__main__:Writing out command to execute", "++ cat /run_command", "+ CMD='/usr/sbin/httpd -DFOREGROUND'", "+ ARGS=", "+ [[ ! -n '' ]]", "+ . kolla_extend_start", "++ set -o errexit", "++ FORCE_GENERATE=", "++ HASH_PATH=/var/lib/kolla/.settings.md5sum.txt", "++ [[ binary == \\b\\i\\n\\a\\r\\y ]]", "++ [[ centos =~ ubuntu ]]", "++ [[ binary == \\b\\i\\n\\a\\r\\y ]]", "++ SITE_PACKAGES=/usr/lib/python2.7/site-packages", "++ [[ -f /var/lib/kolla/venv/bin/python ]]", "++ MANAGE_PY='/usr/bin/python2.7 /usr/bin/manage.py'", "++ [[ binary == \\s\\o\\u\\r\\c\\e ]]", "++ [[ -f /etc/openstack-dashboard/custom_local_settings ]]", "++ CUSTOM_SETTINGS_FILE=/usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py", "++ [[ binary == \\b\\i\\n\\a\\r\\y ]]", "++ [[ centos =~ ubuntu ]]", "++ [[ ! -L /usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py ]]", "++ ln -s /etc/openstack-dashboard/custom_local_settings /usr/lib/python2.7/site-packages/openstack_dashboard/local/custom_local_settings.py", "++ [[ -n 0 ]]", "++ /usr/bin/python2.7 /usr/bin/manage.py migrate --noinput", "Traceback (most recent call last):", " File \"/usr/bin/manage.py\", line 23, in <module>", " execute_from_command_line(sys.argv)", " File \"/usr/lib/python2.7/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line", " utility.execute()", " File \"/usr/lib/python2.7/site-packages/django/core/management/__init__.py\", line 338, in execute", " django.setup()", " File \"/usr/lib/python2.7/site-packages/django/__init__.py\", line 27, in setup", " apps.populate(settings.INSTALLED_APPS)", " File \"/usr/lib/python2.7/site-packages/django/apps/registry.py\", line 108, in populate", " app_config.import_models()", " File \"/usr/lib/python2.7/site-packages/django/apps/config.py\", line 202, in import_models", " self.models_module = import_module(models_module_name)", " File \"/usr/lib64/python2.7/importlib/__init__.py\", line 37, in import_module", " __import__(name)", " File \"/usr/lib/python2.7/site-packages/django/contrib/auth/models.py\", line 4, in <module>", " from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager", " File \"/usr/lib/python2.7/site-packages/django/contrib/auth/base_user.py\", line 52, in <module>", " class AbstractBaseUser(models.Model):", " File \"/usr/lib/python2.7/site-packages/django/db/models/base.py\", line 124, in __new__", " new_class.add_to_class('_meta', Options(meta, app_label))", " File \"/usr/lib/python2.7/site-packages/django/db/models/base.py\", line 325, in add_to_class", " value.contribute_to_class(cls, name)", " File \"/usr/lib/python2.7/site-packages/django/db/models/options.py\", line 214, in contribute_to_class", " self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())", " File \"/usr/lib/python2.7/site-packages/django/db/__init__.py\", line 33, in __getattr__", " return getattr(connections[DEFAULT_DB_ALIAS], item)", " File \"/usr/lib/python2.7/site-packages/django/db/utils.py\", line 211, in __getitem__", " backend = load_backend(db['ENGINE'])", " File \"/usr/lib/python2.7/site-packages/django/db/utils.py\", line 115, in load_backend", " return import_module('%s.base' % backend_name)", " File \"/usr/lib64/python2.7/importlib/__init__.py\", line 37, in import_module", " __import__(name)", " File \"/usr/lib/python2.7/site-packages/django/db/backends/mysql/base.py\", line 30, in <module>", " 'Did you install mysqlclient or MySQL-python?' % e", "django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb.", "Did you install mysqlclient or MySQL-python?"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT **********************************************************************************************************

[Ahmed Ezzat Douban (doubando)]

Options selected on global.yml for Murano deployment

enable_outward_rabbitmq: "{{ enable_murano | bool }}"
enable_horizon_murano: "{{ enable_murano | bool }}"
enable_murano: "yes"
horizon_backend_database: "{{ enable_murano | bool }}"

[Mark Goddard (mgoddard)]

Did you manage to fix this?

[Ahmed Ezzat Douban (doubando)]

No. Didnt figure it out

[Mark Goddard (mgoddard)]

I expect you need to install an additional package in the horizon container.

[Mark Goddard (mgoddard)]

Is it still a problem?

[Radosław Piliszek (yoctozepto)]

Invalidated due to no recent response.

[Aravindh (highlow7)]

This is still a problem. I have the same issue with murano enabled.

I would ideally expect the package to be part of the docker image build.

[James Page (james-page)]

I raised:

https://review.opendev.org/c/openstack/kolla/+/867620

which should resolve this for deb based binary builds. The main challenge is that Django has no way that I can see to configure the driver for MySQL - it just defaults to the native client so pymysql can't be used (which is installed in the container).

Ubuntu did carry a patch in Django for sometime to change the default driver directly in Django but this was dropped in 2022 (around the release of Ubuntu Jammy & Yoga).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla/+/867620
Committed: https://opendev.org/openstack/kolla/commit/5d56db08911441a3e3b603b5c09779514ba6ee88
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 5d56db08911441a3e3b603b5c09779514ba6ee88
Author: James Page <email address hidden>
Date: Wed Dec 14 10:21:12 2022 +0000

    horizon: Ensure mysqldb client is installed

    Impacts ubuntu-binary based built images.

    Django has no way to change the driver for MySQL backed session
    management in Horizon.

    Ensure that the supported MySQLDB driver is installed for use.

    Change-Id: If006a75b97b336242d2d458d0a85f75a5d450635
    Closes-Bug: 1840903

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 14.9.0 release.