Activity log for bug #1947534

Date Who What changed Old value New value Message
2021-10-18 00:47:07 Vahid Alimohamadi bug added bug
2021-10-18 00:49:41 Vahid Alimohamadi tags 13.0.0.0rc1 mariadb 13.0.0.0rc1 all-in-one mariadb
2021-10-18 00:53:39 Vahid Alimohamadi tags 13.0.0.0rc1 all-in-one mariadb 13.0.0.0rc1 all-in-one mariadb mult-node
2021-10-18 01:07:04 Vahid Alimohamadi description TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] I tried to access mariadb using mysql client but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] and count down to zero and failure. I tried to access mariadb using mysql client but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io
2021-10-18 01:09:21 Vahid Alimohamadi description TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] and count down to zero and failure. I tried to access mariadb using mysql client but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] and count down to zero and failure. I tried to access mariadb using mysql client but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 all-in-one Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io
2021-10-18 08:13:18 Radosław Piliszek kolla-ansible: status New Incomplete
2021-10-18 10:06:32 Vahid Alimohamadi description TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] and count down to zero and failure. I tried to access mariadb using mysql client but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 all-in-one Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io TASK [mariadb : Check MariaDB service port liveness] ************************************************ fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 10, "msg": "Timeout when waiting for search string MariaDB in 192.168.54.12:3306"} then stuck at RUNNING HANDLER [mariadb : Wait for first MariaDB service port liveness] and count down to zero and failure. I tried to access mariadb using mysql client and got timeout but there's no error log from container: mariadb image: quay.io/openstack.kolla/ubuntu-source-mariadb-server xena eb5e1822ecce 18 hours ago 624MB mariadb container log output: ... INFO:__main__:Setting permission for /var/lib/mysql/mysql/time_zone.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/index_stats.MAD INFO:__main__:Setting permission for /var/lib/mysql/mysql/procs_priv.MAI INFO:__main__:Setting permission for /var/lib/mysql/mysql/servers.frm INFO:__main__:Setting permission for /var/lib/mysql/performance_schema/db.opt ++ cat /run_command + CMD=/usr/bin/mysqld_safe + ARGS= + sudo kolla_copy_cacerts + [[ ! -n '' ]] + . kolla_extend_start ++ : /var/log/kolla/mariadb ++ [[ ! -d /var/log/kolla/mariadb ]] +++ stat -c %a /var/log/kolla/mariadb ++ [[ 2755 != \7\5\5 ]] ++ chmod 755 /var/log/kolla/mariadb ++ [[ -n '' ]] ++ [[ -n '' ]] ++ [[ -n 0 ]] ++ ARGS=--wsrep-new-cluster + echo 'Running command: '\''/usr/bin/mysqld_safe --wsrep-new-cluster'\''' + exec /usr/bin/mysqld_safe --wsrep-new-cluster Running command: '/usr/bin/mysqld_safe --wsrep-new-cluster' 211018 00:30:19 mysqld_safe Logging to '/var/log/kolla/mariadb/mariadb.log'. 211018 00:30:19 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql/ OS: I tried on CentOS 8 stream, ubuntu 20.04 LTS and ubuntu 21.04 LTS HP G7 bare-metal. Kernel: Linux os 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux kolla-ansible version: 13.0.0.0rc1 all-in-one Docker version:20.10.9, build c2ea9bc kolla global.conf: config_strategy: "COPY_ALWAYS" kolla_base_distro: "ubuntu" kolla_install_type: "source" openstack_release: "xena" kolla_internal_vip_address: "192.168.54.12" network_interface: eno1 neutron_external_interface: eno2 neutron_plugin_agent: "openvswitch" enable_haproxy: "no" enable_ceilometer: "yes" enable_cinder: "yes" enable_cinder_backup: "no" enable_cinder_backend_lvm: "no" enable_cloudkitty: "yes" enable_gnocchi: "yes" enable_neutron_provider_networks: "yes" ceph_cinder_keyring: "ceph.client.admin.keyring" ceph_cinder_user: "admin" ceph_cinder_pool_name: "volumes" fernet_token_expiry: 86400 cinder_backend_ceph: "yes" cinder_volume_group: "volumes" nova_compute_virt_type: "kvm" nova_console: "novnc" This config works on kolla 12.2.0 and wallaby release but those images are on dockerhub not quay.io
2021-10-18 10:42:01 Radosław Piliszek kolla-ansible: status Incomplete Triaged
2021-10-18 10:46:42 Radosław Piliszek kolla-ansible: importance Undecided Medium
2021-10-18 10:49:06 OpenStack Infra kolla-ansible: status Triaged In Progress
2021-10-20 11:27:09 OpenStack Infra kolla-ansible: status In Progress Fix Released
2021-10-21 09:30:35 OpenStack Infra tags 13.0.0.0rc1 all-in-one mariadb mult-node 13.0.0.0rc1 all-in-one in-stable-xena mariadb mult-node
2021-10-27 12:32:09 OpenStack Infra tags 13.0.0.0rc1 all-in-one in-stable-xena mariadb mult-node 13.0.0.0rc1 all-in-one in-stable-wallaby in-stable-xena mariadb mult-node