Activity log for bug #1851847

Date Who What changed Old value New value Message
2019-11-08 14:21:25 Ronelle Landy bug added bug
2019-11-08 14:21:35 Ronelle Landy tripleo: milestone ussuri-1
2019-11-08 14:21:38 Ronelle Landy tripleo: importance Undecided High
2019-11-08 14:21:41 Ronelle Landy tripleo: status New Triaged
2019-11-08 14:21:47 Ronelle Landy tags ci
2019-11-08 14:53:21 Ronelle Landy tags ci ci promotion-blocker
2019-11-08 14:53:50 Ronelle Landy tripleo: importance High Critical
2019-11-08 15:00:48 Ronelle Landy tags ci promotion-blocker ci
2019-11-08 19:45:50 Ronelle Landy tags ci ci promotion-blocker
2019-11-08 19:46:01 Ronelle Landy tripleo: assignee Ronelle Landy (rlandy)
2019-11-12 16:08:45 Ronelle Landy summary rhel-8-scenario004 fails to deploy standalone - error while resolving custom fact \"rabbitmq_nodename\": undefined method `[]' for nil:NilClass" rhel-8-scenario004 fails to deploy standalone - pacemaker/mysql
2019-11-12 16:22:54 Ronelle Landy description tripleo-ci-rhel-8-scenario004-standalone-rdo have been failing consistently with errors related to pacemaker/rabbit. Error: unable to get cib Error: unable to get cib shows up in the container logs. 019-11-08 13:45:41 | "Error: Facter: error while resolving custom fact \"rabbitmq_nodename\": undefined method `[]' for nil:NilClass", 2019-11-08 13:45:41 | "Error: /Stage[main]/Pacemaker::Stonith/Pacemaker::Property[Disable STONITH]/Pcmk_property[property--stonith-enabled]: Could not evaluate: backup_cib: Running: pcs cluster cib /var/lib/pacemaker/cib/puppet-cib-backup20191108-9-1ww1cq7 failed with code: 1 -> Error: unable to get cib", 2019-11-08 13:45:41 | "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Pacemaker::Property[rabbitmq-role-standalone]/Pcmk_property[property-standalone-rabbitmq-role]: Could not evaluate: backup_cib: Running: pcs cluster ci The full deploy log is below: http://logs.rdoproject.org/85/692985/4/openstack-check/tripleo-ci-rhel-8-scenario004-standalone-rdo/d06da86/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz The link below shows the paunch errors: logs.rdoproject.org/85/692985/4/openstack-check/tripleo-ci-rhel-8-scenario004-standalone-rdo/d06da86/logs/undercloud/var/log/extra/errors.txt.txt.gz Updating bug description as check and periodic jobs have two different errors now. We suspect that the check job errors will be fixed by a promotion so to focus on the periodic/promotion errors we have: 2019-11-12 14:39:41 | "Error: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0]", 2019-11-12 14:39:41 | "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Exec[galera-ready]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0]", 2019-11-12 14:39:41 | "Error: Failed to apply catalog: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)", 2019-11-12 14:39:41 | "+ rc=1", Full deploy log: http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario004-standalone-master/39a2b42/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz History of the test pass/fail stats: https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-rhel-8-scenario004-standalone-master periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-master also fails out -------------------------------------------------------------------- tripleo-ci-rhel-8-scenario004-standalone-rdo have been failing consistently with errors related to pacemaker/rabbit. Error: unable to get cib Error: unable to get cib shows up in the container logs. 019-11-08 13:45:41 | "Error: Facter: error while resolving custom fact \"rabbitmq_nodename\": undefined method `[]' for nil:NilClass", 2019-11-08 13:45:41 | "Error: /Stage[main]/Pacemaker::Stonith/Pacemaker::Property[Disable STONITH]/Pcmk_property[property--stonith-enabled]: Could not evaluate: backup_cib: Running: pcs cluster cib /var/lib/pacemaker/cib/puppet-cib-backup20191108-9-1ww1cq7 failed with code: 1 -> Error: unable to get cib", 2019-11-08 13:45:41 | "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Rabbitmq_bundle/Pacemaker::Property[rabbitmq-role-standalone]/Pcmk_property[property-standalone-rabbitmq-role]: Could not evaluate: backup_cib: Running: pcs cluster ci The full deploy log is below: http://logs.rdoproject.org/85/692985/4/openstack-check/tripleo-ci-rhel-8-scenario004-standalone-rdo/d06da86/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz The link below shows the paunch errors: logs.rdoproject.org/85/692985/4/openstack-check/tripleo-ci-rhel-8-scenario004-standalone-rdo/d06da86/logs/undercloud/var/log/extra/errors.txt.txt.gz
2019-11-12 16:23:32 Ronelle Landy summary rhel-8-scenario004 fails to deploy standalone - pacemaker/mysql periodic-tripleo-ci-rhel-8-scenario004-standalone-master fails to deploy standalone - pacemaker/mysql
2019-11-13 07:26:39 OpenStack Infra tripleo: status Triaged In Progress
2019-11-13 07:26:39 OpenStack Infra tripleo: assignee Ronelle Landy (rlandy) yatin (yatinkarel)
2019-11-13 13:42:06 OpenStack Infra tripleo: status In Progress Fix Released
2020-02-13 19:48:08 OpenStack Infra tags ci promotion-blocker ci in-stable-train promotion-blocker