Activity log for bug #1793264

Date Who What changed Old value New value Message
2018-09-19 07:51:48 Quique Llorente bug added bug
2018-09-19 09:04:11 chandan kumar summary Error deploying containerized gnocchi at oc deploy at rocky overcloud deploy script.sh gives false negative result sometimes if it successed
2018-09-19 09:09:25 chandan kumar description Periodic job for featureset020 is failing at gnocchi container installation https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_03_12_38 2018-09-19 03:12:38 | "stdout: 73917fec2e1fd21cb3d200bc19361b44d04813aeebbca34f9cf3263bd0e26a2f", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete", 2018-09-19 03:12:38 | "Error attempting to run <function create_incomplete_consumers at 0x7fd4c6e08d70>", 2018-09-19 03:12:38 | "+---------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| create_incomplete_consumers | 0 | 0 |", 2018-09-19 03:12:38 | "| delete_build_requests_with_no_instance_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_instances_add_request_spec | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_keypairs_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_classes_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_limits_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migration_migrate_to_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_missing_availability_zones | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_queued_for_delete | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_uuids | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete.", 2018-09-19 03:12:38 | "+--------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| attachment_specs_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| backup_service_online_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| shared_targets_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| volume_service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stderr: Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", 2018-09-19 03:12:38 | "stdout: " 2018-09-19 03:12:38 | ] 2018-09-19 03:12:38 | } Periodic job for featureset020 rocky is failing at overcloud deploy but the deployment gets completed successfully but the exist status of the overcloud_deploy.sh script is coming false. From the logs: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_03_12_38 "| attachment_specs_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| backup_service_online_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| shared_targets_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| volume_service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stderr: Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", 2018-09-19 03:12:38 | "stdout: " 2018-09-19 03:12:38 | ] 2018-09-19 03:12:38 | } 2018-09-19 03:12:38 | ok: [overcloud-novacompute-0] => { 2018-09-19 03:12:38 | "failed_when_result": false, 2018-09-19 03:12:38 | "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [] 2018-09-19 03:12:38 | } 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Check if /var/lib/docker-puppet/docker-puppet-tasks5.json exists] ******** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.240) 0:18:56.575 *** 2018-09-19 03:12:38 | ok: [overcloud-controller-0] => {"changed": false, "stat": {"exists": false}} 2018-09-19 03:12:38 | ok: [overcloud-novacompute-0] => {"changed": false, "stat": {"exists": false}} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Run docker-puppet tasks (bootstrap tasks) for step 5] ******************** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.380) 0:18:56.955 *** 2018-09-19 03:12:38 | skipping: [overcloud-controller-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2018-09-19 03:12:38 | skipping: [overcloud-novacompute-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Debug output for task which failed: Run docker-puppet tasks (bootstrap tasks) for step 5] *** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.091) 0:18:57.047 *** 2018-09-19 03:12:38 | skipping: [overcloud-controller-0] => {"skip_reason": "Conditional result was False"} 2018-09-19 03:12:38 | skipping: [overcloud-novacompute-0] => {"skip_reason": "Conditional result was False"} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY [Server Post Deployments] ************************************************* 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [include_tasks] *********************************************************** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.091) 0:18:57.138 *** 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY [External deployment Post Deploy tasks] *********************************** 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY RECAP ********************************************************************* 2018-09-19 03:12:38 | overcloud-controller-0 : ok=188 changed=91 unreachable=0 failed=0 2018-09-19 03:12:38 | overcloud-novacompute-0 : ok=138 changed=66 unreachable=0 failed=0 2018-09-19 03:12:38 | undercloud : ok=10 changed=7 unreachable=0 failed=0 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:37 +0000 (0:00:00.171) 0:18:57.310 *** 2018-09-19 03:12:44 | ==================Waiting for messages on queue 'tripleo' with no timeout. 2018-09-19 04:12:44 | Connection is already closed. 2018-09-19 04:12:44 | ============================================================= 2018-09-19 04:12:44 | 2018-09-19 04:12:44 | Ansible passed. 2018-09-19 04:12:44 | Overcloud configuration completed. 2018-09-19 04:12:45 | + status_code=1 2018-09-19 04:12:45 | + openstack stack list 2018-09-19 04:12:45 | + grep -q overcloud 2018-09-19 04:12:48 | + openstack stack list 2018-09-19 04:12:48 | + grep -Eq '(CREATE|UPDATE)_COMPLETE' 2018-09-19 04:12:50 | + openstack overcloud status 2018-09-19 04:12:50 | + grep -Eq DEPLOY_SUCCESS 2018-09-19 04:12:55 | + exit 1 We need to find out why ok: [overcloud-novacompute-0] => { 2018-09-19 03:12:38 | "failed_when_result": false, 2018-09-19 03:12:38 | "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [] 2018-09-19 03:12:38 | } is returned false while parsing this code http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/overcloud-deploy/templates/overcloud-deploy.sh.j2#n40
2018-09-19 09:52:29 chandan kumar description Periodic job for featureset020 rocky is failing at overcloud deploy but the deployment gets completed successfully but the exist status of the overcloud_deploy.sh script is coming false. From the logs: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_03_12_38 "| attachment_specs_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| backup_service_online_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| shared_targets_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| volume_service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stderr: Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", 2018-09-19 03:12:38 | "stdout: " 2018-09-19 03:12:38 | ] 2018-09-19 03:12:38 | } 2018-09-19 03:12:38 | ok: [overcloud-novacompute-0] => { 2018-09-19 03:12:38 | "failed_when_result": false, 2018-09-19 03:12:38 | "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [] 2018-09-19 03:12:38 | } 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Check if /var/lib/docker-puppet/docker-puppet-tasks5.json exists] ******** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.240) 0:18:56.575 *** 2018-09-19 03:12:38 | ok: [overcloud-controller-0] => {"changed": false, "stat": {"exists": false}} 2018-09-19 03:12:38 | ok: [overcloud-novacompute-0] => {"changed": false, "stat": {"exists": false}} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Run docker-puppet tasks (bootstrap tasks) for step 5] ******************** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.380) 0:18:56.955 *** 2018-09-19 03:12:38 | skipping: [overcloud-controller-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2018-09-19 03:12:38 | skipping: [overcloud-novacompute-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [Debug output for task which failed: Run docker-puppet tasks (bootstrap tasks) for step 5] *** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.091) 0:18:57.047 *** 2018-09-19 03:12:38 | skipping: [overcloud-controller-0] => {"skip_reason": "Conditional result was False"} 2018-09-19 03:12:38 | skipping: [overcloud-novacompute-0] => {"skip_reason": "Conditional result was False"} 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY [Server Post Deployments] ************************************************* 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | TASK [include_tasks] *********************************************************** 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:36 +0000 (0:00:00.091) 0:18:57.138 *** 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY [External deployment Post Deploy tasks] *********************************** 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | PLAY RECAP ********************************************************************* 2018-09-19 03:12:38 | overcloud-controller-0 : ok=188 changed=91 unreachable=0 failed=0 2018-09-19 03:12:38 | overcloud-novacompute-0 : ok=138 changed=66 unreachable=0 failed=0 2018-09-19 03:12:38 | undercloud : ok=10 changed=7 unreachable=0 failed=0 2018-09-19 03:12:38 | 2018-09-19 03:12:38 | Wednesday 19 September 2018 03:12:37 +0000 (0:00:00.171) 0:18:57.310 *** 2018-09-19 03:12:44 | ==================Waiting for messages on queue 'tripleo' with no timeout. 2018-09-19 04:12:44 | Connection is already closed. 2018-09-19 04:12:44 | ============================================================= 2018-09-19 04:12:44 | 2018-09-19 04:12:44 | Ansible passed. 2018-09-19 04:12:44 | Overcloud configuration completed. 2018-09-19 04:12:45 | + status_code=1 2018-09-19 04:12:45 | + openstack stack list 2018-09-19 04:12:45 | + grep -q overcloud 2018-09-19 04:12:48 | + openstack stack list 2018-09-19 04:12:48 | + grep -Eq '(CREATE|UPDATE)_COMPLETE' 2018-09-19 04:12:50 | + openstack overcloud status 2018-09-19 04:12:50 | + grep -Eq DEPLOY_SUCCESS 2018-09-19 04:12:55 | + exit 1 We need to find out why ok: [overcloud-novacompute-0] => { 2018-09-19 03:12:38 | "failed_when_result": false, 2018-09-19 03:12:38 | "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [] 2018-09-19 03:12:38 | } is returned false while parsing this code http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/overcloud-deploy/templates/overcloud-deploy.sh.j2#n40 Periodic job for featureset020 is failing at gnocchi container installation https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_03_12_38 2018-09-19 03:12:38 | "stdout: 73917fec2e1fd21cb3d200bc19361b44d04813aeebbca34f9cf3263bd0e26a2f", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete", 2018-09-19 03:12:38 | "Error attempting to run <function create_incomplete_consumers at 0x7fd4c6e08d70>", 2018-09-19 03:12:38 | "+---------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| create_incomplete_consumers | 0 | 0 |", 2018-09-19 03:12:38 | "| delete_build_requests_with_no_instance_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_instances_add_request_spec | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_keypairs_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_classes_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_limits_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migration_migrate_to_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_missing_availability_zones | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_queued_for_delete | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_uuids | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete.", 2018-09-19 03:12:38 | "+--------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| attachment_specs_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| backup_service_online_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| shared_targets_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| volume_service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stderr: Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", 2018-09-19 03:12:38 | "stdout: " 2018-09-19 03:12:38 | ] 2018-09-19 03:12:38 | }
2018-09-19 09:53:54 chandan kumar summary overcloud deploy script.sh gives false negative result sometimes if it successed periodic fs020 rocky failing at containerized gnocchi
2018-09-19 10:19:09 Quique Llorente description Periodic job for featureset020 is failing at gnocchi container installation https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_03_12_38 2018-09-19 03:12:38 | "stdout: 73917fec2e1fd21cb3d200bc19361b44d04813aeebbca34f9cf3263bd0e26a2f", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete", 2018-09-19 03:12:38 | "Error attempting to run <function create_incomplete_consumers at 0x7fd4c6e08d70>", 2018-09-19 03:12:38 | "+---------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| create_incomplete_consumers | 0 | 0 |", 2018-09-19 03:12:38 | "| delete_build_requests_with_no_instance_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_instances_add_request_spec | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_keypairs_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_classes_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migrate_quota_limits_to_api_db | 0 | 0 |", 2018-09-19 03:12:38 | "| migration_migrate_to_uuid | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_missing_availability_zones | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_queued_for_delete | 0 | 0 |", 2018-09-19 03:12:38 | "| populate_uuids | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stdout: Running batches of 50 until complete.", 2018-09-19 03:12:38 | "+--------------------------------------------+--------------+-----------+", 2018-09-19 03:12:38 | "| Migration | Total Needed | Completed |", 2018-09-19 03:12:38 | "| attachment_specs_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| backup_service_online_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| shared_targets_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "| volume_service_uuids_online_data_migration | 0 | 0 |", 2018-09-19 03:12:38 | "stderr: Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", 2018-09-19 03:12:38 | "stdout: " 2018-09-19 03:12:38 | ] 2018-09-19 03:12:38 | } Periodic job for featureset020 is failing at overcloud deploy, but stack is succesfully installed https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_04_12_44 2018-09-19 04:12:44 | Connection is already closed. 2018-09-19 04:12:44 | ============================================================= 2018-09-19 04:12:44 | 2018-09-19 04:12:44 | Ansible passed. 2018-09-19 04:12:44 | Overcloud configuration completed. Possible issue at python-tripleoclient Possible fix https://review.openstack.org/603633
2018-09-19 11:25:22 OpenStack Infra tripleo: status New In Progress
2018-09-19 11:43:04 Quique Llorente description Periodic job for featureset020 is failing at overcloud deploy, but stack is succesfully installed https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_04_12_44 2018-09-19 04:12:44 | Connection is already closed. 2018-09-19 04:12:44 | ============================================================= 2018-09-19 04:12:44 | 2018-09-19 04:12:44 | Ansible passed. 2018-09-19 04:12:44 | Overcloud configuration completed. Possible issue at python-tripleoclient Possible fix https://review.openstack.org/603633 Periodic job for featureset020 is failing at overcloud deploy, but stack is succesfully installed https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/5e5abd2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-19_04_12_44 2018-09-19 04:12:44 | Connection is already closed. 2018-09-19 04:12:44 | ============================================================= 2018-09-19 04:12:44 | 2018-09-19 04:12:44 | Ansible passed. 2018-09-19 04:12:44 | Overcloud configuration completed. Possible issue at python-tripleoclient Possible fix https://review.openstack.org/#/c/603645/
2018-09-21 16:46:04 OpenStack Infra tripleo: status In Progress Fix Released
2018-09-27 12:24:54 OpenStack Infra tags promotion-blocker in-stable-rocky promotion-blocker
2018-10-20 07:54:57 OpenStack Infra tags in-stable-rocky promotion-blocker in-stable-queens in-stable-rocky promotion-blocker