Activity log for bug #1832509

Date Who What changed Old value New value Message
2019-06-12 11:38:13 Anton Antonov bug added bug
2019-06-12 11:41:41 Anton Antonov bug added subscriber John Fulton
2019-06-12 11:46:58 Anton Antonov description Description: "openstack overcloud external-update run --tags ceph" command fails when trying to do a minor Rocky to Rocky upgrade. Steps: 1. Upgraded the undercloud 2. Upgraded all overcloud nodes 3. Run "openstack overcloud external-update run --tags ceph" Expected results: A successfully updated containerized Red Hat Ceph Storage 3 cluster. Actual results: ''' TASK [set facts for swift back up of ceph-ansible fetch directory] ************* Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.068) 0:01:02.993 ********** ok: [undercloud] => {"ansible_facts": {"new_ceph_ansible_tarball_name": "temporary_dir_new.tar.gz", "old_ceph_ansible_tarball_name" : "temporary_dir_old.tar.gz", "swift_get_url": "https://10.35.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ans ible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a646cc71b094dd6a0ae8bc8df3d4b6&temp_url_expires=1560282811", "swift_put_ url": "https://10.35.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_ url_sig=caea8c31d996d2bc07e5a2152cd85d42d81f2d60&temp_url_expires=1560282839"}, "changed": false} TASK [attempt download of fetch directory tarball from swift backup] *********** Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.079) 0:01:03.073 ********** [WARNING]: Consider using the get_url or uri module rather than running curl. If you need to use command because get_url or uri is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. changed: [undercloud] => {"changed": true, "cmd": "curl -s -o /tmp/temporary_dir_old.tar.gz -w '%{http_code}' -X GET \"https://10.3 5.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a 646cc71b094dd6a0ae8bc8df3d4b6&temp_url_expires=1560282811\"", "delta": "0:00:00.183758", "end": "2019-06-11 22:44:07.674133", "rc": 0, "start": "2019-06-11 22:44:07.490375", "stderr": "", "stderr_lines": [], "stdout": "401", "stdout_lines": ["401"]} TASK [ensure we create a new fetch_directory or use the old fetch_directory] *** Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.409) 0:01:03.482 ********** fatal: [undercloud]: FAILED! => {"changed": false, "msg": "Received HTTP: 401 when attempting to GET from https://10.35.5.2:13808/v 1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a646cc71b094dd 6a0ae8bc8df3d4b6&temp_url_expires=1560282811"} NO MORE HOSTS LEFT ************************************************************* ``` Environment: openstack-tripleo-puppet-elements-9.0.2-0.20190425202749.1ab58f2.el7.noarch ceph-ansible-3.2.5-1.el7.noarch python-tripleoclient-10.6.2-0.20190522234411.12e3f68.el7.noarch openstack-tripleo-common-containers-9.5.1-0.20190507224322.cd24177.el7.noarch ansible-tripleo-ipsec-9.1.1-0.20190513182453.ffe104c.el7.noarch python2-tripleo-common-9.5.1-0.20190507224322.cd24177.el7.noarch openstack-tripleo-common-9.5.1-0.20190507224322.cd24177.el7.noarch ansible-role-tripleo-modify-image-1.0.1-0.20190531141856.f33cad7.el7.noarch openstack-tripleo-image-elements-9.1.1-0.20190420053043.aa75390.el7.noarch puppet-ceph-2.6.1-0.20190425104853.6d67b24.el7.noarch openstack-tripleo-heat-templates-9.3.1-0.20190531051851.bb4fb9d.el7.noarch python-tripleoclient-heat-installer-10.6.2-0.20190522234411.12e3f68.el7.noarch python2-tripleo-repos-0.0.1-0.20190520152004.8a48b48.el7.noarch openstack-tripleo-validations-9.3.2-0.20190523001404.bf11998.el7.noarch puppet-tripleo-9.4.1-0.20190601011855.c5986da.el7.noarch Description: "openstack overcloud external-update run --tags ceph" command fails when trying to do a minor Rocky to Rocky upgrade. Steps: 1. Upgraded the undercloud 2. Prepared the update openstack overcloud update prepare ... 2. Upgraded all overcloud nodes openstack overcloud update run --nodes Controller openstack overcloud update run --nodes Compute openstack overcloud update run --nodes CephStorage 3. Run "openstack overcloud external-update run --tags ceph" Expected results: A successfully updated containerized Red Hat Ceph Storage 3 cluster. Actual results: ''' TASK [set facts for swift back up of ceph-ansible fetch directory] ************* Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.068) 0:01:02.993 ********** ok: [undercloud] => {"ansible_facts": {"new_ceph_ansible_tarball_name": "temporary_dir_new.tar.gz", "old_ceph_ansible_tarball_name" : "temporary_dir_old.tar.gz", "swift_get_url": "https://10.35.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ans ible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a646cc71b094dd6a0ae8bc8df3d4b6&temp_url_expires=1560282811", "swift_put_ url": "https://10.35.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_ url_sig=caea8c31d996d2bc07e5a2152cd85d42d81f2d60&temp_url_expires=1560282839"}, "changed": false} TASK [attempt download of fetch directory tarball from swift backup] *********** Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.079) 0:01:03.073 **********  [WARNING]: Consider using the get_url or uri module rather than running curl. If you need to use command because get_url or uri is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. changed: [undercloud] => {"changed": true, "cmd": "curl -s -o /tmp/temporary_dir_old.tar.gz -w '%{http_code}' -X GET \"https://10.3 5.5.2:13808/v1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a 646cc71b094dd6a0ae8bc8df3d4b6&temp_url_expires=1560282811\"", "delta": "0:00:00.183758", "end": "2019-06-11 22:44:07.674133", "rc":  0, "start": "2019-06-11 22:44:07.490375", "stderr": "", "stderr_lines": [], "stdout": "401", "stdout_lines": ["401"]} TASK [ensure we create a new fetch_directory or use the old fetch_directory] *** Tuesday 11 June 2019 22:44:07 +0100 (0:00:00.409) 0:01:03.482 ********** fatal: [undercloud]: FAILED! => {"changed": false, "msg": "Received HTTP: 401 when attempting to GET from https://10.35.5.2:13808/v 1/AUTH_0d06a24bb33c4b9ebf922cb7c3bcf118/overcloud_ceph_ansible_fetch_dir/temporary_dir.tar.gz?temp_url_sig=7fd5deec39a646cc71b094dd 6a0ae8bc8df3d4b6&temp_url_expires=1560282811"} NO MORE HOSTS LEFT ************************************************************* ``` Environment: openstack-tripleo-puppet-elements-9.0.2-0.20190425202749.1ab58f2.el7.noarch ceph-ansible-3.2.5-1.el7.noarch python-tripleoclient-10.6.2-0.20190522234411.12e3f68.el7.noarch openstack-tripleo-common-containers-9.5.1-0.20190507224322.cd24177.el7.noarch ansible-tripleo-ipsec-9.1.1-0.20190513182453.ffe104c.el7.noarch python2-tripleo-common-9.5.1-0.20190507224322.cd24177.el7.noarch openstack-tripleo-common-9.5.1-0.20190507224322.cd24177.el7.noarch ansible-role-tripleo-modify-image-1.0.1-0.20190531141856.f33cad7.el7.noarch openstack-tripleo-image-elements-9.1.1-0.20190420053043.aa75390.el7.noarch puppet-ceph-2.6.1-0.20190425104853.6d67b24.el7.noarch openstack-tripleo-heat-templates-9.3.1-0.20190531051851.bb4fb9d.el7.noarch python-tripleoclient-heat-installer-10.6.2-0.20190522234411.12e3f68.el7.noarch python2-tripleo-repos-0.0.1-0.20190520152004.8a48b48.el7.noarch openstack-tripleo-validations-9.3.2-0.20190523001404.bf11998.el7.noarch puppet-tripleo-9.4.1-0.20190601011855.c5986da.el7.noarch
2019-06-12 14:49:19 OpenStack Infra tripleo: status New In Progress
2019-06-12 14:49:19 OpenStack Infra tripleo: assignee John Fulton (jfulton-org)
2019-06-12 14:50:30 John Fulton tripleo: importance Undecided High
2019-06-12 14:50:35 John Fulton tripleo: milestone train-2
2019-06-12 14:51:09 John Fulton tags queens-backport-potential
2019-06-13 11:07:54 Anton Antonov attachment added Full output of `openstack overcloud external-upgrade run --tags ceph` command https://bugs.launchpad.net/tripleo/+bug/1832509/+attachment/5270584/+files/ceph_upgrade.txt.gz
2019-06-13 20:54:07 OpenStack Infra tripleo: status In Progress Fix Released
2019-06-20 11:46:19 OpenStack Infra tags queens-backport-potential in-stable-stein queens-backport-potential
2019-06-20 11:56:27 OpenStack Infra tags in-stable-stein queens-backport-potential in-stable-rocky in-stable-stein queens-backport-potential