openstack overcloud image upload 'Client' object has no attribute 'find_image'

[John Fulton]

My undercloud is unable to upload images for the overcloud deployment.

It seems to be failing on code which recently landed to fix https://bugs.launchpad.net/tripleo/+bug/1869736

https://github.com/openstack/python-tripleoclient/commit/62c43e76262a80904a564a71eee256725cca9fc5

(undercloud) [CentOS-8.1 - stack@undercloud ~]$ openstack overcloud image upload \
> --image-path /home/stack/overcloud_imgs \
> --http-boot /var/lib/ironic/httpboot \
> --update-existing
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py", line 541, in take_action
    disk_format='aki'
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py", line 359, in update_or_upload
    updated_image = self._image_try_update(glance_name, file_path)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py", line 300, in _image_try_update
    image = self._get_image(image_name)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py", line 294, in _get_image
    return self.client.find_image(name)
AttributeError: 'Client' object has no attribute 'find_image'
'Client' object has no attribute 'find_image'
(undercloud) [CentOS-8.1 - stack@undercloud ~]$

(undercloud) [CentOS-8.1 - stack@undercloud ~]$ rpm -qa | grep tripleo
ansible-role-tripleo-modify-image-1.1.1-0.20200204003759.bb6f78d.el8.noarch
ansible-tripleo-ipsec-9.2.1-0.20200127183247.0c8693c.el8.noarch
python3-tripleo-common-12.2.1-0.20200404040125.5caa154.el8.noarch
puppet-tripleo-12.1.1-0.20200403175637.8c36fc0.el8.noarch
openstack-tripleo-common-containers-12.2.1-0.20200404040125.5caa154.el8.noarch
python3-tripleoclient-heat-installer-13.1.1-0.20200404091914.624a61f.el8.noarch
openstack-tripleo-heat-templates-12.1.1-0.20200404230415.09d646f.el8.noarch
openstack-tripleo-validations-12.1.1-0.20200403224155.9972cf4.el8.noarch
openstack-tripleo-puppet-elements-12.1.1-0.20200401051948.4f2cac2.el8.noarch
openstack-tripleo-common-12.2.1-0.20200404040125.5caa154.el8.noarch
python3-tripleoclient-13.1.1-0.20200404091914.624a61f.el8.noarch
openstack-tripleo-image-elements-11.0.2-0.20200403224756.b2b4007.el8.noarch
tripleo-ansible-1.2.1-0.20200403212641.3ed3739.el8.noarch
(undercloud) [CentOS-8.1 - stack@undercloud ~]$

[John Fulton (jfulton-org)]

Possibly just a promotion issue. These are my repos: http://paste.openstack.org/show/791695/

[John Fulton (jfulton-org)]

I was able to workaround it by rolling back to:

 https://github.com/openstack/python-tripleoclient/commit/74a6c6eb681b050428dad4f9aeeebd3cf9524428

I suspect my undercloud is simply old. When I reported this I hadn't realized I wasn't running tripleo-ci-testing:

 https://github.com/cjeanner/tripleo-lab/commit/a45e0353e663a3d4cd91f29a340746de17b902d2

My fault. If I can reproduce with tripleo-ci-testing, then I'll re-open.

[John Fulton (jfulton-org)]

I hit this with:

 https://trunk.rdoproject.org/centos8/current-tripleo/delorean.repo

I should test if I get this issue with this repos instead:

 https://trunk.rdoproject.org/centos8/tripleo-ci-testing/delorean.repo

Here's the current values in those repos as of right now:

(undercloud) [CentOS-8.1 - stack@undercloud ~]$ curl https://trunk.rdoproject.org/centos8/tripleo-ci-testing/delorean.repo | curl -F 'f:1=<-' ix.io
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 3825 100 3825 0 0 13235 0 --:--:-- --:--:-- --:--:-- 13235
http://ix.io/2gZM
(undercloud) [CentOS-8.1 - stack@undercloud ~]$ curl https://trunk.rdoproject.org/centos8/current-tripleo/delorean.repo | curl -F 'f:1=<-' ix.io
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 3850 100 3850 0 0 40104 0 --:--:-- --:--:-- --:--:-- 40104
http://ix.io/2gZN
(undercloud) [CentOS-8.1 - stack@undercloud ~]$

[Harald Jensås (harald-jensas)]

I did a deployment using 'current-tripleo-dev' repos as set up by tripleo-repos.

It's using find_image function:
tripleoclient/v1/overcloud_image.py: return self.client.find_image(name)

The repo details below, image upload completed successfully.

[centos@undercloud ~]$ sudo dnf repoinfo delorean-master-testing
Last metadata expiration check: 0:02:44 ago on Mon 06 Apr 2020 06:15:12 PM UTC.

Repo-id : delorean-master-testing
Repo-name : dlrn-master-testing
Repo-status : enabled
Repo-revision: 202004061110
Repo-updated : Mon 06 Apr 2020 11:10:08 AM UTC
Repo-pkgs : 1,253
Repo-size : 599 M
Repo-baseurl : https://trunk.rdoproject.org/centos8-master/deps/latest/
Repo-expire : 172,800 second(s) (last: Mon 06 Apr 2020 06:15:11 PM UTC)
Repo-filename: /etc/yum.repos.d/delorean-master-testing.repo

Repo list:
[centos@undercloud ~]$ sudo dnf repolist
Last metadata expiration check: 0:02:22 ago on Mon 06 Apr 2020 06:15:12 PM UTC.
repo id repo name status
AppStream CentOS-8 - AppStream 5,124
BaseOS CentOS-8 - Base 2,126
advanced-virt Advanced Virtualization 243
delorean-component-baremetal delorean-openstack-ironic-305822c37ce62a31abaacf83e4997c0c93c59af7 56
delorean-component-cinder delorean-openstack-cinder-c2291bead4f1d28808bcc6e68a8c8af1a2d54af3 12
delorean-component-clients delorean-python-os-brick-ebe6a0463d0adca0bd64287cfe783408e150d1db 138
delorean-component-cloudops delorean-openstack-aodh-e71c636dfaf9928073d2064c8f17d9e74380fe7f 39
delorean-component-common delorean-openstack-rally-plugins-c4f0526dd8025956bacfedcd24c8a0e5a1b6005b 306
delorean-component-compute delorean-openstack-nova-c2bd895c6bfc6c797fda0c236fecc3a3ef33963c 20
delorean-component-glance delorean-openstack-glance-53df9ade64cb1ddb84092956f43e260a654347e8 7
delorean-component-manila delorean-python-manila-tests-tempest-b5ed5dfaa5ed4989bdefc30abb00902a46052951 9
delorean-component-network delorean-python-neutron-lib-7798139ffc6905d619ce536c16d188227667cf08 113
delorean-component-octavia delorean-python-octavia-lib-12d678a04943aea6c1987ce098c2ff4659109137 20
delorean-component-security delorean-ansible-tripleo-ipa-bd752a79c2559c493ee8eff5e6cf3af27af0a86d 35
delorean-component-swift delorean-openstack-swift-f87862e69656b98f9c7dd835336e18358c75ee24 8
delorean-component-tempest delorean-python-oswin-tests-tempest-fc5ae5e7...

[Rabi Mishra (rabi)]

Yes, current-tripleo has a pretty old osc i.e python3-openstackclient-4.0.0-0.20191218124517.aa64eb6.el8.noarch.rpm and a recent python-tripleoclient.

We promote tripleolcient without promoting it's deps for some reason.

https://trunk.rdoproject.org/centos8/component/clients/d5/b6/d5b61d10655b3b7a9d32378b5f7bcdb06479e15d_8c8d6250/

[wes hayutin (weshayutin)]

current issue blocking the clients promotion would be.
https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-clients-master/04e70b0/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz

2020-04-06 08:14:18 | Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2
2020-04-06 08:14:18 | Exception occured while running the command
2020-04-06 08:14:18 | Traceback (most recent call last):
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
2020-04-06 08:14:18 | super(Command, self).run(parsed_args)
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
2020-04-06 08:14:18 | return super(Command, self).run(parsed_args)
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
2020-04-06 08:14:18 | return_code = self.take_action(parsed_args) or 0
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_node.py", line 177, in take_action
2020-04-06 08:14:18 | verbosity=oooutils.playbook_verbosity(self=self)
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", line 222, in introspect_manageable_nodes
2020-04-06 08:14:18 | verbosity=verbosity
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", line 190, in introspect
2020-04-06 08:14:18 | "concurrency": concurrency,
2020-04-06 08:14:18 | File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 640, in run_ansible_playbook
2020-04-06 08:14:18 | raise RuntimeError(err_msg)
2020-04-06 08:14:18 | RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed,

[John Fulton (jfulton-org)]

I reproduced this on cent8.1 with tripleo-ci-testing

I'm using the following 3-command workaround until this is solved:

curl https://raw.githubusercontent.com/openstack/python-tripleoclient/74a6c6eb681b050428dad4f9aeeebd3cf9524428/tripleoclient/v1/overcloud_image.py > overcloud_image.py
sudo mv /usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py /root/
sudo mv overcloud_image.py /usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_image.py

[Marios Andreou (marios-b)]

and seeing it there too

        * https://review.rdoproject.org/r/#/c/25796/
        * https://logserver.rdoproject.org/96/25796/14/check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/8198322/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz
        * 2020-04-07 12:55:42 | AttributeError: 'Client' object has no attribute 'find_image'
2020-04-07 12:55:42 | 'Client' object has no attribute 'find_image'

[yatin (yatinkarel)]

So wrong repo files were created due to https://bugs.launchpad.net/tripleo/+bug/1871010, and then there was a false promotion(ovb jobs failed with those wrong repos[2]) of master due to [1]. A new promotion of good hash should fix the issue upstream.

[1] https://review.rdoproject.org/r/#/c/26281
[2] https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=1c419f50bd835a5dd09338469b59b0c7