neutron-tempest-plugin-designate-scenario failes frequently with imageservice doesn't have supported version

Bug #1869862 reported by Lajos Katona
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Invalid
Undecided
Unassigned

Bug Description

neutron-tempest-plugin-designate-scenario job fails frequently with the following error:
...
2020-03-30 18:49:44.170062 | controller | + /opt/stack/neutron-tempest-plugin/devstack/functions.sh:overridden_upload_image:256 : openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create ubuntu-16.04-server-cloudimg-amd64-disk1 --property hw_rng_model=virtio --public --container-format=bare --disk-format qcow2
2020-03-30 18:49:46.242923 | controller | Failed to contact the endpoint at http://10.209.38.120/image for discovery. Fallback to using that endpoint as the base url.
2020-03-30 18:49:46.247351 | controller | Failed to contact the endpoint at http://10.209.38.120/image for discovery. Fallback to using that endpoint as the base url.
2020-03-30 18:49:46.247894 | controller | The image service for devstack-admin:RegionOne exists but does not have any supported versions.
2020-03-30 18:49:46.384047 | controller | + /opt/stack/neutron-tempest-plugin/devstack/customize_image.sh:upload_image:1 : exit_trap
...

Example is here:
https://94d5d118ec3db75721c2-a00e37315b6784119b950c4b112ef30c.ssl.cf2.rackcdn.com/711610/13/check/neutron-tempest-plugin-designate-scenario/b23bb46/job-output.txt

Logstash query:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22The%20image%20service%20for%20devstack-admin%3ARegionOne%20exists%20but%20does%20not%20have%20any%20supported%20versions.%5C%22

Tags: gate-failure
Revision history for this message
Lajos Katona (lajos-katona) wrote :

I checked glance log to see what glance sees from the above problem:
https://94d5d118ec3db75721c2-a00e37315b6784119b950c4b112ef30c.ssl.cf2.rackcdn.com/711610/13/check/neutron-tempest-plugin-designate-scenario/b23bb46/controller/logs/screen-g-api.txt
(this is the same run what I referenced above)

Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: CRITICAL glance [-] Unhandled error: glance_store.exceptions.BadStoreConfiguration: Store swift could not be configured correctly. Reason: Missing dependency python_swiftclient.
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance Traceback (most recent call last):
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance File "/usr/local/bin/glance-wsgi-api", line 52, in <module>
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance application = init_app()
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance File "/opt/stack/glance/glance/common/wsgi_app.py", line 85, in init_app
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance glance_store.create_stores(CONF)
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance File "/usr/local/lib/python3.6/dist-packages/glance_store/backend.py", line 253, in create_stores
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance store_instance.configure(re_raise_bsc=False)
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance File "/usr/local/lib/python3.6/dist-packages/glance_store/_drivers/swift/store.py", line 1246, in configure
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance super(SingleTenantStore, self).configure(re_raise_bsc=re_raise_bsc)
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance File "/usr/local/lib/python3.6/dist-packages/glance_store/_drivers/swift/store.py", line 812, in configure
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance reason=msg)
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance glance_store.exceptions.BadStoreConfiguration: Store swift could not be configured correctly. Reason: Missing dependency python_swiftclient.
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: ERROR glance
Mar 30 18:49:19.145737 ubuntu-bionic-rax-dfw-0015586298 <email address hidden>[11705]: unable to load app 0 (mountpoint='') (callable not found or import error)

Revision history for this message
Bernard Cafarelli (bcafarel) wrote :

I did not check logs in detail, but as this is mentions glance, this may have been another side effect of recent devstack change?

The relevant fix just merged:
https://review.opendev.org/#/c/715835/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-tempest-plugin (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/716272

Revision history for this message
Lajos Katona (lajos-katona) wrote :

@Bernard: thanks for adding this. As I see now the job is passing, so perhaps the issue is solved :-)

Revision history for this message
Lajos Katona (lajos-katona) wrote :

I think this can be closed, there was a mail thread about this as well:
http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013681.html

I check the gate and after if it is really passing now I close this ticket.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron-tempest-plugin (master)

Change abandoned by Lajos Katona (<email address hidden>) on branch: master
Review: https://review.opendev.org/716272
Reason: designate scenario is passing now, as https://review.opendev.org/715835 is merged.

Changed in neutron:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.