Unable to create trust errors in glance-api

Bug #1916052 reported by Rajiv Mucheli on 2021-02-18
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Glance
Undecided
Unassigned

Bug Description

Hi,

I enabled swift_store_expire_soon_interval = 1800 for images taking long time to complete the upload, but it doesnt seem to work as planned, i see a trust issue (based on docu, this is True by default) but i see the below in the glance-api logs :

021-02-18 14:04:52,948.948 30 INFO glance.api.v2.image_data [req-bb9660d8-c24c-4350-9d4e-7cfaffebf8d9 332d21d621e27dd887ff1f3388312be975597e42b755eec00ceff70d033228b8 97bf741678d44e8da33c43f4c4662ade - ec213443e8834473b579f7bea9e8c194 ec213443e8834473b579f7bea9e8c194] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token

2021-02-18 12:02:43,166.166 33 INFO glance.api.v2.image_data [req-8a48bfa1-9d37-4095-8f7f-70438d4daff6 a10475412aa34d05a815fac977df8620 caa6209d2c38450f8266311fd0f05446 - default 582d6603e91d4d3d8193fa9160a599f0] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token.

http://paste.openstack.org/show/802787/

Ref :

https://bugs.launchpad.net/keystone/+bug/1775140
https://review.opendev.org/c/openstack/glance/+/479047

Glance Version : Victoria Release, 21.0.0
Glance_Store Version : 2.3.0
Swift Version (Backend) : Victoria (i have enabled multi-tenant)
Keystone : Train

Please let me know if further information is required.

Regards,
Rajiv

Brian Rosmaita (brian-rosmaita) wrote :

Could you try this with the log level set at debug? I can't find that error message in keystoneauth1 or glance_store, so I'm not sure where it's coming from. glance_store doesn't try to use collect_timing directly, so it must be somewhere else.

Changed in glance:
status: New → Incomplete
Oliver (ciwi) wrote :
Download full text (3.2 KiB)

I'm seeing the same messages on OpenStack Victoria 22.1.0. I set log level to debug, here's my output:

Apr 06 15:04:54 infra1-glance-container-99614ac2 glance-wsgi-api[1824]: 2021-04-06 15:04:54.274 1824 INFO glance.api.v2.image_data [req-c572bb10-58e1-4006-8f0e-9bd721b2986b 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Unable to create trust: no such option collect_timing in group [keystone_authtoken] Use the existing user token.
Apr 06 15:07:14 infra1-glance-container-99614ac2 glance-wsgi-api[1837]: 2021-04-06 15:07:10.195 1837 ERROR keystonemiddleware.auth_token [req-ad8d0db9-b630-4237-9fff-d7ff282155d2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Bad response code while validating token: 500 An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-55767c13-a450-4f81-90ab-5a7af6b3f672): keystoneauth1.exceptions.http.InternalServerError: An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-55767c13-a450-4f81-90ab-5a7af6b3f672)
Apr 06 15:07:40 infra1-glance-container-99614ac2 glance-wsgi-api[1837]: 2021-04-06 15:07:38.197 1837 WARNING keystonemiddleware.auth_token [req-ad8d0db9-b630-4237-9fff-d7ff282155d2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Identity response: {"error": {"code": 500, "title": "Internal Server Error", "message": "An unexpected error prevented the server from fulfilling your request."}}: keystoneauth1.exceptions.http.InternalServerError: An unexpected error prevented the server from fulfilling your request. (HTTP 500) (Request-ID: req-55767c13-a450-4f81-90ab-5a7af6b3f672)
Apr 06 15:07:46 infra1-glance-container-99614ac2 uwsgi[1822]: DAMN ! worker 13 (pid: 1836) died, killed by signal 9 :( trying respawn ...
Apr 06 15:07:46 infra1-glance-container-99614ac2 uwsgi[1822]: Respawned uWSGI worker 13 (new pid: 5799)
Apr 06 15:07:44 infra1-glance-container-99614ac2 glance-wsgi-api[1837]: 2021-04-06 15:07:40.427 1837 CRITICAL keystonemiddleware.auth_token [req-ad8d0db9-b630-4237-9fff-d7ff282155d2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Unable to validate token: Failed to fetch token data from identity server: keystonemiddleware.auth_token._exceptions.ServiceError: Failed to fetch token data from identity server
Apr 06 15:07:46 infra1-glance-container-99614ac2 uwsgi[1837]: Tue Apr 6 15:07:45 2021 - uwsgi_response_writev_headers_and_body_do(): Connection reset by peer [core/writer.c line 306] during GET /v2/images?limit=1000&sort_key=created_at&sort_dir=desc (192.168.110.250)
Apr 06 15:07:45 infra1-glance-container-99614ac2 glance-wsgi-api[1837]: 2021-04-06 15:07:45.629 1837 CRITICAL glance [req-ad8d0db9-b630-4237-9fff-d7ff282155d2 956806468e9f43dbaad1807a5208de52 ebe0fe5f3893495e82598c07716f5d45 - default default] Unhandled error: OSError: write error
                                                                        2021-04-06 15:07:45.629 1837 ERROR glance OSError: write error
                                                                        2021-04-06 15:07:45.629 1...

Read more...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers