[Queens] tokens generated with nocatalog are not usable in some requests
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Identity (keystone) |
In Progress
|
Undecided
|
Unassigned |
Bug Description
NOTE - this is happening only on Queens, and possibly earlier, and is already silently fixed in Rocky as part of the major refactor of token model. I am posing this issue here so that anyone still running Queens has a reference and probably a patch to apply.
In Queens release, if I create a token using a nocatalog option:
curl -X POST <keystone-
and then use this token to e.g. list servers with details
curl -X GET <nova-endpoint>
I get 500 error from nova, with nova api logs containing
ERROR nova.api.openstack EmptyCatalog: The service catalog is empty.
When repeating the same request with the same token after 5-10 minutes, the token starts to work.
Tokens generated with catalog are working as well.
AFAIU this goes down to token caching - in Queens tokens are cached/memoized with catalog - or without it if token was requested w/o catalog. Then on token validation, the token validation response takes the token - with or without catalog - from cache and returns it to caller with minimal processing - e.g. removing catalog if token validation call asked for it. It does not however ensures that the catalog is present otherwise.
This breaks some other services like Nova, that expects the catalog to be present in the request context constructed from the keystonemiddleware results. Nova needs this for example to make API requests to other services - exactly what happens in server/details call where it has to ask Neutron for some network info about instances.
After the cache is invalidated, the catalog starts to be generated for token validation response anew, and everything starts to work as expected.
patch is proposed here https:/ /review. opendev. org/c/openstack /keystone/ +/802751