This is copied from bug 1640920 since that's a pretty old bug and might be unrelated to this issue.
http://logs.openstack.org/06/641806/12/check/nova-next/c7f23f4/logs/screen-c-api.txt.gz?level=TRACE#_Apr_20_23_19_28_844945
Apr 20 23:19:28.844945 ubuntu-bionic-rax-dfw-0005375337 <email address hidden>[398]: ERROR cinder.api.middleware.fault [None req-eed65cf5-f6db-4782-b63f-a7191697eb5e tempest-TestEncryptedCinderVolumes-2130063619 tempest-TestEncryptedCinderVolumes-2130063619] Caught error: <class 'sqlalchemy.orm.exc.DetachedInstanceError'> Parent instance <VolumeAttachment at 0x7f570edb7320> is not bound to a Session; lazy load operation of attribute 'volume' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3): sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <VolumeAttachment at 0x7f570edb7320> is not bound to a Session; lazy load operation of attribute 'volume' cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
The full traceback is here:
http://paste.openstack.org/show/749641/
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Parent%20instance%20%3CVolumeAttachment%5C%22%20AND%20message%3A%5C%22is%20not%20bound%20to%20a%20Session%3B%20lazy%20load%20operation%20of%20attribute%20'volume'%20cannot%20proceed%5C%22%20AND%20tags%3A%5C%22screen-c-api.txt%5C%22&from=7d
This is the request that fails:
2019-04-20 23:19:28,850 7870 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Sat, 20 Apr 2019 23:19:28 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'volume 3.0', 'vary': 'OpenStack-API-Version', 'content-length': '128', 'content-type': 'application/json', 'x-compute-request-id': 'req-eed65cf5-f6db-4782-b63f-a7191697eb5e', 'x-openstack-request-id': 'req-eed65cf5-f6db-4782-b63f-a7191697eb5e', 'connection': 'close', 'status': '500', 'content-location': 'https://104.130.127.82/volume/v3/15cab1f3b3ea4491a5f62d49368897dd/volumes/detail?all_tenants=1'}
Body: b'{"computeFault": {"code": 500, "message": "The server has either erred or is incapable of performing the requested operation."}}'
And right before it:
2019-04-20 23:19:28,670 7870 INFO [tempest.lib.common.rest_client] Request (TestEncryptedCinderVolumes:_run_cleanups): 404 GET https://104.130.127.82/volume/v3/4615ed1db47a4db48e5b4ed79460d6ec/volumes/2ba8052d-8fd0-4b42-8ca7-7313e215cbc5 0.066s
2019-04-20 23:19:28,671 7870 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: None
Response - Headers: {'date': 'Sat, 20 Apr 2019 23:19:28 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'volume 3.0', 'vary': 'OpenStack-API-Version', 'content-length': '109', 'content-type': 'application/json', 'x-compute-request-id': 'req-d030a6e9-4907-4915-af42-ae5a7dbe0cf3', 'x-openstack-request-id': 'req-d030a6e9-4907-4915-af42-ae5a7dbe0cf3', 'connection': 'close', 'status': '404', 'content-location': 'https://104.130.127.82/volume/v3/4615ed1db47a4db48e5b4ed79460d6ec/volumes/2ba8052d-8fd0-4b42-8ca7-7313e215cbc5'}
Body: b'{"itemNotFound": {"code": 404, "message": "Volume 2ba8052d-8fd0-4b42-8ca7-7313e215cbc5 could not be found."}}'
So the volume is gone, and then we're listing volumes with details (attachments) across all_tenants and the all_tenants=1 might be loading up deleted fields? So my guess would be the view builder is trying to load attachments which are trying to load a deleted volume and that blows up.
