Comment 5 for bug 1640920

Revision history for this message
Matt Riedemann (mriedem) wrote :

This is the request that fails:

2019-04-20 23:19:28,850 7870 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 20 Apr 2019 23:19:28 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'volume 3.0', 'vary': 'OpenStack-API-Version', 'content-length': '128', 'content-type': 'application/json', 'x-compute-request-id': 'req-eed65cf5-f6db-4782-b63f-a7191697eb5e', 'x-openstack-request-id': 'req-eed65cf5-f6db-4782-b63f-a7191697eb5e', 'connection': 'close', 'status': '500', 'content-location': 'https://104.130.127.82/volume/v3/15cab1f3b3ea4491a5f62d49368897dd/volumes/detail?all_tenants=1'}
        Body: b'{"computeFault": {"code": 500, "message": "The server has either erred or is incapable of performing the requested operation."}}'

And right before it:

2019-04-20 23:19:28,670 7870 INFO [tempest.lib.common.rest_client] Request (TestEncryptedCinderVolumes:_run_cleanups): 404 GET https://104.130.127.82/volume/v3/4615ed1db47a4db48e5b4ed79460d6ec/volumes/2ba8052d-8fd0-4b42-8ca7-7313e215cbc5 0.066s
2019-04-20 23:19:28,671 7870 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Sat, 20 Apr 2019 23:19:28 GMT', 'server': 'Apache/2.4.29 (Ubuntu)', 'openstack-api-version': 'volume 3.0', 'vary': 'OpenStack-API-Version', 'content-length': '109', 'content-type': 'application/json', 'x-compute-request-id': 'req-d030a6e9-4907-4915-af42-ae5a7dbe0cf3', 'x-openstack-request-id': 'req-d030a6e9-4907-4915-af42-ae5a7dbe0cf3', 'connection': 'close', 'status': '404', 'content-location': 'https://104.130.127.82/volume/v3/4615ed1db47a4db48e5b4ed79460d6ec/volumes/2ba8052d-8fd0-4b42-8ca7-7313e215cbc5'}
        Body: b'{"itemNotFound": {"code": 404, "message": "Volume 2ba8052d-8fd0-4b42-8ca7-7313e215cbc5 could not be found."}}'

So the volume is gone, and then we're listing volumes with details (attachments) across all_tenants and the all_tenants=1 might be loading up deleted fields? So my guess would be the view builder is trying to load attachments which are trying to load a deleted volume and that blows up.