Jenkins Gate failed with "Invalid volume: Volume status must be available or error, but current status is: detaching"
From the logs:
2014-12-03 14:45:36,443 5857 DEBUG [tempest.common.rest_client] Request (AttachVolumeTestJSON:_run_cleanups): 200 GET http://127.0.0.1:8776/v1/81122490792c4235af32e5edfcffdb47/volumes/dd62f37c-e1b3-4813-a5b6-96f2ea52a6db 0.205s
Request - Headers: {'Content-Type': 'application/json', 'X-Auth-Token': '<omitted>', 'Accept': 'application/json'}
Body: None
Response - Headers: {'content-location': 'http://127.0.0.1:8776/v1/81122490792c4235af32e5edfcffdb47/volumes/dd62f37c-e1b3-4813-a5b6-96f2ea52a6db', 'content-type': 'application/json', 'status': '200', 'content-length': '760', 'date': 'Wed, 03 Dec 2014 14:45:36 GMT', 'x-compute-request-id': 'req-43d525f1-f9db-4216-a2d5-a84ae9ba56b7', 'connection': 'close', 'x-openstack-request-id': 'req-43d525f1-f9db-4216-a2d5-a84ae9ba56b7'}
Body: {"volume": {"status": "detaching", "display_name": "test", "attachments": [{"host_name": null, "device": "/dev/vdb", "server_id": "c4aa1ba3-3026-4723-b4c3-b12be35545f5", "id": "dd62f37c-e1b3-4813-a5b6-96f2ea52a6db", "volume_id": "dd62f37c-e1b3-4813-a5b6-96f2ea52a6db"}], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2014-12-03T14:40:27.000000", "os-vol-tenant-attr:tenant_id": "81122490792c4235af32e5edfcffdb47", "os-volume-replication:driver_data": null, "display_description": null, "os-volume-replication:extended_status": null, "volume_type": "lvmdriver-1", "snapshot_id": null, "source_volid": null, "metadata": {"readonly": "False", "attached_mode": "rw"}, "id": "dd62f37c-e1b3-4813-a5b6-96f2ea52a6db", "size": 1}}
2014-12-03 14:45:36,638 5857 DEBUG [tempest.common.rest_client] Request (AttachVolumeTestJSON:_run_cleanups): 400 DELETE http://127.0.0.1:8776/v1/81122490792c4235af32e5edfcffdb47/volumes/dd62f37c-e1b3-4813-a5b6-96f2ea52a6db 0.193s
Request - Headers: {'Content-Type': 'application/json', 'X-Auth-Token': '<omitted>', 'Accept': 'application/json'}
Body: None
Response - Headers: {'content-type': 'application/json; charset=UTF-8', 'status': '400', 'content-length': '134', 'date': 'Wed, 03 Dec 2014 14:45:36 GMT', 'x-compute-request-id': 'req-b39814a4-b2ce-43bb-87b3-ffbea5177072', 'connection': 'close', 'x-openstack-request-id': 'req-b39814a4-b2ce-43bb-87b3-ffbea5177072'}
Body: {"badRequest": {"message": "Invalid volume: Volume status must be available or error, but current status is: detaching", "code": 400}}
}}}
traceback-1: {{{
Traceback (most recent call last):
File "tempest/api/compute/volumes/test_attach_volume.py", line 49, in _delete_volume
self.volumes_client.delete_volume(self.volume['id'])
File "tempest/services/volume/json/volumes_client.py", line 108, in delete_volume
resp, body = self.delete("volumes/%s" % str(volume_id))
File "tempest/common/rest_client.py", line 272, in delete
return self.request('DELETE', url, extra_headers, headers, body)
File "tempest/common/rest_client.py", line 468, in request
resp, resp_body)
File "tempest/common/rest_client.py", line 517, in _error_checker
raise exceptions.BadRequest(resp_body)
BadRequest: Bad request
Details: {u'message': u'Invalid volume: Volume status must be available or error, but current status is: detaching', u'code': 400}
}}}
Traceback (most recent call last):
File "tempest/api/compute/volumes/test_attach_volume.py", line 44, in _detach
self.volumes_client.wait_for_volume_status(volume_id, 'available')
File "tempest/services/volume/json/volumes_client.py", line 181, in wait_for_volume_status
raise exceptions.TimeoutException(message)
TimeoutException: Request timed out
Details: Volume dd62f37c-e1b3-4813-a5b6-96f2ea52a6db failed to reach available status within the required time (196 s).
Traceback (most recent call last):
_StringException: Empty attachments:
stderr
stdout
All Logfiles See:
http://logs.openstack.org/22/138122/1/gate/gate-tempest-dsvm-neutron-full/57aff76/
Logstash is showing this error message a couple of times the last few days for various projects:
http://logstash.openstack.org/index.html#eyJzZWFyY2giOiJtZXNzYWdlOiBcIlZvbHVtZSBzdGF0dXMgbXVzdCBiZSBhdmFpbGFibGUgb3IgZXJyb3IsIGJ1dCBjdXJyZW50IHN0YXR1cyBpczogZGV0YWNoaW5nXCIiLCJmaWVsZHMiOlsiYnVpbGRfY2hhbmdlIiwibWVzc2FnZSJdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MTc2ODUwNDQzMDV9
This issue seems not related to the original devstack fix
https://review.openstack.org/#/c/138122/
