exceptions.ServerFault is raised when creating a server

[Haiwei Xu]

After submitting a patch, I got a lot of similar errors in tempest.

http://logs.openstack.org/52/74552/2/check/check-tempest-dsvm-full/8841b27/console.html

2014-02-19 07:46:41.335 | ======================================================================
2014-02-19 07:46:41.335 | FAIL: setUpClass (tempest.api.compute.v3.images.test_images_oneserver_negative.ImagesOneServerNegativeV3Test)
2014-02-19 07:46:41.335 | setUpClass (tempest.api.compute.v3.images.test_images_oneserver_negative.ImagesOneServerNegativeV3Test)
2014-02-19 07:46:41.335 | ----------------------------------------------------------------------
2014-02-19 07:46:41.335 | _StringException: Traceback (most recent call last):
2014-02-19 07:46:41.336 | File "tempest/api/compute/v3/images/test_images_oneserver_negative.py", line 67, in setUpClass
2014-02-19 07:46:41.336 | resp, server = cls.create_test_server(wait_until='ACTIVE')
2014-02-19 07:46:41.336 | File "tempest/api/compute/base.py", line 121, in create_test_server
2014-02-19 07:46:41.336 | name, image_id, flavor, **kwargs)
2014-02-19 07:46:41.336 | File "tempest/services/compute/v3/json/servers_client.py", line 87, in create_server
2014-02-19 07:46:41.336 | resp, body = self.post('servers', post_body, self.headers)
2014-02-19 07:46:41.336 | File "tempest/common/rest_client.py", line 184, in post
2014-02-19 07:46:41.336 | return self.request('POST', url, headers, body)
2014-02-19 07:46:41.336 | File "tempest/common/rest_client.py", line 360, in request
2014-02-19 07:46:41.336 | resp, resp_body)
2014-02-19 07:46:41.337 | File "tempest/common/rest_client.py", line 453, in _error_checker
2014-02-19 07:46:41.337 | raise exceptions.ServerFault(message)
2014-02-19 07:46:41.338 | ServerFault: Got server fault
2014-02-19 07:46:41.338 | Details: The server has either erred or is incapable of performing the requested operation.
2014-02-19 07:46:41.338 |
2014-02-19 07:46:41.338 |
2014-02-19 07:46:41.338 | ======================================================================
2014-02-19 07:46:41.339 | FAIL: setUpClass (tempest.api.compute.v3.servers.test_server_password.ServerPasswordV3Test)
2014-02-19 07:46:41.339 | setUpClass (tempest.api.compute.v3.servers.test_server_password.ServerPasswordV3Test)
2014-02-19 07:46:41.339 | ----------------------------------------------------------------------
2014-02-19 07:46:41.339 | _StringException: Traceback (most recent call last):
2014-02-19 07:46:41.339 | File "tempest/api/compute/v3/servers/test_server_password.py", line 28, in setUpClass
2014-02-19 07:46:41.340 | resp, cls.server = cls.create_test_server(wait_until="ACTIVE")
2014-02-19 07:46:41.340 | File "tempest/api/compute/base.py", line 121, in create_test_server
2014-02-19 07:46:41.340 | name, image_id, flavor, **kwargs)
2014-02-19 07:46:41.340 | File "tempest/services/compute/v3/json/servers_client.py", line 87, in create_server
2014-02-19 07:46:41.340 | resp, body = self.post('servers', post_body, self.headers)
2014-02-19 07:46:41.342 | File "tempest/common/rest_client.py", line 184, in post
2014-02-19 07:46:41.342 | return self.request('POST', url, headers, body)
2014-02-19 07:46:41.342 | File "tempest/common/rest_client.py", line 360, in request
2014-02-19 07:46:41.342 | resp, resp_body)
2014-02-19 07:46:41.342 | File "tempest/common/rest_client.py", line 453, in _error_checker
2014-02-19 07:46:41.342 | raise exceptions.ServerFault(message)
2014-02-19 07:46:41.343 | ServerFault: Got server fault
2014-02-19 07:46:41.343 | Details: The server has either erred or is incapable of performing the requested operation.
2014-02-19 07:46:41.343 |
2014-02-19 07:46:41.343 |
2014-02-19 07:46:41.343 | ======================================================================
2014-02-19 07:46:41.343 | FAIL: setUpClass (tempest.api.compute.v3.servers.test_create_server.ServersV3TestManualDisk)
2014-02-19 07:46:41.343 | setUpClass (tempest.api.compute.v3.servers.test_create_server.ServersV3TestManualDisk)
2014-02-19 07:46:41.343 | ----------------------------------------------------------------------
2014-02-19 07:46:41.343 | _StringException: Traceback (most recent call last):
2014-02-19 07:46:41.343 | File "tempest/api/compute/v3/servers/test_create_server.py", line 224, in setUpClass
2014-02-19 07:46:41.344 | super(ServersV3TestManualDisk, cls).setUpClass()
2014-02-19 07:46:41.344 | File "tempest/api/compute/v3/servers/test_create_server.py", line 51, in setUpClass
2014-02-19 07:46:41.344 | disk_config=cls.disk_config)
2014-02-19 07:46:41.344 | File "tempest/api/compute/base.py", line 121, in create_test_server
2014-02-19 07:46:41.344 | name, image_id, flavor, **kwargs)
2014-02-19 07:46:41.344 | File "tempest/services/compute/v3/json/servers_client.py", line 87, in create_server
2014-02-19 07:46:41.344 | resp, body = self.post('servers', post_body, self.headers)
2014-02-19 07:46:41.344 | File "tempest/common/rest_client.py", line 184, in post
2014-02-19 07:46:41.344 | return self.request('POST', url, headers, body)
2014-02-19 07:46:41.345 | File "tempest/common/rest_client.py", line 360, in request
2014-02-19 07:46:41.345 | resp, resp_body)
2014-02-19 07:46:41.345 | File "tempest/common/rest_client.py", line 453, in _error_checker
2014-02-19 07:46:41.345 | raise exceptions.ServerFault(message)
2014-02-19 07:46:41.345 | ServerFault: Got server fault
2014-02-19 07:46:41.345 | Details: The server has either erred or is incapable of performing the requested operation.
2014-02-19 07:46:41.345 |
2014-02-19 07:46:41.345 |
2014-02-19 07:46:41.345 | ======================================================================
2014-02-19 07:46:41.345 | FAIL: setUpClass (tempest.api.compute.v3.servers.test_create_server.ServersWithSpecificFlavorV3Test)
2014-02-19 07:46:41.345 | setUpClass (tempest.api.compute.v3.servers.test_create_server.ServersWithSpecificFlavorV3Test)
2014-02-19 07:46:41.346 | ----------------------------------------------------------------------
2014-02-19 07:46:41.346 | _StringException: Traceback (most recent call last):
2014-02-19 07:46:41.346 | File "tempest/api/compute/v3/servers/test_create_server.py", line 140, in setUpClass
2014-02-19 07:46:41.346 | disk_config=cls.disk_config)
2014-02-19 07:46:41.346 | File "tempest/api/compute/base.py", line 121, in create_test_server
2014-02-19 07:46:41.346 | name, image_id, flavor, **kwargs)
2014-02-19 07:46:41.347 | File "tempest/services/compute/v3/json/servers_client.py", line 87, in create_server
2014-02-19 07:46:41.347 | resp, body = self.post('servers', post_body, self.headers)
2014-02-19 07:46:41.347 | File "tempest/common/rest_client.py", line 184, in post
2014-02-19 07:46:41.347 | return self.request('POST', url, headers, body)
2014-02-19 07:46:41.347 | File "tempest/common/rest_client.py", line 360, in request
2014-02-19 07:46:41.347 | resp, resp_body)
2014-02-19 07:46:41.347 | File "tempest/common/rest_client.py", line 453, in _error_checker
2014-02-19 07:46:41.347 | raise exceptions.ServerFault(message)
2014-02-19 07:46:41.347 | ServerFault: Got server fault
2014-02-19 07:46:41.347 | Details: The server has either erred or is incapable of performing the requested operation.
2014-02-19 07:46:41.348 |
2014-02-19 07:46:41.348 |
2014-02-19 07:46:41.348 | ======================================================================

[David Kranz (david-kranz)]

logstash shows this happening a lot, about 7/8 of the time in a cells ( non-voting) run. I searched for

"The server has either erred or is incapable of performing the requested operation. (HTTP 500)" as the message.

Don't see how this could be related to tempest.

[Tracy Jones (tjones-i)]

please attach the nova logs for more information here

[Rossella Sblendido (rossella-o)]

Another example here:

http://logs.openstack.org/81/67281/25/check/check-tempest-dsvm-neutron-full/c47cda8/console.html

It's very frequent in the neutron full job:

http://logstash.openstack.org/#eyJzZWFyY2giOiJidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1kc3ZtLW5ldXRyb24tZnVsbFwiIEFORCBtZXNzYWdlOlwiVGhlIHNlcnZlciBoYXMgZWl0aGVyIGVycmVkIG9yIGlzIGluY2FwYWJsZSBvZiBwZXJmb3JtaW5nIHRoZSByZXF1ZXN0ZWQgb3BlcmF0aW9uXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6Ijg2NDAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDcwNzYwMzg5NH0=

[Rossella Sblendido (rossella-o)]

Actually this is better query
http://logstash.openstack.org/#eyJzZWFyY2giOiJidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1kc3ZtLW5ldXRyb24tZnVsbFwiIEFORCBtZXNzYWdlOlwiRGV0YWlsczogVGhlIHNlcnZlciBoYXMgZWl0aGVyIGVycmVkIG9yIGlzIGluY2FwYWJsZSBvZiBwZXJmb3JtaW5nIHRoZSByZXF1ZXN0ZWQgb3BlcmF0aW9uXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6Ijg2NDAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDcwODEwNzEyM30=

The previous one is actually selecting also the ClientException...which doesn't seem related and for which I opened a bug here:
https://bugs.launchpad.net/neutron/+bug/1291918

[Salvatore Orlando (salvatore-orlando)]

This appears to have the same root cause as bug 1283522.

[Eugene Nikanorov (enikanorov)]

Marking as 'Fix committed' for neutron per Salvatore's comment.