FAIL: setUpClass (tempest.api.compute.servers.test_server_rescue.ServerRescueTestXML Unauthorized)

[Attila Fazekas]

http://logs.openstack.org/44/43444/4/check/gate-grenade-devstack-vm/4f78566/console.html

2013-08-28 06:32:58.510 | ======================================================================
2013-08-28 06:32:58.511 | FAIL: setUpClass (tempest.api.compute.servers.test_server_rescue.ServerRescueTestXML)
2013-08-28 06:32:58.511 | setUpClass (tempest.api.compute.servers.test_server_rescue.ServerRescueTestXML)
2013-08-28 06:32:58.511 | ----------------------------------------------------------------------
2013-08-28 06:32:58.512 | _StringException: Traceback (most recent call last):
2013-08-28 06:32:58.512 | File "tempest/api/compute/servers/test_server_rescue.py", line 52, in setUpClass
2013-08-28 06:32:58.512 | 'test_attach')
2013-08-28 06:32:58.512 | File "tempest/services/compute/xml/volumes_extensions_client.py", line 114, in create_volume
2013-08-28 06:32:58.513 | self.headers)
2013-08-28 06:32:58.513 | File "tempest/common/rest_client.py", line 260, in post
2013-08-28 06:32:58.513 | return self.request('POST', url, headers, body)
2013-08-28 06:32:58.514 | File "tempest/common/rest_client.py", line 388, in request
2013-08-28 06:32:58.514 | resp, resp_body)
2013-08-28 06:32:58.514 | File "tempest/common/rest_client.py", line 430, in _error_checker
2013-08-28 06:32:58.515 | raise exceptions.Unauthorized()
2013-08-28 06:32:58.515 | Unauthorized: Unauthorized

http://logs.openstack.org/23/43723/4/gate/gate-tempest-devstack-vm-full/3fefc90/console.html

The real happening time is close to:
2013-08-28 06:24:24.882 | setUpClass (tempest.api.compute.servers.test_server_rescue.ServerRescueTestXML)

[Attila Fazekas (afazekas)]

http://logs.openstack.org/23/43723/4/gate/gate-tempest-devstack-vm-full/3fefc90/logs/screen-c-api.txt.gz#_2013-08-28_06_24_24_814

[Joe Gordon (jogo)]

Looks like this is an error from cinderclient
http://logs.openstack.org/88/47888/2/check/gate-tempest-devstack-vm-full/eafb237/logs/screen-n-api.txt.gz#_2013-09-24_02_46_49_507

[Joe Gordon (jogo)]

logstash query: @message:"Unauthorized: Unauthorized (HTTP 401)" AND @message:"cinderclient" AND @fields.build_status:"FAILURE"

[Sean Dague (sdague)]

so this actually looks like an issue in cinder validating the keystone token http://logs.openstack.org/23/43723/4/gate/gate-tempest-devstack-vm-full/3fefc90/logs/screen-c-api.txt.gz?level=TRACE#_2013-08-28_06_24_24_814 which Attila linked above.

[Sean Dague (sdague)]

@message:"CalledProcessError: Command 'openssl' returned non-zero exit status" would be better and more narrow

[Sean Dague (sdague)]

I actually think this is really a keystoneclient bug in handling token verify

[Dolph Mathews (dolph)]

It's difficult to diagnose the underlying problem with openssl due to bug 1112784

[Dirk Mueller (dmllr)]

If this is python 2.6, there is a significant chance that this is related to bug 1235252

[Attila Fazekas (afazekas)]

@message:"Unauthorized: Unauthorized (HTTP 401)" AND @message:"cinderclient" AND @fields.build_status:"FAILURE"

The above log-stash query at moment does not gives back any result, is this issue fixed ?

[Sean Dague (sdague)]

if you can't find a gate race in logstash, I'm calling it fixed

[Armando Migliaccio (armando-migliaccio)]

Same failure mode has been observed here:

http://logs.openstack.org/97/62097/1/gate/gate-tempest-dsvm-postgres-full/00bc987/

[Lianhao Lu (lianhao-lu)]

I'm seeing this again http://logs.openstack.org/91/63291/1/gate/gate-tempest-dsvm-full/b7744bb/ , though it failed at tempest.api.compute.servers.test_server_rescue.ServerRescueTestJSON

[Jia Ming Zhang (jmzhang)]

Also seeing it at tempest.api.compute.servers.test_server_rescure.ServerRescureTestJSON at: http://logs.openstack.org/39/65039/3/check/check-tempest-dsvm-full/c5757e5/