This failure was notice when running the gate-tempest-devstack-vm-testr-full job in the gate check.
Job: http://logs.openstack.org/37860/1/check/gate-tempest-devstack-vm-testr-full/338/
2013-07-19 09:58:10.162 | ======================================================================
2013-07-19 09:58:10.162 | FAIL: tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_rebuild_server_with_auto_disk_config[gate]
2013-07-19 09:58:10.162 | tags: worker-3
2013-07-19 09:58:10.162 | ----------------------------------------------------------------------
2013-07-19 09:58:10.162 | Traceback (most recent call last):
2013-07-19 09:58:10.162 | File "tempest/api/compute/servers/test_disk_config.py", line 75, in test_rebuild_server_with_auto_disk_config
2013-07-19 09:58:10.162 | self.client.wait_for_server_status(server['id'], 'ACTIVE')
2013-07-19 09:58:10.162 | File "tempest/services/compute/json/servers_client.py", line 174, in wait_for_server_status
2013-07-19 09:58:10.163 | raise exceptions.TimeoutException(message)
2013-07-19 09:58:10.163 | TimeoutException: Request timed out
2013-07-19 09:58:10.163 | Details: Server 4ad5fcd0-6f3b-4eb0-a79e-5753ffe58a09 failed to reach ACTIVE status within the required time (400 s). Current status: REBUILD.
Investigating the logs, we have request for rebuild during the test execution. Here are the most relevant parts I could find
n-api:
2013-07-19 09:49:59.479 DEBUG nova.api.openstack.wsgi [req-c1f3b0bc-047f-461a-a2d8-1a7d5551ed55 ServerDiskConfigTestJSON46324236-user ServerDiskConfigTestJSON95279157-tenant] Action: 'action', body: {"rebuild": {"OS-DCF:diskConfig": "AUTO", "imageRef": "9bb4c3b7-5036-4e4a-8092-abd2dc515087"}} _process_stack /opt/stack/new/nova/nova/api/openstack/wsgi.py:926
In n-cpu, we find the following debug information
2013-07-19 09:50:00.080 AUDIT nova.compute.manager [req-c1f3b0bc-047f-461a-a2d8-1a7d5551ed55 ServerDiskConfigTestJSON46324236-user ServerDiskConfigTestJSON95279157-tenant] [instance: 4ad5fcd0-6f3b-4eb0-a79e-5753ffe58a09] Rebuilding instance
and later
2013-07-19 09:56:46.068 DEBUG nova.virt.libvirt.driver [req-c1f3b0bc-047f-461a-a2d8-1a7d5551ed55 ServerDiskConfigTestJSON46324236-user ServerDiskConfigTestJSON95279157-tenant] couldn't obtain the vpu count from domain id: 125 get_vcpu_used /opt/stack/new/nova/nova/virt/libvirt/driver.py:2775
2013-07-19 09:56:46.074 INFO nova.virt.libvirt.driver [req-c1f3b0bc-047f-461a-a2d8-1a7d5551ed55 ServerDiskConfigTestJSON46324236-user ServerDiskConfigTestJSON95279157-tenant] libvirt can't find a domain with id: 128
Could this be related to the issues pointed up in this patch for nova: https://review.openstack.org/#/c/26341 (Cache vcpu used num of domain in periodic task)
I have not seen this one again and I was not able to reproduce it. Closing this bug