compute exceptions when doing multiple nova client commands

[Chris Behrens]

If you have a few VMs and you want to delete them all, Ie:

nova delete 1
nova delete 2
nova delete 3

Some of the deletes may fail because of exceptions in compute

'AssertionError: Trying to re-send() an already-triggered event.'

Full trace:
http://paste.openstack.org/show/1225/

Note: Same exception can happen if you kick off simultaneous builds via 'nova boot', also.

[Brian Lamar (blamar)]

I can't reproduce this with libvirt as a hypervisor, can you? I'm thinking it might be a Xen issue/bug?

[Chris Behrens (cbehrens)]

Ya, very possibly something specific to using XenAPI. Sorry, forgot to mention in the comment above that this was XenServer, although you'll see xenapi_conn.py in the traceback in the paste URL. Haven't dug into it yet.

[Chris Behrens (cbehrens)]

Definitely an issue in xenapi_conn.py

Nova creates a single instance of XenAPIConnection(). XenAPIConnection() creates a single instance of XenAPISession(). The problem is that the same XenAPISession() will be used for multiple outstanding queries. XenAPISession.wait_for_task() sets various variables within the instance to track a query from which it is waiting for a response. If another query is started, those variables are overwritten. Doh. Solution is to make xenapi_conn.py handle multiple outstanding queries correctly.