Race condition when running two ec2test instances very close together

[Jonathan Lange]

This seems to happen when I try o run two tests at the same time:

$ ~/src/lp-dev-utils/trunk/ec2test.py --<email address hidden> --headless
No handlers could be found for logger "boto"
Using machine image version 11
Traceback (most recent call last):
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 1543, in <module>
    runner.start()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 1027, in start
    self._instance.start()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 563, in start
    self.private_key = self._controller.acquire_private_key()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 311, in acquire_private_key
    key_pair = self.conn.create_key_pair(self.name)
  File "/var/lib/python-support/python2.5/boto/ec2/connection.py", line 506, in create_key_pair
    return self.get_object('CreateKeyPair', params, KeyPair)
  File "/var/lib/python-support/python2.5/boto/connection.py", line 433, in get_object
    raise self.ResponseError(response.status, response.reason, body)
EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0"?>
<Response><Errors><Error><Code>InvalidKeyPair.Duplicate</Code><Message>The keypair 'ec2-test-runner' already exists.</Message></Error></Errors><RequestID>0d976833-9ab5-413b-8828-793755c996ac</RequestID></Response>
no instance created

[Michael Hudson-Doyle (mwhudson)]

I've had this, or something very similar, happen to me too.

[Michael Hudson-Doyle (mwhudson)]

The problem is that when ec2 test is starting up, it looks for a key pair called 'ec2-test-runner'. If it finds it, it deletes it. Then it creates a key pair with this name and uses it to start an instance. The race occurs when one invocation deletes the key pair after another has created it but before it uses it.

I think a fix would be to instead of deleting an existing key pair, to mutate the key pair name until one can be created, and then deleting it again after the instance is safely booted.

[Diogo Matsubara (matsubara)]

Fixed in devel r9631 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/9631>

[Diogo Matsubara (matsubara)]

Fixed in devel r9672 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/9672>