Race condition when running two ec2test instances very close together

Bug #422433 reported by Jonathan Lange on 2009-09-01
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
High
Gavin Panella

Bug Description

This seems to happen when I try o run two tests at the same time:

$ ~/src/lp-dev-utils/trunk/ec2test.py --<email address hidden> --headless
No handlers could be found for logger "boto"
Using machine image version 11
Traceback (most recent call last):
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 1543, in <module>
    runner.start()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 1027, in start
    self._instance.start()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 563, in start
    self.private_key = self._controller.acquire_private_key()
  File "/home/jml/src/lp-dev-utils/trunk/ec2test.py", line 311, in acquire_private_key
    key_pair = self.conn.create_key_pair(self.name)
  File "/var/lib/python-support/python2.5/boto/ec2/connection.py", line 506, in create_key_pair
    return self.get_object('CreateKeyPair', params, KeyPair)
  File "/var/lib/python-support/python2.5/boto/connection.py", line 433, in get_object
    raise self.ResponseError(response.status, response.reason, body)
EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0"?>
<Response><Errors><Error><Code>InvalidKeyPair.Duplicate</Code><Message>The keypair 'ec2-test-runner' already exists.</Message></Error></Errors><RequestID>0d976833-9ab5-413b-8828-793755c996ac</RequestID></Response>
no instance created

Related branches

Michael Hudson-Doyle (mwhudson) wrote :

I've had this, or something very similar, happen to me too.

Changed in launchpad-foundations:
status: New → Triaged
importance: Undecided → High
Michael Hudson-Doyle (mwhudson) wrote :

The problem is that when ec2 test is starting up, it looks for a key pair called 'ec2-test-runner'. If it finds it, it deletes it. Then it creates a key pair with this name and uses it to start an instance. The race occurs when one invocation deletes the key pair after another has created it but before it uses it.

I think a fix would be to instead of deleting an existing key pair, to mutate the key pair name until one can be created, and then deleting it again after the instance is safely booted.

Gavin Panella (allenap) on 2009-09-28
Changed in launchpad-foundations:
assignee: nobody → Gavin Panella (allenap)
milestone: none → 3.1.10
status: Triaged → In Progress
Changed in launchpad-foundations:
status: In Progress → Fix Committed
Gavin Panella (allenap) on 2009-10-05
Changed in launchpad-foundations:
status: Fix Committed → In Progress
Diogo Matsubara (matsubara) wrote :
Changed in launchpad-foundations:
status: In Progress → Fix Committed
Gavin Panella (allenap) on 2009-10-13
Changed in launchpad-foundations:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers