3.2.1 Ubuntu HA OSTF launch instance fails

[Matthew Mosesohn]

ISO #91
{"release": "3.2.1", "nailgun_sha": "71bb3cbc6aeda57d6676f5a8e87eb08b80f1ea11", "ostf_sha": "c54f0ff0e5765f0c89497100eb34386829b00bbd", "astute_sha": "cb4868f0481d6c7d2d213ceb065ae3c40ab699ce", "fuellib_sha": "2d45962a302b311c3aa8b7688e80f2ca1d81b2ed"}

1. Create new environment (Ubuntu, HA)
2. Choose Ceph for volumes
3. Add 3 controllers, compute, cinder, and 3 ceph nodes (8 total)
4. Choose VLAN Manager in network settings (Nova-network)
5. Start deployment. It was successful
6. Start OSTF tests.

Result:
Launch instance
Failed to get to expected status. In error state. Please refer to OpenStack logs for more details.

Target component: Nova

Scenario:
1. Create a new security group (if it doesn`t exist yet).
2. Create an instance using the new security group.
3. Delete instance.

Additional information:
I can use Horizon to launch a Tiny TestVM instance with no problems at all.

[Tatyanka (tatyana-leontovich)]

There are several problems. First creation of security groups with rules fails:
2013-11-29T13:57:31.183012+00:00 err: ERROR: Caught error: Request Timeout
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line 81, in __call__
    return req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 451, in __call__
    return self.app(env, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 890, in __call__
    content_type, body, accept)
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 942, in _process_stack
    action_result = self.dispatch(meth, request, action_args)
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 1022, in dispatch
    return method(req=request, **action_args)
  File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/contrib/security_groups.py", line 295, in create
    context, group_name, group_description)
  File "/usr/lib/python2.7/dist-packages/nova/network/security_group/quantum_driver.py", line 60, in create_security_group
    raise e
QuantumClientException: Request Timeout
2013-11-29T13:57:31.198311+00:00 info: INFO: http://172.16.0.8:8774/v2/3a4817d4d35b4d368450ff352fa912ef/os-security-groups returned with HTTP 500

see nova-nova.log on 13 node

also return connection.drain_events(**kwargs)
  File "/usr/lib/python2.6/site-packages/kombu/transport/amqplib.py", line 161, in drain_events
    chanmap, None, timeout=timeout)
  File "/usr/lib/python2.6/site-packages/kombu/transport/amqplib.py", line 219, in _wait_multiple
    channel, method_sig, args, content = read_timeout(timeout)
  File "/usr/lib/python2.6/site-packages/kombu/transport/amqplib.py", line 186, in read_timeout
    return self.method_reader.read_method()
  File "/usr/lib/python2.6/site-packages/amqplib/cli...

[Matthew Mosesohn (raytrac3r)]

this repeats in ISO 94 with Ubuntu simple (CentOS is fine)

[Tatyanka (tatyana-leontovich)]

Update OSTF packages to havanna release, Verify on ubuntu HA, ubuntu simple on KVM and Vbox - issue do not reproduce