consistent tox test failures with "rate-limit" error

[Amrith Kumar]

This is the first error, goes south from here (fast).

======================================================================
ERROR: proboscis.case.MethodTest (test_restart_service_after_unassign_return_active)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/proboscis/decorators.py", line 59, in new_method
    return func(*kargs, **kwargs)
  File "/opt/stack/trove/trove/tests/api/configurations.py", line 685, in test_restart_service_after_unassign_return_active
    poll_until(result_is_not_active)
  File "/opt/stack/trove/trove/tests/util/event_simulator.py", line 113, in fake_poll_until
    resource = retriever()
  File "/opt/stack/trove/trove/tests/api/configurations.py", line 680, in result_is_not_active
    instance_info.id)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/v1/instances.py", line 137, in get
    "instance")
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/base.py", line 169, in _get
    resp, body = self.api.client.get(url)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/compat/client.py", line 237, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/compat/client.py", line 231, in _cs_request
    return request()
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/compat/client.py", line 221, in request
    **kwargs)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/compat/client.py", line 209, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/opt/stack/trove/.tox/py27/local/lib/python2.7/site-packages/troveclient/compat/client.py", line 187, in request
    raise exceptions.from_response(resp, body)
OverLimit: This request was rate-limited. (HTTP 413)
-------------------- >> begin captured logging << --------------------
troveclient.compat.client: DEBUG: REQ: curl -i http://localhost:8779/v1.0/2500/instances/a7d7d9ea-530f-431b-97b0-3ea5caf8fb26 -X GET -H "Content-Type: application/json" -H "User-Agent: python-troveclient" -H "X-Auth-Project-Id: 2500" -H "X-Auth-Token: 2500" -H "Accept: application/json"

troveclient.compat.client: DEBUG: RESP:{'content-type': 'application/json; charset=UTF-8', 'retry-after': '1', 'content-length': '161', 'status': '413'} {"overLimit": {"message": "This request was rate-limited.", "code": 413, "details": "Only 200 GET request(s) can be made to * every minute.", "retryAfter": "1"}}

--------------------- >> end captured logging << ---------------------

[Amrith Kumar (amrith)]

patch is trivial, will send it up once we close Juno and open for business with Kilo

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/125611

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/125611
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=a67c0ca7030362c35ea8dd183229bc73173cf60b
Submitter: Jenkins
Branch: master

commit a67c0ca7030362c35ea8dd183229bc73173cf60b
Author: Amrith Kumar <email address hidden>
Date: Thu Oct 2 08:19:38 2014 -0400

    Increase test rate limit to avoid rate limit error

    The current rate limit is 200 requests per minute. I'm getting
    consistent tox failures as the machine is exceeding this rate. The fix
    impacts only test code and adjusts the limit and a test that has a
    hard-coded reference to the old limit. Why 500 you may ask? Because
    600 worked and 450 failed consistently with the rate limit error.

    In addition, the change addresses the fact that some test
    configuration values are duplicated in the test; the change makes the
    test reference the configuration value.

    Change-Id: I4bb290d8de6253d65b7877c743bb288ee2bce536
    Closes-Bug: #1376689
    Closes-Bug: #1378932

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/149208

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/149208
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=874ec543a2174dc60f8da4429646a830445b6f02
Submitter: Jenkins
Branch: stable/juno

commit 874ec543a2174dc60f8da4429646a830445b6f02
Author: Amrith Kumar <email address hidden>
Date: Thu Oct 2 08:19:38 2014 -0400

    Increase test rate limit to avoid rate limit error

    The current rate limit is 200 requests per minute. I'm getting
    consistent tox failures as the machine is exceeding this rate. The fix
    impacts only test code and adjusts the limit and a test that has a
    hard-coded reference to the old limit. Why 500 you may ask? Because
    600 worked and 450 failed consistently with the rate limit error.

    In addition, the change addresses the fact that some test
    configuration values are duplicated in the test; the change makes the
    test reference the configuration value.

    Change-Id: I4bb290d8de6253d65b7877c743bb288ee2bce536
    Closes-Bug: #1376689
    Closes-Bug: #1378932
    (cherry picked from commit a67c0ca7030362c35ea8dd183229bc73173cf60b)