NeutronLoadBalancerTest.test_neutron_loadbalancer timesout occasionally

[Angus Salkeld]

http://logs.openstack.org/20/168220/1/check/check-heat-dsvm-functional-mysql/2ac753b/console.html#_2015-03-27_05_40_27_456

2015-03-27 05:40:27.459 | 2015-03-27 05:40:27.409 |
2015-03-27 05:40:27.462 | 2015-03-27 05:40:27.412 | Captured traceback:
2015-03-27 05:40:27.463 | 2015-03-27 05:40:27.413 | ~~~~~~~~~~~~~~~~~~~
2015-03-27 05:40:27.466 | 2015-03-27 05:40:27.416 | Traceback (most recent call last):
2015-03-27 05:40:27.468 | 2015-03-27 05:40:27.418 | File "heat_integrationtests/scenario/test_neutron_loadbalancer.py", line 74, in test_neutron_loadbalancer
2015-03-27 05:40:27.486 | 2015-03-27 05:40:27.419 | self.check_connectivity(server1_ip)
2015-03-27 05:40:27.486 | 2015-03-27 05:40:27.421 | File "heat_integrationtests/common/test.py", line 125, in check_connectivity
2015-03-27 05:40:27.486 | 2015-03-27 05:40:27.422 | raise exceptions.TimeoutException()
2015-03-27 05:40:27.487 | 2015-03-27 05:40:27.424 | heat_integrationtests.common.exceptions.TimeoutException: Request timed out
2015-03-27 05:40:27.487 | 2015-03-27 05:40:27.425 |
2015-03-27 05:40:27.487 | 2015-03-27 05:40:27.427 |

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/170765
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=7436a003b1d033f126dbbbe188db5643e5fbf928
Submitter: Jenkins
Branch: master

commit 7436a003b1d033f126dbbbe188db5643e5fbf928
Author: Rabi Mishra <email address hidden>
Date: Mon Apr 6 07:57:34 2015 +0530

    Fix NeutronLoadBalancerTest.test_neutron_loadbalancer timeout

    NeutronLoadBalancerTest.test_neutron_loadbalancer timeout
    happens occasionally. Added WaitCondition resource to
    signal back after SimpleHTTPServer is started.

    Change-Id: I73085d1b20277fba9f28460abaa991d489d2447a
    Closes-Bug: #1437203

[Sirushti Murugesan (sirushtim)]

Ok, so this happened again -> http://logs.openstack.org/45/173045/7/check/check-heat-dsvm-functional-mysql/c4b36f0/console.html

2015-04-30 04:29:13.366 | 2015-04-30 04:29:13.359 | Captured traceback:
2015-04-30 04:29:13.367 | 2015-04-30 04:29:13.361 | ~~~~~~~~~~~~~~~~~~~
2015-04-30 04:29:13.369 | 2015-04-30 04:29:13.363 | Traceback (most recent call last):
2015-04-30 04:29:13.371 | 2015-04-30 04:29:13.364 | File "/opt/stack/new/heat/heat_integrationtests/scenario/test_neutron_loadbalancer.py", line 66, in test_neutron_loadbalancer
2015-04-30 04:29:13.402 | 2015-04-30 04:29:13.367 | parameters=parameters
2015-04-30 04:29:13.402 | 2015-04-30 04:29:13.368 | File "/opt/stack/new/heat/heat_integrationtests/scenario/scenario_base.py", line 55, in launch_stack
2015-04-30 04:29:13.402 | 2015-04-30 04:29:13.370 | expected_status=expected_status
2015-04-30 04:29:13.402 | 2015-04-30 04:29:13.372 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 414, in stack_create
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.374 | self._wait_for_stack_status(**kwargs)
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.376 | File "/opt/stack/new/heat/heat_integrationtests/common/test.py", line 307, in _wait_for_stack_status
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.378 | raise exceptions.TimeoutException(message)
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.380 | heat_integrationtests.common.exceptions.TimeoutException: Request timed out
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.381 | Details: Stack NeutronLoadBalancerTest-1163902613/534b1d2e-eb39-428d-8d3f-7d85c277449b failed to reach CREATE_COMPLETE status within the required time (1200 s).
2015-04-30 04:29:13.403 | 2015-04-30 04:29:13.383 |
2015-04-30 04:37:24.443 | Build timed out (after 75 minutes). Marking the build as failed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/165944
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=1bc886c89d54fb0b706f724fa202d61604b14562
Submitter: Jenkins
Branch: master

commit 1bc886c89d54fb0b706f724fa202d61604b14562
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Thu Mar 19 17:54:52 2015 +0000

    Merge Neutron AutoScaling and LoadBalancer tests

    The tested use case is an autoscaling group of web app servers
    behind a loadbalancer.

    Test template was rewritten to use AutoScalingGroup, wait conditions and
    outputs, so no other client than heat one is used.
    Now there is no need to check for VM connectivity,
    as stack/resources status COMPLETE now quite reliably means
    that the listening servers are running.

    This patch should probably also help with narrowing down the causes of
    bug 1437203, since it does not checks instance connectivity with SSH.

    Change-Id: Iec8e8061f9ab3f3841fa221722b7b7805760cdf7
    Related-Bug: #1437203
    Closes-Bug: #1435285

[Steven Hardy (shardy)]

This happened again so moving this back to Triaged:

http://logs.openstack.org/99/219799/1/check/gate-heat-dsvm-functional-orig-mysql/9736b86/

[Pavlo Shchelokovskyy (pshchelo)]

In my recent patch I replaced two autoscaling/loadbalancing tests with single combined one

https://review.openstack.org/#/c/165944/

merged today. We'll need to watch out for that instead.

[Pavlo Shchelokovskyy (pshchelo)]

Setting to Incomplete as this particular test does no longer exist. Feel free to change otherwise if you feel it is more appropriate

[Zane Bitter (zaneb)]

Still seeing this on stable/kilo.

[Steve Baker (steve-stevebaker)]

stable/kilo has gone, I'm not finding this in logstash