gate-neutron-python27 fails: "greenlet.GreenletExit" and then "WARNING: Unable to find to confirm results!"

Bug #1282452 reported by Akihiro Motoki
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Core Infrastructure
Invalid
Undecided
Unassigned
neutron
Fix Released
Critical
Salvatore Orlando
Havana
Fix Released
Critical
Armando Migliaccio

Bug Description

gate-neutron-python27 fails with "WARNING: Unable to find to confirm results!".
The original cause is "greenlet.GreenletExit" and testr aborted.

message:"greenlet.GreenletExit" AND filename:console.html
87 hits in this 48 hours. (84 are in gate-neutron-python27 and 3 are in gate-oslo-incubator-python27)

I am not sure it is a bug of neutron or CI infra, but the hit rate above says it is only related to neutron python 2.7.

logstash: http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZ3JlZW5sZXQuR3JlZW5sZXRFeGl0XCIgQU5EIGZpbGVuYW1lOmNvbnNvbGUuaHRtbCIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiY3VzdG9tIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7ImZyb20iOiIyMDE0LTAyLTE4VDA4OjA5OjQ3KzAwOjAwIiwidG8iOiIyMDE0LTAyLTIwVDA4OjA5OjQ3KzAwOjAwIiwidXNlcl9pbnRlcnZhbCI6IjAifSwic3RhbXAiOjEzOTI4ODU1NDA1MjAsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=

Revision history for this message
Salvatore Orlando (salvatore-orlando) wrote :

It is a pure-neutron bug. I can reproduce it 100% of time with a few tweaks in the code.
The root cause is that greenlet.GreenletExit seem to not be controlled properly in vshield test cases and this is now causing the test runner to commit suicide

Changed in neutron:
assignee: nobody → Salvatore Orlando (salvatore-orlando)
Changed in openstack-ci:
status: New → Incomplete
status: Incomplete → Invalid
Changed in neutron:
status: New → Confirmed
milestone: none → icehouse-3
importance: High → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/75054

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/75062

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/75062
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=433287b4f51e45fbb7a73f68dd63b433ea5d9066
Submitter: Jenkins
Branch: master

commit 433287b4f51e45fbb7a73f68dd63b433ea5d9066
Author: Salvatore Orlando <email address hidden>
Date: Thu Feb 20 06:30:11 2014 -0800

    vshield task manager: abort tasks in stop() on termination

    This patch kills the manager thread, and aborts active tasks
    rather than sending an exception to the manager thread and
    have it do the abort on termination.

    Unit tests involving vshield task manager might take longer
    as a side effect of this patch.

    Related-bug: #1282452

    Change-Id: I9e9e41ce7e8969a2ea51bfce96b1303125a24308

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/havana)

Related fix proposed to branch: stable/havana
Review: https://review.openstack.org/75660

tags: added: havana-backport-potential
Changed in neutron:
status: Confirmed → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/havana)

Reviewed: https://review.openstack.org/75660
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=28ee2ac315c226ce096190879077121702e4d644
Submitter: Jenkins
Branch: stable/havana

commit 28ee2ac315c226ce096190879077121702e4d644
Author: Salvatore Orlando <email address hidden>
Date: Thu Feb 20 06:30:11 2014 -0800

    vshield task manager: abort tasks in stop() on termination

    This patch kills the manager thread, and aborts active tasks
    rather than sending an exception to the manager thread and
    have it do the abort on termination.

    Unit tests involving vshield task manager might take longer
    as a side effect of this patch.

    Related-bug: #1282452

    Conflicts:

            neutron/tests/unit/nicira/test_vcns_driver.py

    Change-Id: I9e9e41ce7e8969a2ea51bfce96b1303125a24308
    (cherry picked from commit 433287b4f51e45fbb7a73f68dd63b433ea5d9066)

tags: added: in-stable-havana
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in neutron:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.