py34 intermittent failure

Bug #1492505 reported by Armando Migliaccio
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Critical
Robert Collins
oslo.messaging
Fix Released
Undecided
Davanum Srinivas (DIMS)
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

@Cedric: can you please triage this? Many thanks.

Changed in neutron:
assignee: nobody → Cedric Brandily (cbrandily)
importance: Undecided → High
milestone: none → liberty-rc1
status: New → Confirmed
description: updated
Changed in neutron:
importance: High → Critical
Revision history for this message
Henry Gessau (gessau) wrote :
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

Tagged oslo.messaging because the logstash query shows it within the affected build names.

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :
description: updated
description: updated
Revision history for this message
Robert Collins (lifeless) wrote :

Hi, so I looked into a similar thing for designate recently. The bug appears to be related to stream generation to testr on 3.4 only - I haven't managed to reproduce in a small self contained environment yet. It could be related to eventlet even, since its coming from eventlet patched environments.

putting | cat seems to fix it (in .testr.conf)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/220713

Changed in neutron:
assignee: Cedric Brandily (cbrandily) → Robert Collins (lifeless)
status: Confirmed → In Progress
Changed in neutron:
assignee: Robert Collins (lifeless) → Armando Migliaccio (armando-migliaccio)
Changed in neutron:
assignee: Armando Migliaccio (armando-migliaccio) → Robert Collins (lifeless)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/220713
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=065275e51ff3852462586d01f5d3dd750bf2d663
Submitter: Jenkins
Branch: master

commit 065275e51ff3852462586d01f5d3dd750bf2d663
Author: Robert Collins <email address hidden>
Date: Sat Sep 5 16:04:42 2015 +1200

    Workaround test stream corruption issue.

    Change-Id: I4c88f1891f53c6559bca71bf657aa30df2101280
    Closes-Bug: #1492505

Changed in neutron:
status: In Progress → Fix Committed
Changed in oslo.messaging:
status: New → Incomplete
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

@Davanum: what else do you need? A oslo.messaging failure is right here:

http://logs.openstack.org/06/189006/4/check/gate-oslo.messaging-python34/4da6512/console.html

Changed in oslo.messaging:
status: Incomplete → New
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

Feel free to remove oslo.messaging from here: I was only tagging out of courtesy.

You should patch your testr.conf [1] like we did for neutron.

[1] https://github.com/openstack/oslo.messaging/blob/master/.testr.conf

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (feature/pecan)

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/224334

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: feature/pecan
Review: https://review.openstack.org/224357

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (feature/pecan)
Download full text (73.6 KiB)

Reviewed: https://review.openstack.org/224357
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fdc3431ccd219accf6a795079d9b67b8656eed8e
Submitter: Jenkins
Branch: feature/pecan

commit fe236bdaadb949661a0bfb9b62ddbe432b4cf5f1
Author: Miguel Angel Ajo <email address hidden>
Date: Thu Sep 3 15:40:12 2015 +0200

    No network devices on network attached qos policies

    Network devices, like internal router legs, or dhcp ports
    should not be affected by bandwidth limiting rules.

    This patch disables application of network attached policies
    to network/neutron owned ports.

    Closes-bug: #1486039
    DocImpact

    Change-Id: I75d80227f1e6c4b3f5fa7762b8dc3b0c0f1abd46

commit db4a06f7caa20a4c7879b58b20e95b223ed8eeaf
Author: Ken'ichi Ohmichi <email address hidden>
Date: Wed Sep 16 10:04:32 2015 +0000

    Use tempest-lib's token_client

    Now tempest-lib provides token_client modules as library and the
    interface is stable. So neutron repogitory doesn't need to contain
    these modules.
    This patch makes neutron use tempest-lib's token_client and removes
    the own modules for the maintenance.

    Change-Id: Ieff7eb003f6e8257d83368dbc80e332aa66a156c

commit 78aed58edbe6eb8a71339c7add491fe9de9a0546
Author: Jakub Libosvar <email address hidden>
Date: Thu Aug 13 09:08:20 2015 +0000

    Fix establishing UDP connection

    Previously, in establish_connection() for UDP protocol data were sent
    but never read on peer socket. That lead to successful read on peer side
    if this connection was filtered. Having constant testing string masked
    this issue as we can't distinguish to which test of connectivity data
    belong.

    This patch makes unique data string per test_connectivity() and
    also makes establish_connection() to create an ASSURED entry in
    conntrack table. Finally, in last test after firewall filter was
    removed, connection is re-established in order to avoid troubles with
    terminated processes or TCP continuing sending packets which weren't
    successfully delivered.

    Closes-Bug: 1478847
    Change-Id: I2920d587d8df8d96dc1c752c28f48ba495f3cf0f

commit e6292fcdd6262434a7b713ad8802db6bc8a6d3dc
Author: YAMAMOTO Takashi <email address hidden>
Date: Wed Sep 16 13:20:51 2015 +0900

    ovsdb: Fix a few docstring

    Change-Id: I53e1e21655b28fe5da60e58aeeb7cbbd103ae014

commit c22949a4449d96a67caa616290cf76b67b182917
Author: fumihiko kakuma <email address hidden>
Date: Wed Sep 16 11:52:59 2015 +0900

    Remove requirements.txt for the ofagent mechanism driver

    It is no longer used.

    Related-Blueprint: core-vendor-decomposition
    https://blueprints.launchpad.net/neutron/+spec/core-vendor-decomposition

    Change-Id: Ib31fb3febf8968e50d86dd66e1e6e1ea2313f8ac

commit d1d4de19d85f961d388c91e70f31b3bafec418c5
Author: Kevin Benton <email address hidden>
Date: Thu Sep 3 20:25:57 2015 -0700

    Always return iterables in L3 get_candidates

    The caller of this function expects iterables.

    Closes-Bug: #1494996
    Change-Id: I3d103e63f4e127a77268502415c0ddb0d804b54a

commit 1ad6ac448067306...

tags: added: in-feature-pecan
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (feature/pecan)

Change abandoned by Doug Wiegley (<email address hidden>) on branch: feature/pecan
Review: https://review.openstack.org/224334

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (master)

Fix proposed to branch: master
Review: https://review.openstack.org/227003

Changed in oslo.messaging:
assignee: nobody → Davanum Srinivas (DIMS) (dims-v)
status: New → In Progress
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (master)

Reviewed: https://review.openstack.org/227003
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=34d24b79aba55b7a5d26cfeaf063426c0678cc69
Submitter: Jenkins
Branch: master

commit 34d24b79aba55b7a5d26cfeaf063426c0678cc69
Author: Davanum Srinivas <email address hidden>
Date: Wed Sep 23 17:17:05 2015 -0400

    Workaround test stream corruption issue

    We don't see it right now in logstash as we fail with
    a Segmentation fault (core dumped) because of other
    issues. But will end up seeing it once that is fixed.

    Closes-Bug: #1492505
    Change-Id: I6507e693fc929e03884f933bbda241f744d3a7c0

Changed in oslo.messaging:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in neutron:
milestone: liberty-rc1 → 7.0.0
Changed in oslo.messaging:
milestone: none → 2.7.0
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to oslo.messaging (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/281178

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/282337

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to oslo.messaging (master)

Reviewed: https://review.openstack.org/281178
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=e384dca5a1af89daa3e72476b2cd35d043d3a8f7
Submitter: Jenkins
Branch: master

commit e384dca5a1af89daa3e72476b2cd35d043d3a8f7
Author: Victor Stinner <email address hidden>
Date: Wed Feb 17 11:30:21 2016 +0100

    .testr.conf: revert workaround of testtools bug

    Revert the change I6507e693fc929e03884f933bbda241f744d3a7c0. The
    testtools bug was fixed, the "| cat" workaround is no more needed.

    Fix for subunit (for testtools), workaround the eventlet bug:
    https://github.com/testing-cabal/subunit/pull/14

    Fix of the root cause in eventlet:
    https://github.com/eventlet/eventlet/issues/248

    The bug was limited to Python 3 and related to non-blocking stdout.

    Change-Id: I207b7a7e82cbd7e5848231115b3577be7b600638
    Related-Bug: 1492505

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/282337
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5eaae59450667e039250b44ff4f454881c5e2bed
Submitter: Jenkins
Branch: master

commit 5eaae59450667e039250b44ff4f454881c5e2bed
Author: Victor Stinner <email address hidden>
Date: Fri Feb 19 14:27:40 2016 +0100

    .testr.conf: revert workaround of testtools bug

    Revert the change I4c88f1891f53c6559bca71bf657aa30df2101280.
    The testtools bug was fixed, the "| cat" workaround is no more
    needed.

    Fix for subunit (for testtools), workaround the eventlet bug:
    https://github.com/testing-cabal/subunit/pull/14

    Fix of the root cause in eventlet:
    https://github.com/eventlet/eventlet/issues/248

    The bug was limited to Python 3 and related to non-blocking stdout.

    See also the similar change for oslo.messaging:
    I207b7a7e82cbd7e5848231115b3577be7b600638.

    Change-Id: Iad759af7fbd8a404b7ac361763b27214f3a49ac3
    Related-Bug: 1492505

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.