Failing to delete an linux bridge causes log littering

Bug #1501779 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Low
Matt Riedemann
Juno
Fix Released
Undecided
Unassigned
Kilo
Fix Released
Undecided
Unassigned

Bug Description

I saw this in some ansible jobs in the gate:

2015-09-30 22:37:21.805 26634 ERROR neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [req-23466df3-f59e-4897-9a22-1abb7c99dfd9 9a365636c1b44c41a9770a26ead28701 cbddab88045d45eeb3d2027a3e265b78 - - -] Cannot delete bridge brq33213e3f-2b, does not exist

http://logs.openstack.org/57/227957/3/gate/gate-openstack-ansible-dsvm-commit/de3daa3/logs/aio1-neutron/neutron-linuxbridge-agent.log

https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py#L533

That should not be an ERROR message, it could be INFO at best. If you're racing with RPC and a thing is already gone, which you were going to delete anyway, it's not an error.

Matt Riedemann (mriedem)
tags: added: linuxbridge ml2
Changed in neutron:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Matt Riedemann (mriedem)
Revision history for this message
Matt Riedemann (mriedem) wrote :
tags: added: logging
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/229911

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote : Re: Failing to delete an ML2 linux bridge b/c it does not exist should not be an ERROR in the logs

@Matt: please get yourself familiar with some changes in tag management. ML2 is a retired tag.

https://review.openstack.org/#/c/228733/

tags: removed: ml2
Changed in neutron:
importance: Medium → Low
milestone: none → mitaka-1
summary: - Failing to delete an ML2 linux bridge b/c it does not exist should not
- be an ERROR in the logs
+ Failing to delete an linux bridge causes log littering
tags: added: liberty-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/229911
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a9dc48f5a4616a505824c3e8520c3dce7436f1b1
Submitter: Jenkins
Branch: master

commit a9dc48f5a4616a505824c3e8520c3dce7436f1b1
Author: Matt Riedemann <email address hidden>
Date: Thu Oct 1 06:55:14 2015 -0700

    Do not log an error when deleting a linuxbridge does not exist

    When you're racing to teardown and relying on RPC, it should not be
    considered an error that a bridge does not exist when you're trying to
    delete it anyway. This is better served as debug level information.

    For context, this shows up over 1000 times in successful
    gate-tempest-dsvm-neutron-linuxbridge job runs in a week so it's
    probably safe to say it's not contributing to test failures.

    Closes-Bug: #1501779

    Change-Id: I8e9b27e010d033291497f4e660a97bbae92e75b1

Changed in neutron:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/230828

Revision history for this message
Thierry Carrez (ttx) wrote :

mestery/armax: please target this bug to liberty-rc2 if you want it in liberty

Changed in neutron:
milestone: mitaka-1 → none
Kyle Mestery (mestery)
Changed in neutron:
milestone: none → liberty-rc2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/liberty)

Reviewed: https://review.openstack.org/230828
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=08971b0fb61170e86319c6801d6b1021d5944e3c
Submitter: Jenkins
Branch: stable/liberty

commit 08971b0fb61170e86319c6801d6b1021d5944e3c
Author: Matt Riedemann <email address hidden>
Date: Thu Oct 1 06:55:14 2015 -0700

    Do not log an error when deleting a linuxbridge does not exist

    When you're racing to teardown and relying on RPC, it should not be
    considered an error that a bridge does not exist when you're trying to
    delete it anyway. This is better served as debug level information.

    For context, this shows up over 1000 times in successful
    gate-tempest-dsvm-neutron-linuxbridge job runs in a week so it's
    probably safe to say it's not contributing to test failures.

    Closes-Bug: #1501779

    Change-Id: I8e9b27e010d033291497f4e660a97bbae92e75b1
    (cherry picked from commit a9dc48f5a4616a505824c3e8520c3dce7436f1b1)

tags: added: in-stable-liberty
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
tags: added: juno-backport-potential kilo-backport-potential
removed: liberty-rc-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/kilo)

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/233171

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/juno)

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/233204

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/kilo)

Reviewed: https://review.openstack.org/233171
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6c55f584c1d149d2f726754262c579b6d090beff
Submitter: Jenkins
Branch: stable/kilo

commit 6c55f584c1d149d2f726754262c579b6d090beff
Author: Matt Riedemann <email address hidden>
Date: Thu Oct 1 06:55:14 2015 -0700

    Do not log an error when deleting a linuxbridge does not exist

    When you're racing to teardown and relying on RPC, it should not be
    considered an error that a bridge does not exist when you're trying to
    delete it anyway. This is better served as debug level information.

    For context, this shows up over 1000 times in successful
    gate-tempest-dsvm-neutron-linuxbridge job runs in a week so it's
    probably safe to say it's not contributing to test failures.

    Closes-Bug: #1501779

    Change-Id: I8e9b27e010d033291497f4e660a97bbae92e75b1
    (cherry picked from commit a9dc48f5a4616a505824c3e8520c3dce7436f1b1)

tags: added: in-stable-kilo
tags: added: in-stable-juno
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/juno)

Reviewed: https://review.openstack.org/233204
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0aec542de2efee8b7f175b4dd3077839d785cd59
Submitter: Jenkins
Branch: stable/juno

commit 0aec542de2efee8b7f175b4dd3077839d785cd59
Author: Matt Riedemann <email address hidden>
Date: Thu Oct 1 06:55:14 2015 -0700

    Do not log an error when deleting a linuxbridge does not exist

    When you're racing to teardown and relying on RPC, it should not be
    considered an error that a bridge does not exist when you're trying to
    delete it anyway. This is better served as debug level information.

    For context, this shows up over 1000 times in successful
    gate-tempest-dsvm-neutron-linuxbridge job runs in a week so it's
    probably safe to say it's not contributing to test failures.

    Closes-Bug: #1501779

    Change-Id: I8e9b27e010d033291497f4e660a97bbae92e75b1
    (cherry picked from commit a9dc48f5a4616a505824c3e8520c3dce7436f1b1)

Thierry Carrez (ttx)
Changed in neutron:
milestone: liberty-rc2 → 7.0.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/235300

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)
Download full text (16.6 KiB)

Reviewed: https://review.openstack.org/235300
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=140ccc36d172bead2605968b3d61b36cca8a0040
Submitter: Jenkins
Branch: master

commit 6dcfe3a9362ae5fcf18e5cfb59663e43446cd59c
Author: Kevin Benton <email address hidden>
Date: Tue Oct 6 19:28:47 2015 -0700

    Mock oslo policy HTTPCheck instead of urllib

    We were mocking internal behavior of oslo policy by
    patching urllib. This will break with the upcoming oslo
    release that switches to requests.

    This patch changes the mock to the HTTPCheck level and we
    can leave implementation details testing up to oslo_policy.

    Change-Id: I07957f01307e25f1547197c720eea6e3e7f0ef5a
    Closes-Bug: #1503890
    (cherry picked from commit a0f1d9d6de1560be91d3001c8ac9f880a7a5a7e0)

    Add testresources used by oslo.db fixture

    If we use oslo.db fixtures, we'll need the package or
    the next version of oslo.db release will break us.

    Closes-Bug: #1503501
    Change-Id: I7dfbf240333095d91a414ba15a439bdc4804eb25
    (cherry picked from commit 86ad967e40c2c6752ec0fb46cfd3098ede0c7178)

    Fix functional test_server tests

    Now oslo.service 0.10.0 no longer sends SIGHUP to parent and
    children services.

    This was a chance introduced by 286a6ea, and since it invalidated
    the very logic under test, this must be revised.

    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Change-Id: I18a11283925369bc918002477774f196010a1bc3
    Closes-bug: #1505438
    (cherry picked from commit 090fe713592c2b6398d999bfa03b80cbb2054609)

    Make test_server work with older versions of oslo.service

    Change I18a11283925369bc918002477774f196010a1bc3 fixed the test for
    oslo.service >= 0.10.0, but it also broke it for older versions of
    oslo.service. Since the library has minimal version of >= 0.7.0 in
    requirements.txt, test should pass for those versions too.

    Now, instead of validating that either reset() or restart() of workers
    are triggered on SIGHUP, just validate that .start() is triggered the
    expected number of times (either way, no matter how oslo.service decide
    to clean up the children, they exit and then are respawned).

    Change-Id: I41f9d3af780b3178b075bc1e7084f417a2bd1378
    Closes-Bug: #1505645
    (cherry picked from commit 7bb40921660cf29beb68e338e205499efd6ffa36)

    Fixed multiple py34 gate issues

    1. Scope mock of 'open' to module

    By mocking 'open' at the module level, we can avoid affecting
    'open' calls from other modules.

    2. Stop using LOG.exception in contexts with no sys.exc_info set

    Python 3.4 logger fills in record.exc_info with sys.exc_info() result
    [1], and then it uses it to determine the current exception [2] to
    append to the log message. Since there is no exception, exc_info[1] is
    None, and we get AttributeError inside traceback module.

    It's actually a bug in Python interpreter that it attempt to access the
    attribute when there is no exception. It turns out that it's fixed in
    latest master of cPython [3] (...

Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/neutron 8.0.0.0b1

This issue was fixed in the openstack/neutron 8.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.