some test_l3 unit test failures

Bug #1605894 reported by YAMAMOTO Takashi
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
networking-midonet
Fix Released
Critical
YAMAMOTO Takashi
neutron
Fix Released
High
Sreekumar S

Bug Description

networking-midonet gate fails due to recent l3_db change. [1]
any plugins which have a surrounding transaction for add_router_interface
would be affected in the same way.

[1] I797df266dafc41843408dc95a6ce9f986db5c21c

http://logs.openstack.org/00/344100/2/check/gate-networking-midonet-python27/f55680d/console.html

2016-07-23 10:56:05.220890 | ======================================================================
2016-07-23 10:56:05.220921 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin.TestMidonetL3NatDBIntTest.test_router_add_interface_dup_subnet2_returns_400
2016-07-23 10:56:05.220941 | ----------------------------------------------------------------------
2016-07-23 10:56:05.220953 | Traceback (most recent call last):
2016-07-23 10:56:05.220984 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1425, in test_router_add_interface_dup_subnet2_returns_400
2016-07-23 10:56:05.221003 | expected_code=exc.HTTPBadRequest.code)
2016-07-23 10:56:05.221029 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.221046 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.221081 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.221101 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.221136 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.221146 | raise mismatch_error
2016-07-23 10:56:05.221160 | testtools.matchers._impl.MismatchError: 400 != 500
2016-07-23 10:56:05.221178 | ======================================================================
2016-07-23 10:56:05.221211 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin.TestMidonetL3NatDBIntTest.test_router_add_interface_ipv6_port_existing_network_returns_400
2016-07-23 10:56:05.221235 | ----------------------------------------------------------------------
2016-07-23 10:56:05.221247 | Traceback (most recent call last):
2016-07-23 10:56:05.221281 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1316, in test_router_add_interface_ipv6_port_existing_network_returns_400
2016-07-23 10:56:05.221297 | expected_code=exp_code)
2016-07-23 10:56:05.221324 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.221349 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.221386 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.221400 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.221435 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.221445 | raise mismatch_error
2016-07-23 10:56:05.221459 | testtools.matchers._impl.MismatchError: 400 != 500
2016-07-23 10:56:05.221477 | ======================================================================
2016-07-23 10:56:05.221509 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin.TestMidonetL3NatDBIntTest.test_router_add_interface_multiple_ipv4_subnet_port_returns_400
2016-07-23 10:56:05.221528 | ----------------------------------------------------------------------
2016-07-23 10:56:05.221540 | Traceback (most recent call last):
2016-07-23 10:56:05.221574 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1287, in test_router_add_interface_multiple_ipv4_subnet_port_returns_400
2016-07-23 10:56:05.221584 | expected_code=exp_code)
2016-07-23 10:56:05.221610 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.221626 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.221660 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.221675 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.221709 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.221719 | raise mismatch_error
2016-07-23 10:56:05.221734 | testtools.matchers._impl.MismatchError: 400 != 500
2016-07-23 10:56:05.221752 | ======================================================================
2016-07-23 10:56:05.221782 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin_v2.TestMidonetL3NatExtraRoute.test_router_add_interface_dup_subnet2_returns_400
2016-07-23 10:56:05.221801 | ----------------------------------------------------------------------
2016-07-23 10:56:05.221812 | Traceback (most recent call last):
2016-07-23 10:56:05.221843 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1425, in test_router_add_interface_dup_subnet2_returns_400
2016-07-23 10:56:05.221856 | expected_code=exc.HTTPBadRequest.code)
2016-07-23 10:56:05.221882 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.221898 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.222052 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.222076 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.222111 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.222121 | raise mismatch_error
2016-07-23 10:56:05.222135 | testtools.matchers._impl.MismatchError: 400 != 500
2016-07-23 10:56:05.222154 | ======================================================================
2016-07-23 10:56:05.222186 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin_v2.TestMidonetL3NatExtraRoute.test_router_add_interface_ipv6_port_existing_network_returns_400
2016-07-23 10:56:05.222205 | ----------------------------------------------------------------------
2016-07-23 10:56:05.222217 | Traceback (most recent call last):
2016-07-23 10:56:05.222259 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1316, in test_router_add_interface_ipv6_port_existing_network_returns_400
2016-07-23 10:56:05.222272 | expected_code=exp_code)
2016-07-23 10:56:05.222298 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.222314 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.222348 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.222363 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.222397 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.222407 | raise mismatch_error
2016-07-23 10:56:05.222422 | testtools.matchers._impl.MismatchError: 400 != 500
2016-07-23 10:56:05.222440 | ======================================================================
2016-07-23 10:56:05.222473 | FAIL: midonet.neutron.tests.unit.test_midonet_plugin_v2.TestMidonetL3NatExtraRoute.test_router_add_interface_multiple_ipv4_subnet_port_returns_400
2016-07-23 10:56:05.222491 | ----------------------------------------------------------------------
2016-07-23 10:56:05.222503 | Traceback (most recent call last):
2016-07-23 10:56:05.222536 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 1287, in test_router_add_interface_multiple_ipv4_subnet_port_returns_400
2016-07-23 10:56:05.222547 | expected_code=exp_code)
2016-07-23 10:56:05.222573 | File "/tmp/openstack/neutron/neutron/tests/unit/extensions/test_l3.py", line 403, in _router_interface_action
2016-07-23 10:56:05.222589 | self.assertEqual(expected_code, res.status_int, msg)
2016-07-23 10:56:05.222623 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 411, in assertEqual
2016-07-23 10:56:05.222637 | self.assertThat(observed, matcher, message)
2016-07-23 10:56:05.222672 | File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 498, in assertThat
2016-07-23 10:56:05.222682 | raise mismatch_error
2016-07-23 10:56:05.222696 | testtools.matchers._impl.MismatchError: 400 != 500

tags: added: gate-failure
Changed in networking-midonet:
importance: Undecided → Critical
description: updated
description: updated
Revision history for this message
YAMAMOTO Takashi (yamamoto) wrote :
Download full text (4.3 KiB)

http://logs.openstack.org/74/334874/7/check/gate-networking-midonet-python27/b6e71e0/testr_results.html.gz

   ERROR [root] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 683, in _add_interface_by_port\n return self._validate_router_port_info(context, router, port_id)\n', ' File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 664, in _validate_router_port_info\n subnets)\n', ' File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 576, in _check_for_dup_router_subnets\n raise n_exc.BadRequest(resource=\'router\', msg=msg)\n', 'BadRequest: Bad router request: Router already has a port on subnet 876deb79-9cdc-47fe-90aa-d607b400294f.\n']
   ERROR [neutron.api.v2.resource] add_router_interface failed: No details.
Traceback (most recent call last):
  File "/tmp/openstack/neutron/neutron/api/v2/resource.py", line 79, in resource
    result = method(request=request, **args)
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/oslo_db/api.py", line 151, in wrapper
    ectxt.value = e.inner_exc
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/oslo_db/api.py", line 139, in wrapper
    return f(*args, **kwargs)
  File "/tmp/openstack/neutron/neutron/api/v2/base.py", line 217, in _handle_action
    ret_value = getattr(self._plugin, name)(*arg_list, **kwargs)
  File "midonet/neutron/plugin_v1.py", line 388, in add_router_interface
    context, router_id, interface_info)
  File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 1707, in add_router_interface
    context, router_id, interface_info)
  File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 763, in add_router_interface
    context, router, interface_info['port_id'], device_owner)
  File "/tmp/openstack/neutron/neutron/db/l3_db.py", line 688, in _add_interface_by_port
    'device_owner': prev_owner}})
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1062, in __call__
    return _mock_self._mock_call(*args, **kwargs)
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1128, in _mock_call
    ret_val = effect(*args, **kwargs)
  File "midonet/neutron/plugin_v1.py", line 270, in update_port
    context.session.begin(subtransactions=True):
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 725, in begin
    nested=nested)
  File "/home/jenkins/workspace/gate-networking-midonet-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", li...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to networking-midonet (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/346576

Changed in networking-midonet:
assignee: nobody → YAMAMOTO Takashi (yamamoto)
milestone: none → 3.0.0
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-midonet (master)

Fix proposed to branch: master
Review: https://review.openstack.org/346585

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-midonet (master)

Change abandoned by YAMAMOTO Takashi (<email address hidden>) on branch: master
Review: https://review.openstack.org/346576
Reason: https://review.openstack.org/#/c/346585/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/346954

Changed in neutron:
assignee: nobody → Sreekumar S (sreesiv)
status: New → In Progress
Changed in neutron:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-midonet (master)

Reviewed: https://review.openstack.org/346585
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=0fa57a25a683bb1c85a839dba4bb253aa9551ad0
Submitter: Jenkins
Branch: master

commit 0fa57a25a683bb1c85a839dba4bb253aa9551ad0
Author: YAMAMOTO Takashi <email address hidden>
Date: Mon Jul 25 13:37:29 2016 +0900

    l3: Avoid breaking transaction in _validate_router_port_info

    The recent Neutron change [1] broke plugins which have a surrounding
    transaction for the method. This commit workarounds the breakage by
    avoid breaking the whole transaction in _validate_router_port_info.

    [1] I797df266dafc41843408dc95a6ce9f986db5c21c

    Closes-Bug: #1605894
    Change-Id: Ib2fff32642013af2523b159a48c7e2bc8c854131

Changed in networking-midonet:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/346954
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=22a341a204e84b83094899730602975baa469af9
Submitter: Jenkins
Branch: master

commit 22a341a204e84b83094899730602975baa469af9
Author: Sreekumar S <email address hidden>
Date: Mon Jul 25 22:43:24 2016 +0530

    Fixes the midonet test_l3 unit test failures

    Failures for midonet unit tests seems to be caused due
    to nested rollback.
    Referred https://review.openstack.org/#/c/230481/
    for this fix.

    Change-Id: Ic6b37bf3f799055c93e9eeb9b7f51758ceecbeb5
    Closes-Bug: #1605894
    Related-Bug: #1501686

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/352917

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/mitaka)

Change abandoned by Sreekumar S (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/352917
Reason: The merging got all wrong. Abandoning.

Revision history for this message
Sreekumar S (sreesiv) wrote :

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/346376

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 9.0.0.0b3

This issue was fixed in the openstack/neutron 9.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/346376
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=51ebedb2c654efd69adde83bcd26578c79720c51
Submitter: Jenkins
Branch: stable/mitaka

commit 51ebedb2c654efd69adde83bcd26578c79720c51
Author: Sreekumar S <email address hidden>
Date: Mon Jun 20 21:06:19 2016 +0530

    Fixes port device_id/device_owner change in failed operation

    If a router is a attached to a neutron port and that operation
    fails at a late stage in the processing then the 'device_id' and
    'device_owner' attributes of the port are still changed. The
    failed operation should not change them.

    The reason is, as of commit 0947458018725b241603139f4ec6f92e84b2f29b,
    the update of those attributes happens outside of the DB transaction.
    For failed operation, the values are not reset back to their original
    values.

    (cherry picked from commit cb8af9bd50659daca4c1dea4070a011190ef3461)

    Also fixes: Fixes the midonet test_l3 unit test failures

    Failures for midonet unit tests seems to be caused due
    to nested rollback.
    Referred https://review.openstack.org/#/c/230481/
    for this fix.

    Conflicts:
     neutron/db/l3_db.py
     neutron/tests/unit/extensions/test_l3.py

    (cherry picked from commit 22a341a204e84b83094899730602975baa469af9)

    NOTE: This commit merged two patches from master branch.

    Change-Id: I797df266dafc41843408dc95a6ce9f986db5c21c
    Closes-Bug: #1523859
    Closes-Bug: #1605894
    Related-Bug: #1501686

tags: added: in-stable-mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 8.3.0

This issue was fixed in the openstack/neutron 8.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-midonet 3.0.0

This issue was fixed in the openstack/networking-midonet 3.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.