some functional ovs tests are not mutually isolated

Bug #1625724 reported by Ihar Hrachyshka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
High
Slawek Kaplonski

Bug Description

Some tests, like ARPSpoofTestCase or OVSFlowTestCase, don't set int_peer_patch_port and tun_peer_patch_port to unique values, making tests clash on attempt to add the same port to different bridges:

http://logs.openstack.org/17/372817/1/check/gate-neutron-dsvm-functional-ubuntu-trusty/dc73766/testr_results.html.gz

Traceback (most recent call last):
  File "neutron/tests/functional/agent/test_ovs_flows.py", line 351, in setUp
    self.br_tun.setup_default_table(self.tun_p, True)
  File "neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_tun.py", line 119, in setup_default_table
    actions=actions)
  File "neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 201, in install_apply_actions
    **match_kwargs)
  File "neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 188, in install_instructions
    self._send_msg(msg)
  File "neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 89, in _send_msg
    raise RuntimeError(m)
RuntimeError: ofctl request version=0x4,msg_type=0xe,msg_len=None,xid=0xefa1def,OFPFlowMod(buffer_id=4294967295,command=0,cookie=11167842442732864518L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[OFPInstructionActions(actions=[NXActionLearn(cookie=11167842442732864518L,experimenter=8992,fin_hard_timeout=0,fin_idle_timeout=0,flags=0,hard_timeout=300,idle_timeout=0,len=96,priority=1,specs=[NXFlowSpecMatch(dst=('vlan_tci', 0),n_bits=12,src=('vlan_tci', 0)), NXFlowSpecMatch(dst=('eth_dst', 0),n_bits=48,src=('eth_src', 0)), NXFlowSpecLoad(dst=('vlan_tci', 0),n_bits=16,src=0), NXFlowSpecLoad(dst=('tunnel_id', 0),n_bits=64,src=('tunnel_id', 0)), NXFlowSpecOutput(dst='',n_bits=32,src=('in_port', 0))],subtype=16,table_id=20,type=65535), OFPActionOutput(len=16,max_len=0,port=-1,type=0)],type=4)],match=OFPMatch(oxm_fields={}),out_group=0,out_port=0,priority=1,table_id=10) timed out

In the test log:

http://logs.openstack.org/17/372817/1/check/gate-neutron-dsvm-functional-ubuntu-trusty/dc73766/logs/dsvm-functional-logs/neutron.tests.functional.agent.test_ovs_flows.OVSFlowTestCase.test_install_flood_to_tun_native,vsctl_.txt.gz#_2016-09-20_06_49_39_366

2016-09-20 06:49:39.366 26997 ERROR neutron.agent.ovsdb.impl_vsctl [req-0efe376e-dfa5-407b-a462-55084cdb57ad - - - - -] Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--', '--may-exist', 'add-port', 'test-brdd5b39b5', 'patch-int', '--', 'set', 'Interface', 'patch-int', 'type=patch', 'options:peer=patch-tun']. Exception: Exit code: 1; Stdin: ; Stdout: ; Stderr: ovs-vsctl: "--may-exist add-port test-brdd5b39b5 patch-int" but patch-int is actually attached to bridge test-br3454b438

Some other test used the same name for patch-int port.

tags: added: functional-tests
Changed in neutron:
importance: Undecided → High
tags: added: gate-failure
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/386971

Changed in neutron:
assignee: nobody → Slawek Kaplonski (slaweq)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/386971
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ca62e7ec9d09eac2b516b3014fc4d678ac85a6db
Submitter: Jenkins
Branch: master

commit ca62e7ec9d09eac2b516b3014fc4d678ac85a6db
Author: Sławek Kapłoński <email address hidden>
Date: Sat Oct 15 20:03:45 2016 +0000

    Make ovs functional tests mutually isolated

    Patch ports between br-int and br-tun have now got
    randomly generated names so two different tests
    shouldn't try to create ports with same name in same time.

    Change-Id: I6107bc3bc2e2a504e8ec5b4f0e3abaa68d51c01f
    Closes-Bug: #1625724

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 10.0.0.0b1

This issue was fixed in the openstack/neutron 10.0.0.0b1 development milestone.

tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/421974

tags: removed: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/newton)

Reviewed: https://review.openstack.org/421974
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1cfc49c365b7245e97e5046d3a8ad587f5ab3c66
Submitter: Jenkins
Branch: stable/newton

commit 1cfc49c365b7245e97e5046d3a8ad587f5ab3c66
Author: Sławek Kapłoński <email address hidden>
Date: Sat Oct 15 20:03:45 2016 +0000

    Make ovs functional tests mutually isolated

    Patch ports between br-int and br-tun have now got
    randomly generated names so two different tests
    shouldn't try to create ports with same name in same time.

    Change-Id: I6107bc3bc2e2a504e8ec5b4f0e3abaa68d51c01f
    Closes-Bug: #1625724
    (cherry picked from commit ca62e7ec9d09eac2b516b3014fc4d678ac85a6db)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 9.2.0

This issue was fixed in the openstack/neutron 9.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.