test_gateway_chassis_rebalance fails randomly because of port not being bound yet

Bug #1842291 reported by Jakub Libosvar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-ovn
Fix Released
Undecided
Jakub Libosvar

Bug Description

The test creates lots of routers and then binds a port. It may happen ovn-northd hasn't calculated from NB db to SB db yet and thus port binding for given port hasn't been created yet.

Traceback example:
2019-09-01 10:39:45.989734 | primary | 2019-09-01 10:39:45.988 | Traceback (most recent call last):
2019-09-01 10:39:45.993299 | primary | 2019-09-01 10:39:45.991 | File "/opt/stack/new/networking-ovn/.tox/dsvm-functional-py27/local/lib/python2.7/site-packages/neutron/tests/base.py", line 176, in func
2019-09-01 10:39:45.996311 | primary | 2019-09-01 10:39:45.995 | return f(self, *args, **kwargs)
2019-09-01 10:39:45.999119 | primary | 2019-09-01 10:39:45.998 | File "networking_ovn/tests/functional/test_router.py", line 353, in test_gateway_chassis_rebalance
2019-09-01 10:39:46.002344 | primary | 2019-09-01 10:39:46.001 | may_exist=True).execute(check_error=True)
2019-09-01 10:39:46.005762 | primary | 2019-09-01 10:39:46.004 | File "/opt/stack/new/networking-ovn/.tox/dsvm-functional-py27/local/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 40, in execute
2019-09-01 10:39:46.008808 | primary | 2019-09-01 10:39:46.008 | t.add(self)
2019-09-01 10:39:46.011568 | primary | 2019-09-01 10:39:46.010 | File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2019-09-01 10:39:46.013817 | primary | 2019-09-01 10:39:46.013 | self.gen.next()
2019-09-01 10:39:46.016190 | primary | 2019-09-01 10:39:46.015 | File "/opt/stack/new/networking-ovn/.tox/dsvm-functional-py27/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 119, in transaction
2019-09-01 10:39:46.018197 | primary | 2019-09-01 10:39:46.017 | del self._nested_txns_map[cur_thread_id]
2019-09-01 10:39:46.020931 | primary | 2019-09-01 10:39:46.020 | File "/opt/stack/new/networking-ovn/.tox/dsvm-functional-py27/local/lib/python2.7/site-packages/ovsdbapp/api.py", line 69, in __exit__
2019-09-01 10:39:46.023694 | primary | 2019-09-01 10:39:46.023 | self.result = self.commit()
2019-09-01 10:39:46.026191 | primary | 2019-09-01 10:39:46.025 | File "/opt/stack/new/networking-ovn/.tox/dsvm-functional-py27/local/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 62, in commit
2019-09-01 10:39:46.029181 | primary | 2019-09-01 10:39:46.028 | raise result.ex
2019-09-01 10:39:46.032121 | primary | 2019-09-01 10:39:46.031 | ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Binding with logical_port=cr-lrp-450f9ee1-79fe-4da2-bf9b-cd20f2375deb

Changed in networking-ovn:
assignee: nobody → Jakub Libosvar (libosvar)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (master)

Fix proposed to branch: master
Review: https://review.opendev.org/679663

Changed in networking-ovn:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-ovn (master)

Reviewed: https://review.opendev.org/679663
Committed: https://git.openstack.org/cgit/openstack/networking-ovn/commit/?id=602f7790de7beda5b3ee97d0c24cd0a39f4b243d
Submitter: Zuul
Branch: master

commit 602f7790de7beda5b3ee97d0c24cd0a39f4b243d
Author: Jakub Libosvar <email address hidden>
Date: Mon Sep 2 09:31:54 2019 +0000

    functional: Fix test_router tests LSP binding

    On busy machines when running funcional tests, it may take time to
    process new logical port and create port bindings entry in the
    southbound database.

    Because the tests need to wait for gateway router port that is created
    along with router object, we don't know the name of the logical port
    beforehand. The patch creates a new wait event that matches on ports
    starting with "cr-lrp-" string and caches all such ports. Later, the
    wait() method implements parameter for the logical port names and
    compares whether event with this logical port name occured and if so,
    the waiting ends and port is removed from the cache.

    This makes effective waiting for gateway router ports.

    Change-Id: I5d2209906e70c0220ce20c4dc6593461d5dc7ffa
    Closes-bug: #1842291

Changed in networking-ovn:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.