My previous comment is wrong. It looks like race-condition. Full log: http://paste.openstack.org/show/796117/
There are many errors linked to failed tests. All of those fails on similar issue:
Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin [None req-74120644-67fe-4b11-964e-86299ee5407f tempest-RoutersIpV6Test-2067377291 tempest-RoutersIpV6Test-2067377291] Unable to update lrouter for deb1a165-eaaa-4406-aee5-bacc5d106260: sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin Traceback (most recent call last): Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin File "/opt/stack/neutron/neutron/services/ovn_l3/plugin.py", line 157, in update_router Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin original_router) ... Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/persistence.py", line 1028, in _emit_update_statements Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin % (table.description, len(records), rows) Jul 18 00:33:03.396324 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR neutron.services.ovn_l3.plugin sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched.
Looks like main cause is somewhere around this function:
Jul 18 01:09:50.692083 ubuntu-bionic-rax-iad-0018417357 neutron-server[7625]: ERROR oslo_db.api File "/opt/stack/neutron/neutron/services/ovn_l3/plugin.py", line 163, in update_router
My previous comment is wrong. It looks like race-condition. Full log: paste.openstack .org/show/ 796117/
http://
There are many errors linked to failed tests. All of those fails on similar issue:
Jul 18 00:33:03.396324 ubuntu- bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin [None req-74120644- 67fe-4b11- 964e-86299ee540 7f tempest- RoutersIpV6Test -2067377291 tempest- RoutersIpV6Test -2067377291] Unable to update lrouter for deb1a165- eaaa-4406- aee5-bacc5d1062 60: sqlalchemy. orm.exc. StaleDataError: UPDATE statement on table 'standardattrib utes' expected to update 1 row(s); 0 were matched. bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin Traceback (most recent call last): bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin File "/opt/stack/ neutron/ neutron/ services/ ovn_l3/ plugin. py", line 157, in update_router bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin original_router) bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin File "/usr/local/ lib/python3. 6/dist- packages/ sqlalchemy/ orm/persistence .py", line 1028, in _emit_update_ statements bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin % (table.description, len(records), rows) bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR neutron. services. ovn_l3. plugin sqlalchemy. orm.exc. StaleDataError: UPDATE statement on table 'standardattrib utes' expected to update 1 row(s); 0 were matched.
Jul 18 00:33:03.396324 ubuntu-
Jul 18 00:33:03.396324 ubuntu-
Jul 18 00:33:03.396324 ubuntu-
...
Jul 18 00:33:03.396324 ubuntu-
Jul 18 00:33:03.396324 ubuntu-
Jul 18 00:33:03.396324 ubuntu-
Looks like main cause is somewhere around this function:
Jul 18 01:09:50.692083 ubuntu- bionic- rax-iad- 0018417357 neutron- server[ 7625]: ERROR oslo_db.api File "/opt/stack/ neutron/ neutron/ services/ ovn_l3/ plugin. py", line 163, in update_router