Description:
============
When OpenDaylight controller is disconnected with OpenvSwitch, then BGPVPN Router association created and after ODL controller is connected back
the BGPVPN network association is not upated in ODL data store and The journal entry remains in failed state in openstack.
mysql> select * from opendaylightjournal;
+--------+---------------------------+--------------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------+---------------------+---------------------+------------+
| seqnum | object_type | object_uuid | operation | data | state | retry_count | created_at | last_retried | version_id |
+--------+---------------------------+--------------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------+---------------------+---------------------+------------+
| 256 | bgpvpn_router_association | 6d063bbb-0e58-4a38-b3eb-ec6c1872b3b2 | create | }q(U router_idqX$ 747d1a4e-772a-48bb-acde-eadfec12cdbdU tenant_idqX af85ed01c8504cdebe0da05601b2fea4qU bgpvpn_idqX$ c20a8f06-975b-497c-83d9-894455e5280fU
project_idqhUidqX$ 6d063bbb-0e58-4a38-b3eb-ec6c1872b3b2u. | failed | 5 | 2018-06-25 22:58:30 | 2018-06-25 22:59:54 | 13 |
+--------+---------------------------+--------------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------+---------------------+---------------------+------------+
1 row in set (0.00 sec)
Following Error message noticed in q-svc log:
---------------------------------------------
2018-06-26 01:28:28.970 ^[[01;31mERROR networking_odl.journal.periodic_task [^[[01;36mreq-a154ed62-e9b9-407a-bdc2-cdeac044beb6 ^[[00;36mNone None^[[01;31m] ^[[01;35m^[[01;31mFailed during periodic task operation journal_recovery.^[[00m: TypeError: get_bgpvpn_router_association() takes at least 4 arguments (3 given)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00mTraceback (most recent call last):
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/networking-odl/networking_odl/journal/periodic_task.py", line 64, in _execute_op
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m operation(context=context)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/neutron/neutron/db/api.py", line 163, in wrapped
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m return method(*args, **kwargs)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/neutron/neutron/db/api.py", line 93, in wrapped
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m setattr(e, '_RETRY_EXCEEDED', True)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m self.force_reraise()
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m six.reraise(self.type_, self.value, self.tb)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/neutron/neutron/db/api.py", line 89, in wrapped
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 150, in wrapper
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m ectxt.value = e.inner_exc
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m self.force_reraise()
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m six.reraise(self.type_, self.value, self.tb)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 138, in wrapper
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m return f(*args, **kwargs)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/neutron/neutron/db/api.py", line 128, in wrapped
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m LOG.debug("Retry wrapper got retriable exception: %s", e)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m self.force_reraise()
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m six.reraise(self.type_, self.value, self.tb)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/neutron/neutron/db/api.py", line 124, in wrapped
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m return f(*dup_args, **dup_kwargs)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/networking-odl/networking_odl/journal/recovery.py", line 56, in journal_recovery
661324,1 99%
[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m _handle_non_existing_resource(context, row)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/networking-odl/networking_odl/journal/recovery.py", line 102, in _handle_non_existing_resource
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m _sync_resource_to_odl(context, row, odl_const.ODL_CREATE, False)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/networking-odl/networking_odl/journal/recovery.py", line 77, in _sync_resource_to_odl
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m resource = _get_latest_resource(context, row)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m File "/opt/stack/networking-odl/networking_odl/journal/recovery.py", line 71, in _get_latest_resource
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m return obj_getter(context, row.object_uuid)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00mTypeError: get_bgpvpn_router_association() takes at least 4 arguments (3 given)
^[[01;31m2018-06-26 01:28:28.970 TRACE networking_odl.journal.periodic_task ^[[01;35m^[[00m
2018-06-26 01:28:29.019 ^[[00;36mINFO networking_odl.journal.periodic_task [^[[01;36mreq-a154ed62-e9b9-407a-bdc2-cdeac044beb6 ^[[00;36mNone None^[[00;36m] ^[[01;35m^[[00;36mmaintenance task has been finished^[[00m
Configuration:
=============
Devstack enviornment-
=>stable/pike version
=>v2 driver is enabled
Openstack Controller& compute Node:10.164.224.235
Openstack Compute Node:10.164.224.230
ODL Controller Node:10.164.224.59 [ ODL-Oxyagen build]
TOR Node:10.164.224.59
Steps to Recreate:
------------------
1. Create BGPVPN
2. Create Router
2. Create Network1 and Network2
3. Create Subnet1 and Subnet2
4. Launch VM in each subnet
5. Disconnect the ODL controller from Openvswitch
6. Associate Router to BGPVPN created above
7. Reconnect the ODL
8. Verify the sync of BGPVPN router association with ODL data store
Logs Attached:
q-svc log
