[OVN] Modifying FIP that is no associated causes ovn_revision_numbers to go stale

[Flavio Fernandes]

NOTE: This is a low priority issue, mostly because it eventually gets fixed by maintenance task. Also because while fip is not associated, there is no
real harm done to the NAT functionality.

CheckRevisionNumberCommand relies in finding a corresponding entry in OVN's NAT table
in order to update the OVN_REV_NUM_EXT_ID_KEY to keep ovn and neutron databases in sync.

Ref: http://lucasgom.es/posts/neutron_ovn_database_consistency.html

Trouble is that unless the floating ip is associated, there will be no entries in OVN's NAT table, causing the call to

 db_rev.bump_revision(context, floatingip, ovn_const.TYPE_FLOATINGIPS)

to not take place.

Steps to reproduce it:

# create a floating ip but do not associate it with anything so router_id is None
FIP=172.24.4.8
openstack floating ip create --floating-ip-address ${FIP} public
FIP_UUID=$(openstack floating ip show ${FIP} -f value -c id) ; echo $FIP_UUID

# Mess with its name, which will bump revision on fip object
openstack floating ip set --description foo ${FIP_UUID}

Code when there is no NAT for a given FIP makes line 1044 skip line 1045

https://github.com/openstack/neutron/blob/15088b39bab715e40d8161a85c95ca400708c83f/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py#L1044

check_rev_cmd.result is None

The dbs are now the inconsistent state

mysql> use neutron;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from standardattributes where resource_type="floatingips";
+----+---------------+---------------------+---------------------+-------------+-----------------+
| id | resource_type | created_at | updated_at | description | revision_number |
+----+---------------+---------------------+---------------------+-------------+-----------------+
| 49 | floatingips | 2020-05-18 20:56:51 | 2020-05-18 20:58:58 | foo2 | 2 |
+----+---------------+---------------------+---------------------+-------------+-----------------+
1 row in set (0.01 sec)

mysql> select * from ovn_revision_numbers where resource_type="floatingips";
+------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+
| standard_attr_id | resource_uuid | resource_type | revision_number | created_at | updated_at |
+------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+
| 49 | 5a1e1ffa-0312-4e78-b7a0-551c396bcf6b | floatingips | 0 | 2020-05-18 20:56:51 | 2020-05-18 20:57:08 |
+------------------+--------------------------------------+---------------+-----------------+---------------------+---------------------+
1 row in set (0.00 sec)

Maintenance task fixes it up later

May 18 21:50:29 stack neutron-server[909]: DEBUG futurist.periodics [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBIn\
consistenciesPeriodics.check_for_inconsistencies' {{(pid=3186) _process_scheduled /usr/local/lib/python3.6/dist-packages/futurist/periodics.py:642}}
May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronizing Neutron and OVN datab\
ases {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:347}}
May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Number of inconsistencies found at \
create/update: floatingips=1 {{(pid=3186) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:325}}
May 18 21:50:29 stack neutron-server[909]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Fixing resource 6b876a35-d286-4407-\
b538-9ce07ab1a281 (type: floatingips) at create/update {{(pid=3186) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:359}}
May 18 21:50:29 stack neutron-server[909]: INFO neutron.db.ovn_revision_numbers_db [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Successfully bumped revision number for resource 6b876a35-d286-4407-b538-9ce07ab1\
a281 (type: floatingips) to 1
May 18 21:50:29 stack neutron-server[909]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-35091ee8-f2fe-47cc-b757-8bb70f750b47 None None] Maintenance task: Synchronization finished (took 0.08 \
seconds)

[Flavio Fernandes (ffernand)]

This is what the transaction "noop" log looks like:

May 18 20:59:45 stack neutron-server[17346]: DEBUG ovsdbapp.backend.ovs_idl.transaction [None req-cbdde0e4-d2fc-4bb2-91a7-dc82fe682127 None None] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5a1e1ffa-0312-4e78-b7a0-551c396bcf6b, resource={'id': '5a1e1ffa-0312-4e78-b7a0-551c396bcf6b', 'tenant_id': '159b77c3839247468362e6841e351413', 'floating_ip_address': '172.24.4.8', 'floating_network_id': 'ef1acdcf-25af-4ec3-8326-a2922870fedd', 'router_id': None, 'port_id': None, 'fixed_ip_address': None, 'status': 'DOWN', 'description': 'foo2', 'port_details': None, 'dns_domain': '', 'dns_name': '', 'port_forwardings': [], 'tags': [], 'created_at': '2020-05-18T20:56:51Z', 'updated_at': '2020-05-18T20:58:58Z', 'revision_number': 2, 'project_id': '159b77c3839247468362e6841e351413'}, resource_type=floatingips, if_exists=True) {{(pid=17354) do_commit /opt/stack/ovsdbapp/ovsdbapp/backend/ovs_idl/transaction.py:87}}

May 18 20:59:45 stack neutron-server[17346]: DEBUG ovsdbapp.backend.ovs_idl.transaction [None req-cbdde0e4-d2fc-4bb2-91a7-dc82fe682127 None None] Transaction caused no change {{(pid=17354) do_commit /opt/stack/ovsdbapp/ovsdbapp/backend/ovs_idl/transaction.py:124}}

[Brian Haley (brian-haley)]

I am inclined to leave this as-is since there are other resources that follow the same pattern, and either the maintenance task will fix it, otherwise when it's associated to a port.

Thanks for the bug Flavio :)