Comment 1 for bug 1835029

Looks like a core ovn issue. Still investigating but ovn-northd gets stuck retrying the same transaction (to set the FIP on the port) over and over and over:

2019-07-12T15:01:36.979Z|2796980|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: send request, method="transact", params=["OVN_Southbound",{"lock":"ovn_northd","op":"assert"},{"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"row":{"nat_addresses":"fa:16:3e:de:ea:d5 172.24.5.169 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"},"op":"update","table":"Port_Binding"},{"mutations":[["nat_addresses","insert",["set",["fa:16:3e:de:ea:d5 172.24.5.125 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"]]]],"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"op":"mutate","table":"Port_Binding"},{"where":[["_uuid","==",["uuid","104adb06-d1db-4617-95c6-3af720271e30"]]],"row":{"options":["map",[["ipv6_ra_address_mode","slaac"],["ipv6_ra_max_interval","600"],["ipv6_ra_min_interval","200"],["ipv6_ra_mtu","1392"],["ipv6_ra_prefixes","fd8b:814:608f::/64"],["ipv6_ra_send_periodic","true"],["ipv6_ra_src_addr","fe80::f816:3eff:fe8f:98c5"],["ipv6_ra_src_eth","fa:16:3e:8f:98:c5"],["peer","fa25893c-d654-47a2-bbee-f3acc89464c3"]]]},"op":"update","table":"Port_Binding"}], id=1456621
2019-07-12T15:01:36.979Z|2796981|poll_loop|DBG|wakeup due to [POLLIN] on fd 11 (<->/usr/local/var/run/openvswitch/ovnsb_db.sock) at lib/stream-fd.c:157 (83% CPU usage)
2019-07-12T15:01:36.979Z|2796982|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: received reply, result=[{},{"count":1},{"count":1},{"count":1}], id=1456621
2019-07-12T15:01:36.979Z|2796983|poll_loop|DBG|wakeup due to 0-ms timeout at lib/ovsdb-idl.c:5397 (83% CPU usage)
2019-07-12T15:01:36.981Z|2796984|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: send request, method="transact", params=["OVN_Southbound",{"lock":"ovn_northd","op":"assert"},{"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"row":{"nat_addresses":"fa:16:3e:de:ea:d5 172.24.5.169 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"},"op":"update","table":"Port_Binding"},{"mutations":[["nat_addresses","insert",["set",["fa:16:3e:de:ea:d5 172.24.5.125 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"]]]],"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"op":"mutate","table":"Port_Binding"},{"where":[["_uuid","==",["uuid","104adb06-d1db-4617-95c6-3af720271e30"]]],"row":{"options":["map",[["ipv6_ra_address_mode","slaac"],["ipv6_ra_max_interval","600"],["ipv6_ra_min_interval","200"],["ipv6_ra_mtu","1392"],["ipv6_ra_prefixes","fd8b:814:608f::/64"],["ipv6_ra_send_periodic","true"],["ipv6_ra_src_addr","fe80::f816:3eff:fe8f:98c5"],["ipv6_ra_src_eth","fa:16:3e:8f:98:c5"],["peer","fa25893c-d654-47a2-bbee-f3acc89464c3"]]]},"op":"update","table":"Port_Binding"}], id=1456622
2019-07-12T15:01:36.981Z|2796985|poll_loop|DBG|wakeup due to [POLLIN] on fd 11 (<->/usr/local/var/run/openvswitch/ovnsb_db.sock) at lib/stream-fd.c:157 (83% CPU usage)
2019-07-12T15:01:36.981Z|2796986|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: received reply, result=[{},{"count":1},{"count":1},{"count":1}], id=1456622
2019-07-12T15:01:36.981Z|2796987|poll_loop|DBG|wakeup due to 0-ms timeout at lib/ovsdb-idl.c:5397 (83% CPU usage)
2019-07-12T15:01:36.982Z|2796988|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: send request, method="transact", params=["OVN_Southbound",{"lock":"ovn_northd","op":"assert"},{"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"row":{"nat_addresses":"fa:16:3e:de:ea:d5 172.24.5.169 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"},"op":"update","table":"Port_Binding"},{"mutations":[["nat_addresses","insert",["set",["fa:16:3e:de:ea:d5 172.24.5.125 is_chassis_resident(\"cr-lrp-01adeeb9-99f5-43ec-8da4-a7bb343c82f1\")"]]]],"where":[["_uuid","==",["uuid","44c638c5-7b92-41e3-9a25-a1f4bf4c6ae1"]]],"op":"mutate","table":"Port_Binding"},{"where":[["_uuid","==",["uuid","104adb06-d1db-4617-95c6-3af720271e30"]]],"row":{"options":["map",[["ipv6_ra_address_mode","slaac"],["ipv6_ra_max_interval","600"],["ipv6_ra_min_interval","200"],["ipv6_ra_mtu","1392"],["ipv6_ra_prefixes","fd8b:814:608f::/64"],["ipv6_ra_send_periodic","true"],["ipv6_ra_src_addr","fe80::f816:3eff:fe8f:98c5"],["ipv6_ra_src_eth","fa:16:3e:8f:98:c5"],["peer","fa25893c-d654-47a2-bbee-f3acc89464c3"]]]},"op":"update","table":"Port_Binding"}], id=1456623
2019-07-12T15:01:36.983Z|2796989|poll_loop|DBG|wakeup due to [POLLIN] on fd 11 (<->/usr/local/var/run/openvswitch/ovnsb_db.sock) at lib/stream-fd.c:157 (83% CPU usage)
2019-07-12T15:01:36.983Z|2796990|jsonrpc|DBG|unix:/usr/local/var/run/openvswitch/ovnsb_db.sock: received reply, result=[{},{"count":1},{"count":1},{"count":1}], id=1456623
2019-07-12T15:01:36.983Z|2796991|poll_loop|DBG|wakeup due to 0-ms timeout at lib/ovsdb-idl.c:5397 (83% CPU usage)

We're still investigating the issue.