ovs-fw does not reinstate GRE conntrack entry .
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Fix Released
|
High
|
Jakub Libosvar |
Bug Description
*High level description:*
We have VMs running GRE tunnels between them with OVSFW and SG implemented along with GRE conntrack helper loaded on the hypervisor. GRE works as expected but the tunnel breaks whenever there is a neutron ovs agent event causing some exception like the below AMQP timeouts or OVSFW port not found :
AMQP Timeout :
2017-04-07 19:07:03.001 5275 ERROR neutron.
2017-04-07 19:07:03.001 5275 ERROR neutron.
2017-04-07 19:07:03.003 5275 WARNING oslo.service.
2017-04-07 19:07:03.041 5275 INFO neutron.
2017-04-07 19:07:06.747 5275 INFO neutron.
2017-04-07 19:07:06.841 5275 INFO neutron.
OVSFWPortNOtFound:
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.048 5160 ERROR neutron.
2017-03-30 18:31:05.072 5160 INFO neutron.
The agent throws out of sync messages and starts to initialize neutron ports once again along with fresh SG rules.
2017-04-07 19:07:07.110 5275 INFO neutron.
2017-04-07 19:07:07.215 5275 ERROR neutron.
During this process, when it prepares new filters for all ports, its marking the conntrack entry for certain GRE connection(high traffic) as invalid.
root@server:
ipv4 2 gre 47 178 src=1.1.1.203 dst=2.2.2.66 srckey=0x0 dstkey=0x0 src=2.2.2.66 dst=1.1.1.203 srckey=0x0 dstkey=0x0 [ASSURED] mark=1 zone=5 use=1
ipv4 2 gre 47 179 src=5.5.5.104 dst=4.4.4.187 srckey=0x0 dstkey=0x0 src=4.4.4.187 dst=5.5.5.104 srckey=0x0 dstkey=0x0 [ASSURED] mark=0 zone=5 use=1
And that connection state remains invalid, unless someone reboots the VM, or flushes the connection directly on the conntrack or through OVS.
We have a blanket any protocol any port any IP SG rule during this scenario, we even tried adding specific rules to allow IP 47 for GRE. But nothing fixed this problem.
Was checking for ovs-conntrack helper specific bugs and came across https:/
OpenStack Version : Newton.
Hypervisor OS : Ubuntu 16.04.2
Kernel version : 4.4.0-70-generic
OVS version : 2.6.1
affects: | neutron → null-and-void |
information type: | Public → Private |
Changed in null-and-void: | |
status: | New → Invalid |
affects: | null-and-void → neutron |
Changed in neutron: | |
status: | Invalid → New |
information type: | Private → Public |
Changed in neutron: | |
importance: | Undecided → High |
Assigning to Jakub for further investigation.