I am observing that OVS is dropping packet while flooding to multiple VXLAN tunnel. I have three compute node (compute A, compute B, compute C).
src(compute A) -> SF (compute B) -> dst (compute A)
To make it more clear:
Execute:
sudo ovs-ofctl dump-flows br-tun table=22
cookie=0xb4718d8402f201be, duration=19741.982s, table=22, n_packets=320, n_bytes=37484, idle_age=25, priority=1,dl_vlan=1 actions=strip_vlan,load:0x49->NXM_NX_TUN_ID[],output:2,output:4
2 & 4 are tunnel ports.
The issue looks like in OVS and still in OVS 2.6.1 code. datapath/flow_netlink.c +2247
** Note **: - The behavior is weird also, as it may not be observed for some time. If you don't see the behavior, try deleting and recreating chain. for eg.
neutron port-chain-delete pc1
neutron port-chain-create pc1 --port-pair-group ppg1 --flow-classifier fc1
uname -a
Linux vikash-E5 4.8.0-46-generic #49~16.04.1-Ubuntu SMP Fri Mar 31 14:51:03 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
sudo ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.6.1
DB Schema 7.14.0
Below are the logs:
2017-04-20T05:47:57.103Z|00207|dpif(handler30)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:d9d2c3c7-a9ad-4268-9e00-cb5bfe9e8c82 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(11),skb_mark(0x4000000/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:0b:cf:35),eth_type(0x0800),ipv4(src=76.0.0.5,dst=77.0.0.9,proto=1,tos=0/0xfc,ttl=63,frag=no),icmp(type=8,code=0), actions:set(eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:18:b0:f1)),set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.177,ttl=64,flags(df|key))),push_mpls(label=511,tc=0,ttl=255,bos=1,eth_type=0x8847),8,set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.182,ttl=64,flags(df|key))),8
2017-04-20T05:47:58.103Z|00208|dpif(handler30)|WARN|Dropped 1 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2017-04-20T05:47:58.103Z|00209|dpif(handler30)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:d9d2c3c7-a9ad-4268-9e00-cb5bfe9e8c82 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(11),skb_mark(0x4000000/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:0b:cf:35),eth_type(0x0800),ipv4(src=76.0.0.5,dst=77.0.0.9,proto=1,tos=0/0xfc,ttl=63,frag=no),icmp(type=8,code=0), actions:set(eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:18:b0:f1)),set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.177,ttl=64,flags(df|key))),push_mpls(label=511,tc=0,ttl=255,bos=1,eth_type=0x8847),8,set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.182,ttl=64,flags(df|key))),8
2017-04-20T05:47:59.103Z|00210|dpif(handler30)|WARN|Dropped 1 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2017-04-20T05:47:59.103Z|00211|dpif(handler30)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:d9d2c3c7-a9ad-4268-9e00-cb5bfe9e8c82 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(11),skb_mark(0x4000000/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:0b:cf:35),eth_type(0x0800),ipv4(src=76.0.0.5,dst=77.0.0.9,proto=1,tos=0/0xfc,ttl=63,frag=no),icmp(type=8,code=0), actions:set(eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:18:b0:f1)),set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.177,ttl=64,flags(df|key))),push_mpls(label=511,tc=0,ttl=255,bos=1,eth_type=0x8847),8,set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.182,ttl=64,flags(df|key))),8
2017-04-20T05:48:00.103Z|00212|dpif(handler30)|WARN|Dropped 1 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2017-04-20T05:48:00.103Z|00213|dpif(handler30)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:d9d2c3c7-a9ad-4268-9e00-cb5bfe9e8c82 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(11),skb_mark(0x4000000/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:0b:cf:35),eth_type(0x0800),ipv4(src=76.0.0.5,dst=77.0.0.9,proto=1,tos=0/0xfc,ttl=63,frag=no),icmp(type=8,code=0), actions:set(eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:18:b0:f1)),set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.177,ttl=64,flags(df|key))),push_mpls(label=511,tc=0,ttl=255,bos=1,eth_type=0x8847),8,set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.182,ttl=64,flags(df|key))),8
2017-04-20T05:48:01.103Z|00214|dpif(handler30)|WARN|Dropped 1 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2017-04-20T05:48:01.103Z|00215|dpif(handler30)|WARN|system@ovs-system: failed to put[create] (Invalid argument) ufid:d9d2c3c7-a9ad-4268-9e00-cb5bfe9e8c82 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(11),skb_mark(0x4000000/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:0b:cf:35),eth_type(0x0800),ipv4(src=76.0.0.5,dst=77.0.0.9,proto=1,tos=0/0xfc,ttl=63,frag=no),icmp(type=8,code=0), actions:set(eth(src=fa:16:3e:37:fa:66,dst=fa:16:3e:18:b0:f1)),set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.177,ttl=64,flags(df|key))),push_mpls(label=511,tc=0,ttl=255,bos=1,eth_type=0x8847),8,set(tunnel(tun_id=0x53,src=192.168.2.91,dst=192.168.2.182,ttl=64,flags(df|key))),8
I have met the same problem, is there anyone know how to solve this? thx