[DVR] br-int in compute node will send unknown unicast to sg-xxx

[Hong Hui Xiao]

Environment:

    Installation: devstack
    Dataplane: OpenVSwitch
    Version: Ocata/Stable
    Nodes: Two nodes. One node with controller services and network services(dvr_snat), the other node with compute service and network services(dvr)

Setups to reproduce:

    1. Create networks and DVR and connect them, enable snat.
    2. Boot one VM in compute node
    3. Ping 8.8.8.8 inside the VM
    4. tcpdump the tap device of VM

Observation:

    $ sudo tcpdump -nei tap8b25d590-09
    fa:16:3e:63:0c:57 > fa:16:3e:c8:7a:67, ethertype IPv4 (0x0800), length 98: 10.0.0.6 > 8.8.8.8: ICMP echo request, id 22273, seq 343, length 64
    fa:16:3e:c8:7a:67 > fa:16:3e:ba:67:74, ethertype IPv4 (0x0800), length 98: 10.0.0.6 > 8.8.8.8: ICMP echo request, id 22273, seq 343, length 64
    fa:16:3e:ba:67:74 > fa:16:3e:63:0c:57, ethertype IPv4 (0x0800), length 98: 8.8.8.8 > 10.0.0.6: ICMP echo reply, id 22273, seq 343, length 64

Relationship between IP address and MAC address:

    VM 10.0.0.6 fa:16:3e:63:0c:57
    qr-xxx 10.0.0.1 fa:16:3e:c8:7a:67
    sg-xxx 10.0.0.8 fa:16:3e:ba:67:74

Error:

    VM should not capture "fa:16:3e:c8:7a:67 > fa:16:3e:ba:67:74", because it should be an unicast from qr-xxx to sg-xxx. It appears that in br-int, there is no fdb record for fa:16:3e:ba:67:74, so br-int will flood frames destined to "fa:16:3e:ba:67:74" to every port in the same local VLAN. So, VM can capture this unknown unicast.

    Since every device in the same local VLAN on the same br-int can capture the flooded unknown unicast, it will have impact on performance and security.

Expect:

    "qr-xxx to sg-xxx" should mainly be unicast.

[Pawel Suder (pasuder)]

Thank you for your information.

I would like to ask you to provide few more data:

- could you collect packet capture from instance, please?
- could you check bridge type of br-int, please?
- what is the connection between tapxxx and qr-xxx?

As per my understanding: tap and qr ports might be linked. From compute host perspective it might happen that packets will be visible on both interfaces.

Flooding might happen if br-int is a regular bridge (not switch). What is more, if information about IP-MAC is not known, ARP will be sent to get information which MAC has IP address or flood will be done one time to get reply which MAC has IP.

[Hongbin Lu (hongbin.lu)]

Awaiting response for Pawel's comments.

[Hong Hui Xiao (xiaohhui)]

@pasuder, sorry for the delay. for your questions:

- could you collect packet capture from instance, please?

It is exactly the same as I observed from tap-xxx.

- could you check bridge type of br-int, please?

I don't know how to check the bridge type, I just use devstack to build an openvswitch environment. If you know how to check the bridge type of openvswitch device, pls let me know.
But I believe it is a switch not a bridge, or else, every thing will be flooded, not just the frame from qr-xxx to sg-xxx, right?

- what is the connection between tapxxx and qr-xxx?
tap-xxx connects to a linuxbridge then connects to br-int as qvo-xxx device. It is a typical openvswitch connection with iptables as securitygroup implementation. I believe you know what I am talking about :)
The ports on br-int are listed in [1]. So tap-xxx and qr-xxx can be seen as being connected to br-int.

Actually, I have a temporary workaround for it. The root cause is that the mac table in br-int doesn't have record for sg-xxx. You can check it by

    sudo ovs-appctl fdb/show br-int

The workaround is to add an openflow rule

    sudo ovs-ofctl add-flow br-int "table=60,priority=5,dl_vlan=<local vlan id>,dl_src=<mac address of sg device> actions=NORMAL"

So, br-int will have the chance the learn the mac address of sg-xxx device. And frame between qr-xxx and sg-xxx will be unicast.

[1] http://paste.openstack.org/show/720335/