Duplicate packets when pinging between Floating IPs

[Everett Toews]

When you ping between VMs on their Floating IPs you get duplicate packets.

1. Default Security Group should have a rule for ping and ssh
  Protocal: ICMP
  From: -1
  To: -1
  CIDR: 0.0.0.0/0

  Protocal: TCP
  From: 22
  To: 22
  CIDR: 0.0.0.0/0
2. Start 2 VMs (test1 and test2)
3. Allocate and assign a Floating IP to each VM
4. ssh to test1
5. ping test2 by its Floating IP
6. ping reports duplicate packets

Output:

ubuntu@i-0000047a:/opt/stack/nova$ ssh cirros@172.24.4.225

The authenticity of host '172.24.4.225 (172.24.4.225)' can't be established.
RSA key fingerprint is d2:58:5a:77:28:46:f5:19:79:34:d5:d4:fc:46:37:dd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.24.4.225' (RSA) to the list of known hosts.
cirros@172.24.4.225's password:

$ ping 172.24.4.226

PING 172.24.4.226 (172.24.4.226): 56 data bytes
64 bytes from 172.24.4.226: seq=0 ttl=64 time=2.226 ms
64 bytes from 172.24.4.226: seq=0 ttl=63 time=4.523 ms (DUP!)
64 bytes from 172.24.4.226: seq=0 ttl=64 time=5.533 ms (DUP!)
64 bytes from 172.24.4.226: seq=0 ttl=63 time=6.188 ms (DUP!)
64 bytes from 172.24.4.226: seq=1 ttl=64 time=2.180 ms
64 bytes from 172.24.4.226: seq=1 ttl=63 time=4.320 ms (DUP!)
64 bytes from 172.24.4.226: seq=1 ttl=64 time=6.045 ms (DUP!)
64 bytes from 172.24.4.226: seq=1 ttl=63 time=7.456 ms (DUP!)
^C
--- 172.24.4.226 ping statistics ---
2 packets transmitted, 2 packets received, 6 duplicates, 0% packet loss
round-trip min/avg/max = 2.180/4.808/7.456 ms

Version Info:

devstack
commit 97eb92880e0886a3f257f32a88ba9e55b8ec8a90
Date: Thu Mar 15 15:32 2012 (MDT)

[Evan Callicoat (diopter)]

Can you try turning off promisc(uous) mode on the bridge and see if you still A) get working floating IP traffic from/to/between VMs, and B) don't get dupes anymore?

ip link set dev brXXX promisc off <- (for whatever your bridge device is)

[Anthony Young (sleepsonthefloor)]

You can actually reproduce a similar scenario with just one vm:

> Boot instance
> Associate floating ip
> From within the instance, try to ping an external address (e.g. google.com)

This results in the following:

(from vm)
$ ping google.com
PING google.com (74.125.225.65): 56 data bytes
64 bytes from 74.125.225.65: seq=0 ttl=128 time=85.824 ms
64 bytes from 74.125.225.65: seq=0 ttl=127 time=88.867 ms (DUP!)

Doing as Evan suggests on the host machine corrects the issue:

(on host)
$ sudo ip link set br100 promisc off

(from vm)
$ ping google.com
PING google.com (74.125.225.99): 56 data bytes
64 bytes from 74.125.225.99: seq=0 ttl=128 time=81.363 ms

[Vish Ishaya (vishvananda)]

so it looks like the promisc bridge mode that we were using to work around having the bridge be the public interface works now that we have hairpin mode. Will propose removing the workaround.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/5429

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/5429
Committed: http://github.com/openstack/nova/commit/682f612b2c35ab3c967b76c457bde80af0caec54
Submitter: Jenkins
Branch: master

commit 682f612b2c35ab3c967b76c457bde80af0caec54
Author: Vishvananda Ishaya <email address hidden>
Date: Thu Mar 15 16:32:13 2012 -0700

    Stop setting promisc on bridge

     * With hairpin mode the old workaround is now broken
     * Fixes bug 956474

    Change-Id: Ib06baa5292ad59138041b2c641673a96263b5ff8

[Everett Toews (everett-toews)]

For what it's worth, I just fired up a new devstack environment and I no longer get duplicate packets when running my test case above.