Cannot ping to a same subnet VM via floating IP

[Kahou Lei]

Suppose I have two VMs running and they are under the same subnet, they are assigned with the floating IPs. (See attached image). I am using nova network model.

I cannot get ping working if I ping from one VM to another VM via the floating IP.

Ping to another vm which resides in another subnet via floating IP seems fine.

I did some investigation and looks like the packet is being dropped after the PREROUTING rules. Here is the modprobe iptable log:

Jul 17 10:15:40 localhost kernel: [ 1846.629048] TRACE: raw:PREROUTING:rule:2 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0
Jul 17 10:15:40 localhost kernel: [ 1846.629055] TRACE: raw:PREROUTING:policy:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0
Jul 17 10:15:40 localhost kernel: [ 1846.629063] TRACE: mangle:PREROUTING:policy:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0
Jul 17 10:15:40 localhost kernel: [ 1846.629068] TRACE: nat:PREROUTING:rule:1 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0
Jul 17 10:15:40 localhost kernel: [ 1846.629074] TRACE: nat:nova-network-PREROUTING:rule:3 IN=br100 OUT= PHYSIN=vlan100 MAC=fa:16:3e:c2:b9:7d:fa:16:3e:dd:e7:c9:08:00 SRC=10.0.0.3 DST=172.24.4.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=3620 DF PROTO=ICMP TYPE=8 CODE=0 ID=8705 SEQ=0

And from the iptables counter, nothing got incremented after the PREROUTING rule:

sudo iptables -t nat -L -v -n
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
    1 84 nova-network-PREROUTING all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 nova-api-PREROUTING all -- * * 0.0.0.0/0 0.0.0.0/0

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
    0 0 nova-network-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 nova-api-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target prot opt in out source destination
    0 0 nova-network-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 nova-api-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 nova-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0

Chain nova-api-OUTPUT (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-POSTROUTING (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-PREROUTING (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-float-snat (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-snat (1 references)
 pkts bytes target prot opt in out source destination
    0 0 nova-api-float-snat all -- * * 0.0.0.0/0 0.0.0.0/0

Chain nova-network-OUTPUT (1 references)
 pkts bytes target prot opt in out source destination
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.1 to:11.0.0.3
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.2 to:10.0.0.4
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.3 to:10.0.0.3
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.4 to:11.0.0.4

Chain nova-network-POSTROUTING (1 references)
 pkts bytes target prot opt in out source destination
    0 0 ACCEPT all -- * * 10.0.0.0/24 192.168.62.100
    0 0 ACCEPT all -- * * 10.0.0.0/24 10.0.0.0/24 ! ctstate DNAT
    0 0 ACCEPT all -- * * 11.0.0.0/24 192.168.62.100
    0 0 ACCEPT all -- * * 11.0.0.0/24 11.0.0.0/24 ! ctstate DNAT
    0 0 SNAT all -- * * 11.0.0.3 0.0.0.0/0 ctstate DNAT to:172.24.4.1
    0 0 SNAT all -- * * 10.0.0.4 0.0.0.0/0 ctstate DNAT to:172.24.4.2
    0 0 SNAT all -- * * 10.0.0.3 0.0.0.0/0 ctstate DNAT to:172.24.4.3
    0 0 SNAT all -- * * 11.0.0.4 0.0.0.0/0 ctstate DNAT to:172.24.4.4

Chain nova-network-PREROUTING (1 references)
 pkts bytes target prot opt in out source destination
    0 0 DNAT tcp -- * * 0.0.0.0/0 169.254.169.254 tcp dpt:80 to:192.168.62.100:8775
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.1 to:11.0.0.3
    1 84 DNAT all -- * * 0.0.0.0/0 172.24.4.2 to:10.0.0.4
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.3 to:10.0.0.3
    0 0 DNAT all -- * * 0.0.0.0/0 172.24.4.4 to:11.0.0.4

Chain nova-network-float-snat (1 references)
 pkts bytes target prot opt in out source destination
    0 0 SNAT all -- * * 11.0.0.3 11.0.0.3 to:172.24.4.1
    0 0 SNAT all -- * eth1 11.0.0.3 0.0.0.0/0 to:172.24.4.1
    0 0 SNAT all -- * * 10.0.0.4 10.0.0.4 to:172.24.4.2
    0 0 SNAT all -- * eth1 10.0.0.4 0.0.0.0/0 to:172.24.4.2
    0 0 SNAT all -- * * 10.0.0.3 10.0.0.3 to:172.24.4.3
    0 0 SNAT all -- * eth1 10.0.0.3 0.0.0.0/0 to:172.24.4.3
    0 0 SNAT all -- * * 11.0.0.4 11.0.0.4 to:172.24.4.4
    0 0 SNAT all -- * eth1 11.0.0.4 0.0.0.0/0 to:172.24.4.4

Chain nova-network-snat (1 references)
 pkts bytes target prot opt in out source destination
    0 0 nova-network-float-snat all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 SNAT all -- * eth1 10.0.0.0/24 0.0.0.0/0 to:192.168.62.100
    0 0 SNAT all -- * eth1 11.0.0.0/24 0.0.0.0/0 to:192.168.62.100

Chain nova-postrouting-bottom (1 references)
 pkts bytes target prot opt in out source destination
    0 0 nova-network-snat all -- * * 0.0.0.0/0 0.0.0.0/0
    0 0 nova-api-snat all -- * * 0.0.0.0/0 0.0.0.0/0

[Markus Zoeller (markus_z) (mzoeller)]

@Kahou Lei:

It's been over 2 months since you are set as assignee but without
a commit to solve this bug. To signalize to other contributors that
this is not in progress and can be worked on, I remove you as assignee.
If you still plan to work on this, please set yourself as assignee
again and provide a patch in Gerrit in the near future.

Please consider updating your Launchpad profile with your IRC nickname
and hanging around in #openstack-nova on irc.freenode.net this makes
it easier to communicate with each other (see [1] for more).

If you have any questions about this process, just ping me (markus_z)
in IRC.

[1] https://wiki.openstack.org/wiki/Nova/Mentoring#Top_Tips_for_working_with_the_Nova_community

[Augustina Ragwitz (auggy)]

I'm working on triaging this and attempting to reproduce the issue.

[Augustina Ragwitz (auggy)]

Not sure if this is related - https://bugs.launchpad.net/nova/+bug/1349590

[onlyellow (onlyellow)]

HI, I test the scenario as below and works fine (using Liberty)

tenant_net : 193.160.15.0/24
ext_net : 192.168.0.0/16

2 vm for testing( both have floating IP ):
[root@controller ~(keystone_admin)]# nova list
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-----------+--------+------------+-------------+-----------------------------------+
| 2803a146-7b30-4ac8-ab0e-f28eb950a0c4 | test1 | ACTIVE | - | Running | t1=193.160.15.43, 192.168.100.134
| 9fac083f-4e73-44d2-9166-10a0e67f8b6c | test2 | ACTIVE | - | Running | t1=193.160.15.45, 192.168.100.135

The result of test is in the attachment.

[Luis (luis-ramirez)]

This bug lacks the necessary information to effectively reproduce and fix it, therefore it has been closed. Feel free to reopen the bug by providing the requested information and set the bug status back to ''New''.