Comment 10 for bug 1060559

Revision history for this message
Koaps (koaps) wrote :

Hi Gary,

Here's the network info:

br-ex Link encap:Ethernet
          inet addr:10.2.1.201 Bcast:10.2.1.207 Mask:255.255.255.248
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

br-int Link encap:Ethernet
          inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

br-omg Link encap:Ethernet
          inet addr:10.0.1.1 Bcast:10.0.1.255 Mask:255.255.255.0
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

eth0 Link encap:Ethernet
          inet addr:10.2.1.175 Bcast:10.2.1.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

eth1 Link encap:Ethernet
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

eth2 Link encap:Ethernet
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1

eth3 Link encap:Ethernet
          UP BROADCAST PROMISC MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          UP LOOPBACK RUNNING MTU:16436 Metric:1

    Bridge br-tun
        Port br-tun
            Interface br-tun
                type: internal
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-omg
        Port br-omg
            Interface br-omg
                type: internal
        Port "eth2"
            Interface "eth2"
    Bridge br-int
        Port "eth1"
            Interface "eth1"
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "1.7.1"

No I can't ping the floating ip assigned to the VM.
The VM can ping the public and private IPs assigned to the controller node.

I don't really see anything in IPtables doing forwarding.

iptables -L -n -v

Chain INPUT (policy ACCEPT 10M packets, 2399M bytes)
 pkts bytes target prot opt in out source destination
4183K 977M nova-api-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0

Chain FORWARD (policy ACCEPT 13 packets, 1092 bytes)
 pkts bytes target prot opt in out source destination
   11 924 nova-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
   11 924 nova-api-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 10M packets, 2420M bytes)
 pkts bytes target prot opt in out source destination
8482K 2035M nova-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
4094K 990M nova-api-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0

Chain nova-api-FORWARD (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-INPUT (1 references)
 pkts bytes target prot opt in out source destination
    0 0 ACCEPT tcp -- * * 0.0.0.0/0 10.0.1.1 tcp dpt:8775

Chain nova-api-OUTPUT (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-local (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-filter-top (2 references)
 pkts bytes target prot opt in out source destination
4094K 990M nova-api-local all -- * * 0.0.0.0/0 0.0.0.0/0

iptables -L -n -v -t nat

Chain PREROUTING (policy ACCEPT 37570 packets, 4517K bytes)
 pkts bytes target prot opt in out source destination
11894 1545K nova-api-PREROUTING all -- * * 0.0.0.0/0 0.0.0.0/0

Chain POSTROUTING (policy ACCEPT 52108 packets, 3129K bytes)
 pkts bytes target prot opt in out source destination
26717 1604K nova-api-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0
52108 3129K nova-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 52104 packets, 3129K bytes)
 pkts bytes target prot opt in out source destination
26713 1603K nova-api-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0

Chain nova-api-OUTPUT (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-POSTROUTING (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-PREROUTING (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-float-snat (1 references)
 pkts bytes target prot opt in out source destination

Chain nova-api-snat (1 references)
 pkts bytes target prot opt in out source destination
26717 1604K nova-api-float-snat all -- * * 0.0.0.0/0 0.0.0.0/0

Chain nova-postrouting-bottom (1 references)
 pkts bytes target prot opt in out source destination
26717 1604K nova-api-snat all -- * * 0.0.0.0/0 0.0.0.0/0

These are the command I used in quantum:

## Create network and subnet
quantum net-create --shared priv_net
quantum subnet-create --name int_sub --allocation-pool start=10.0.0.10,end=10.0.0.100 priv_net 10.0.0.0/24

## Create a router, and add the private subnet as one of its interface
quantum router-create vr1
quantum router-interface-add vr1 <PRIV_SUBNET_ID>

## Create an external network, and a subnet. Note that this is on a different tenant, the service tenant, and the DHCP is disabled for the subnet.
quantum net-create --shared pub_net -- --router:external=True
quantum subnet-create pub_net 10.2.1.200/29 -- --enable_dhcp=False
quantum router-gateway-set vr1 <EXTERNAL_NETWORK_ID>

## Get the external gateway IP
quantum subnet-show <EXTERNAL_SUBNET_ID>

## Update the gateway IP for the external bridge (using the gateway subnet)
vi /etc/sysconfig/network-scripts/ifcfg-br-ex
ifdown br-ex
ifup br-ex

## Assign a floating IP to a VM
quantum port-list
quantum floatingip-create --port_id <internal VM port-id> <ext-net-id>

I tried adding explict iptables masquerading rules but that didn't work either.

The compute node is similar in setup with the addition of the tap device:

tapb9c14f03-95 Link encap:Ethernet HWaddr 7A:BA:77:71:E6:BF
          inet6 addr: fe80::78ba:77ff:fe71:e6bf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:272 errors:0 dropped:0 overruns:0 frame:0
          TX packets:332 errors:0 dropped:6 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:35517 (34.6 KiB) TX bytes:36220 (35.3 KiB)

Both the compute and controller have a script that setups of the bridges so they are almost exact, except the compute doesn't have a br-ex setup.

The VM can only ping the br-int IP on the compute node, but like I said, it can ping all IP's assigned on the controller node.

Thanks