l3 agent gw port missing vlan tag for vlan provider network

[Robert Collins]

Hi, I have a provider network with my floating NAT range on it and a vlan segmentation id:
neutron net-show ext-net
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | f8ea424f-fcbe-4d57-9f17-5c576bf56e60 |
| name | ext-net |
| provider:network_type | vlan |
| provider:physical_network | datacentre |
| provider:segmentation_id | 25 |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | 391829e1-afc5-4280-9cd9-75f554315e82 |
| tenant_id | e23f57e1d6c54398a68354adf522a36d |
+---------------------------+--------------------------------------+

My ovs agent config:

cat /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini
[DATABASE]
sql_connection = mysql://.....@localhost/ovs_neutron?charset=utf8

reconnect_interval = 2

[OVS]
bridge_mappings = datacentre:br-ex
network_vlan_ranges = datacentre

tenant_network_type = gre
tunnel_id_ranges = 1:1000
enable_tunneling = True
integration_bridge = br-int
tunnel_bridge = br-tun
local_ip = 10.10.16.151

[AGENT]
polling_interval = 2

[SECURITYGROUP]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
root@ci-overcloud-controller0-ydt5on7wojsb:~#

But, the thing is, the port created in ovs is missing the tag:
    Bridge br-ex
        Port "qg-d8c27507-14"
            Interface "qg-d8c27507-14"
                type: internal

And we (As expected) are seeing tagged frames in tcpdump:
19:37:16.107288 20:fd:f1:b6:f5:16 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 68: vlan 25, p 0, ethertype ARP, Request who-has 138.35.77.67 tell 138.35.77.1, length 50

rather than untagged frames for the vlan 25.

Running ovs-vsctl set port qg-d8c27507-14 tag=25 makes things work, but the agent should do this, no?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/108494

[Cedric Brandily (cbrandily)]

If i understand correctly, setting in l3_agent.ini[1]:
  gateway_external_network_id =
  external_network_bridge =
responds to your need?

If so, the bug seems invalid

[1] https://github.com/openstack/neutron/blob/master/etc/l3_agent.ini#L30-L35

[Robert Collins (lifeless)]

l3 agent ini file.

[DEFAULT]
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

state_path = /var/run/neutron

auth_url = http://10.10.16.155:35357/v2.0
admin_tenant_name = service
admin_user = neutron
admin_password = xxxxxxxxxxxxxxxxxxxxxx
use_namespaces = True

ovs_set_vlan_tag = True

[Chuck Carlino (ccarlino)]

I'll investigate using external_network_bridge=

[Chuck Carlino (ccarlino)]

The recommended configuration appears to work. I can ping from a vm to 2 different external networks, each based on the same physical network with distinct vlan id. I used the packet counts of ovs-ofctl dump-flows and tcpdump to verify the packet flow was as expected.

I must note that I could not get floating ips to work consistently, but I believe it is some security groups issue with my configuration. So traffic initiated from the VM was used to verify connectivity.

stack@ds1:~$ cat ml2_conf.ini
[ml2]
tenant_network_types = vxlan,vlan
type_drivers = local,flat,vlan,gre,vxlan
mechanism_drivers = openvswitch,linuxbridge
[ml2_type_flat]
[ml2_type_vlan]
network_vlan_ranges = tenantsnet:500:600,externalnet1:1:100
[ml2_type_gre]
tunnel_id_ranges = 1:1000
[ml2_type_vxlan]
vni_ranges = 1001:2000
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
[ovs]
bridge_mappings = tenantsnet:br-eth1,externalnet1:br-eth2
enable_tunneling = True
local_ip = 192.168.122.131
[agent]
tunnel_types = vxlan
root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf

stack@ds1:~$ cat l3_agent.ini
[DEFAULT]
l3_agent_manager = neutron.agent.l3_agent.L3NATAgentWithStateReport
external_network_bridge =
gateway_external_network_id =
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
ovs_use_veth = False
root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
use_namespaces = True
debug = True
verbose = True

 stack@ds1:~$ neutron net-list
 +--------------------------------------+---------+----------------------------------------------------+
 | id | name | subnets |
 +--------------------------------------+---------+----------------------------------------------------+
 | 2cbadc7f-3b09-4307-9c6f-896952fdaaa4 | dnet1 | ff980428-78ec-4876-9f8b-45c40a92aa7d 10.3.0.0/24 |
 | 96185d8e-b3d8-4510-8077-952b390438a0 | private | ad32116d-3019-4f85-95fd-0b1338d59acd 10.4.128.0/24 |
 | bda25235-5f3a-4ac7-9ac7-5870d6bac09a | extnet1 | 2a3f9d6d-c8ce-47ef-8cae-aca7bb7a7491 10.1.0.0/24 |
 | f4b39b5a-b3d1-44ba-8488-aa4e7f9834d8 | extnet2 | 916a5243-161f-480c-b0a7-670cafd3d6c8 10.6.0.0/24 |
 +--------------------------------------+---------+----------------------------------------------------+

 stack@ds1:~$ neutron net-show extnet1 extnet2
 +---------------------------+--------------------------------------+
 | Field | Value |
 +---------------------------+--------------------------------------+
 | admin_state_up | True |
 | id | bda25235-5f3a-4ac7-9ac7-5870d6bac09a |
 | name | extnet1 |
 | provider:network_type | vlan |
 | provider:physical_network | externalnet1 |
 | provider:segmentation_id | 20 |
 | router:external | True |
 | shared | False ...

[Jian Wen (wenjianhn)]

Hello, Chuck

There is a problem.
A general user(not an admin) is able to create a network in physical_network externalnet1.
Because network_vlan_ranges are available for allocation as tenant networks.

I think we need a new feature(or a new cfg option) to reserve the VLAN ids in the external network.

[Jian Wen (wenjianhn)]

I figured out how to forbid a general user to create a network in physical_network externalnet1.
It's as simple as removing the trailing vlan ids from "externalnet1:1:100".
Instead of "network_vlan_ranges = tenantsnet:500:600,externalnet1:1:100"
using "network_vlan_ranges = tenantsnet:500:600,externalnet1".

After an admin delete a network outside of the tenantsnet vlan pool, the db
record is deleted instead of marked as not allocated. As a result, a general user
is not able to reuse the released vlan id to create a tenant network. That's is to say
only an admin is able to create and delete a network in the physical_network
 externalnet1.

[Cedric Brandily (cbrandily)]

Accordin to Chuck last comment #5, this bug is invalid

[Chuck Carlino (ccarlino)]

Thanks Cedric.

Jian, I'm not sure if the behavior you're seeing is a bug, but for sure it's not this bug.

[Jian Wen (wenjianhn)]

Chuck, agreed.
It's not a bug, it's a feature that I didn't know about.

[James Polley (tchaypo)]

Status reset to invalid; I *think* Jian Wen meant to say that the unrelated issue discussed in the middle of this bug is not a valid bug; but the issue originally raised by Rob does seem to be a bug.

[Chuck Carlino (ccarlino)]

James, is there new data around this? I thought we had resolved this by putting 'external_network_bridge=' in l3_agent.ini? See comment #5.

[James Polley (tchaypo)]

You're right, I misread the history.

I've been fighting other bugs, but I'm now up to confirming that "external_network_bridge=" does work for the scenario rob had in mind. I'll leave this invalid for now and re-open if anything comes up.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kyle Mestery (<email address hidden>) on branch: master
Review: https://review.openstack.org/108494
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.