[RFE] There should be a way to set ethertype for the vlan_transparent networks

[Slawek Kaplonski]

Networks with enabled vlan_transparency can be any of any type available in Neutron (vxlan, vlan, flat in case of Linuxbridge, etc.). In most cases it is fine when packets in such case have ethertype 0x8100 (802.1q).
The same ethertype is configured always for vlan network with vlan_transparent=True. But in such case as we have 2 vlans already (S-Tag and C-Tag) it is really QinQ. Neutron still configures ethertype to be 0x8100 in such case and it works for most of the cases.
But in case of some hardware it needs to be 0x8a88 (802.1ad) which is real QinQ type.

So I think that it would be good to add operators possibility to configure what ethertype will be used in such case.
It could be simple config option and all vlan networks with enabled vlan transparency would use it.
Default value of that option would be 0x8100 to be compatible with what we have now.

[sean mooney (sean-k-mooney)]

i dont really think is is valid.

vlan transparancy and QinQ transparnace are two different things
so really i think you should have two seperate extentions.

if you were to extend it to QinQ then vlan_transparent=qinq or vlan_transparent=0x8a88 might be valid

the current way that neutron extension are reported is also kind of problematic.

neutron currently reports all available extension not just enabled extensions so you cant as an end user determine if the cloud support an extension by look at the extension list.
you have to actually try to use it which can lead to very strange behaviour due to how the correct port create logic works. on in this case network create.

the response form the api is not guaranteed to match the value in the db depending on how the extension was implemented.

im not sure exposing the Ethernet type as something the operator set is really what we want to do. it really should be configured by the tenant. the operator might want a value to say which ones are allowed but i think it would be better for each ml2 driver to report what it supports and fail to bind ports to then network if it can fullfil that requirement.

it really sucks to get the failure that late but i think that the best approach we could do today.
if the operator has an api level config value like the allowed vnic_types to specify allowed Ethernet types you could reject it at the api level instead.

alternately if you have a separate qinq extension and you only report it if the extension driver is listed then operators can control enabling it via the extension driver config option in the ml2_conf.ini and end-user can detect it by looking at the extension list. this also allows individual ml2 drivers to opt into support.

i think ^ is a much better approach as all the other approach that extend the existing extension are config driver api behaviour that causes interoperate issues between clouds.

e.g. how would you detect if two cloud both support QinQ if you extend the vlan transparency extension instead of adding a new QinQ transparency extension. as an end user you cannot.

[Slawek Kaplonski (slaweq)]

Thx Sean for Your input,

I agree that qinq is really different thing than vlan transparency.
So IIUC Your proposal is to do new extension which will be similar to vlan_transparency and will work only for vlan networks. So user can then do something like:

neutron net-create qinq-network --qinq True

And in such case ethtype for such network would be 0x8a88 instead of 0x8100 and user would be able to control directly in VM C-Tag. S-Tag would be still controlled by Neutron, as it is now for vlan networks.
If my understanding is correct, I like that solution :) We can move in that direction IMO.

[sean mooney (sean-k-mooney)]

yep exactly add a new extension for qinq
i don't know if you want to also allow the user to control the s-tag if they requested both?

im not sure if the use case is for the tenant to be able to send fully encapulated qinq packets where they have contoler of both or just to use QinQ as an overly.

there have been seperate proposal for added a QinQ segmenation type in the past which to me was a different usecase. in the qinq segmenation type it was a way to have neutron contol both the S-tag and C-tag so that you increase teh number of network you could support on a phsynet without needing to use l3 tunnel encapsulation.

my understanding of the vlan_transparent extenion is that vlan transparent network allow user to send singly tagged vlan packets. transitivly i would have infered a qinq_transparent network would have allowed tenants to send double tagged packets contoling both the S and C tags.
neutron would then have layered another level of encaplolation over that for tenant isolation based on the segmentation type of the neutron network.

does that make sense?

having neutron conrtol the S-tag basically shoudl very similar to the current vlan-aware-vms feature aka trunk ports so for me i think it add much less value then instead allowing the tenatn to set both tags.

[sean mooney (sean-k-mooney)]

by the way both are valid e.g. tenant control of S-tag or neutron contol. it just depend on the usecase you wanted to enable. i dont really object to either approch although i would probly lean towards tenat contol to make this feature sufficently different form the exsiting vlan transparend networks and vlan aware vms features.

[Slawek Kaplonski (slaweq)]

Hi Sean,

Yes, this RFE is exactly about use case as You described in comment #3 - so Neutron will control S-Tag, in similar way like it is now in vlan networks and user will be able to control C-Tag on their own in the VM.
So from VM PoV it will be single tagged packet but neutron will add another vlan id for it. That is use case which we currently have and wants to address :)

According to the Akihiro's question from last drivers meeting I did some comparison about how ethertypes looks currently in case of trunks and network with vlan_transparent=True.

- vlan_transparent = True, packet captured on the compute node looks like:

fa:16:3e:80:36:be > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 50: vlan 10, p 0, ethertype 802.1Q, vlan 100, p 0, ethertype ARP, Request who-has 192.168.100.92 tell 192.168.100.94, length 28

So double 802.1Q as ethertype.

- trunk

16:13:36.793004 fa:16:3e:90:ee:fd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 50: vlan 10, p 0, ethertype 802.1Q, vlan 1500, p 0, ethertype ARP, Request who-has 192.168.50.223 tell 192.168.50.117, length 28

So again, 802.1Q as ethertype.

This is all based on the linuxbridge agent.
I then changed eth1.10 interface to be 0x8a88 like this RFE actually propose and packet looked like:

15:58:05.373213 fa:16:3e:80:36:be > ff:ff:ff:ff:ff:ff, ethertype 802.1Q-QinQ (0x88a8), length 50: vlan 10, p 0, ethertype 802.1Q, vlan 100, p 0, ethertype ARP, Request who-has 192.168.100.92 tell 192.168.100.94, length 28

As You can see, "outer" ethertype is 0x88a8 and "inner" ethertype is still 802.1Q.

And that is basically whole this RFE about. To be able to change this "outer" ethertype to QinQ.
Does it makes sense for You?

[Slawek Kaplonski (slaweq)]

On our last drivers meeting we discussed that RFE: http://eavesdrop.openstack.org/meetings/neutron_drivers/2021/neutron_drivers.2021-02-19-14.00.log.html#l-15 and we decided to approve it.

[Slawek Kaplonski (slaweq)]

Ohh, I forgot to mention. Next step is to propose spec for it.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/798704

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-specs/+/798704
Committed: https://opendev.org/openstack/neutron-specs/commit/40f6c1663d045fbb23dcb64f4fc0227f6f063dd4
Submitter: "Zuul (22348)"
Branch: master

commit 40f6c1663d045fbb23dcb64f4fc0227f6f063dd4
Author: Slawek Kaplonski <email address hidden>
Date: Tue Jun 29 17:46:46 2021 +0200

    Neutron VLAN networks with QinQ enabled

    Related-Bug: #1915151
    Change-Id: I4ebac838df82a64a5a6f2c773d69050ade5668ce

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/936841

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/936841
Committed: https://opendev.org/openstack/neutron-lib/commit/8396d75b31ea4ac4622de1b64b77ab8dfb8ac2e1
Submitter: "Zuul (22348)"
Branch: master

commit 8396d75b31ea4ac4622de1b64b77ab8dfb8ac2e1
Author: Slawek Kaplonski <email address hidden>
Date: Mon Dec 2 15:51:09 2024 +0100

    Add definition of the 'qinq' api extension

    This new extension allows to enable QinQ VLAN type for the networks.

    Related-bug: #1915151
    Change-Id: If6436c048bf51d503c4f433a900b67e9945965ac

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/937543

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/937603

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/937633

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/937543
Committed: https://opendev.org/openstack/neutron-lib/commit/238e94f11dedc2543241900b1ed7284824d0be8c
Submitter: "Zuul (22348)"
Branch: master

commit 238e94f11dedc2543241900b1ed7284824d0be8c
Author: Slawek Kaplonski <email address hidden>
Date: Wed Dec 11 16:48:17 2024 +0100

    Add VlanQinqDriverError exception

    This is follow up for the [1] which introduced API definition for the
    network's ``qinq`` attribute.

    [1] https://review.opendev.org/c/openstack/neutron-lib/+/936841

    Related-bug: #1915151
    Change-Id: I902aac9a2231c84bd8be2d16730397f1657d8ae5

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/937778

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/937603
Committed: https://opendev.org/openstack/neutron-lib/commit/6ede4989f0e118ceb6d250646cbeedeea7f28d26
Submitter: "Zuul (22348)"
Branch: master

commit 6ede4989f0e118ceb6d250646cbeedeea7f28d26
Author: Slawek Kaplonski <email address hidden>
Date: Thu Dec 12 10:10:59 2024 +0100

    Add check_vlan_qinq method to the ML2 MechanismDriver base class

    This is follow up for the [1] and introduces base method to check if
    mechanism driver supports vlan QinQ for the specific network.

    [1] https://review.opendev.org/c/openstack/neutron-lib/+/936841

    Related-bug: #1915151
    Change-Id: Ic8713df93e42fb505bc18eda5a8bd92c1b6f9c12

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/939708

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/939708
Committed: https://opendev.org/openstack/neutron-lib/commit/4b30a94c883ebddeb38b097ca050e472a0e6d487
Submitter: "Zuul (22348)"
Branch: master

commit 4b30a94c883ebddeb38b097ca050e472a0e6d487
Author: Slawek Kaplonski <email address hidden>
Date: Tue Jan 21 12:43:24 2025 +0100

    [API-REF] Add clarification about QinQ and transparent vlan differences

    Related-bug: #1915151
    Change-Id: I1e3865345fc5f4aaeaa96fcd7cf24e528317fccc

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/937372
Committed: https://opendev.org/openstack/neutron/commit/e20ef3fa86cd61c20559441fb47eba62515885f2
Submitter: "Zuul (22348)"
Branch: master

commit e20ef3fa86cd61c20559441fb47eba62515885f2
Author: Slawek Kaplonski <email address hidden>
Date: Mon Dec 9 15:13:45 2024 +0100

    QinQ API extension implementation in the Neutron server

    This patch implements 'qinq' API extension in the Neutron server. It
    means that this new attribute is now added for the "vlan" networks to
    the network dict and returned through the API.
    This patch also adds validation that both "vlan_transparent" and "qinq"
    aren't enabled for the same network at the same time as this is not
    supported.
    This patch adds all the necessary bits to store value of the new
    attribute in the Neutron DB and to add support for it to the Network
    OVO.
    Finally it also adds check_vlan_qinq() method to all mechanism drivers
    which are in-tree. For now all of them declare that QinQ vlans are not
    supported.

    Related-Bug: #1915151
    Change-Id: I427edfd580eb06aa4f6904f90ff28cf8b5267397

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/937633
Committed: https://opendev.org/openstack/neutron/commit/8cf622d83dceedc20a0fae66d790380823e43676
Submitter: "Zuul (22348)"
Branch: master

commit 8cf622d83dceedc20a0fae66d790380823e43676
Author: Slawek Kaplonski <email address hidden>
Date: Thu Dec 12 15:56:36 2024 +0100

    QinQ implementation for the ML2/OVN backend

    This patch implements support for the 'vlan_qinq' network parameter in
    the ML2/OVN backend. It is done in almost similar way to the
    'vlan_transparent' parameter, the difference is in the 'ethtype' set for
    the provnet port for the network. For QinQ it is set to '802.1ad'.

    It also adds functional tests for the 'vlan_transparent' setting for the
    OVN mechanism driver. The reason why those 2 are tested together is that
    both are using the same options on the OVN side and are mutually
    exclusive so we have to make sure we set those options as expected in
    each case.

    Related-Bug: #1915151
    Change-Id: I110c366a37a65d625083a7112f1adb9a3dc5e7cc

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/937778
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/d4c707e68d488885f4164b015d7adc797a151851
Submitter: "Zuul (22348)"
Branch: master

commit d4c707e68d488885f4164b015d7adc797a151851
Author: Slawek Kaplonski <email address hidden>
Date: Mon Dec 16 14:46:16 2024 +0100

    Add scenario tests for the Vlan QinQ feature

    This patch adds 2 new scenario tests to test connectivity between VMs
    using vlan network with enabled QinQ and with vlan configured inside
    VMs.
    This is very similar to the "vlan_transparent" option which was already
    tested since some time. The only difference is in the ethtype of the
    outer vlan packets send through such network. Unfortunately we can't
    test ethtype of the outer vlan in the neutron-tempest-plugin tests as
    that would require access to the host where VMs runs and test packets on
    the physical NICs.
    So those new tests are actually the same as the existing tests for the
    network with "vlan_transparent" enabled to at least make sure that QinQ
    functionality works from the end user point of view.

    This patch also enables those new tests for the ML2/OVN job as this is
    currently only backend with implemented support for QinQ.

    Depends-On: https://review.opendev.org/c/openstack/neutron/+/937633
    Related-Bug: #1915151
    Change-Id: Icc2b175cf4cfe5c4ca73bb7ccc5a9089a27798c9

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/945556

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/945556
Committed: https://opendev.org/openstack/neutron/commit/6351f9012c1d0c34fbddb1622b7e871ea3e7c668
Submitter: "Zuul (22348)"
Branch: master

commit 6351f9012c1d0c34fbddb1622b7e871ea3e7c668
Author: Slawek Kaplonski <email address hidden>
Date: Wed Mar 26 12:14:42 2025 +0100

    Implement check_vlan_qinq to return True in SR-IOV mech driver

    As the SR-IOV mechanism driver does not impact VLAN QinQ
    support, this patch overrides the check_vlan_qinq method
    from the MechanismDriver base class and returns 'True'. In this
    way, inclusion of the SR-IOV mechanism driver in the list of
    configured mechanism_drivers does not prevent the creation of
    networks with enabled vlan QinQ.

    Additionally this patch adds comment with explanation why
    "check_vlan_transparency" method is also returning `True` always.

    Related-Bug: #1915151
    Change-Id: I3a237a57b10b6e9df4dd698f6cd7e61c7dba5be3

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2025.1
Review: https://review.opendev.org/c/openstack/neutron/+/945681

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/945681
Committed: https://opendev.org/openstack/neutron/commit/06822ede3b401b4bb5e91220d8b77b6fae0af793
Submitter: "Zuul (22348)"
Branch: stable/2025.1

commit 06822ede3b401b4bb5e91220d8b77b6fae0af793
Author: Slawek Kaplonski <email address hidden>
Date: Wed Mar 26 12:14:42 2025 +0100

    Implement check_vlan_qinq to return True in SR-IOV mech driver

    As the SR-IOV mechanism driver does not impact VLAN QinQ
    support, this patch overrides the check_vlan_qinq method
    from the MechanismDriver base class and returns 'True'. In this
    way, inclusion of the SR-IOV mechanism driver in the list of
    configured mechanism_drivers does not prevent the creation of
    networks with enabled vlan QinQ.

    Additionally this patch adds comment with explanation why
    "check_vlan_transparency" method is also returning `True` always.

    Related-Bug: #1915151
    Change-Id: I3a237a57b10b6e9df4dd698f6cd7e61c7dba5be3
    (cherry picked from commit 6351f9012c1d0c34fbddb1622b7e871ea3e7c668)