[RFE][QoS] Add minimum guaranteed packet rate QoS rule

[Balazs Gibizer]

Quote from "[RFE] [QoS] add qos rule type packet per second (pps)" [1]:

For cloud providers, to limit the packet per second (pps) of VM NIC is popular and sometimes essential. Transit large set packets for VM in physical compute hosts will consume the CPU/phy-nic performance. And for small packets, even the bandwidth is low, the pps can still be higher, which can be an attact point inside the cloud while some VMs are becoming hacked.

--

Neutron already supports bandwidth_limit and minimum_bandwidth QoS rules.

So [1] proposes a packet rate limit QoS rule. With similar reasoning and aligning with the existing bandwidth rules providing minimum packet rate QoS rule could also make sense.

The new minimum_packet_rate rule has a similar structure and semantic as the minimum_bandwidth rule:

  * It defines a guaranteed minimum packet rate in kpps

  * It defines a direction (egress / ingress) to which direction the guarantee is applied.

  E.g.:

  POST /v2.0/qos/policies/{policy_id}/minimum_packet_rate_rules
  {
      "minimum_packet_rate_rule": {
          "min_kpps": 10000,
          "direction": "egress"
      }
  }

  * Ports with such QoS rule expected to be scheduled on compute nodes where the networking backend (typically OVS) still has enough packet processing capacity to fulfill the guarantee.

This RFE is focusing on supporting min guaranteed packet rate for OVS network backend only.

A new config option is introduced in the OVS agent configuration where the admin can define the available packet processing capacity of the OVS deployed on the given compute host. This information is sent to the neutron server in the agent hearth beat. The neutron server uses this information to create NET_KILOPACKET_PER_SEC resource inventory on the OVS Agent RP in Placement.

Note that the resource inventory is directionless, while the proposed QoS rule has direction. From scheduling perspective the direction of the QoS rule does not matter and both direction will be counted against the single directionless resource inventory. However for data plane enforcement directions should be handled separately.

Note that while bandwidth inventory is define per bridge / physical device the packet processing capacity is applied globally to the whole OVS instance.

A port that has such QoS policy rule needs to express the related NET_KILOPACKET_PER_SEC resource request in the port.resource_request attribute. As a single port can have both bandwidth and packet rate QoS applied and because bandwidth is allocated from the bridge / physical device while the packet rate is allocated from the whole OVS instance the two sets of resources need to be requested separately. (A deeper technical reason to this is that a single resource request group is always allocated from a single resource provider in Placement. So if bandwidth and packet rate needs to be allocated from different providers then they should be requested in different resource request groups.) To accommodate such separation the structure of the resource_request field of the neutron port needs to be changed from:

"resource_request":
{
    "required": ["CUSTOM_PHYSNET_PUBLIC", "CUSTOM_VNIC_TYPE_NORMAL"],
    "resources": {"NET_BW_EGR_KILOBIT_PER_SEC": 1000}
},

to:

"resource_request":
{
    {
        "name": <some port unique name, e.g. the policy rule id that requesting the resource>
        "required": [],
        "resources": {"NET_KILOPACKET_PER_SEC": 1000}
    },
    {
        "name": <some port unique name, e.g. the policy rule id that requesting the resource>
        "required": ["CUSTOM_PHYSNET_PUBLIC", "CUSTOM_VNIC_TYPE_NORMAL"],
        "resources": {"NET_BW_EGR_KILOBIT_PER_SEC": 1000}
    },
},

As a consequence the port bindig:profile.allocation key needs to be change too. Today it contains the single UUID of the resource provider the port's resources are allocated from. Now that a port can allocate from multiple providers this key needs to be transformed to a dict where the resource provider UUIDs are keyed by the resource_request.name value.

Enforcing the packet rate guarantees on the data plane is out of scope of this RFE. In the future a basic guarantee can be provided in the networking backend by re-using the data plane enforcement implementation of the packet_rate_limit rule from [1].

I will propose a nova-spec under the bp[2] that will define both the high level solution and the details of the nova impact. Also I will propose a neutron-spec defining the detailed impact on neutron.

[1] https://bugs.launchpad.net/neutron/+bug/1912460
[2] https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate

[LIU Yulong (dragon889)]

+1

We will have pps support, then we should add ability to support minimum guaranteed pps. Otherwise, each instances will break the limitation for no guarantee anymore.

[Balazs Gibizer (balazs-gibizer)]

@LIU: from the data plane enforcement point of view you are absolutely right, we need the pps limit first and based on that we can implement a basic form of minimum pps guarantee on the data plane side. However I would like to keep the data plane enforcement out of scope of this RFE so your pps limit and the scheduling part of the minimum pps can be worked on in parallel.

[Balazs Gibizer (balazs-gibizer)]

A WIP specification has been proposed https://review.opendev.org/c/openstack/neutron-specs/+/785236

[Slawek Kaplonski (slaweq)]

I would like to discuss it on next Neutron drivers meeting which will be on Friday 09.04.2021: http://eavesdrop.openstack.org/#Neutron_drivers_Meeting - so it would be great if You could join there if there would be any additional questions. But RFE should be discussed even if You will not be able to attend this meeting.

[Balazs Gibizer (balazs-gibizer)]

@Slaweq: Thanks for the drivers meeting slot. I will join to the discussion tomorrow.

[Slawek Kaplonski (slaweq)]

We discussed that RFE on the last drivers meeting http://eavesdrop.openstack.org/meetings/neutron_drivers/2021/neutron_drivers.2021-04-09-14.01.log.html#l-17 and we decided to approve it. Please now work on the spec and the implementation in Neutron.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-specs/+/785236
Committed: https://opendev.org/openstack/neutron-specs/commit/ceb39afd7dddd0322834e5f25be72c788744913b
Submitter: "Zuul (22348)"
Branch: master

commit ceb39afd7dddd0322834e5f25be72c788744913b
Author: Balazs Gibizer <email address hidden>
Date: Wed Apr 7 18:44:34 2021 +0200

    QoS minimum guaranteed packet rate

    Similarly to how bandwidth can be a limiting factor of a network
    interface, packet processing capacity tend to be a limiting factor
    of the soft switching solutions like OVS. In the same time certain
    applications are dependent on not just guaranteed bandwidth but also
    on guaranteed packet rate to function properly. OpenStack already
    supports bandwidth guarantees via the minimum bandwidth QoS policy
    rules. This specification is aiming for adding support for a similar
    minimum packet rate QoS policy rule.

    Related-Bug: 1922237
    Change-Id: Id17ee01dc288517a05f746a479500a6218ad55f4

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-specs/+/797877
Committed: https://opendev.org/openstack/neutron-specs/commit/6fbc6bd8a4156641d2e8335b3a4c15a2ee7819a2
Submitter: "Zuul (22348)"
Branch: master

commit 6fbc6bd8a4156641d2e8335b3a4c15a2ee7819a2
Author: Balazs Gibizer <email address hidden>
Date: Thu Jun 24 13:13:33 2021 +0200

    Fix up the pps spec config name

    There was a comment from Rodolfo to add the resource_provider_ prefix
    for the new packet processing config options but I forgot to add it to
    the packet_processing_inventory_defaults config name. Fixed it now so
    all the three new config option names have the same prefix.

    Related-Bug: 1922237
    Change-Id: I3018715e1265aff70002711a06b2bf5ee9ba53a5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/803045

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/803160

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/803160
Committed: https://opendev.org/openstack/neutron/commit/2356f0eb874b01d4bab696e49e0286de51554f35
Submitter: "Zuul (22348)"
Branch: master

commit 2356f0eb874b01d4bab696e49e0286de51554f35
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Aug 2 10:54:14 2021 +0200

    Bump oslo.log to version 4.5.0

    In this version versionutils were updated for Xena release.

    Related-bug: #1922237
    Change-Id: I89f52929cec067bbe55b183f5dcdbc64a92805de

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/802486
Committed: https://opendev.org/openstack/neutron-lib/commit/83b6727dad09f69ce05574babbb288204907e858
Submitter: "Zuul (22348)"
Branch: master

commit 83b6727dad09f69ce05574babbb288204907e858
Author: Przemyslaw Szczerbik <email address hidden>
Date: Thu Jul 15 08:49:43 2021 +0200

    Add API extension definition for QoS minimum pps rule

    This patch adds a new API extension definition for QoS minimum
    packet rate (pps) rule.

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ia03cebdd030dbfa489bccb3b104e5db71dac0aef

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/803905

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/803914

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/802487
Committed: https://opendev.org/openstack/neutron-lib/commit/ec29bdf065d9653d791c8dee933fe0f6fd1c43bc
Submitter: "Zuul (22348)"
Branch: master

commit ec29bdf065d9653d791c8dee933fe0f6fd1c43bc
Author: Przemyslaw Szczerbik <email address hidden>
Date: Fri Jul 16 12:28:40 2021 +0200

    Add FlowDirectionAndAnyEnumField common type

    This common type is going to be used by QoS min PPS, which supports
    not only '[e|i]gress' direction, but also 'any'.

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I8666db65ad337bd7b003453fce9ffbaa76a7d291

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/804203

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/804210

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/804210
Committed: https://opendev.org/openstack/neutron-lib/commit/77723c9370ea3a7b6fdbbaf8653a08b1894af938
Submitter: "Zuul (22348)"
Branch: master

commit 77723c9370ea3a7b6fdbbaf8653a08b1894af938
Author: Przemyslaw Szczerbik <email address hidden>
Date: Wed Aug 11 09:11:22 2021 +0200

    Rephrase QoSRuleParameterConflict message

    The name of QoSRuleParameterConflict exception class suggests that
    it's a generic exception that can be used by multiple QoS rules to
    raise a parameter conflict. On the other hand, exception's message
    refers to bandwidth which could be confusing when used with QoS
    minimum packet rate rule.

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I05d33fd3337b755b1b0b8a5986739c76a1db3d84

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Przemyslaw Szczerbik <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/804203
Reason: After a discussion with @Lajos, @Bence and @Balazs we agreed that additional extension is not needed. If port-resource-request-groups extension is supported, it already implies that Nova has to use a new format of 'allocation' key in the 'binding:profile'.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/803914
Committed: https://opendev.org/openstack/neutron-lib/commit/fb4775cf8362588775467f4f92e6418f379369db
Submitter: "Zuul (22348)"
Branch: master

commit fb4775cf8362588775467f4f92e6418f379369db
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Aug 9 14:07:40 2021 +0200

    Allow PUT on direction param in QoS minimum pps rule

    Based on discussion in the previous review [1] and IRC [2], it was
    decided to allow direction parameter in QoS minimum packet rate
    rule to be updated with a PUT request. This behavior matches current
    behavior of other QoS rules.

    If the rule to be updated is associated with a bound port, the request
    is going to be rejected by the server.

    Since direction-oriented and direction-less QoS min pps rules are
    mutually exclusive, the request is going to be rejected if it would
    result in mixing direction-oriented and direction-less rules in a single
    policy.

    [1] https://review.opendev.org/802486
    [2] https://meetings.opendev.org/irclogs/%23openstack-neutron/latest.log.html#t2021-08-09T07:56:39

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: If1f7b28184c740c30388f564b704ba42c8967527

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/803905
Committed: https://opendev.org/openstack/neutron-lib/commit/1d64353340f68ced2499787999d5f8e87092652a
Submitter: "Zuul (22348)"
Branch: master

commit 1d64353340f68ced2499787999d5f8e87092652a
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Aug 2 13:14:50 2021 +0200

    Add port-resource-request-groups shim API ext

    Add a new shim API extension definition called
    port-resource-request-groups. This extension indicates that
    Neutron supports the new format of resource_request, that allows
    to request multiple groups of resources and traits from the same RP
    subtree.

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Idbfa3998930374b8de1e65d586ab54f85707c713

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/805010

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/805010
Committed: https://opendev.org/openstack/neutron-lib/commit/794fc7032efca33c24e229d94f914be632911124
Submitter: "Zuul (22348)"
Branch: master

commit 794fc7032efca33c24e229d94f914be632911124
Author: Przemyslaw Szczerbik <email address hidden>
Date: Wed Aug 18 13:24:06 2021 +0200

    Add util to generate resource request group UUID

    Add a new utility function that can be used to generate a stable UUID
    for a resource request group.

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ic3718defd74c2165077bb5a942e46474d3582cf5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/805391

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/805395

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/805637

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/805395
Committed: https://opendev.org/openstack/neutron-lib/commit/e8b398ed6b09e733a7c228766cb6744868402432
Submitter: "Zuul (22348)"
Branch: master

commit e8b398ed6b09e733a7c228766cb6744868402432
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Fri Aug 20 14:29:45 2021 +0000

    Add missing default "direction" value to "qos-pps-minimum"

    All QoS rule types with "direction" parameter have "egress" as default
    value.

    Change-Id: Icd66a4ce29fd3bdf3610157006a42416bb12cd54
    Related-Bug: #1922237

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/801066
Committed: https://opendev.org/openstack/neutron/commit/31c09cd3b76d43a485002011df596a45d066eeb4
Submitter: "Zuul (22348)"
Branch: master

commit 31c09cd3b76d43a485002011df596a45d066eeb4
Author: Przemyslaw Szczerbik <email address hidden>
Date: Fri Jul 16 15:34:25 2021 +0200

    Bump os-resource-classes lib to 1.1.0

    Newer version provides access to packet rate related resource classes:

    * NET_PACKET_RATE_KILOPACKET_PER_SEC
    * NET_PACKET_RATE_EGR_KILOPACKET_PER_SEC
    * NET_PACKET_RATE_IGR_KILOPACKET_PER_SEC

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I505911412fd903d351f39ebb733ef8e872cb16fb

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/806689

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/809202

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/811411

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/803045
Committed: https://opendev.org/openstack/neutron/commit/56044db26d97c65c7baec1a16623eff3a1614949
Submitter: "Zuul (22348)"
Branch: master

commit 56044db26d97c65c7baec1a16623eff3a1614949
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Jul 12 09:02:02 2021 +0200

    Add API extension for QoS minimum pps rule

    This patch implements support for CRUD operations for QoS minimum
    packet rate, for example:

    DELETE /qos/policies/$POLICY_ID/minimum_packet_rate_rules/$RULE_ID

    Placement or dataplane enforcement is not implemented yet.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ie994bdab62bab33737f25287e568519c782dea9a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/800444
Committed: https://opendev.org/openstack/neutron/commit/1ea26616b41335b24b9c162062740ee3de78372d
Submitter: "Zuul (22348)"
Branch: master

commit 1ea26616b41335b24b9c162062740ee3de78372d
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Jun 29 09:11:45 2021 +0200

    ovs-agent: Report pkt processing info in heartbeat

    OVS agent configuration is extended to support new configuration
    options:
      - 'resource_provider_packet_processing_without_direction'
      - 'resource_provider_packet_processing_with_direction'
      - 'resource_provider_packet_processing_inventory_defaults'

    OVS agent RPC hearthbeat now reports this information to neutron
    server in 'configuration' field .

    Example config:

    ml2_conf.ini:
    [ovs]
    resource_provider_packet_processing_with_direction = :1000:1000

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ief554bc445dfd93ea6995bb42b4d010674c7a091

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/813650

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/813660

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/806689
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/814361

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/800445
Committed: https://opendev.org/openstack/neutron/commit/76a99f9a1458b4691a94f9508f56d4f05f500df9
Submitter: "Zuul (22348)"
Branch: master

commit 76a99f9a1458b4691a94f9508f56d4f05f500df9
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Jul 5 16:19:53 2021 +0200

    Report CUSTOM_VNIC_TYPE_ traits on Neutron agent RP

    Create a resource inventory on Neutron agent RP and report the same
    'CUSTOM_VNIC_TYPE_' traits on it, as the ones reported on the bridge
    RPs. These are the vnic types this agent configured to support.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I8ae6d2714474d0ca32441d07d7b4dc84571d14a6

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/800446
Committed: https://opendev.org/openstack/neutron/commit/c2bc676183a202743fa80f9cb0271af818958441
Submitter: "Zuul (22348)"
Branch: master

commit c2bc676183a202743fa80f9cb0271af818958441
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Jul 6 12:24:59 2021 +0200

    Report pkt processing capacity on Neutron agent RP

    Report the packet processing capacity on the Neutron agent resource
    provider to Placement as the new 'NET_PACKET_RATE_KILOPACKET_PER_SEC'
    or 'NET_PACKET_RATE_[E|I]GR_KILOPACKET_PER_SEC' resource inventory.
    This is similar to how the bandwidth resource is reported today.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I8deefbeed4b4b51dd20062df62c8891fee3ebf9d

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/815116

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/815120

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/813650
Committed: https://opendev.org/openstack/neutron-lib/commit/65e188eebd23e9d3527f94a84fcac8ee42a1104a
Submitter: "Zuul (22348)"
Branch: master

commit 65e188eebd23e9d3527f94a84fcac8ee42a1104a
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Sep 27 11:50:53 2021 +0200

    Add binding-profile.allocation converter

    With the introduction of port-resource-request-groups extension,
    format of binding-profile.allocation has changed. This patch
    adds a converter that allows to upgrade binding-profile.allocation
    to the new format.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I1b090a0d7a9639aa03f45c6fbd0efea9420ff71f

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/815959

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Przemyslaw Szczerbik <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/815959
Reason: Duplicate

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/805637
Committed: https://opendev.org/openstack/neutron/commit/8db15cb2f3dec37441103df15ff28c7e72de682c
Submitter: "Zuul (22348)"
Branch: master

commit 8db15cb2f3dec37441103df15ff28c7e72de682c
Author: Przemyslaw Szczerbik <email address hidden>
Date: Thu Aug 5 13:48:50 2021 +0200

    Add port-resource-request-groups extension

    port-resource-request-groups extension provides support for the
    new format of resource_request. The new format allows to request
    multiple groups of resources and traits from the same RP subtree.

    Closes-Bug: #1943724
    Partial-Bug: #1922237
    Depends-On: https://review.opendev.org/c/openstack/tempest/+/809168/
    See-Also: https://review.opendev.org/785236
    Change-Id: I99a49b107b1872ddf83d1d8497a26a8d728feb07

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/816306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/815116
Committed: https://opendev.org/openstack/neutron-lib/commit/44de25534daf244ea1d7eb8ef372e1ce3bd687a1
Submitter: "Zuul (22348)"
Branch: master

commit 44de25534daf244ea1d7eb8ef372e1ce3bd687a1
Author: Przemyslaw Szczerbik <email address hidden>
Date: Fri Oct 22 12:34:20 2021 +0200

    Add qos-pps-minimum-rule-alias api-def and api-ref

    Introduce a new API extension definition to enable GET, PUT
    and DELETE operations on QoS minimum packet rate rule without
    specifying policy ID.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/78523
    Change-Id: I1a708cf2935cb10a09c9088abc94bab1a2248157

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/816306
Committed: https://opendev.org/openstack/neutron-lib/commit/63869951ea5fddbb7dc60115c79c45c7b9786941
Submitter: "Zuul (22348)"
Branch: master

commit 63869951ea5fddbb7dc60115c79c45c7b9786941
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Sep 14 15:06:38 2021 +0200

    Move QosPlacementAllocationUpdateConflict to neutron-lib

    Addresses TODO comment from [1].

    [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/exceptions/qos.py#L20

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I0a0f37b671b7c8d41677f44808e10be767904c51

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-lib/+/816447

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/816447
Committed: https://opendev.org/openstack/neutron-lib/commit/6b2fe94b2ccb49a7906fbda8cfe48892cf37833c
Submitter: "Zuul (22348)"
Branch: master

commit 6b2fe94b2ccb49a7906fbda8cfe48892cf37833c
Author: Przemyslaw Szczerbik <email address hidden>
Date: Wed Nov 3 11:18:00 2021 +0100

    Move RULE_TYPE_MINIMUM_PACKET_RATE to neutron-lib

    Addresses TODO comment from:

    neutron/services/qos/constants.py

    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ie6fffb7829ac1f212f4c4e40fc2e5161f2ccd6fe

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817356

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817357

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817358

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/811411
Committed: https://opendev.org/openstack/neutron/commit/d699a955cd55a6401a787630d37f04c5691eea78
Submitter: "Zuul (22348)"
Branch: master

commit d699a955cd55a6401a787630d37f04c5691eea78
Author: Przemyslaw Szczerbik <email address hidden>
Date: Fri Sep 24 13:39:15 2021 +0200

    Sanitize profile column of ml2_port_bindings table in the DB

    With the introduction of port-resource-request-groups extension,
    format of binding-profile.allocation has changed. Since the DB,
    may contain port bindings that were created before the introduction
    of the new format, it's necessary to perform upgrade check and
    sanitize those rows that are still using an older format.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I95e9e1bc553ac499d75c9280e45dfea61d135279

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/805391
Committed: https://opendev.org/openstack/neutron/commit/aada855f6dc4e0e8a6d087b47382fc4e30ac5dc5
Submitter: "Zuul (22348)"
Branch: master

commit aada855f6dc4e0e8a6d087b47382fc4e30ac5dc5
Author: Przemyslaw Szczerbik <email address hidden>
Date: Wed Aug 18 09:12:05 2021 +0200

    Enable QoS minimum packet rate rule for OVS backend

    This patch does *not* implement dataplane enforcement.

    QoS minimum packet rate rule is enabled in OVS backend driver and
    create/delete/update empty methods are added to enable placement
    enforcement.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ie283ad3a4ec433c88ac23f798908cd143159394b

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/818067

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814361
Committed: https://opendev.org/openstack/neutron/commit/77b3a1a7749341ff4cd8b809edf84459f95fe276
Submitter: "Zuul (22348)"
Branch: master

commit 77b3a1a7749341ff4cd8b809edf84459f95fe276
Author: Przemyslaw Szczerbik <email address hidden>
Date: Thu Sep 23 16:08:17 2021 +0200

    Networking guide: Add Guaranteed Minimum Packet Rate

    Add Neutron doc for the Guaranteed Minimum Packet Rate feature.

    Co-Authored-By: Balazs Gibizer <email address hidden>
    Co-Authored-By: Bence Romsics <email address hidden>

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: I8bbd503488ae0997c0e5f33e0ef7709841c310b5

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/813660
Committed: https://opendev.org/openstack/neutron-lib/commit/d2bd7760efdef2c47ae67923aec37f914f8400ce
Submitter: "Zuul (22348)"
Branch: master

commit d2bd7760efdef2c47ae67923aec37f914f8400ce
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Sep 7 15:37:07 2021 +0200

    Make update_qos_minbw_allocation() more generic

    update_qos_minbw_allocation() function is used to update Placement
    allocation. Initially, only minimum bandwidth rule allocated resources
    in Placement, but with the introduction of minimum packet rate rule
    that's changed. Because of that, we should rename the function to make
    it more generic and to avoid confusion.

    The second problem with this function is that it allows to update
    resources only of a single RP at a time, even if multiple RPs are
    associated with the same consumer UUID. With introduction of a
    minimum packet rate rule it makes sense to allow to update resources of
    multiple RPs in a single API call. To accommodate this, we need to
    slightly modify arguments this function takes, and embed RP UUID
    in alloc_diff, rather than pass it as a separate parameter.

    Addresses TODO comment from [1].

    [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/services/qos/qos_plugin.py#L68

    Partial-Bug: #1943724
    Related-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ie28b95e8ed351ab88db1fc75c83a02c474582e0b

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/818067
Committed: https://opendev.org/openstack/neutron/commit/4909c8c18dc13a19d7760ec27c4c61de508a188a
Submitter: "Zuul (22348)"
Branch: master

commit 4909c8c18dc13a19d7760ec27c4c61de508a188a
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Nov 16 11:26:21 2021 +0000

    Bump neutron-lib to 2.17.0

    Remove the QoS constants from Neutron code. QoS constants are now
    located in ``neutron_lib.services.qos.constants``.

    This patch also reverts [1]. This patch was merged in order to
    allow a newer neutron-lib release in "requirements". This test
    was failing because the element order of the "VALID_RULE_TYPES"
    list was different between Neutron and neutron-lib. That was
    modifying the "QosRuleType" OVO hash.

    [1]https://review.opendev.org/c/openstack/neutron/+/817940

    Closes-Bug: #1950977
    Related-Bug: #1922237
    Change-Id: I31edea3cc0f4a284a773a35302997ca6069efc95

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-specs/+/809202
Committed: https://opendev.org/openstack/neutron-specs/commit/8f136023f52887e67c0a0d43e2a309c9c41f3c56
Submitter: "Zuul (22348)"
Branch: master

commit 8f136023f52887e67c0a0d43e2a309c9c41f3c56
Author: Balazs Gibizer <email address hidden>
Date: Wed Sep 15 16:27:01 2021 +0200

    Move and extend qos-minimum-guaranteed-packet-rate spec

    The implementation of the qos-minimum-guaranteed-packet-rate spec did
    not land in Xena so this patch moves it to the Yoga folder. Also it
    extends the space based on findings during the implementation work.

    Related-Bug: 1922237
    Change-Id: I5ed384be490ad3962e274ad94ce28060bd6fe96b

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/820363

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/820363
Committed: https://opendev.org/openstack/neutron/commit/6288dc7259a9032ef528c364f419b1a079a19cd9
Submitter: "Zuul (22348)"
Branch: master

commit 6288dc7259a9032ef528c364f419b1a079a19cd9
Author: elajkat <email address hidden>
Date: Fri Dec 3 10:49:48 2021 +0100

    Keep binding:profile keys during placement allocation change

    [1] overwrites the port's binding:profile with the allocation dict,but
    that dict can contain other fields also.

    [1]: https://review.opendev.org/c/openstack/neutron/+/805637#617

    Change-Id: I687686a673979cd9b95bac7282e836a6435f0521
    Partial-Bug: #1922237

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817357
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/9b8be88438df22a56a130dc05d50a90e1c24b311
Submitter: "Zuul (22348)"
Branch: master

commit 9b8be88438df22a56a130dc05d50a90e1c24b311
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Nov 9 11:58:44 2021 +0100

    Update PortTestCasesResourceRequest for new Neutron API ext

    The port-resource-request-groups Neutron API extension, introduced
    in I99a49b107b1872ddf83d1d8497a26a8d728feb07, changed format of
    port.resource_request attribute. This patch adaps
    PortTestCasesResourceRequest tests to accept both formats.

    Partial-Bug: #1922237
    Change-Id: I55b52e360ee99174d804c9bf25b5ccfdc959a1ed

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/806689
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/4a1357b86d9ea50459c0a73f1b59b7aedc39009c
Submitter: "Zuul (22348)"
Branch: master

commit 4a1357b86d9ea50459c0a73f1b59b7aedc39009c
Author: elajkat <email address hidden>
Date: Tue Aug 31 10:16:25 2021 +0200

    QoS min pps API tests

    Co-Authored-By: Przemyslaw Szczerbik <email address hidden>
    Partial-Bug: #1922237
    Change-Id: I39f521a11de4e9ff4d447d7a6e85f9cbee6ee62a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/815120
Committed: https://opendev.org/openstack/neutron/commit/084bb163f2427d9ba260dc376b1c0b831387e503
Submitter: "Zuul (22348)"
Branch: master

commit 084bb163f2427d9ba260dc376b1c0b831387e503
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Oct 19 09:42:03 2021 +0200

    Add qos-pps-minimum-rule-alias API extension

    Introduce a new API extension to enable GET, PUT and DELETE
    operations on QoS minimum packet rate rule without specifying
    policy ID.

    Partial-Bug: #1922237
    See-Also: https://review.opendev.org/785236
    Change-Id: Ia083b5ac98c9e18ddbcdd2e0fc46f2f8432a628c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817356
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/1f3d6b92783aa20bdfe751ec64c9699e1ac61dfd
Submitter: "Zuul (22348)"
Branch: master

commit 1f3d6b92783aa20bdfe751ec64c9699e1ac61dfd
Author: Przemyslaw Szczerbik <email address hidden>
Date: Tue Nov 9 11:13:56 2021 +0100

    Fix test_port_resource_request_inherited_policy test

    In 'test_port_resource_request_inherited_policy' test a QoS policy
    is assigned to a network. The network used in this test is created
    in resource_setup(), which is run as part of setUpClass() phase.
    Resources allocated in resource_setup() are shared by all tests
    in the class, and are cleaned up in tearDownClass() only after all
    test had been run. This means that
    test_port_resource_request_inherited_policy test leaks network QoS
    policy, that can impact all subsequent tests. Depending on the
    order in which tests are run, this can break tests that are
    checking port's resource_request attribute.

    To avoid affecting other tests, create a dedicated provider network
    in test_port_resource_request_inherited_policy test.

    Related-Bug: #1922237
    Change-Id: Id8758ef4fe56fe7e0e7792270d6ee585313592a6

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/817358
Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/d2ecabb4fc1f35aa6374260f8b89cea0c467534b
Submitter: "Zuul (22348)"
Branch: master

commit d2ecabb4fc1f35aa6374260f8b89cea0c467534b
Author: Przemyslaw Szczerbik <email address hidden>
Date: Mon Nov 8 15:25:02 2021 +0100

    Test port.resource_request format with min bw and min pps rules

    Add a new test to verify port.resource_request format when associated
    QoS policy contains both minimum bandwidth and minimum packet rate
    rules.

    Partial-Bug: #1922237
    Change-Id: Ia5aa134a3b2a254fdb9c111eac95e0eb5ea87217

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron-lib/+/832811

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron-lib/+/832812

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron-lib/+/832812
Reason: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/832911

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron-lib/+/832811
Reason: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/832911

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/841118

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/841118
Committed: https://opendev.org/openstack/neutron/commit/701900ac3392af98add27ffd16e3d62956a1258c
Submitter: "Zuul (22348)"
Branch: master

commit 701900ac3392af98add27ffd16e3d62956a1258c
Author: elajkat <email address hidden>
Date: Mon May 9 15:18:31 2022 +0200

    FUP: remove convert_to_sanitized_binding_profile_allocation

    convert_to_sanitized_binding_profile_allocation was added to Neutron
    temporarily before [1] was merged and released in neutron-lib.

    [1]: https://review.opendev.org/c/openstack/neutron-lib/+/813650
    Related-Bug: #1922237

    Change-Id: I953b96d97076cd6a80fff6e97e2fd956da737d46

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-openstackclient 6.0.0 release.