Succeed to create new security group, even if security group rule quota is exceeded

[Rodolfo Alonso]

Description of problem:
Even if the quota for SG rules is exhausted, new security groups (with two default rules by default) can be created successfully.

How to reproduce:
  OS_PROJECT_NAME=admin
  CREATED=`openstack security group list --project $OS_PROJECT_NAME -f \
    json | jq -r '.[] | .ID' | xargs -I {} openstack security \
    group rule list {} -f value | wc -l`
  let "CREATED +=1"
  SG=`openstack security group list --project $OS_PROJECT_NAME -f json \
    | jq -r '.[0] | .ID'`
  QUOTA=`openstack quota show $OS_PROJECT_NAME -f json | jq -r '. \
    "secgroup-rules"]'`

  for ((i=CREATED; i<=QUOTA; i++)); do
    PORT=`printf "%04d" $i`
    openstack security group rule create --ingress --protocol tcp \
      --dst-port 5$PORT:5$PORT $SG
  done

  openstack security group create --project $OS_PROJECT_NAME \
    sec_group_with_excess_rules

Actual results:
The number of SG rules after the last command exceeds in 2 the maximum quota assigned for SG rules.

Related bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1787933

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

First of all, this is happening when the SG "is_default" flag is True. Then, when a SG group is created, two new rules (ingress and egress) are created.

The problem I found was the way the quota is enforced (actually when the quota is checked). When an API call arrives, "api.v2.resource.Resource.resource" is called. In this case, the action used is "create" and it will call "api.v2.base.Controller.create" and then "api.v2.base.Controller._create". This method will check the quota for the resource being created [1].

The problem here is that when a SG rule is created, the resource checked is "security_group_rule" and the related quota is enforced. But when a new SG group is created, the resource checked is "security_group", not "security_group_rule".

We can:
- In the resource check (same as the current code), add some kind of logic to handle resources and sub resources. But in this case we don't know if the SG is going to need those two new SG rules.
- Add a quota reservation check in the plugin, when the DB registers are created. This will delay the command failure and will generate a DB rollback to delete the SG register, but will know exactly if the SG rules are needed or not.

Regards.

[1] https://github.com/openstack/neutron/blob/ac63c570a1c630ac4405e4caf3d516d069165d69/neutron/api/v2/base.py#L481-L487

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/701565

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/701565
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=936bd67aa42b2e06241d309315b895c9c9c49dcc
Submitter: Zuul
Branch: master

commit 936bd67aa42b2e06241d309315b895c9c9c49dcc
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jan 8 13:31:29 2020 +0000

    Check "security_group_rule" quota during security group creation

    The tracked resources quota check is done at the beginning of an API
    call to the Neutron server. The API call contains a resource and an
    action over the resource. In case of creation, the server checks if
    the number of items requested fits in the existing quota.

    In case of security group creation, the tracked resource checked is
    "security_group". But "SecurityGroupDbMixin.create_security_group"
    method also creates several default rules for the new group and the
    quota for "security_group_rule" is not enforced.

    This patch checks the number of security group rules being created
    ("delta") and checks in the plugin method (not in the API method) if
    there is enough room for those new rules (tracked resource
    "security_group_rule").

    Change-Id: I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    Closes-Bug: #1858680

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/705267

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705267
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=967d3dea9142c4784302a637b752c673ebe6fc67
Submitter: Zuul
Branch: stable/train

commit 967d3dea9142c4784302a637b752c673ebe6fc67
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jan 8 13:31:29 2020 +0000

    Check "security_group_rule" quota during security group creation

    The tracked resources quota check is done at the beginning of an API
    call to the Neutron server. The API call contains a resource and an
    action over the resource. In case of creation, the server checks if
    the number of items requested fits in the existing quota.

    In case of security group creation, the tracked resource checked is
    "security_group". But "SecurityGroupDbMixin.create_security_group"
    method also creates several default rules for the new group and the
    quota for "security_group_rule" is not enforced.

    This patch checks the number of security group rules being created
    ("delta") and checks in the plugin method (not in the API method) if
    there is enough room for those new rules (tracked resource
    "security_group_rule").

    Change-Id: I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    Closes-Bug: #1858680
    (cherry picked from commit 936bd67aa42b2e06241d309315b895c9c9c49dcc)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/722652

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/722652
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ff607c3f2275c3f772228a50de51f04132ab0b23
Submitter: Zuul
Branch: stable/stein

commit ff607c3f2275c3f772228a50de51f04132ab0b23
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Jan 8 13:31:29 2020 +0000

    Check "security_group_rule" quota during security group creation

    The tracked resources quota check is done at the beginning of an API
    call to the Neutron server. The API call contains a resource and an
    action over the resource. In case of creation, the server checks if
    the number of items requested fits in the existing quota.

    In case of security group creation, the tracked resource checked is
    "security_group". But "SecurityGroupDbMixin.create_security_group"
    method also creates several default rules for the new group and the
    quota for "security_group_rule" is not enforced.

    This patch checks the number of security group rules being created
    ("delta") and checks in the plugin method (not in the API method) if
    there is enough room for those new rules (tracked resource
    "security_group_rule").

    Change-Id: I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    Closes-Bug: #1858680
    (cherry picked from commit 936bd67aa42b2e06241d309315b895c9c9c49dcc)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/864765

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/864766

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/864767

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/864435
Committed: https://opendev.org/openstack/neutron/commit/02bdd0470246dd768227affa2d6a8dd8328d3463
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 02bdd0470246dd768227affa2d6a8dd8328d3463
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Nov 14 17:43:27 2022 +0000

    [stable-only] Do not fail making reservation when creating a SG

    Do not fail during the creation of a security group when trying to
    make a quota reservation for the security group rules. This feature
    was added in [1], in order to prevent the rule quota excess during
    the security group creation.

    However, as reported in LP#1992161, this method can be called from
    the RPC worker. If this RPC worker is spawned alone (not with the API
    workers), the extensions are not loaded and the security group rule
    quota resources are not created. That means the quota engine does not
    have the security group rules as managed resources (in this worker).

    When a new network (and the first subnet) is created, the DHCP agent
    (or agents) handling this network will try to create the DHCP port.
    If, as commented in the LP bug, the default security group is not
    created, the RPC worker will try to create it. In this case this
    patch skips the quota check.

    This patch is for stable releases only. Since Xena, this check is
    done using a new method called "quota_limit_check" [2]. This method
    does not fail in the related case.

    [1]https://review.opendev.org/q/I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    [2]https://review.opendev.org/q/Id73368576a948f78a043d7cf0be16661a65626a9

    Closes-Bug: #1992161
    Related-Bug: #1858680
    Change-Id: I0f20b17c1b13c3cf56de70588fca4a6956d276df

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/864765
Committed: https://opendev.org/openstack/neutron/commit/90865c06afe9780ac3116be9e527da9a75944c96
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 90865c06afe9780ac3116be9e527da9a75944c96
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Nov 14 17:43:27 2022 +0000

    [stable-only] Do not fail making reservation when creating a SG

    Do not fail during the creation of a security group when trying to
    make a quota reservation for the security group rules. This feature
    was added in [1], in order to prevent the rule quota excess during
    the security group creation.

    However, as reported in LP#1992161, this method can be called from
    the RPC worker. If this RPC worker is spawned alone (not with the API
    workers), the extensions are not loaded and the security group rule
    quota resources are not created. That means the quota engine does not
    have the security group rules as managed resources (in this worker).

    When a new network (and the first subnet) is created, the DHCP agent
    (or agents) handling this network will try to create the DHCP port.
    If, as commented in the LP bug, the default security group is not
    created, the RPC worker will try to create it. In this case this
    patch skips the quota check.

    This patch is for stable releases only. Since Xena, this check is
    done using a new method called "quota_limit_check" [2]. This method
    does not fail in the related case.

    [1]https://review.opendev.org/q/I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    [2]https://review.opendev.org/q/Id73368576a948f78a043d7cf0be16661a65626a9

    Conflicts:
          neutron/db/securitygroups_db.py

    Closes-Bug: #1992161
    Related-Bug: #1858680
    Change-Id: I0f20b17c1b13c3cf56de70588fca4a6956d276df
    (cherry picked from commit 02bdd0470246dd768227affa2d6a8dd8328d3463)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/864766
Committed: https://opendev.org/openstack/neutron/commit/49980e3d120ca1f0cdd761477a96b27415b3aefe
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 49980e3d120ca1f0cdd761477a96b27415b3aefe
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Nov 14 17:43:27 2022 +0000

    [stable-only] Do not fail making reservation when creating a SG

    Do not fail during the creation of a security group when trying to
    make a quota reservation for the security group rules. This feature
    was added in [1], in order to prevent the rule quota excess during
    the security group creation.

    However, as reported in LP#1992161, this method can be called from
    the RPC worker. If this RPC worker is spawned alone (not with the API
    workers), the extensions are not loaded and the security group rule
    quota resources are not created. That means the quota engine does not
    have the security group rules as managed resources (in this worker).

    When a new network (and the first subnet) is created, the DHCP agent
    (or agents) handling this network will try to create the DHCP port.
    If, as commented in the LP bug, the default security group is not
    created, the RPC worker will try to create it. In this case this
    patch skips the quota check.

    This patch is for stable releases only. Since Xena, this check is
    done using a new method called "quota_limit_check" [2]. This method
    does not fail in the related case.

    [1]https://review.opendev.org/q/I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    [2]https://review.opendev.org/q/Id73368576a948f78a043d7cf0be16661a65626a9

    Conflicts:
          neutron/db/securitygroups_db.py

    Closes-Bug: #1992161
    Related-Bug: #1858680
    Change-Id: I0f20b17c1b13c3cf56de70588fca4a6956d276df
    (cherry picked from commit 02bdd0470246dd768227affa2d6a8dd8328d3463)
    (cherry picked from commit 90865c06afe9780ac3116be9e527da9a75944c96)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/864767
Committed: https://opendev.org/openstack/neutron/commit/03abe3848bdc3d2c1edb4b26ff5545cbdd5e4bc3
Submitter: "Zuul (22348)"
Branch: stable/train

commit 03abe3848bdc3d2c1edb4b26ff5545cbdd5e4bc3
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Nov 14 17:43:27 2022 +0000

    [stable-only] Do not fail making reservation when creating a SG

    Do not fail during the creation of a security group when trying to
    make a quota reservation for the security group rules. This feature
    was added in [1], in order to prevent the rule quota excess during
    the security group creation.

    However, as reported in LP#1992161, this method can be called from
    the RPC worker. If this RPC worker is spawned alone (not with the API
    workers), the extensions are not loaded and the security group rule
    quota resources are not created. That means the quota engine does not
    have the security group rules as managed resources (in this worker).

    When a new network (and the first subnet) is created, the DHCP agent
    (or agents) handling this network will try to create the DHCP port.
    If, as commented in the LP bug, the default security group is not
    created, the RPC worker will try to create it. In this case this
    patch skips the quota check.

    This patch is for stable releases only. Since Xena, this check is
    done using a new method called "quota_limit_check" [2]. This method
    does not fail in the related case.

    [1]https://review.opendev.org/q/I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
    [2]https://review.opendev.org/q/Id73368576a948f78a043d7cf0be16661a65626a9

    Conflicts:
          neutron/db/securitygroups_db.py

    Closes-Bug: #1992161
    Related-Bug: #1858680
    Change-Id: I0f20b17c1b13c3cf56de70588fca4a6956d276df
    (cherry picked from commit 02bdd0470246dd768227affa2d6a8dd8328d3463)
    (cherry picked from commit 90865c06afe9780ac3116be9e527da9a75944c96)