Neutron security group rule creation is not rejected after exceeding the quota

[Yvonne Ding]

Brief Description
-----------------
Security rule creation should be failed after max quota is reached, however it is successful.

Severity
--------
Major

Steps to Reproduce
------------------
1. Retrieve quota and usage information
2. Retrieve usage for security groups
3. Create enough groups to reach the quota
4. Verify security group creation fail as quota is reached
5. Retrieve usage for security group rules"
6. Create enough rules to reach the quota
7. Verify security rule creation fail as quota is reached

TC-name:
test_security_group_and_rule_create_reject_when_max_reached

Expected Behavior
-----------------
Security rule creation after max quota reached failed

Actual Behavior
----------------
Security rule creation after max quota reached succeeded

Reproducibility
---------------
reproducible

System Configuration
--------------------
Regular standard 2+2

Lab-name:
wcp_7_10

Branch/Pull Time/Commit
-----------------------
BUILD_ID="r/stx.4.0"

Timestamp/Logs
--------------
[2020-07-17 05:49:10,605] 314 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group rule create --remote-ip=0.0.0.1/32 --protocol=udp --ingress 59f8114d-2546-43e0-9310-053dd3fc9254'

......

[2020-07-17 05:52:13,886] 785 INFO MainThread network_helper.create_security_group_rule:: Creating security group rule for group 59f8114d-2546-43e0-9310-053dd3fc9254 with args: --remote-ip=255.255.255.254/32 --protocol=udp --ingress 59f8114d-2546-43e0-9310-053dd3fc9254
[2020-07-17 05:52:13,888] 314 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group rule create --remote-ip=255.255.255.254/32 --protocol=udp --ingress 59f8114d-2546-43e0-9310-053dd3fc9254'
[2020-07-17 05:52:16,333] 436 DEBUG MainThread ssh.expect :: Output:
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2020-07-17T05:51:22Z |
| description | |
| direction | ingress |
| ether_type | IPv4 |
| id | 20c54dc6-a762-4ae0-aad5-993168c521c4 |
| location | cloud='', project.domain_id=, project.domain_name='Default', project.id='23a6f438af8e493ab5b13501d5b08d4f', project.name='tenant1', region_name='RegionOne', zone= |
| name | None |
| port_range_max | None |
| port_range_min | None |
| project_id | 23a6f438af8e493ab5b13501d5b08d4f |
| protocol | udp |
| remote_group_id | None |
| remote_ip_prefix | 255.255.255.254/32 |
| revision_number | 0 |
| security_group_id | 59f8114d-2546-43e0-9310-053dd3fc9254 |
| tags | [] |
| updated_at | 2020-07-17T05:51:22Z |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+

[2020-07-17 05:52:16,488] 61 DEBUG MainThread conftest.update_results:: ***Failure at test call: /home/svc-cgcsauto/wassp-repos.new/testcases/cgcs/CGCSAuto/testcases/functional/neutron/test_qos_security.py:272: AssertionError: creation after max quota reached succeeded

Logs of .tar and automation log as below,
https://files.starlingx.kube.cengn.ca/launchpad/1888171

Test Activity
-------------
Test neutron function with the openstack install system

[Frank Miller (sensfan22)]

Assigning to the PL for distro.openstack to re-assign to an STX openstack developer.

[zhipeng liu (zhipengs)]

From log, secgroups works as expected when reach quotas = 10.
secgroup-rules failed to exceed quotas = 100
Could you double check the rule number after test failed, Thanks!

Zhipeng

[Yvonne Ding (yding)]

Please refer to logs at https://files.starlingx.kube.cengn.ca/launchpad/1888171

[Yan Chen (ychen2u)]

May need to upgrade openstack client for ussuri.
The test case need to be updated too. Can we defer it to the maintenance release?

There are new flags for security group.
https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/security-group.html
Ussuri Release Notes also mentioned several changes on security group:
https://docs.openstack.org/releasenotes/neutron/ussuri.html

[Yan Chen (ychen2u)]

With a very simple script, we can test the security group rule quota:

## Test Script Start ##
openstack security group test_sec_group_quota
for i in {0..255}; do
openstack security group rule create --protocol=udp --ingress --remote-ip=11.0.$i.10/32 --dst-port=1:65535 test_sec_group_quota
done
## Test Script End ##

And we see the following error log after totally 100 rules created:
Error while executing command: ConflictException: 409, Quota exceeded for resources: ['security_group_rule'].

This should be expected.
What is the criteria of this case?

[Yan Chen (ychen2u)]

See from the Neutron document, you can modify the quotas in the neutron.conf, in stx-openstack there's no override for the quotas, so it should be limited by the default values.

==== Neutron Doc ====
OpenStack Networking also supports quotas for security group
resources: number of security groups and the number of rules for
each security group. Add these lines to the
``quotas`` section in the ``/etc/neutron/neutron.conf`` file:

.. code-block:: ini

   [quotas]
   # number of security groups per tenant, and minus means unlimited
   quota_security_group = 10

   # number of security rules allowed per tenant, and minus means unlimited
   quota_security_group_rule = 100