kuryr-kubernetes

NP CRD unable to be patched because of missing sg rule ID

Bug #1887167 reported by Maysa de Macedo Souza
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kuryr-kubernetes
Fix Released
Undecided
Maysa de Macedo Souza

Bug Description

During the Network Policy creation it's possible that the CRD is patched with repeated sg rules, which is not allowed, resulting in validation error as the repeated sg rules will not have the sg rule id.

2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.controller.drivers.utils [-] Patching KuryrNetPolicy CRD np-allow-to-server-a-pod-selector patch_kuryrnetworkpolicy_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py:221
2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.k8s_client [-] Patch /apis/openstack.org/v1/namespaces/network-policy-6479/kuryrnetpolicies/np-allow-to-server-a-pod-selector: [{'op': 'replace', 'path': '/spec/ingressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '79c924de-0983-4b60-8f92-7beb1cac17cd'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '703b7159-d3f7-409e-b19d-c44eba3b1201'}}]}, {'op': 'replace', 'path': '/spec/egressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '14eedd8d-3a67-4828-be1f-64b8d5220ea9'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '3616868c-c52d-4fc6-a596-9197ecea3cba'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.2.244', 'id': 'f553e38e-6240-46d9-b3b1-03e071d2340d'}, 'namespace': 'network-policy-6479'}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.150', 'id': 'c92a7d7e-0a96-4e5d-a572-74b9fbed3ec3'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.145', 'id': '6663def7-5c7e-4600-8aa2-e48ebed8cc38'}}]}, {'op': 'replace', 'path': '/spec/podSelector', 'value': {'matchLabels': {'pod-name': 'client-a'}}}, {'op': 'replace', 'path': '/spec/networkpolicy_spec', 'value': {'podSelector': {'matchLabels': {'pod-name': 'client-a'}}, 'egress': [{'ports': [{'protocol': 'UDP', 'port': 53}]}, {'to': [{'podSelector': {'matchLabels': {'pod-name': 'server'}}}]}], 'policyTypes': ['Egress']}}] patch_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py:134
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils [-] Error updating kuryrnetpolicy CRD np-allow-to-server-a-pod-selector: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils Traceback (most recent call last):
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils self._raise_from_response(response)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils 
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry [-] Report handler unhealthy NetworkPolicyHandler: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry Traceback (most recent call last):
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._handler(event, *args, **kwargs)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 90, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.on_present(obj)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/policy.py", line 53, in on_present
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry project_id)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 58, in ensure_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.update_security_group_rules_from_network_policy(policy))
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 119, in update_security_group_rules_from_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry np_spec=policy['spec'])
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._raise_from_response(response)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry 
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging [-] Failed to handle event {'type': 'ADDED', 'object': {'kind': 'NetworkPolicy', 'apiVersion': 'networking.k8s.io/v1', 'metadata': {'name': 'allow-to-server-a-pod-selector', 'namespace': 'network-policy-6479', 'selfLink': '/apis/networking.k8s.io/v1/namespaces/network-policy-6479/networkpolicies/allow-to-server-a-pod-selector', 'uid': '257fea0e-1135-4309-98cd-43765b3ee705', 'resourceVersion': '9366', 'generation': 1, 'creationTimestamp': '2020-07-09T16:43:33Z', 'annotations': {'kuryrnetpolicy_selfLink': '/apis/openstack.org/v1/namespaces/network-policy-6479/kuryrnetpolicies/np-allow-to-server-a-pod-selector'}}, 'spec': {'podSelector': {'matchLabels': {'pod-name': 'client-a'}}, 'egress': [{'ports': [{'protocol': 'UDP', 'port': 53}]}, {'to': [{'podSelector': {'matchLabels': {'pod-name': 'server'}}}]}], 'policyTypes': ['Egress']}}}: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging Traceback (most recent call last):
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/logging.py", line 37, in __call__
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging self._handler(event, *args, **kwargs)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging self._handler(event, *args, **kwargs)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 90, in __call__
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging self.on_present(obj)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/policy.py", line 53, in on_present
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging project_id)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 58, in ensure_network_policy
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging self.update_security_group_rules_from_network_policy(policy))
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 119, in update_security_group_rules_from_network_policy
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging np_spec=policy['spec'])
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging self._raise_from_response(response)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.982 1 ERROR kuryr_kubernetes.handlers.logging kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}

OpenStack Infra (hudson-openstack)
Changed in kuryr-kubernetes:
assignee: nobody → Maysa de Macedo Souza (maysa)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kuryr-kubernetes (master)

Reviewed: https://review.opendev.org/740381
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=abc679c9f23dfc8a784eb350534b0e01e372bdab
Submitter: Zuul
Branch: master

commit abc679c9f23dfc8a784eb350534b0e01e372bdab
Author: Maysa Macedo <email address hidden>
Date: Thu Jul 9 22:21:57 2020 +0000

    Fix duplicated sg rules on NP crd

    While handling the creation of a Network
    Policy it's possible that the CRD is patched
    with repeated sg rules, which is not allowed
    resulting in validation error as the repeated
    sg rules will not have the sg rule id.

    Closes-Bug: #1887167
    Change-Id: Ia7814ddcea0d6948ff280a3e03a896bbc442891c

Changed in kuryr-kubernetes:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kuryr-kubernetes 2.1.0

This issue was fixed in the openstack/kuryr-kubernetes 2.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.