Wrongly Iteration over the remote_ip_prefixes dict

[Maysa de Macedo Souza]

The remote_ip_prefixes is used to keep track of pods that has a container port matching a Network Policy rule with named port specified, and it has the following format {'remote_ip_prefixes': {'10.128.106.20': 'network-policy-5545'}}. Right now it's being tried to iterator over each remote_ip_prefixes dicts and fetch its keys and values without fetching the dict items, causing the following error:

2020-01-04 18:06:22.839 1 DEBUG kuryr_kubernetes.controller.drivers.utils [-] Return Kuryr Network Policies with label {'apiVersion': 'openstack.org/v1', 'items': [{'apiVersion': 'openstack.org/v1', 'kind': 'KuryrNetPolicy', 'metadata': {'annotations': {'networkpolicy_name': 'allow-client-a-via-named-port-ingress-rule', 'networkpolicy_namespace': 'network-policy-5545', 'networkpolicy_uid': 'ab27ccf2-daf7-4316-86a5-645c43c7679e'}, 'creationTimestamp': '2020-01-04T18:03:47Z', 'generation': 3, 'name': 'np-allow-client-a-via-named-port-ingress-rule', 'namespace': 'network-policy-5545', 'resourceVersion': '81157', 'selfLink': '/apis/openstack.org/v1/namespaces/network-policy-5545/kuryrnetpolicies/np-allow-client-a-via-named-port-ingress-rule', 'uid': '7aef0f43-0c0b-4f3f-bd82-1f8a04c41269'}, 'spec': {'egressSgRules': [{'security_group_rule': {'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'ethertype': 'IPv4', 'id': '24d088d3-83c0-4874-acd3-9325b6971633', 'security_group_id': '7a726447-9bda-41ca-a72e-c478d73c99ec'}}], 'ingressSgRules': [{'remote_ip_prefixes': {'10.128.106.20': 'network-policy-5545'}, 'security_group_rule': {'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'ingress', 'ethertype': 'IPv4', 'id': '154bd8a2-7834-4b70-a88b-5fdddc754d5d', 'port_range_max': 80, 'port_range_min': 80, 'protocol': 'tcp', 'security_group_id': '7a726447-9bda-41ca-a72e-c478d73c99ec'}}], 'networkpolicy_spec': {'ingress': [{'ports': [{'port': 'serve-80', 'protocol': 'TCP'}]}], 'podSelector': {'matchLabels': {'pod-name': 'server'}}, 'policyTypes': ['Ingress']}, 'podSelector': {'matchLabels': {'pod-name': 'server'}}, 'securityGroupId': '7a726447-9bda-41ca-a72e-c478d73c99ec', 'securityGroupName': 'sg-allow-client-a-via-named-port-ingress-rule'}}], 'kind': 'KuryrNetPolicyList', 'metadata': {'continue': '', 'resourceVersion': '81931', 'selfLink': '/apis/openstack.org/v1/kuryrnetpolicies'}} get_kuryrnetpolicy_crds /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py:331
2020-01-04 18:06:22.839 1 DEBUG kuryr_kubernetes.controller.drivers.network_policy_security_groups [-] Parsing ingress Rule {'remote_ip_prefixes': {'10.128.106.20': 'network-policy-5545'}, 'security_group_rule': {'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'ingress', 'ethertype': 'IPv4', 'id': '154bd8a2-7834-4b70-a88b-5fdddc754d5d', 'port_range_max': 80, 'port_range_min': 80, 'protocol': 'tcp', 'security_group_id': '7a726447-9bda-41ca-a72e-c478d73c99ec'}} _parse_rules_on_delete_namespace /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy_security_groups.py:381
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry [-] Report handler unhealthy NamespaceHandler: ValueError: too many values to unpack (expected 2)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry Traceback (most recent call last):
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry self._handler(event)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 72, in __call__
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry self.on_present(obj)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/namespace.py", line 86, in on_present
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry crd_selectors = self._drv_sg.update_namespace_sg_rules(namespace)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy_security_groups.py", line 562, in update_namespace_sg_rules
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry crd_selectors.extend(self.delete_namespace_sg_rules(namespace))
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy_security_groups.py", line 526, in delete_namespace_sg_rules
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry ingress_rule_list, "ingress", ns_name)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy_security_groups.py", line 388, in _parse_rules_on_delete_namespace
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry for remote_ip, namespace in remote_ip_prefixes:
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry ValueError: too many values to unpack (expected 2)
2020-01-04 18:06:22.840 1 ERROR kuryr_kubernetes.handlers.retry

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/701100

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/701100
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=674344b1826b4e2724b3fc397f20984fe6a8ad9c
Submitter: Zuul
Branch: master

commit 674344b1826b4e2724b3fc397f20984fe6a8ad9c
Author: Maysa Macedo <email address hidden>
Date: Sat Jan 4 23:11:05 2020 +0000

    Fix iteration over remote_ip_prefixes field

    The remote_ip_prefixes filed of a KuryrNetPolicy has the
    format: {'remote_ip_prefixes': {'pod_ip': 'np-namespace'}}.
    Right now, we're iterating over each remote_ip_prefixes
    dicts and retrieving its keys and values without fetching the
    dict items causing a ValueError. This commit fixes the
    issue by iterating over the dict items.

    Closes-bug: 1858301

    Change-Id: Ic40878a830bcc32da06c0ab2763f593595e81bf2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/701964

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/701964
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=0f0608d4c56701d31a5356961e2e1b905d78a941
Submitter: Zuul
Branch: stable/train

commit 0f0608d4c56701d31a5356961e2e1b905d78a941
Author: Maysa Macedo <email address hidden>
Date: Sat Jan 4 23:11:05 2020 +0000

    Fix iteration over remote_ip_prefixes field

    The remote_ip_prefixes filed of a KuryrNetPolicy has the
    format: {'remote_ip_prefixes': {'pod_ip': 'np-namespace'}}.
    Right now, we're iterating over each remote_ip_prefixes
    dicts and retrieving its keys and values without fetching the
    dict items causing a ValueError. This commit fixes the
    issue by iterating over the dict items.

    Closes-bug: 1858301

    Change-Id: Ic40878a830bcc32da06c0ab2763f593595e81bf2
    (cherry picked from commit 674344b1826b4e2724b3fc397f20984fe6a8ad9c)