L3 agent prefix delegation - adding new subnet to the router fails

Bug #1892364 reported by Slawek Kaplonski
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Brian Haley

Bug Description

Adding new subnet with prefix delegation enabled to the router fails with error like:

2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent [-] Error while deleting router 7ef0983f-60a4-4389-909c-785eb6c9a51e: KeyError: '999e404c-3c33-4123-9c61-99426f2e5c16'
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 438, in _safe_router_removed
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self._router_removed(ri, router_id)
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 474, in _router_removed
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.router_info[router_id] = ri
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.force_reraise()
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent raise value
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 471, in _router_removed
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent ri.delete()
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 433, in delete
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.process_delete()
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.logger(e)
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.force_reraise()
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent raise value
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1167, in process_delete
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self._process_internal_ports()
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 591, in _process_internal_ports
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent del self.pd_subnets[subnet['id']]
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent KeyError: '999e404c-3c33-4123-9c61-99426f2e5c16'
2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent

I think we should simply handle the case when self.pd_subnets don't have key with subnet['id'] there.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/747290

Changed in neutron:
assignee: nobody → Slawek Kaplonski (slaweq)
status: Confirmed → In Progress
Changed in neutron:
assignee: Slawek Kaplonski (slaweq) → Brian Haley (brian-haley)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/747290
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=13b894288e9354b9eb0833a635fe4a624bb068a9
Submitter: Zuul
Branch: master

commit 13b894288e9354b9eb0833a635fe4a624bb068a9
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/747844

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/747845

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/747846

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/747847

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/747848

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/747844
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=29b4f9561cdbf6eed3e11f1fdaffed88edd78317
Submitter: Zuul
Branch: stable/ussuri

commit 29b4f9561cdbf6eed3e11f1fdaffed88edd78317
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364
    (cherry picked from commit 13b894288e9354b9eb0833a635fe4a624bb068a9)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/747846
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=907bd92595e6593f35c65b5faa2311300ace0c95
Submitter: Zuul
Branch: stable/stein

commit 907bd92595e6593f35c65b5faa2311300ace0c95
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364
    (cherry picked from commit 13b894288e9354b9eb0833a635fe4a624bb068a9)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/rocky)

Reviewed: https://review.opendev.org/747847
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5bb84a40f8d79789f4d471d385babed177a4a835
Submitter: Zuul
Branch: stable/rocky

commit 5bb84a40f8d79789f4d471d385babed177a4a835
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364
    (cherry picked from commit 13b894288e9354b9eb0833a635fe4a624bb068a9)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/queens)

Reviewed: https://review.opendev.org/747848
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ca1d3496faa706e4bed80f91aae8a4f52a1ad2be
Submitter: Zuul
Branch: stable/queens

commit ca1d3496faa706e4bed80f91aae8a4f52a1ad2be
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364
    (cherry picked from commit 13b894288e9354b9eb0833a635fe4a624bb068a9)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/train)

Reviewed: https://review.opendev.org/747845
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a95ddf8ae853d0bd3b5dba89dc3e63a916bd52b2
Submitter: Zuul
Branch: stable/train

commit a95ddf8ae853d0bd3b5dba89dc3e63a916bd52b2
Author: Slawek Kaplonski <email address hidden>
Date: Thu Aug 20 22:39:04 2020 +0200

    Fix deletion of subnet_id from pd_subnets

    In the RouterInfo._process_internal_ports() method when it process
    old ports and port belongs to the subnet with CIDR assigned by
    Prefix Delegation it will try to remove subnet_id key from the
    pd_subnets dict.
    However it seems that in some case it may happen that such subnet_id key
    is not added to the pd_subnets dict and processing of ports fails.

    We shouldn't fail in such case, if there is no subnet_id key in this
    dict we should be good as we want to delete it simply. So this patch
    changes that to not raise KeyError in such case.

    Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79
    Closes-Bug: #1892364
    (cherry picked from commit 13b894288e9354b9eb0833a635fe4a624bb068a9)

tags: added: in-stable-train
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers