Fail to recreate namespace when previous KuryrNet CRD is not deleted

Bug #1833712 reported by Maysa de Macedo Souza
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kuryr-kubernetes
Fix Released
Undecided
Maysa de Macedo Souza

Bug Description

When a namespace/project is recreated right after its deletion, it's possible that the previous kuryrNet CRD is not yet removed, and Kuryr will try to create another CRD, resulting in k8s conflict exception. In the meantime, other events handling resources under the same namespace in question could be inquiring for an annotation on the namespace, that was not annotated due to the conflict error, resulting in restart of the controller.

description: updated
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kuryr-kubernetes (master)

Fix proposed to branch: master
Review: https://review.opendev.org/666850

Changed in kuryr-kubernetes:
assignee: nobody → Maysa de Macedo Souza (maysa)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kuryr-kubernetes (master)

Reviewed: https://review.opendev.org/666850
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=37b8e3e6b37c7ffd3136a1fbefdd7bb0c1739f6e
Submitter: Zuul
Branch: master

commit 37b8e3e6b37c7ffd3136a1fbefdd7bb0c1739f6e
Author: Maysa Macedo <email address hidden>
Date: Fri Jun 21 14:21:29 2019 +0000

    Fix fail to recreate namespace when previous KuryrNet CRD is not deleted

    When a namespace/project is recreated right after its deletion,
    it's possible that the previous kuryrNet CRD is not yet removed,
    and Kuryr will try to create another CRD, resulting in k8s conflict
    exception. In the meantime, other events handling resources under
    the same namespace in question could be inquiring for an annotation
    on the namespace, that was not annotated due to the conflict error,
    causing the controller to restart.

    This commit fixes the issue by adding an annotation to the KuryrNet
    CRD with the respective namespace uid, which allows to check
    if the existing CRD corresponds to the namespace.

    Change-Id: I85d903902b516bfd109078705860540d731886bc
    Closes-Bug: 1833712

Changed in kuryr-kubernetes:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kuryr-kubernetes (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/669800

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kuryr-kubernetes (stable/stein)

Reviewed: https://review.opendev.org/669800
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=2a58fa8fb560481b1a070a5c0543c63c1fffbcfe
Submitter: Zuul
Branch: stable/stein

commit 2a58fa8fb560481b1a070a5c0543c63c1fffbcfe
Author: Maysa Macedo <email address hidden>
Date: Fri Jun 21 14:21:29 2019 +0000

    Fix fail to recreate namespace when previous KuryrNet CRD is not deleted

    When a namespace/project is recreated right after its deletion,
    it's possible that the previous kuryrNet CRD is not yet removed,
    and Kuryr will try to create another CRD, resulting in k8s conflict
    exception. In the meantime, other events handling resources under
    the same namespace in question could be inquiring for an annotation
    on the namespace, that was not annotated due to the conflict error,
    causing the controller to restart.

    This commit fixes the issue by adding an annotation to the KuryrNet
    CRD with the respective namespace uid, which allows to check
    if the existing CRD corresponds to the namespace.

    Change-Id: I85d903902b516bfd109078705860540d731886bc
    Closes-Bug: 1833712
    (cherry picked from commit 37b8e3e6b37c7ffd3136a1fbefdd7bb0c1739f6e)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kuryr-kubernetes (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/676681

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kuryr-kubernetes (stable/rocky)

Reviewed: https://review.opendev.org/676681
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=46b1407deac0e0b44a09fb747ac130f1cd31a2a1
Submitter: Zuul
Branch: stable/rocky

commit 46b1407deac0e0b44a09fb747ac130f1cd31a2a1
Author: Maysa Macedo <email address hidden>
Date: Fri Jun 21 14:21:29 2019 +0000

    Fix fail to recreate namespace when previous KuryrNet CRD is not deleted

    When a namespace/project is recreated right after its deletion,
    it's possible that the previous kuryrNet CRD is not yet removed,
    and Kuryr will try to create another CRD, resulting in k8s conflict
    exception. In the meantime, other events handling resources under
    the same namespace in question could be inquiring for an annotation
    on the namespace, that was not annotated due to the conflict error,
    causing the controller to restart.

    This commit fixes the issue by adding an annotation to the KuryrNet
    CRD with the respective namespace uid, which allows to check
    if the existing CRD corresponds to the namespace.

    Depends-On: https://review.opendev.org/#/c/677487/
    Change-Id: I85d903902b516bfd109078705860540d731886bc
    Closes-Bug: 1833712
    (cherry picked from commit 37b8e3e6b37c7ffd3136a1fbefdd7bb0c1739f6e)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kuryr-kubernetes 1.1.0

This issue was fixed in the openstack/kuryr-kubernetes 1.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kuryr-kubernetes 0.5.4

This issue was fixed in the openstack/kuryr-kubernetes 0.5.4 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.