Murano environment hanging on deletion

Bug #1576703 reported by Victor Ryzhenkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Murano
Fix Released
High
Stan Lagun
Mitaka
Fix Released
High
Stan Lagun
Newton
Fix Released
High
Stan Lagun

Bug Description

Using Mitaka branch of murano

1. Deploy kubernetes app in murano environment
2. Wait for successful deployment
3. Remove environment

Expected result:
Environment successfully removed

Environment removal hangs for 1h, but all resources removed.

Changed in murano:
importance: Medium → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to murano (master)

Fix proposed to branch: master
Review: https://review.openstack.org/311134

Changed in murano:
assignee: nobody → Stan Lagun (slagun)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to murano (master)

Reviewed: https://review.openstack.org/311134
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=440b4b104996337bf07be75ffb412ddf5adc9a5d
Submitter: Jenkins
Branch: master

commit 440b4b104996337bf07be75ffb412ddf5adc9a5d
Author: Stan Lagun <email address hidden>
Date: Fri Apr 29 07:52:56 2016 -0700

    Revert the destroy execution order

    Restores back the order of .destroy execution so that
    child objects get destroyed first and Environment get
    destroyed last.

    Both orders have their cons and pros. With
    the current order Environment and thus Heat stack
    are delete first so if the app sends a command to
    murano-agent in its destroy method it will never
    receive an answer since the VM of the agent was
    already terminated. This broke Kubernetes and might
    broke other apps as well. However the downside is that
    apps are going to perform uninstall of the software
    on VM that is going to be terminated anyway which
    takes time. But until we implement a controllable
    resource deallocation this looks like a safer approach.

    Change-Id: I8265a69938658e89e4fc36824502c75a75a75c66
    Closes-Bug: #1576703

Changed in murano:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to murano (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/312508

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to murano (stable/mitaka)

Reviewed: https://review.openstack.org/312508
Committed: https://git.openstack.org/cgit/openstack/murano/commit/?id=3443d01e09e079801b0660c82f5b57b230cc97d6
Submitter: Jenkins
Branch: stable/mitaka

commit 3443d01e09e079801b0660c82f5b57b230cc97d6
Author: Stan Lagun <email address hidden>
Date: Fri Apr 29 07:52:56 2016 -0700

    Revert the destroy execution order

    Restores back the order of .destroy execution so that
    child objects get destroyed first and Environment get
    destroyed last.

    Both orders have their cons and pros. With
    the current order Environment and thus Heat stack
    are delete first so if the app sends a command to
    murano-agent in its destroy method it will never
    receive an answer since the VM of the agent was
    already terminated. This broke Kubernetes and might
    broke other apps as well. However the downside is that
    apps are going to perform uninstall of the software
    on VM that is going to be terminated anyway which
    takes time. But until we implement a controllable
    resource deallocation this looks like a safer approach.

    Change-Id: I8265a69938658e89e4fc36824502c75a75a75c66
    Closes-Bug: #1576703
    (cherry picked from commit 440b4b104996337bf07be75ffb412ddf5adc9a5d)

Revision history for this message
Mykola Stolyarenko (mstolyarenko) wrote :

Still reproduce. Fix not help

tags: added: mitaka-backport-potential
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/murano 3.0.0.0b1

This issue was fixed in the openstack/murano 3.0.0.0b1 development milestone.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/murano 2.0.1

This issue was fixed in the openstack/murano 2.0.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.