Environments in delete failed state are in progress forever

[Stan Lagun]

When environment deletion fails with exception API doesn't recognizes it and deployment continued forever. There is also "None has no attribute state" exception raised in API.

The reason was that the deployment session was in "deploying" state instead of "deleting". That's why RPC result handler didn't found it (and didn't checked it, so exception happened) and didn't changed its status

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/131052

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/133539

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/131052
Committed: https://git.openstack.org/cgit/stackforge/murano/commit/?id=c301b5e7633a85829774691f3361230633b4fd89
Submitter: Jenkins
Branch: master

commit c301b5e7633a85829774691f3361230633b4fd89
Author: Stan Lagun <email address hidden>
Date: Mon Oct 27 11:52:22 2014 +0300

    Environment in delete failed state was in progress forever

    Deletion sessions were in 'deploying' state instead of 'deleting'.
    Because of this such sessions could not be found by RPC result
    processing code and remained in 'deploying' status causing UI
    to display progress bar forever.

    Also there were 2 duplicate SessionStates enums in the code with
    and the second copy was outdated and didn't contained DELETING
    status as well as other new session statuses. Because buggy code
    was using that outdated enum it was necessary to merge both enums
    into single declaration

    Change-Id: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104
    Closes-Bug: #1386068

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/133539
Committed: https://git.openstack.org/cgit/stackforge/murano/commit/?id=bbfa9cbfdaf837b3aec11bf7c00b8bc15a766ce0
Submitter: Jenkins
Branch: stable/juno

commit bbfa9cbfdaf837b3aec11bf7c00b8bc15a766ce0
Author: Stan Lagun <email address hidden>
Date: Mon Oct 27 11:52:22 2014 +0300

    Environment in delete failed state was in progress forever

    Deletion sessions were in 'deploying' state instead of 'deleting'.
    Because of this such sessions could not be found by RPC result
    processing code and remained in 'deploying' status causing UI
    to display progress bar forever

    Also there were 2 duplicate SessionStates enums in the code
    and the second copy was outdated and didn't contained DELETING
    status as well as other new session statuses. Because buggy code
    was using that outdated enum it was necessary to merge both enums
    into single declaration

    Closes-Bug: #1386068
    Cherry-pick: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104
    Change-Id: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104

[smarta94 (smarta94)]

This still seems to be an issue in juno -- I have checked my code and have the fixes released as of this bug report and my environments are still stuck in a state of either Delete in progress or Deploy in progress (with no option to cancel/delete, but it won't deploy due to an error:
OS::Nova::Server 21 hours, 17 minutes Create Failed BadRequest: Port id dbe841d2-13a6-4046-859c-d2ac988a7003 could not be found. (HTTP 400) (Request-ID: req-4ae87094-c4c6-4141-9c09-a8eeaea0b85e) [Although all the routers/networks/subnets get created and associated just fine). I have submitted a bug requrest on the nova::server issue but haven't heard back on that yet.

The Deploy failed ones are all stuck in a state of delete in progress continually.

[smarta94 (smarta94)]

Once a delete has been started and one remains on the environments page for some time, the murno-api log continually spits out the following:
2015-05-27 08:28:40.574 32614 DEBUG keystonemiddleware.auth_token [-] Authenticating user token __call__ /usr/lib/python2.7/site-packages/keystonemiddleware/auth_
token.py:708
2015-05-27 08:28:40.575 32614 DEBUG keystonemiddleware.auth_token [-] Returning cached token _cache_get /usr/lib/python2.7/site-packages/keystonemiddleware/auth_t
oken.py:1793
2015-05-27 08:28:40.575 32614 DEBUG keystonemiddleware.auth_token [-] Authenticating service token __call__ /usr/lib/python2.7/site-packages/keystonemiddleware/au
th_token.py:727
2015-05-27 08:28:40.575 32614 DEBUG keystonemiddleware.auth_token [-] Received request from user: user_id None, project_id None, roles None service: user_id None,
 project_id None, roles None __call__ /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token.py:746
2015-05-27 08:28:40.576 32614 DEBUG routes.middleware [-] Matched GET /environments/95a94faa773448588f739e5afd933acd __call__ /usr/lib/python2.7/site-packages/rou
tes/middleware.py:100
2015-05-27 08:28:40.576 32614 DEBUG routes.middleware [-] Route path: '/environments/{environment_id}', defaults: {'action': u'show', 'controller': <murano.common
.wsgi.Resource object at 0x312fad0>} __call__ /usr/lib/python2.7/site-packages/routes/middleware.py:102
2015-05-27 08:28:40.576 32614 DEBUG routes.middleware [-] Match dict: {'action': u'show', 'controller': <murano.common.wsgi.Resource object at 0x312fad0>, 'enviro
nment_id': u'95a94faa773448588f739e5afd933acd'} __call__ /usr/lib/python2.7/site-packages/routes/middleware.py:103
2015-05-27 08:28:40.576 32614 DEBUG murano.common.wsgi [-] Empty body provided in request deserialize_body /usr/lib/python2.7/site-packages/murano/common/wsgi.py:
664
2015-05-27 08:28:40.577 32614 DEBUG murano.api.v1.environments [-] Environments:Show <Id: 95a94faa773448588f739e5afd933acd> show /usr/lib/python2.7/site-packages/
murano/api/v1/environments.py:78
2015-05-27 08:28:40.577 32614 DEBUG murano.openstack.common.policy [-] Rule show_environment will be now enforced enforce /usr/lib/python2.7/site-packages/murano/
openstack/common/policy.py:288
2015-05-27 08:28:40.577 32614 AUDIT murano.common.policy [-] Policy check succeeded for rule 'show_environment' on target {'environment_id': u'95a94faa773448588f7
39e5afd933acd'}
2015-05-27 08:28:40.585 32614 DEBUG murano.api.v1.request_statistics [-] Updating count stats for Environments, Show on object <murano.api.v1.request_statistics.R
equestStatisticsCollection object at 0xf7c990> update_count /usr/lib/python2.7/site-packages/murano/api/v1/request_statistics.py:74
2015-05-27 08:28:40.585 32614 INFO eventlet.wsgi [-] 172.16.128.11 - - [27/May/2015 08:28:40] "GET /v1/environments/95a94faa773448588f739e5afd933acd HTTP/1.1" 200
 2929 0.012533
2015-05-27 08:28:40.602 32614 INFO eventlet.wsgi [-] (32614) accepted ('172.16.128.11', 39072)
2015-05-27 08:28:40.603 32614 DEBUG murano.api.middleware.version_negotiation [-] Determining version of request: GET /v1/environments/8dacc96957f94f06b0037e619a3
edf09 Accept: */* process_request /usr/lib/python2.7/site-...