Failed to remove vim monitor due to Execution is in RUNNING state

[Nguyen Hai]

2018-05-14 22:42:12.206 DEBUG tacker.common.log [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] tacker.nfvo.nfvo_plugin.NfvoPlugin method delete_vim called with arguments (<tacker.context.Context object at 0x7ff96e75bfd0>, u'760bda5b-2107-41e5-88dc-f10772c6ceba') {} from (pid=1783) wrapper /opt/stack/tacker/tacker/common/log.py:34
2018-05-14 22:42:12.218 DEBUG tacker.common.log [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] tacker.nfvo.drivers.vim.openstack_driver.OpenStack_Driver method deregister_vim called with arguments () {'vim_obj': {'status': u'REACHABLE', 'description': u'', 'tenant_id': u'f182686e5a8b4243b3c2b10d5c9eae04', 'created_at': datetime.datetime(2018, 5, 14, 13, 42, 2), 'vim_project': {u'name': u'nfv', u'project_domain_name': u'Default'}, 'updated_at': datetime.datetime(2018, 5, 14, 13, 42, 2), 'is_default': True, 'auth_url': u'http://220.70.2.41/identity/v3', 'auth_cred': {u'username': u'nfv_user', 'password': u'***', u'project_name': u'nfv', u'cert_verify': u'False', u'user_domain_name': u'Default', u'key_type': u'barbican_key', u'secret_uuid': u'***', u'auth_url': u'http://220.70.2.41/identity/v3', u'project_id': None, u'project_domain_name': u'Default'}, 'placement_attr': {u'regions': [u'RegionOne']}, 'shared': False, 'type': u'openstack', 'id': u'760bda5b-2107-41e5-88dc-f10772c6ceba', 'name': u'VIM0'}, 'context': <tacker.context.Context object at 0x7ff96e75bfd0>} from (pid=1783) wrapper /opt/stack/tacker/tacker/common/log.py:34
2018-05-14 22:42:12.219 DEBUG tacker.common.log [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] tacker.nfvo.drivers.vim.openstack_driver.OpenStack_Driver method delete_vim_auth called with arguments (<tacker.context.Context object at 0x7ff96e75bfd0>, u'760bda5b-2107-41e5-88dc-f10772c6ceba', {u'username': u'nfv_user', 'password': u'***', u'project_name': u'nfv', u'cert_verify': u'False', u'user_domain_name': u'Default', u'key_type': u'barbican_key', u'secret_uuid': u'***', u'auth_url': u'http://220.70.2.41/identity/v3', u'project_id': None, u'project_domain_name': u'Default'}) {} from (pid=1783) wrapper /opt/stack/tacker/tacker/common/log.py:34
2018-05-14 22:42:12.219 DEBUG tacker.nfvo.drivers.vim.openstack_driver [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] Attempting to delete key for vim id 760bda5b-2107-41e5-88dc-f10772c6ceba from (pid=1783) delete_vim_auth /opt/stack/tacker/tacker/nfvo/drivers/vim/openstack_driver.py:225
2018-05-14 22:42:12.241 DEBUG barbicanclient.client [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] Creating Client object from (pid=1783) Client /opt/stack/python-barbicanclient/barbicanclient/client.py:156
2018-05-14 22:42:12.291 DEBUG barbicanclient.client [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] Response status 204 from (pid=1783) _check_status_code /opt/stack/python-barbicanclient/barbicanclient/client.py:87
2018-05-14 22:42:12.292 DEBUG tacker.nfvo.drivers.vim.openstack_driver [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] VIM key deleted successfully for vim 760bda5b-2107-41e5-88dc-f10772c6ceba from (pid=1783) delete_vim_auth /opt/stack/tacker/tacker/nfvo/drivers/vim/openstack_driver.py:234
2018-05-14 22:42:12.321 DEBUG mistralclient.api.httpclient [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] HTTP GET http://220.70.2.41:8989/v2/executions?workflow_name=vim_id_760bda5b-2107-41e5-88dc-f10772c6ceba 200 from (pid=1783) decorator /usr/local/lib/python2.7/dist-packages/mistralclient/api/httpclient.py:56
2018-05-14 22:42:12.334 DEBUG mistralclient.api.httpclient [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] HTTP DELETE http://220.70.2.41:8989/v2/executions/c4be52ff-c443-4ad4-848b-62b19b21e2b1 403 from (pid=1783) decorator /usr/local/lib/python2.7/dist-packages/mistralclient/api/httpclient.py:56
2018-05-14 22:42:12.335 ERROR tacker.nfvo.nfvo_plugin [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] Failed to remove vim monitor: APIException: Only completed executions can be deleted. Execution c4be52ff-c443-4ad4-848b-62b19b21e2b1 is in RUNNING state
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin Traceback (most recent call last):
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/opt/stack/tacker/tacker/nfvo/nfvo_plugin.py", line 216, in delete_vim
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin vim_monitor_utils.delete_vim_monitor(context, auth_dict, vim_obj)
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/opt/stack/tacker/tacker/nfvo/workflows/vim_monitor/vim_monitor_utils.py", line 91, in delete_vim_monitor
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin delete_executions(mc, vim_obj['id'])
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/opt/stack/tacker/tacker/nfvo/workflows/vim_monitor/vim_monitor_utils.py", line 55, in delete_executions
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin mistral_client.executions.delete(execution.id)
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/usr/local/lib/python2.7/dist-packages/mistralclient/api/v2/executions.py", line 119, in delete
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin self._delete('/executions/%s' % id)
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/usr/local/lib/python2.7/dist-packages/mistralclient/api/base.py", line 148, in _delete
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin self._raise_api_exception(resp)
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin File "/usr/local/lib/python2.7/dist-packages/mistralclient/api/base.py", line 160, in _raise_api_exception
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin error_message=error_data)
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin APIException: Only completed executions can be deleted. Execution c4be52ff-c443-4ad4-848b-62b19b21e2b1 is in RUNNING state
2018-05-14 22:42:12.335 TRACE tacker.nfvo.nfvo_plugin
2018-05-14 22:42:12.340 DEBUG tacker.common.log [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] tacker.db.common_services.common_services_db_plugin.CommonServicesPluginDb method create_event called with arguments (<tacker.context.Context object at 0x7ff96e75bfd0>,) {'res_type': 'vim', 'evt_type': 'DELETE', 'res_id': u'760bda5b-2107-41e5-88dc-f10772c6ceba', 'res_state': u'REACHABLE', 'tstamp': datetime.datetime(2018, 5, 14, 13, 42, 12, 339995)} from (pid=1783) wrapper /opt/stack/tacker/tacker/common/log.py:34
2018-05-14 22:42:12.345 INFO tacker.wsgi [req-b3adad87-7939-455d-a993-12b14483bfbb nfv nfv_user] 220.70.2.41 - - [14/May/2018 22:42:12] "DELETE /v1.0/vims/760bda5b-2107-41e5-88dc-f10772c6ceba.json HTTP/1.1" 204 173 0.149414

[Nguyen Hai (nguyentrihai93-deactivatedaccount)]

The problem raises because of this: https://bugs.launchpad.net/mistral/+bug/1598135 - Deleting executions shouldn't be allowed unless they have finished (SUCCESS/ERROR)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/568300

[Adriano Petrich (apetrich)]

This is probably due to the python-mistralclient not being up to date. I've made a release https://review.openstack.org/568530 so that the force functionality is added

[Nguyen Hai (nguyentrihai93-deactivatedaccount)]

Hi Adriano Petrich,

The python-mistralclient not being up to date is one of the problems. The other problem is that we need to add force parameter to functions that use executions.delete.

[Yan Xing'an (yanxingan)]

I have tested the patch, i works fine.

yanxa@yanxa-ubuntu:~/mytest$ mistral workflow-list
+--------------------------------------+---------------------------------------------+-----------+----------------------------------+--------+------------------------------+---------+---------------------+------------+
| ID | Name | Namespace | Project ID | Tags | Input | Scope | Created at | Updated at |
+--------------------------------------+---------------------------------------------+-----------+----------------------------------+--------+------------------------------+---------+---------------------+------------+
| 525ff5d5-6456-4e7f-9b21-9e47337404c3 | std.create_instance | | <default-project> | <none> | name, image_id, flavor_id... | public | 2018-05-15 16:44:09 | None |
| e4fc1f35-01a3-4f0b-8b1f-439b48bc6a62 | std.delete_instance | | <default-project> | <none> | instance_id | public | 2018-05-15 16:44:09 | None |
| 16fb01b8-c615-4fb4-ac1f-d1678a638738 | vim_id_cb7ac02e-1445-48c2-a3cb-4fcddd49fce5 | | a5346a4d3c464b4f8776ee2f4bfb86af | <none> | ping_vim | private | 2018-06-17 16:26:57 | None |
| dfe29872-0bd4-472f-b7bc-d9c67d029ee8 | vim_id_28dbc4f0-80e2-4006-b26c-724a1202e82d | | a5346a4d3c464b4f8776ee2f4bfb86af | <none> | ping_vim | private | 2018-06-18 04:13:32 | None |
| dfc04828-e931-4ca0-8691-9cea7c38d247 | vim_id_21be1f30-6aaf-48a6-b3f8-7a49c309916d | | a5346a4d3c464b4f8776ee2f4bfb86af | <none> | ping_vim | private | 2018-06-18 04:20:46 | None |
| 4c8d2bde-62c1-428a-bc39-cf5e26a62f05 | vim_id_46f1300e-51e8-44fc-992d-02f9ff5fdca7 | | a5346a4d3c464b4f8776ee2f4bfb86af | <none> | ping_vim | private | 2018-06-19 08:58:18 | None |
+--------------------------------------+---------------------------------------------+-----------+----------------------------------+--------+------------------------------+---------+---------------------+------------+
yanxa@yanxa-ubuntu:~/mytest$ tacker vim-list
Deprecated: tacker command line is deprecated, will be deleted after Rocky is released.
+--------------------------------------+----------------------------------+------+-----------+------------+------------------------------+-----------+
| id | tenant_id | name | type | is_default | placement_attr | status |
+--------------------------------------+----------------------------------+------+-----------+------------+------------------------------+-----------+
| 46f1300e-51e8-44fc-992d-02f9ff5fdca7 | a5346a4d3c464b4f8776ee2f4bfb86af | VIM2 | openstack | False | {u'regions': [u'RegionOne']} | PENDING |
| cb7ac02e-1445-48c2-a3cb-4fcddd49fce5 | a5346a4d3c464b4f8776ee2f4bfb86af | VIM0 | openstack | False | {u'regions': [u'RegionOne']} | REACHABLE |
+------------------------------...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/568300
Committed: https://git.openstack.org/cgit/openstack/tacker/commit/?id=4d167c2d747920ed3a5f83bd7716d70b4b4378bf
Submitter: Zuul
Branch: master

commit 4d167c2d747920ed3a5f83bd7716d70b4b4378bf
Author: Nguyen Hai <email address hidden>
Date: Mon May 14 23:21:46 2018 +0900

    Fix failed to remove VIM monitor due to running execution

    Mistral recently add the force parameter to delete
    non finished executions, therefore, Tacker cannot delete
    VIM monitor execution from Mistral.

    https://review.openstack.org/#/c/560802/
    https://review.openstack.org/#/c/561159/

    Change-Id: I5b81d220b34b5fb349b71133856ba1202d46d415
    Closes-Bug: #1771126