deleting stack with mistral workflow resource (already deleted) fails with error

[Rabi Mishra]

This results in an undeletable stack.

steps to reproduce:

1. create stack with below template
2. Delete the workflow
3. Delete the the stack

ubuntu@devstack-ubuntu:~/example_mistral$ cat workflow.yaml
heat_template_version: ocata
resources:
  workflow1:
    type: OS::Mistral::Workflow
    properties:
      type: direct
      input:
        name: rabi
      tasks:
        - name: task1
          action: std.echo output=<% $.name %>
          on_success:
            - task2
        - name: task2
          action: std.echo output="Done"

traceback:

2017-09-02 13:30:39.606 TRACE heat.engine.resource Traceback (most recent call last):
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resource.py", line 840, in _action_recorder
2017-09-02 13:30:39.606 TRACE heat.engine.resource yield
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resource.py", line 1924, in delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource *action_args)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/scheduler.py", line 334, in wrapper
2017-09-02 13:30:39.606 TRACE heat.engine.resource step = next(subtask)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resource.py", line 893, in action_handler_task
2017-09-02 13:30:39.606 TRACE heat.engine.resource handler_data = handler(*args)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resources/openstack/mistral/workflow.py", line 614, in handle_delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource return super(Workflow, self).handle_delete()
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resources/signal_responder.py", line 58, in handle_delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource return super(SignalResponder, self).handle_delete()
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resources/stack_user.py", line 75, in handle_delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource return super(StackUser, self).handle_delete()
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resource.py", line 1851, in handle_delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource self.client_plugin().ignore_not_found(ex)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/oslo_utils/excutils.py", line 342, in __call__
2017-09-02 13:30:39.606 TRACE heat.engine.resource six.reraise(exc_type, exc_val, traceback)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/lib/python3/dist-packages/six.py", line 686, in reraise
2017-09-02 13:30:39.606 TRACE heat.engine.resource raise value
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/opt/stack/heat/heat/engine/resource.py", line 1848, in handle_delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource obj.delete(self.resource_id)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/mistralclient/api/v2/workflows.py", line 110, in delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource self._delete('/workflows/%s' % identifier)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/mistralclient/api/base.py", line 128, in _delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource resp = self.http_client.delete(url)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/mistralclient/api/httpclient.py", line 52, in decorator
2017-09-02 13:30:39.606 TRACE heat.engine.resource resp = func(self, *args, **kwargs)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/mistralclient/api/httpclient.py", line 137, in delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource **options)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/keystoneauth1/session.py", line 863, in delete
2017-09-02 13:30:39.606 TRACE heat.engine.resource return self.request(url, 'DELETE', **kwargs)
2017-09-02 13:30:39.606 TRACE heat.engine.resource File "/usr/local/lib/python3.5/dist-packages/keystoneauth1/session.py", line 736, in request
2017-09-02 13:30:39.606 TRACE heat.engine.resource raise exceptions.from_response(resp, method, url)
2017-09-02 13:30:39.606 TRACE heat.engine.resource keystoneauth1.exceptions.http.NotFound: Not Found (HTTP 404)

[Rabi Mishra (rabi)]

Seems mistralclient was changed to use keystone sessions, but the keystoneauth errors are not translated in 3.1.2 (currently released python-mistralclient). This has been fixed in https://github.com/openstack/python-mistralclient/commit/19c89ca49f46c9bf7d21a255182aeff26197bdc8 but has not made it to pike, which would break pike heat too.

So there are 2 options.

1. Revert the patch to use session with mistral client plugin.
2. Check for for keystoneauth exceptions in the plugin.

Whatever option we choose we've to backport it to pike.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/500346

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500346
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=b93e3c637e9a272f54804e52ededa708af78d83b
Submitter: Jenkins
Branch: master

commit b93e3c637e9a272f54804e52ededa708af78d83b
Author: rabi <email address hidden>
Date: Sun Sep 3 10:11:38 2017 +0530

    Check for keystoneauth exceptions in mistral client plugin

    We started to use keyston esession with mistal client plugin
    since pike, but it seems the patch to translate keystoneauth
    exceptions to mistal api exceptions did not get to
    python-mistalclient==3.1.2.

    Change-Id: I198cdbb8ee02623f2f5fb03a784135ce6f94a046
    Closes-Bug: #1714679

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/501111

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501111
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=a88ae514c6ecee60ee622262c7fa850f8e15556d
Submitter: Jenkins
Branch: stable/pike

commit a88ae514c6ecee60ee622262c7fa850f8e15556d
Author: rabi <email address hidden>
Date: Sun Sep 3 10:11:38 2017 +0530

    Check for keystoneauth exceptions in mistral client plugin

    We started to use keyston esession with mistal client plugin
    since pike, but it seems the patch to translate keystoneauth
    exceptions to mistal api exceptions did not get to
    python-mistalclient==3.1.2.

    Change-Id: I198cdbb8ee02623f2f5fb03a784135ce6f94a046
    Closes-Bug: #1714679
    (cherry picked from commit b93e3c637e9a272f54804e52ededa708af78d83b)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/heat 10.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/heat 9.0.1 release.