Cinder

cinder manage fails with taskflow 1.32.0

Bug #1581655 reported by Andrew Kerr on 2016-05-13

This bug affects 3 people

Affects		Status	Importance	Assigned to	Milestone
	Cinder	Fix Released	Undecided	Eric Harney
	Mitaka	Fix Released	Undecided	Eric Harney

Bug Description

Attempting to manage and existing volume in cinder using taskflow 1.32.0 will produce the below traceback in the c-api.log

http://paste.openstack.org/show/497120/

2016-05-13 18:29:45.025 ERROR cinder.api.middleware.fault [req-f6e70f8a-c164-4db3-962f-2f896b10d7eb tempest-NetAppCDOTManageUnmanageTest-1390677315] Caught error: <class 'taskflow.exceptions.MissingDependencies'> 'linear_flow.Flow: volume_manage_existing_api(len=2)' requires ['optional_args'] but no other entity produces said requirements
MissingDependencies: 'cinder.volume.flows.api.manage_existing.EntryCreateTask;volume:manage_existing==1.0' requires ['optional_args'] but no other entity produces said requirements
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault Traceback (most recent call last):
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/api/middleware/fault.py", line 79, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return req.get_response(self.application)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1299, in send
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault application, catch_exc_info=False)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1263, in call_application
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **self.kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/osprofiler/web.py", line 108, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return request.get_response(self.application)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1299, in send
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault application, catch_exc_info=False)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1263, in call_application
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **self.kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token/__init__.py", line 467, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault response = req.get_response(self._app)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1299, in send
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault application, catch_exc_info=False)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1263, in call_application
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 141, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault response = self.app(environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **self.kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/api/openstack/wsgi.py", line 841, in __call__
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault content_type, body, accept)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/api/openstack/wsgi.py", line 903, in _process_stack
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault action_result = self.dispatch(meth, request, action_args)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/api/openstack/wsgi.py", line 998, in dispatch
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return method(req=request, **action_args)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/api/contrib/volume_manage.py", line 140, in create
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/opt/stack/new/cinder/cinder/volume/api.py", line 1585, in manage_existing
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault flow_engine.run()
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 246, in run
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault for _state in self.run_iter(timeout=timeout):
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 272, in run_iter
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault self.validate()
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/fasteners/lock.py", line 306, in wrapper
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return f(self, *args, **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 79, in wrapper
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault return meth(self, *args, **kwargs)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 428, in validate
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault cause=last_cause)
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault MissingDependencies: 'linear_flow.Flow: volume_manage_existing_api(len=2)' requires ['optional_args'] but no other entity produces said requirements
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault MissingDependencies: 'cinder.volume.flows.api.manage_existing.EntryCreateTask;volume:manage_existing==1.0' requires ['optional_args'] but no other entity produces said requirements
2016-05-13 18:29:45.025 27301 ERROR cinder.api.middleware.fault

You can get it to work again by installing taskflow 1.31.0 or lower

Sean McGinnis (sean-mcginnis) on 2016-05-13

affects:

cinder → taskflow

Revision history for this message

Sean McGinnis (sean-mcginnis) wrote on 2016-05-13:

Some notes from IRC discussion:

14:53 < jimbobhickville> so, revert and execute have always required the same argument list, it was just never enforced
                         during compilation previously. this code:
https://github.com/openstack/cinder/blob/9567844f0e180a7278767fdcd4378b19913933be/cinder/volume/flows/api/manage_existing.py#L86
14:53 < jimbobhickville> `optional_args` will never have a value because it isn't required by the `execute` method
14:54 < jimbobhickville> we fixed it so that `revert` *can* take additional arguments, so long as the flow provides them
                         somehow, so we verify that all the required arguments *are* provided as part of the flow
                         compilation
14:57 < jimbobhickville> so, basically, this was a bug in cinder that taskflow just let slide previously, but if this task
                         *was* reverted, the revert would have failed because it wouldn't pass in anything for
                         `optional_args`. and if a revert of a task blows up, taskflow stops processing reverts, so any
                         tasks prior to this task would not be reverted, which is generally undesirable (and fun to debug)

affects:

taskflow → cinder

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-13: Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/316273

Changed in cinder:
assignee:	nobody → Eric Harney (eharney)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-14: Fix merged to cinder (master)

Reviewed: https://review.openstack.org/316273
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=202caf2dedd7c7b8ed6c7790f125417228418ccb
Submitter: Jenkins
Branch: master

commit 202caf2dedd7c7b8ed6c7790f125417228418ccb
Author: Eric Harney <email address hidden>
Date: Fri May 13 16:14:31 2016 -0400

Fix taskflow parameters for manage_existing

Make optional_args not a required parameter.

    Taskflow validates that these parameters match
    between execute and revert. Since optional_args
    is not used, we can fix this by making it not
    required in revert.

    This fixes the taskflow validation failure, we
    should investigate further into whether this
    flow works completely correctly.

Change-Id: I13f5e4103b320a7b73191c85486bb1f96cb77e88
Closes-Bug: #1581655

Changed in cinder:
status:	In Progress → Fix Released

Revision history for this message

Andrew Kerr (andrew-kerr) wrote on 2016-05-16:

Should this be backported to Mitaka? Or at least should Mitaka be pinned to taskflow <= 1.31.0 in requirements.txt?

tags:

added: mitaka-backport-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-05-16: Fix proposed to cinder (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/316854

Eric Harney (eharney) on 2016-05-16

tags:

removed: mitaka-backport-potential

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2016-06-01: Fix included in openstack/cinder 9.0.0.0b1

This issue was fixed in the openstack/cinder 9.0.0.0b1 development milestone.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-06-07: Fix merged to cinder (stable/mitaka)

Reviewed: https://review.openstack.org/316854
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=71efebc06f0354df0b900103f98e50627a0d2bf4
Submitter: Jenkins
Branch: stable/mitaka

commit 71efebc06f0354df0b900103f98e50627a0d2bf4
Author: Eric Harney <email address hidden>
Date: Fri May 13 16:14:31 2016 -0400

Fix taskflow parameters for manage_existing

Make optional_args not a required parameter.

    Taskflow validates that these parameters match
    between execute and revert. Since optional_args
    is not used, we can fix this by making it not
    required in revert.

    This fixes the taskflow validation failure, we
    should investigate further into whether this
    flow works completely correctly.

    Change-Id: I13f5e4103b320a7b73191c85486bb1f96cb77e88
    Closes-Bug: #1581655
    (cherry picked from commit 202caf2dedd7c7b8ed6c7790f125417228418ccb)

Revision history for this message

Yafei Yu (yu-yafei) wrote on 2016-06-22:

Thank you very much, Eric.

Revision history for this message

Doug Hellmann (doug-hellmann) wrote on 2016-08-09: Fix included in openstack/cinder 8.1.0

This issue was fixed in the openstack/cinder 8.1.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

Duplicates of this bug

Bug #1594693

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.