Workflow execution fails when catalog is too big

Bug #1785654 reported by Pierre Gaxatte on 2018-08-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
High
Pierre Gaxatte

Bug Description

When running a workflow with a very large catalog (more than 65535 characters in the json returned by keystone), the execution will fail with a message of the form:

2018-08-06 11:55:08.284 18031 ERROR oslo_messaging.rpc.server DBDataError: (pymysql.err.DataError) (1406, u"Data too long for column 'auth_context' at row 1") [SQL: u'INSERT INTO delayed_calls_v2 (created_at, updated_at, id, factory_method_path, target_method_name, method_
arguments, serializers, `key`, auth_context, execution_time, processing) VALUES (%(created_at)s, %(updated_at)s, %(id)s, %(factory_method_path)s, %(target_method_name)s, %(method_arguments)s, %(serializers)s, %(key)s, %(auth_context)s, %(execution_time)s, %(processing)s)']
 [parameters: {'factory_method_path': None, 'serializers': None, 'method_arguments': '{"wf_ex_id": "XXXX"}', 'execution_time': datetime.datetime(2018, 8, 6, 11, 55, 8, 220336), 'created_at': datetime.datetime(2018, 8, 6, 11, 55, 8, 234867),
'processing': 0, 'updated_at': None, 'auth_context': '{"insecure": false, "is_target": false, "domain": null, "project_name": "services", "global_request_id": null, "region_name": null, "project_domain": ... (351996 characters truncated) ... , "redelivered": false, "user_d
omain": "default", "user_name": "XXXX", "user_domain_name": "Default", "project_domain_name": "Default"}', 'target_method_name': 'mistral.engine.workflow_handler._check_and_complete', 'key': u'XXXX', 'id': 'XXXX'}]

On mysql/mariadb the delayed_calls_v2.auth_context is created as a TEXT field and cannot hold the entire catalog (which is huge here)

Change abandoned by Huang.Xiangdong (<email address hidden>) on branch: master
Review: https://review.openstack.org/589954
Reason: wrong branch

Dougal Matthews (d0ugal) on 2018-08-13
Changed in mistral:
status: New → Confirmed
milestone: none → rocky-rc2
importance: Undecided → High
assignee: nobody → Pierre Gaxatte (pierre-gaxatte)
status: Confirmed → In Progress
Dougal Matthews (d0ugal) on 2018-08-17
Changed in mistral:
milestone: rocky-rc2 → stein-1

Reviewed: https://review.openstack.org/589377
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=1f8ae7fc863d77a9ad722bd0dec7c41c26235400
Submitter: Zuul
Branch: master

commit 1f8ae7fc863d77a9ad722bd0dec7c41c26235400
Author: Pierre Gaxatte <email address hidden>
Date: Tue Aug 7 08:39:24 2018 +0200

    Increase delayed_calls_v2.auth_context

    When the keystone catalog gets big, no execution can be made because the
    catalog cannot fit in the `auth_context` field.

    Change-Id: I100494e467f43386be17eb397fbdf0a0edd24453
    Closes-Bug: 1785654

Changed in mistral:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/599606
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=31705ea0a0d1ffd26c5203dd46760195e65f573e
Submitter: Zuul
Branch: stable/rocky

commit 31705ea0a0d1ffd26c5203dd46760195e65f573e
Author: Pierre Gaxatte <email address hidden>
Date: Tue Aug 7 08:39:24 2018 +0200

    Increase delayed_calls_v2.auth_context

    When the keystone catalog gets big, no execution can be made because the
    catalog cannot fit in the `auth_context` field.

    Change-Id: I100494e467f43386be17eb397fbdf0a0edd24453
    Closes-Bug: 1785654
    (cherry picked from commit 1f8ae7fc863d77a9ad722bd0dec7c41c26235400)

tags: added: in-stable-rocky

This issue was fixed in the openstack/mistral 7.0.1 release.

This issue was fixed in the openstack/mistral 8.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers