project_name and user_name no longer set in mistral action context

Bug #1740891 reported by Alex Schultz
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Mistral
Fix Released
Undecided
Steven Hardy
tripleo
Fix Released
Critical
Emilien Macchi

Bug Description

We appear to be losing the credentials which are passed via environment vars for the AnsibleGenerateInventoryAction. This started failing around the time of a promotion so there may be a regression

http://logs.openstack.org/95/528695/1/check/tripleo-ci-centos-7-containers-multinode/a454f72/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-01-02_15_49_50

2018-01-02 15:49:50 | Deploying overcloud configuration
2018-01-02 15:49:50 | Started Mistral Workflow tripleo.deployment.v1.config_download_deploy. Execution ID: 331b53de-ab06-4eb7-91d2-0f790c99c91c
2018-01-02 15:49:50 | Config downloaded at /var/lib/mistral/331b53de-ab06-4eb7-91d2-0f790c99c91c
2018-01-02 15:49:50 | Failed to run action [action_ex_id=32f0e406-0a5f-442b-9817-7b3dfdcd03cb, action_cls='<class 'mistral.actions.action_factory.AnsibleGenerateInventoryAction'>', attributes='{}', params='{u'ansible_ssh_user': u'tripleo-admin', u'work_dir': u'/var/lib/mistral/331b53de-ab06-4eb7-91d2-0f790c99c91c'}']
2018-01-02 15:49:50 | execve() arg 3 contains a non-string value

Tags: alert ci
Revision history for this message
Steven Hardy (shardy) wrote :

This is the error:

http://logs.openstack.org/90/529590/4/check/tripleo-ci-centos-7-containers-multinode/0132355/logs/undercloud/var/log/mistral/executor.log.txt.gz

2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor [req-154a5dd8-44fd-40d6-853e-c15a91b7c8af 4e51ff82f53c4e569a1f8cfb009ed6eb e68eaa648d4942aea66c0c7f4f95c69a - default default] Failed to run action [action_ex_id=25d23c4b-cee6-4e75-8a7b-7d9d735490bb, action_cls='<class 'mistral.actions.action_factory.AnsibleGenerateInventoryAction'>', attributes='{}', params='{u'ansible_ssh_user': u'tripleo-admin', u'work_dir': u'/var/lib/mistral/5ebb0ccf-67b4-4666-addf-f31f0c523366'}']
 execve() arg 3 contains a non-string value: TypeError: execve() arg 3 contains a non-string value
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor Traceback (most recent call last):
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 110, in run_action
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor result = action.run(action_ctx)
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/actions/ansible.py", line 551, in run
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor log_errors=processutils.LogErrors.ALL)
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 386, in execute
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor env=env_variables)
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 55, in __init__
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor errread, errwrite)
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor raise child_exception
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor TypeError: execve() arg 3 contains a non-string value
2018-01-02 13:33:35.255 3080 ERROR mistral.executors.default_executor

Changed in tripleo:
assignee: nobody → Steven Hardy (shardy)
description: updated
Revision history for this message
Alex Schultz (alex-schultz) wrote :
Revision history for this message
Steven Hardy (shardy) wrote :

Yes I can confirm this is a regression in mistral, the refactoring to use mistral_lib context classes missed setting project_name and user_name, so these are now None in our environment, which breaks the exec of the inventory (which actually works fine, despite the recent changes AFAICS)

summary: - deploy fails with execve() arg 3 contains a non-string value
+ project_name and user_name no longer set in context
Changed in mistral:
assignee: nobody → Steven Hardy (shardy)
summary: - project_name and user_name no longer set in context
+ project_name and user_name no longer set in mistral action context
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to mistral (master)

Fix proposed to branch: master
Review: https://review.openstack.org/530781

Changed in mistral:
status: New → In Progress
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (master)

Fix proposed to branch: master
Review: https://review.openstack.org/530783

Changed in tripleo:
assignee: Steven Hardy (shardy) → Emilien Macchi (emilienm)
Changed in tripleo:
assignee: Emilien Macchi (emilienm) → Steven Hardy (shardy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to mistral (master)

Reviewed: https://review.openstack.org/530781
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=6b81707ce1ea49134ae0def3f05a2cf15b9626db
Submitter: Zuul
Branch: master

commit 6b81707ce1ea49134ae0def3f05a2cf15b9626db
Author: Steven Hardy <email address hidden>
Date: Tue Jan 2 17:49:12 2018 +0000

    Add missing user/project name in action context

    These were removed in Ife653558bfcda794e7f37086832f70b0ad7c28a4
    but that breaks any actions which require this data, such as
    some TripleO actions.

    Closes-Bug: 1740891
    Change-Id: I777b1f7c7012b735805e8585938b5ce5dec31d26

Changed in mistral:
status: In Progress → Fix Released
Changed in tripleo:
assignee: Steven Hardy (shardy) → Emilien Macchi (emilienm)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (master)

Reviewed: https://review.openstack.org/530783
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=04317fff33612a94d26856ba00c0f3b7dc0a6562
Submitter: Zuul
Branch: master

commit 04317fff33612a94d26856ba00c0f3b7dc0a6562
Author: Steven Hardy <email address hidden>
Date: Tue Jan 2 17:58:36 2018 +0000

    Update AnsibleGenerateInventoryAction to match new mistral interfaces

    Ife653558bfcda794e7f37086832f70b0ad7c28a4 changed the way this data is
    represented, now there's a nested context with the security data in,
    and the old interfaces are deprecated.

    Change-Id: Id0ba10d1192314187bedb14998d4aadc5fb91ef5
    Closes-Bug: 1740891

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/mistral 6.0.0.0b3

This issue was fixed in the openstack/mistral 6.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 8.4.0

This issue was fixed in the openstack/tripleo-common 8.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.