Unable to launch remote action

[Loïc LE GAL]

Launching a local single action works, but not launching the same action on a remote Openstack VIM (which implements keystoneV3).

Running the command remotely via Openstack endpoints works.

Find samples below and versions are:
stack@tacker:~$ mistral --version
mistral 2.1.2
stack@tacker:~$ cd python-mistralclient/
stack@tacker:~/python-mistralclient$ git branch
* master => today (20170113)

-----------------------------------------------------

Local action that works:
stack@tacker:~$ mistral --debug -vvv run-action keystone.projects_list

DEBUG (extension) found extension EntryPoint.parse('keystone = mistralclient.auth.keystone:KeystoneAuthHandler')
DEBUG (extension) found extension EntryPoint.parse('keycloak-oidc = mistralclient.auth.keycloak:KeycloakAuthHandler')
DEBUG (v2) Making authentication request to http://127.0.0.1:5000/v2.0/tokens
INFO (connectionpool) Starting new HTTP connection (1): 127.0.0.1
DEBUG (connectionpool) "POST /v2.0/tokens HTTP/1.1" 200 1357
DEBUG (extension) found extension EntryPoint.parse('table = cliff.formatters.table:TableFormatter')
DEBUG (extension) found extension EntryPoint.parse('json = cliff.formatters.json_format:JSONFormatter')
DEBUG (extension) found extension EntryPoint.parse('shell = cliff.formatters.shell:ShellFormatter')
DEBUG (extension) found extension EntryPoint.parse('value = cliff.formatters.value:ValueFormatter')
DEBUG (extension) found extension EntryPoint.parse('yaml = cliff.formatters.yaml_format:YAMLFormatter')
DEBUG (command) run(Namespace(columns=[], formatter='table', input=None, max_width=0, name='keystone.projects_list', noindent=False, prefix='', run_sync=False, save_result=False, target=None, variables=[]))
INFO (connectionpool) Starting new HTTP connection (1): 10.35.1.54
DEBUG (connectionpool) "POST /v2/action_executions HTTP/1.1" 201 3591
DEBUG (httpclient) HTTP POST http://10.35.1.54:8989/v2/action_executions 201
{"result": [{"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/59f36ed689084e329ef64361c903cc6a"}, "enabled": true, "id": "59f36ed689084e329ef64361c903cc6a", "parent_id": "default", "manager": {"client": null}, "_info": {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/59f36ed689084e329ef64361c903cc6a"}, "enabled": true, "domain_id": "default", "parent_id": "default", "id": "59f36ed689084e329ef64361c903cc6a", "name": "invisible_to_admin"}, "_loaded": true, "domain_id": "default", "name": "invisible_to_admin"}, {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/7863e719694349feb58ff10ca57245e6"}, "enabled": true, "id": "7863e719694349feb58ff10ca57245e6", "parent_id": "default", "manager": {"client": null}, "_info": {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/7863e719694349feb58ff10ca57245e6"}, "enabled": true, "domain_id": "default", "parent_id": "default", "id": "7863e719694349feb58ff10ca57245e6", "name": "service"}, "_loaded": true, "domain_id": "default", "name": "service"}, {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/bd1c08c467104065ad5a6ba7dd743f1a"}, "enabled": true, "id": "bd1c08c467104065ad5a6ba7dd743f1a", "parent_id": "default", "manager": {"client": null}, "_info": {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/bd1c08c467104065ad5a6ba7dd743f1a"}, "enabled": true, "domain_id": "default", "parent_id": "default", "id": "bd1c08c467104065ad5a6ba7dd743f1a", "name": "alt_demo"}, "_loaded": true, "domain_id": "default", "name": "alt_demo"}, {"is_domain": false, "description": "Bootstrap project for initializing the cloud.", "links": {"self": "http://127.0.0.1/identity/v3/projects/d2510e7923c547089c632e49f152813e"}, "enabled": true, "id": "d2510e7923c547089c632e49f152813e", "parent_id": "default", "manager": {"client": null}, "_info": {"is_domain": false, "description": "Bootstrap project for initializing the cloud.", "links": {"self": "http://127.0.0.1/identity/v3/projects/d2510e7923c547089c632e49f152813e"}, "enabled": true, "domain_id": "default", "parent_id": "default", "id": "d2510e7923c547089c632e49f152813e", "name": "admin"}, "_loaded": true, "domain_id": "default", "name": "admin"}, {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/ec15e46359e44e3f8a4cb18a66e2e450"}, "enabled": true, "id": "ec15e46359e44e3f8a4cb18a66e2e450", "parent_id": "default", "manager": {"client": null}, "_info": {"is_domain": false, "description": "", "links": {"self": "http://127.0.0.1/identity/v3/projects/ec15e46359e44e3f8a4cb18a66e2e450"}, "enabled": true, "domain_id": "default", "parent_id": "default", "id": "ec15e46359e44e3f8a4cb18a66e2e450", "name": "demo"}, "_loaded": true, "domain_id": "default", "name": "demo"}]}

--------------------------------------------------------
=> Setting a remote Openstack to be seen as a remote VIM (with KeystoneV3) by defining
export OS_TARGET_PROJECT_DOMAIN_ID=default
export OS_TARGET_USER_DOMAIN_ID=default
export OS_TARGET_PROJECT_NAME=default

export OS_TARGET_TENANT_NAME=tacker_project
export OS_TARGET_USERNAME=tacker
export OS_TARGET_PASSWORD=tacker
export OS_TARGET_AUTH_URL=http://10.35.8.91:5000/v3
-------------------------------------------------------
=> Remote action launch that fails (althought the tacker user/password and project are defined)
stack@tacker:~$ mistral --debug -vvv run-action keystone.projects_list

DEBUG (extension) found extension EntryPoint.parse('keystone = mistralclient.auth.keystone:KeystoneAuthHandler')
DEBUG (extension) found extension EntryPoint.parse('keycloak-oidc = mistralclient.auth.keycloak:KeycloakAuthHandler')
DEBUG (v2) Making authentication request to http://127.0.0.1:5000/v2.0/tokens
INFO (connectionpool) Starting new HTTP connection (1): 127.0.0.1
DEBUG (connectionpool) "POST /v2.0/tokens HTTP/1.1" 200 1357
DEBUG (base) Making authentication request to http://10.35.8.91:5000/v3/auth/tokens
INFO (connectionpool) Starting new HTTP connection (1): 10.35.8.91
DEBUG (connectionpool) "POST /v3/auth/tokens HTTP/1.1" 201 6603
DEBUG (base) {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "15d733f707ff4953a67dfa0333d7eec0", "name": "admin"}], "is_admin_project": false, "project": {"domain": {"id": "default", "name": "Default"}, "id": "4c0ece1b63064ac3a801eb969483d3a8", "name": "tacker_project"}, "catalog": [{"endpoints": [{"url": "http://10.35.8.91:9292", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "a481ea3354d641e780fb8cf175d40669"}, {"url": "http://10.35.8.91:9292", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "a9b649f53c39488da2b4e2c0cced2fb4"}, {"url": "http://10.35.8.91:9292", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c318f519bcf444708b49bb8039cb772e"}], "type": "image", "id": "5b16b1ccab50405ca04375ed025e62b7", "name": "glance"}, {"endpoints": [{"url": "http://10.35.8.91/identity", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "0ef2e8aea57e41399da6a6c80cce6b58"}, {"url": "http://10.35.8.91/identity_v2_admin", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "4b4cadf6f069472ca29d0bf4cf30eb8d"}, {"url": "http://10.35.8.91/identity", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "816691b2c8954b3c83421153c9716f06"}], "type": "identity", "id": "5f2d5bdd6c6b4aa881cfc391f34d9120", "name": "keystone"}, {"endpoints": [{"url": "http://10.35.8.91:8776/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "9a59b3d2c48b4eebb4fe487e852554af"}, {"url": "http://10.35.8.91:8776/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "bc284fd947824f76beb706d1968c151a"}, {"url": "http://10.35.8.91:8776/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "fb04f1dfd23d40558882c2a878e25c3b"}], "type": "volume", "id": "6967a13de9e349088ea9a5b9b46a600b", "name": "cinder"}, {"endpoints": [{"url": "http://10.35.8.91:8000/v1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "b5f99d7ea9a44082a48323bfa2183b8b"}, {"url": "http://10.35.8.91:8000/v1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "cb4d33c2a20143e0a8804c58d22ab51a"}, {"url": "http://10.35.8.91:8000/v1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "df2c24869c48413d97967aff4927d409"}], "type": "cloudformation", "id": "7488b3aec06f403c85f8ed62ac5d738d", "name": "heat-cfn"}, {"endpoints": [{"url": "http://10.35.8.91:8776/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "3958537cd37e454b9414b489cc7b1686"}, {"url": "http://10.35.8.91:8776/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "4da8c24c987d4894a2471d15fbd95ad3"}, {"url": "http://10.35.8.91:8776/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "c1f853ae997945a386f30774199f8f9e"}], "type": "volumev2", "id": "757d9f9af6a64ae9b3c82fda03ce6067", "name": "cinderv2"}, {"endpoints": [{"url": "http://10.35.8.91:8776/v3/4c0ece1b63064ac3a801eb969483d3a8", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "242eb654c8bf46fab5e6d4dbe3a44352"}, {"url": "http://10.35.8.91:8776/v3/4c0ece1b63064ac3a801eb969483d3a8", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "474a7fb3863f4e9d9304d846c07e5a2d"}, {"url": "http://10.35.8.91:8776/v3/4c0ece1b63064ac3a801eb969483d3a8", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "8344708c01fc4818ba7e033f9ba3d7cd"}], "type": "volumev3", "id": "a2fbb4ab489346c9af6775333a2126cf", "name": "cinderv3"}, {"endpoints": [{"url": "http://10.35.8.91:8774/v2.1", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "06e1b65142184faa99729928f929f742"}, {"url": "http://10.35.8.91:8774/v2.1", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "29dd812bfad24b39bca6dadc9da09546"}, {"url": "http://10.35.8.91:8774/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c5f8ce8b07124d658ed228224ba345b1"}], "type": "compute", "id": "c77cd52b5fb641b9898104b2e190e61f", "name": "nova"}, {"endpoints": [{"url": "http://10.35.8.91:8004/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "17ef2a2c0d1840d5a4b47122ee316419"}, {"url": "http://10.35.8.91:8004/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "4837a5918f5e48fdaa0a6c37e06787dc"}, {"url": "http://10.35.8.91:8004/v1/4c0ece1b63064ac3a801eb969483d3a8", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "5ba3a26a4ae44f67ad9db805ce30900a"}], "type": "orchestration", "id": "cf3aa67e3d5a49699c8755cce1679363", "name": "heat"}, {"endpoints": [{"url": "http://10.35.8.91:9696/", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "81f676bb336e4a84bb6de113fbf8ffa8"}, {"url": "http://10.35.8.91:9696/", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8867e9825fab4318b3121235a1327d97"}, {"url": "http://10.35.8.91:9696/", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "d8577772cd3843cb997b3318271f45b6"}], "type": "network", "id": "e25a4071047c4586a5c46f19f68d41ca", "name": "neutron"}, {"endpoints": [{"url": "http://10.35.8.91:8774/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "825eb571bc834eee9cd397d58e3671c4"}, {"url": "http://10.35.8.91:8774/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "bbf70398825b4ee6b779a095cd137381"}, {"url": "http://10.35.8.91:8774/v2/4c0ece1b63064ac3a801eb969483d3a8", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "cf61cb915b8f40e5be675b258d20b012"}], "type": "compute_legacy", "id": "eed7d122f81d40d19b43e112ccc622fc", "name": "nova_legacy"}], "expires_at": "2017-01-13T17:13:52.000000Z", "audit_ids": ["DeDWqLwsSp6OkBIB-NdYXA"], "issued_at": "2017-01-13T16:13:52.000000Z", "user": {"domain": {"id": "default", "name": "Default"}, "id": "e2b6e6e03912427497446de5642a486f", "name": "tacker"}}}
DEBUG (extension) found extension EntryPoint.parse('table = cliff.formatters.table:TableFormatter')
DEBUG (extension) found extension EntryPoint.parse('json = cliff.formatters.json_format:JSONFormatter')
DEBUG (extension) found extension EntryPoint.parse('shell = cliff.formatters.shell:ShellFormatter')
DEBUG (extension) found extension EntryPoint.parse('value = cliff.formatters.value:ValueFormatter')
DEBUG (extension) found extension EntryPoint.parse('yaml = cliff.formatters.yaml_format:YAMLFormatter')
DEBUG (command) run(Namespace(columns=[], formatter='table', input=None, max_width=0, name='keystone.projects_list', noindent=False, prefix='', run_sync=False, save_result=False, target=None, variables=[]))
INFO (connectionpool) Starting new HTTP connection (1): 10.35.1.54
DEBUG (connectionpool) "POST /v2/action_executions HTTP/1.1" 201 598
DEBUG (httpclient) HTTP POST http://10.35.1.54:8989/v2/action_executions 201
{"result": "Failed to run action [action_ex_id=None, action_cls='<class 'mistral.actions.action_factory.KeystoneAction'>', attributes='{u'client_method_name': u'projects.list'}', params='{}']\n KeystoneAction.projects.list failed: <class 'keystoneauth1.exceptions.http.NotFound'>: Could not find token: 5783dd016ff747baa2adaad5d74ab055"}

---------------------------------------------------------
=> The same curl command succeed launch from Mistral host that works:
stack@tacker:~$ curl -si -d @token-tacker.json -H "Content-type: application/json" http://10.35.8.91:35357/v3/auth/tokens | awk '/X-Subject-Token/ {print $2}'
93686ffae87f403d8c950e9b53237442
stack@tacker:~$ curl -s -H "X-Auth-Token: 93686ffae87f403d8c950e9b53237442" http://10.35.8.91:5000/v3/projects | python -mjson.tool
{
    "links": {
        "next": null,
        "previous": null,
        "self": "http://10.35.8.91/identity/v3/projects"
    },
    "projects": [
        {
            "description": "Bootstrap project for initializing the cloud.",
            "domain_id": "default",
            "enabled": true,
            "id": "1f7605ef07034c13b9819b390c736728",
            "is_domain": false,
            "links": {
                "self": "http://10.35.8.91/identity/v3/projects/1f7605ef07034c13b9819b390c736728"
            },
            "name": "admin",
            "parent_id": "default"
        },
...

[Loïc LE GAL (loic35)]

Hi, are you reproducing this ?
This is a blocking point for us to use Tacker+Mistral as a VNFM. I've also set a similar bug regarding Mistral GUI after chating with Akovi.

Thanks.

[Andras Kovi (akovi)]

Hi Loïc,

sorry, the Dashboard does not support multi-vim. If this is a blocker issue, please submit a feature request.

Thank you,
Andras

[Loïc LE GAL (loic35)]

Hi Andras,
Here it is not a Dashboard issue. The problem is also with the CLI.
BR.
Loïc

[Loïc LE GAL (loic35)]

Hi,
Seems to be fixed in Mistral 3.0.0.
Thanks!