Create softwareconfig resource in multiregion stack fails

[Ethan Lynn]

I use below template to create a softwareconfig resource in regionone,
https://github.com/lynic/templates/tree/master/bug/20150306-mr-sc

heat stack-create fd-sc -f update.yaml

And I got this failure message.
HTTPBadRequest: ERROR: Property error : sc: config No content found in the "files" section for get_file path: test.sh

If I don't use OS::Heat::Stack resource, it works fine.

[Ethan Lynn (ethanlynn)]

It seems heatclient didn't pass softwareconfig scripts to heat services.

[elynn@devstack 20150306-mr-sc]$ heat --debug stack-create region -f update.yaml
DEBUG (session) REQ: curl -g -i --cacert "/opt/stack/data/CA/int-ca/ca-chain.pem" -X GET http://192.168.31.215:5000/v2.0 -H "Accept: application/json" -$
"User-Agent: python-keystoneclient"
INFO (connectionpool) Starting new HTTP connection (1): 192.168.31.215
DEBUG (connectionpool) "GET /v2.0 HTTP/1.1" 200 340
DEBUG (session) RESP: [200] content-length: 340 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.10 (Fedora) OpenSSL/1.0.1e-fips mod$
wsgi/3.5 Python/2.7.5 connection: Keep-Alive date: Tue, 10 Mar 2015 10:14:01 GMT content-type: application/json x-openstack-request-id: req-7e11d98d-1e1$
-4efd-812d-8fdb332229b2
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.ope$
stack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://192.168.31.215:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/",
"type": "text/html", "rel": "describedby"}]}}

DEBUG (v2) Making authentication request to http://192.168.31.215:5000/v2.0/tokens
DEBUG (connectionpool) "POST /v2.0/tokens HTTP/1.1" 200 9419
DEBUG (session) REQ: curl -g -i --cacert "/opt/stack/data/CA/int-ca/ca-chain.pem" -X POST http://192.168.31.215:8004/v1/f18e7e37f9f44a28b43a3d74d417cbf7/
stacks -H "Accept: application/json" -H "X-Auth-Token: {SHA1}23cac33e24bae758af10f7b6cc42d7194134bd6b" -H "X-Region-Name: RegionOne" -H "X-Auth-Key: Pass
w0rd" -H "X-Auth-User: admin" -H "User-Agent: python-heatclient" -H "Content-Type: application/json" -H "X-Auth-Url: http://192.168.31.215:5000/v2.0" -d
'{"files": {"file:///home/elynn/source/templates/bug/20150306-mr-sc/web.yaml": "heat_template_version: 2013-05-23\nparameters:\n image:\n type: strin
g\n flavor:\n type: string\n key:\n type: string\nresources:\n server:\n type: OS::Nova::Server\n properties:\n flavor: {get_param: f
lavor}\n image: {get_param: image}\n key_name: {get_param: key}\n user_data_format: SOFTWARE_CONFIG\n sc:\n type: OS::Heat::SoftwareCo
nfig\n properties:\n group: script\n config: {get_file: test.sh}\n sc-deploy:\n type: OS::Heat::SoftwareDeployment\n properties:\n
   config: {get_resource: sc}\n server: {get_resource: server} \n"}, "disable_rollback": true, "parameters": {}, "stack_name": "region", "environmen
t": {}, "template": {"heat_template_version": "2013-05-23", "parameters": {"flavor": {"default": "m1.small", "type": "string"}, "image": {"default": "fed
ora-sc", "type": "string"}, "key": {"default": "lxiaojun", "type": "string"}}, "resources": {"stack": {"type": "OS::Heat::Stack", "properties": {"context
": {"region_name": "RegionOne"}, "template": {"get_file": "file:///home/elynn/source/templates/bug/20150306-mr-sc/web.yaml"}, "parameters": {"flavor": {"
get_param": "flavor"}, "image": {"get_param": "image"}, "key": {"get_param": "key"}}}}}}}'
INFO (connectionpool) Starting new HTTP connect...

[Ethan Lynn (ethanlynn)]

From debug message , we can see that softwareconfig script test.sh didn't pass to heat api in files section.

[Steve Baker (steve-stevebaker)]

Moved to python-heatclient

[Steve Baker (steve-stevebaker)]

The fix will be to make resolve_template_get_files[1] be aware that it is evaluating an OS::Heat::Stack template property so that it sets file_is_template=True on the call to get_file_contents[2].

[1] https://github.com/openstack/python-heatclient/blob/master/heatclient/common/template_utils.py#L76
[2] https://github.com/openstack/python-heatclient/blob/master/heatclient/common/template_utils.py#L109

[Steve Baker (steve-stevebaker)]

Actually get_file_contents needs to be a function ref instead of a boolean, so it is called like ignore_if and recurse_if

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/164417

[Ethan Lynn (ethanlynn)]

Upload two patches for this bug:
https://review.openstack.org/#/c/164416/ for OS::Heat::Stack resource
https://review.openstack.org/#/c/164417/ for heatclient

After above patches merged, these two type of template will both work
template1:
heat_template_version: 2013-05-23
resources:
    remote_stack:
        type: OS::Heat::Stack
        properties:
            context:
                region_name: RegionOne
            template: { get_file: remote_template.yaml }
            timeout: 60
            parameters:
                name: foo

template2:
heat_template_version: 2013-05-23
resources:
    remote_stack:
        type: OS::Heat::Stack
        properties:
            context:
                region_name: RegionOne
            template: remote_template.yaml
            timeout: 60
            parameters:
                name: foo

The only difference between template1 and template2 is that the remote_template.yaml in template1 will not be parsed as a template but just a file, so the nested stack in template1 will not work.

I still insist that get_file should return the original file and this file should not be parsed as a template, maybe we should add a new function like get_template so that heat can know that this file is a template which should be parsed.

[Ethan Lynn (ethanlynn)]

Make heatclient more smarter to automatically judgement if a file is a template or not.
Abandon https://review.openstack.org/#/c/164416/.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/164417
Committed: https://git.openstack.org/cgit/openstack/python-heatclient/commit/?id=22660e943e23e515cd83c2d22e864b35890ffc18
Submitter: Jenkins
Branch: master

commit 22660e943e23e515cd83c2d22e864b35890ffc18
Author: Ethan Lynn <email address hidden>
Date: Mon Mar 16 19:15:35 2015 +0800

    Parse nested files if they are template

    Parse nested files if they are in template format.

    Closes-Bug: #1429036
    Change-Id: I6b01ed853edac815510e89fbfaf775084a9c5ac7

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-heatclient 0.4.0 release.