resource_registry broken for relative paths

[Steven Hardy]

I'm trying to do this:

-bash-4.3$ cat test.yaml
heat_template_version: 2015-04-30
resources:
  foo:
    type: OS::Foo
-bash-4.3$ cat registry.yaml
resource_registry:
  OS::Foo: foo.yaml
-bash-4.3$ cat foo.yaml
heat_template_version: 2015-04-30
-bash-4.3$ heat stack-create test -f test.yaml -e registry.yaml
WARNING (shell) "heat stack-create" is deprecated, please use "openstack stack create" instead
ERROR: Could not fetch remote template "foo.yaml": Invalid URL scheme

The cause appears related to the switch to pass environment data via the files map, as adding this debug indicates:

$ git diff
diff --git a/heatclient/common/template_utils.py b/heatclient/common/template_utils.py
index fae54ef..74ad1b2 100644
--- a/heatclient/common/template_utils.py
+++ b/heatclient/common/template_utils.py
@@ -247,6 +247,8 @@ def process_multiple_environments_and_files(env_paths=None, template=None,
                 env_url = utils.normalise_file_path_to_url(env_path)
                 env_list_tracker.append(env_url)

+ print "SHDEBUG merged_env=%s" % merged_env
+ print "SHDEBUG merged_files=%s" % merged_files
     return merged_files, merged_env

-bash-4.3$ heat stack-create test -f test.yaml -e registry.yaml
WARNING (shell) "heat stack-create" is deprecated, please use "openstack stack create" instead
ERROR: Could not fetch remote template "foo.yaml": Invalid URL scheme
-bash-4.3$ heat stack-create test -f test.yaml -e registry.yaml
WARNING (shell) "heat stack-create" is deprecated, please use "openstack stack create" instead
SHDEBUG merged_env={u'resource_registry': {u'OS::Foo': u'file:///opt/stack/test_templates/test2/reproduce/foo.yaml'}}
SHDEBUG merged_files={'file:///opt/stack/test_templates/test2/reproduce/registry.yaml': 'resource_registry:\n OS::Foo: foo.yaml\n', u'file:///opt/stack/test_templates/test2/reproduce/foo.yaml': '{"heat_template_version": "2015-04-30"}'}
ERROR: Could not fetch remote template "foo.yaml": Invalid URL scheme

Here we can see the foo.yaml mapping in the merged_env is correct, but in the merged_files, it's lacking the resolved path - presumably this ends up taking precedence in heat and we try (then fail) to retrieve the template from file:///foo.yaml

I'm confused by how this passed CI, as it's fundamental functionality I thought we had coverage for - possibly I've failed to pull a required additional change in to my environment..

[Steven Hardy (shardy)]

Reverting https://github.com/openstack/python-heatclient/commit/7627e1bae53747e1471caf632dfc1b6edf2e26cf appears to fix it - I'm still unclear how we landed that without CI failing?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/295412

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/296768

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/295412
Committed: https://git.openstack.org/cgit/openstack/python-heatclient/commit/?id=f13d36ae0114e159663ea2a3fa07260df9d9407d
Submitter: Jenkins
Branch: master

commit f13d36ae0114e159663ea2a3fa07260df9d9407d
Author: Steven Hardy <email address hidden>
Date: Mon Mar 21 17:03:43 2016 +0000

    Fix regression handling relative paths in environment

    7627e1bae53747e1471caf632dfc1b6edf2e26cf introduced new code to add the
    environment to the files mapping, so server side resolution of multiple
    environments is possible.

    However, the environment added is wrong when you're handling those with
    relative paths inside, we must add the resolved version or the keys
    don't map to keys in the files map, an you get an error when we try
    to locate the file on the server.

    Evidently we lack adequate functional test coverage of this, which can
    be addressed via a heat patch, but I added a unit test illustrating
    the problem.

    Change-Id: I3bd2cd48623e4ad1b4067f4cefc7ddff30b9e88d
    Closes-Bug: #1560106

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/296768
Committed: https://git.openstack.org/cgit/openstack/python-heatclient/commit/?id=4d253e7a9108a8e6e014e18c2f6412ece7b3bd0d
Submitter: Jenkins
Branch: stable/mitaka

commit 4d253e7a9108a8e6e014e18c2f6412ece7b3bd0d
Author: Steven Hardy <email address hidden>
Date: Mon Mar 21 17:03:43 2016 +0000

    Fix regression handling relative paths in environment

    7627e1bae53747e1471caf632dfc1b6edf2e26cf introduced new code to add the
    environment to the files mapping, so server side resolution of multiple
    environments is possible.

    However, the environment added is wrong when you're handling those with
    relative paths inside, we must add the resolved version or the keys
    don't map to keys in the files map, an you get an error when we try
    to locate the file on the server.

    Evidently we lack adequate functional test coverage of this, which can
    be addressed via a heat patch, but I added a unit test illustrating
    the problem.

    Change-Id: I3bd2cd48623e4ad1b4067f4cefc7ddff30b9e88d
    Closes-Bug: #1560106

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-heatclient 1.1.0 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/python-heatclient 1.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-heatclient 1.1.0 release.