Heat environments don't work for local files

[Steven Hardy]

There is an issue with environment files (and heat templates which contain a file reference to a nested stack directly), which means that they don't work via Horizon.

The issue is you pass a template and optionally an environment file to Horizon via the file dialog, but Horizon doesn't have any way to resolve the file references and populate the "files" part of the API call to create the stack.

Additionally, the environment is not passed at all when trying to create the stack, so template validation actually fails before you hit the problem above.. ;)

Here are some simple examples which demonstrate the problem:

In all of the following examples, a default nova keypair of "stack_key" is expected, as is a glance image of "cirros-0.3.2-x86_64-disk", these can be overridden or modified obviously.

1. Template referencing a nested stack directly (no environment)

https://github.com/hardys/demo_templates/tree/master/juno_summit_intro_to_heat/example2_server_with_volume_nested

This can be launched on the CLI via:

heat --debug template-validate -f server_with_volume.yaml
heat --debug stack-create test1 -f server_with_volume.yaml

The --debug will show the content of the API call, which should hopefully help see what is missing from the horizon call (files section missing from the create)

2. Template referencing nested stack via environment

https://github.com/hardys/demo_templates/tree/master/juno_summit_intro_to_heat/example4_provider_environment

This can be launched on the CLI via:

heat --debug template-validate -f server_with_volume_env.yaml -e env_server_with_volume.yaml
heat --debug stack-create test2 -f server_with_volume_env.yaml -e env_server_with_volume.yaml

Currently it's not possible to launch either stack via horizon, unless the local file references are replaced with URLs.

[Steven Hardy (shardy)]

Correction "Additionally, the environment is not passed at all when trying to *validate* the stack"

[Steven Hardy (shardy)]

https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/stacks/forms.py#L142

Found it, that's where we get the environment but don't update the kwargs for validate

[Dale Aavang (dale-aavang)]

If the environment file is not being passed to the validation routine, it seems this issue also impacts template validation if the template depends on a resource_registry entry in the environment file?

I have an issue where I cannot launch a stack via horizon where the HOT template depends on a resource_registry declaration that is in the environment file. The same template and environment file work successful with the heat command line API.

[Steven Hardy (shardy)]

> If the environment file is not being passed to the validation routine, it seems this issue also impacts template validation if the template depends on a resource_registry entry in the environment file?

Yes, it probably does, although I've not tested with URL's in the environment myself it does seem that validation will fail unless the environment is overriding existing resource types, so if your resource_registry defines a new resource type then it probably won't work atm.

[Dale Aavang (dale-aavang)]

You are correct in my issue. The environment file is creating a new resource type and associating the new resource type with a URL to retrieve a file that defines the resource type. Since my template references the new resource type, the template validation is failing.

Do you feel we can cover this same issue with this bug, or should a new bug be created. The root cause is the same so I am thinking a single bug report that documents template validation failures due to the env file not being incorporated into the template validation would suffice.

[Steven Hardy (shardy)]

> Do you feel we can cover this same issue with this bug, or should a new bug be created.

We're going to have to fix validation to solve the issue reported in this bug, so I think one bug should suffice.

[Jordan OMara (jomara)]

After some digging (with the help of #heat), I also realized that the heat-client handles URLs differently for environment and template files. #heat mentioned that template_url is the "unfavored" way of sending template data. It was recommend that we do 2 things:

1) Fix the environment data in validation for now, and remove the environment URL option
2) In a later patch, add back the environment URL and change it + template URL to download the file client side and send that as template data, instead of sending the URLs directly

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/97899

[Steven Hardy (shardy)]

Moving back to in progress, as during IRC discussion with jomara we agreed the above is only a partial fix

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/97899
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=ecdc73a33954bb9e7b24f72dafd1cfba42f69631
Submitter: Jenkins
Branch: master

commit ecdc73a33954bb9e7b24f72dafd1cfba42f69631
Author: Jordan OMara <email address hidden>
Date: Wed Jun 4 15:45:39 2014 -0400

    Ensuring environment data is passed to heat template validate

     - environment data was being passed to heat API on stack create,
       but not stack validate - meaning that templates that referenced
       environment files were not validating correctly

     - heat client does not support URLs for environment files - we will
       need to do this client side before POSTing back to horizon. future
       patch

    Partial-Bug: #1322258

    Change-Id: I40982b777abaec743ed6cbc527008522a619e817

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/111379

[Jordan OMara (jomara)]

This bug will be completely fixed by:

https://review.openstack.org/#/c/111379/

[Steven Hardy (shardy)]

I'm a bit confused, this is marked fix committed, but https://review.openstack.org/#/c/111379/ is still under review - Jordan is that correct?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/118416

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jordan OMara (<email address hidden>) on branch: master
Review: https://review.openstack.org/111379
Reason: For whatever reason, my next "review" created a new change!

Please see: https://review.openstack.org/#/c/118416/

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Lyle (<email address hidden>) on branch: master
Review: https://review.openstack.org/118416
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Sachin (deoks24)]

Hi,

Is the support for this added to the rest end point as well? I want to use the same functionality via rest service.

Also, I see the fix being abandoned in the latest build ... any plans when this will be released?

Sachin

[Nikunj Aggarwal (nikunj2512)]

Hi Sachin,

As you can see from the logs, i took this bug few days ago only. As of now a partial fix have been released and i am working towards implementing the rest of the fix.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/151111

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/165354

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jeremy Stanley (<email address hidden>) on branch: stable/icehouse
Review: https://review.openstack.org/165354
Reason: This branch has reached end of life and is being deleted.

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Lyle (<email address hidden>) on branch: master
Review: https://review.openstack.org/151111
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.