j2 templates are not rendered during upgrade

[Sergii Golovatiuk]

A few templates for services such as apache, haproxy-internal-tls-certmonger were transformed to jinja2 templates

[stack@undercloud services]$ pwd
/home/stack/tripleo-heat-templates/puppet/services
[stack@undercloud services]$ ls -la *j2*
-rw-rw-r--. 1 stack stack 4879 Apr 3 18:29 apache.j2.yaml
-rw-rw-r--. 1 stack stack 3331 Apr 3 18:29 haproxy-internal-tls-certmonger.j2.yaml

However, during 'openstack overcloud upgrade prepare' they are not rendered to yaml thus upgrade fails with the following error

https://paste.fedoraproject.org/paste/ZnqabEvmDP39cQ4Co2QNTQ

When I render manually using

cd /usr/share/openstack-tripleo-heat-templates/environments/
sudo ./tools/process-templates.py
cd

then I was able to run

openstack overcloud upgrade prepare

without any issues.

[Sergii Golovatiuk (sgolovatiuk)]

I performed the RCA of this bug

When operator runs

openstack overcloud upgrade prepare \
  --templates ~/tripleo-heat-templates
  -e /usr/share/openstack-tripleo-heat-templates/*.yaml

the j2 templates from '~/tripleo-heat-templates' will never be rendered and uploaded to swift.

As a workaround --templates and -e should point to the same directory. In that case, j2 templates will be rendered correctly.

[Sergii Golovatiuk (sgolovatiuk)]

There is insufficient validation in framework as operator may run whatever framework allows to.

[Steven Hardy (shardy)]

> openstack overcloud upgrade prepare \
  --templates ~/tripleo-heat-templates
  -e /usr/share/openstack-tripleo-heat-templates/*.yaml

This won't work - the --templates path must match the -e path, e.g:

openstack overcloud upgrade prepare \
  --templates ~/tripleo-heat-templates
  -e ~/tripleo-heat-templates/*.yaml

It's a known issue, this has never worked due to this path comparison in tripleoclient:

https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L327

It may be that could be done differently, but personally I think mixing t-h-t trees like this is very bad practice, you could easily end up with weird problems due to mismatched templates and environments, so probably the proper fix here is to enforce that any environments that are j2 rendered are inside the --templates tree via some additional validation in the client?

[Sergii Golovatiuk (sgolovatiuk)]

I am trying to think as operator. I might not know good/bad practices. I might be a new operator to OOO. So, I agree there should be additional validation in client. At least there should be a alert that combination is not good and may lead to unknown/untested behaviour.

[Bogdan Dobrelya (bogdando)]

There are totally valid cases when --templates is mixed with auto generated -e files residing in either temp dirs or the user's home dir. For example, heat undercloud installer generates such files. We can not blindly prohibit non matching --templates and -e's. We should define additional some constraints.

[Bogdan Dobrelya (bogdando)]

I'll add that additional contstraint as "The non-matching path of a "-e" file shall not contain a resource_registry"

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/561192

[Bogdan Dobrelya (bogdando)]

We have a very similar issue for undercloud heat installer https://bugs.launchpad.net/tripleo/+bug/1765358

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/561192
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=038ed010587e4c91d2197faccaed2e45930255aa
Submitter: Zuul
Branch: master

commit 038ed010587e4c91d2197faccaed2e45930255aa
Author: Bogdan Dobrelya <email address hidden>
Date: Fri Apr 13 11:00:24 2018 +0200

    UC: validate paths for tht templates and env files

    The --templates path must match the -e path, e.g:

    openstack overcloud upgrade prepare \
      --templates ~/tripleo-heat-templates
      -e ~/tripleo-heat-templates/*.yaml

    Though, there are totally valid cases when --templates is
    mixed with auto generated -e files residing in either temp
    dirs or the user's home dir. For example, heat undercloud
    installer generates such files. So we can not blindly
    prohibit non matching --templates and -e's.

    To adress that UX issue, add additional contstraints for
    undercloud tht env files validation:

    * "-e" files can not refer external files normally
      processed from jinja2 in t-h-t. Those must come from
      the templates path only.

    NOTE: the similar change for overcloud should be done
    in follow-ups.

    Partial-bug: #1762403

    Depends-On: I10d4dffcd3802f62fc824c808728c0b5b4f1002c
    Change-Id: Ia9c62c787d6c581a66b2fde030a60499cfa18b82
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[Emilien Macchi (emilienm)]

Sergii, can we close this one?

[Sergii Golovatiuk (sgolovatiuk)]

Yes, we can close this. Bogdan's patch fixes the issue.