tripleo

Bug #1638428
Comment #3

Comment 3 for bug 1638428

Revision history for this message

Carlos Camacho (ccamacho) wrote on 2016-11-02:

Hey,

I just have finished some test related to how to use the --environment-directory option.

### STEP 1 ### Reproducing the issue

[stack@instack ~]$ cd
[stack@instack ~]$ pwd
/home/stack

# Creating a temp folder to move an env file
[stack@instack ~]$ mkdir kk
[stack@instack ~]$ cp tripleo-heat-templates/environments/puppet-pacemaker.yaml kk/

# We will deploy puppet-pacemaker.yaml inside the templates folder
[stack@instack ~]$ openstack overcloud deploy --libvirt-type qemu --ntp-server pool.ntp.org --templates /home/stack/tripleo-heat-templates --environment-directory /home/stack/kk/ --debug

# You have the error described in the bug.

Environment directory: kk/
Environment directory file: kk/puppet-pacemaker.yaml
Processing environment files ['/home/stack/overcloud-env.json', u'kk/puppet-pacemaker.yaml']
Processing environment files /home/stack/overcloud-env.json
Processing environment files kk/puppet-pacemaker.yaml
Error Could not fetch contents for file:///home/stack/puppet/services/pacemaker/rabbitmq.yaml processing environment file kk/puppet-pacemaker.yaml

### STEP 2 ### Checking some behaviour

The thing here is that rabbitmq.yaml is referenced using a relative path in puppet-pacemaker.yaml, relative to the --templates folder.
i.e.
OS::TripleO::Services::RabbitMQ: ../puppet/services/pacemaker/rabbitmq.yaml

A quick workaround might be to update the path to the templates in your `kk` folder.

resource_registry:
  OS::TripleO::ControllerConfig: /home/stack/tripleo-heat-templates/puppet/controller-config-pacemaker.yaml
  OS::TripleO::Tasks::ControllerPrePuppet: /home/stack/tripleo-heat-templates/extraconfig/tasks/pre_puppet_pacemaker.yaml
  OS::TripleO::Tasks::ControllerPostPuppet: /home/stack/tripleo-heat-templates/extraconfig/tasks/post_puppet_pacemaker.yaml
  OS::TripleO::Tasks::ControllerPostPuppetRestart: /home/stack/tripleo-heat-templates/extraconfig/tasks/post_puppet_pacemaker_restart.yaml
  OS::TripleO::Services::CinderVolume: /home/stack/tripleo-heat-templates/puppet/services/pacemaker/cinder-volume.yaml
  OS::TripleO::Services::RabbitMQ: /home/stack/tripleo-heat-templates/puppet/services/pacemaker/rabbitmq.yaml
  OS::TripleO::Services::HAproxy: /home/stack/tripleo-heat-templates/puppet/services/pacemaker/haproxy.yaml
  OS::TripleO::Services::Pacemaker: /home/stack/tripleo-heat-templates/puppet/services/pacemaker.yaml
  OS::TripleO::Services::Redis: /home/stack/tripleo-heat-templates/puppet/services/pacemaker/database/redis.yaml
  OS::TripleO::Services::MySQL: /home/stack/tripleo-heat-templates/puppet/services/pacemaker/database/mysql.yaml

# Try to re-deploy.
[stack@instack ~]$ openstack overcloud deploy --libvirt-type qemu --ntp-server pool.ntp.org --templates /home/stack/tripleo-heat-templates --environment-directory /home/stack/kk/ --debug

As you can see the templates were found (We just hard wired the path)

Deploying templates in the directory /tmp/tripleoclient-CZhMjc/tripleo-heat-templates
Creating Environment file
Environment directory: /home/stack/kk/
Environment directory file: /home/stack/kk/puppet-pacemaker.yaml
Processing environment files [u'/home/stack/kk/puppet-pacemaker.yaml']
Processing environment files /home/stack/kk/puppet-pacemaker.yaml
Adding files {u'file:///home/stack/tripleo-heat-templates/puppet/services/pacemaker/haproxy.yaml': '{"outputs": {"role_data": {"description": "Role data for the HAproxy with pacemaker role.", "value": {"service_name": "haproxy", "step_config": "include ::tripleo::profile::pacemaker::haproxy", "config_settings": {"map_merge": [{"get_attr": ["LoadbalancerServiceBase", "role_data", "config_settings"]}, {"trip.....

# Ideally, those env. files should be referenced with the -e option if you are not going to modify them i.e.

openstack overcloud deploy \
--libvirt-type qemu \
--ntp-server pool.ntp.org \
--templates /home/stack/tripleo-heat-templates \
-e /home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml \
-e /home/stack/tripleo-heat-templates/environments/puppet-pacemaker.yaml

# Now if you want to use the --environment-directory let's move a few env files to the `kk` folder.
The env. files that we are going to use does not have any dependency with any other file.

[stack@instack kk]$ ls
config-debug.yaml debug.yaml use-dns-for-vips.yaml

# And redeploy with.

# Those env. files were loaded correctly.

Creating Environment file
Environment directory: /home/stack/kk/
Environment directory file: /home/stack/kk/config-debug.yaml
Environment directory file: /home/stack/kk/debug.yaml
Environment directory file: /home/stack/kk/use-dns-for-vips.yaml
Processing environment files [u'/home/stack/kk/config-debug.yaml', u'/home/stack/kk/debug.yaml', u'/home/stack/kk/use-dns-for-vips.yaml', u'/home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml', u'/home/stack/tripleo-heat-templates/environments/puppet-pacemaker.yaml']
Processing environment files /home/stack/kk/config-debug.yaml
Processing environment files /home/stack/kk/debug.yaml
Processing environment files /home/stack/kk/use-dns-for-vips.yaml
Processing environment files /home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml
Redirecting env file /home/stack/tripleo-heat-templates/overclo......

# Now let's try with some env files updating the resource registry.
Create inside `kk` custom_timezone.yaml and services/timezone.yaml:

custom_timezone.yaml with:

resource_registry:
OS::TripleO::Services::Timezone: ./services/timezone.yaml

services/timezone.yaml with:

heat_template_version: 2016-04-08

description: >
Composable Timezone service

parameters:
   ServiceNetMap:
     default: {}
     description: Mapping of service_name -> network name. Typically set
           via parameter_defaults in the resource registry. This
           mapping overrides those in ServiceNetMapDefaults.
     type: json
   DefaultPasswords:
     default: {}
     type: json
   EndpointMap:
     default: {}
     description: Mapping of service endpoint -> protocol. Typically set
           via parameter_defaults in the resource registry.
     type: json
   TimeZone:
     default: 'UTC'
     description: The timezone to be set on the overcloud.
     type: string

outputs:
   role_data:
     description: Timezone role using composable services.
     value:
       service_name: timezone
       config_settings:
  timezone::timezone: {get_param: TimeZone}
       step_config: |
  include ::timezone

# And redeploy with..

# This will load your files correctly

Environment directory: /home/stack/kk/
Environment directory file: /home/stack/kk/custom_timezone.yaml
Processing environment files [u'/home/stack/kk/custom_timezone.yaml', u'/home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml', u'/home/stack/tripleo-heat-templates/environments/puppet-pacemaker.yaml']
Processing environment files /home/stack/kk/custom_timezone.yaml
Adding files {u'file:///home/stack/kk/services/timezone.yaml': '{"outputs": {"role_data": {"description": "Timezone role using composable services.", "value": {"service_name": "timezone", "step_config": "include ::timezone", "config_settings": {"timezone::timezone": {"get_param": "TimeZone"}}}}}, "heat_template_version": "2016-04-08", "description": "Composable Timezone service\\n", "parameters": {"TimeZone": {"default": "UTC", "type": "string", "description": "The timezone to be set on the overcloud."}, "DefaultPasswords": {"default": {}, "type": "json"}, "EndpointMap": {"default": {}, "type": "json", "description": "Mapping of service endpoint -> protocol. Typically set via parameter_defaults in the resource registry."}, "ServiceNetMap": {"default": {}, "type": "json", "description": "Mapping of service_name -> network name. Typically set via parameter_defaults in the resource registry. This mapping overrides those in ServiceNetMapDefaults."}}}'} for /home/stack/kk/custom_timezone.yaml

# Note:
# Take into account that when using --environment-directory it's not recursing into the subfolders.

Hey,

I just have finished some test related to how to use the --environment-directory option.

### STEP 1 ### Reproducing the issue

[stack@instack ~]$ cd
[stack@instack ~]$ pwd
/home/stack

# Creating a temp folder to move an env file
[stack@instack ~]$ mkdir kk
[stack@instack ~]$ cp tripleo-heat-templates/environments/puppet-pacemaker.yaml kk/

# You have the error described in the bug.

### STEP 2 ### Checking some behaviour

The thing here is that rabbitmq.yaml is referenced using a relative path in puppet-pacemaker.yaml, relative to the --templates folder.
i.e.
  OS::TripleO::Services::RabbitMQ: ../puppet/services/pacemaker/rabbitmq.yaml

A quick workaround might be to update the path to the templates in your `kk` folder.

As you can see the templates were found (We just hard wired the path)

# Ideally, those env. files should be referenced with the -e option if you are not going to modify them  i.e.

# Now if you want to use the --environment-directory let's move a few env files to the `kk` folder.
  The env. files that we are going to use does not have any dependency with any other file.

[stack@instack kk]$ ls
config-debug.yaml  debug.yaml  use-dns-for-vips.yaml

# And redeploy with.

# Those env. files were loaded correctly.

# Now let's try with some env files updating the resource registry.
  Create inside `kk` custom_timezone.yaml and services/timezone.yaml:

custom_timezone.yaml with:

resource_registry:
	  OS::TripleO::Services::Timezone: ./services/timezone.yaml

services/timezone.yaml with:

heat_template_version: 2016-04-08

description: >
	  Composable Timezone service

parameters:
	  ServiceNetMap:
	    default: {}
	    description: Mapping of service_name -> network name. Typically set
		         via parameter_defaults in the resource registry.  This
		         mapping overrides those in ServiceNetMapDefaults.
	    type: json
	  DefaultPasswords:
	    default: {}
	    type: json
	  EndpointMap:
	    default: {}
	    description: Mapping of service endpoint -> protocol. Typically set
		         via parameter_defaults in the resource registry.
	    type: json
	  TimeZone:
	    default: 'UTC'
	    description: The timezone to be set on the overcloud.
	    type: string

outputs:
	  role_data:
	    description: Timezone role using composable services.
	    value:
	      service_name: timezone
	      config_settings:
		timezone::timezone: {get_param: TimeZone}
	      step_config: |
		include ::timezone

# And redeploy with..

# This will load your files correctly

Environment directory: /home/stack/kk/
Environment directory file: /home/stack/kk/custom_timezone.yaml
Processing environment files [u'/home/stack/kk/custom_timezone.yaml', u'/home/stack/tripleo-heat-templates/overcloud-resource-registry-puppet.yaml', u'/home/stack/tripleo-heat-templates/environments/puppet-pacemaker.yaml']
Processing environment files /home/stack/kk/custom_timezone.yaml
Adding files {u'file:///home/stack/kk/services/timezone.yaml': '{"outputs": {"role_data": {"description": "Timezone role using composable services.", "value": {"service_name": "timezone", "step_config": "include ::timezone", "config_settings": {"timezone::timezone": {"get_param": "TimeZone"}}}}}, "heat_template_version": "2016-04-08", "description": "Composable Timezone service\\n", "parameters": {"TimeZone": {"default": "UTC", "type": "string", "description": "The timezone to be set on the overcloud."}, "DefaultPasswords": {"default": {}, "type": "json"}, "EndpointMap": {"default": {}, "type": "json", "description": "Mapping of service endpoint -> protocol. Typically set via parameter_defaults in the resource registry."}, "ServiceNetMap": {"default": {}, "type": "json", "description": "Mapping of service_name -> network name. Typically set via parameter_defaults in the resource registry.  This mapping overrides those in ServiceNetMapDefaults."}}}'} for /home/stack/kk/custom_timezone.yaml

# Note:
# Take into account that when using --environment-directory it's not recursing into the subfolders.