unable to leverage swap-partition extraconfig

Bug #1625783 reported by Alex Schultz on 2016-09-20
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
High
Steven Hardy

Bug Description

With the merging of https://review.openstack.org/#/c/367295/, the following deployment configuration no longer works.

cat > ~/swap.yaml <<EOF
resource_registry:
  OS::TripleO::AllNodesExtraConfig: /usr/share/openstack-tripleo-heat-templates/extraconfig/all_nodes/swap-partition.yaml
EOF

openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e ~/swap.yaml

The swap-partition.yaml was renamed to swap-partition.j2.yaml but it is unclear how to deploy with this configuration. Attempting to switch out the file to use swap-parition.j2.yaml leads to a mistral error about invalid format, see http://paste.openstack.org/show/582306/ and http://paste.openstack.org/show/582307/.

Changed in tripleo:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → newton-rc2
assignee: nobody → Steven Hardy (shardy)
Emilien Macchi (emilienm) wrote :

I could reproduce the bug, and I tested with tripleoclient from master.

Changed in tripleo:
status: Confirmed → Triaged
Steven Hardy (shardy) wrote :

This is an ordering issue, caused by the change to j2 templating combined with moving all the templates into swift via mistral.

The problem is at the time of resolving the environment on the tripleoclient side, the swap-partition.j2.yaml file has not yet been rendered, so we either need the environment path resolution to happen only inside mistral, or for tripleoclient to resolve the paths on a local copy of the post-jinja templates (e.g download the entire plan from swift).

Thanks for the report, will look into it (note this is somewhat related to https://bugs.launchpad.net/tripleo/+bug/1623552)

Changed in tripleo:
importance: Critical → High
Steven Hardy (shardy) wrote :

I think the fix for this will be the same as for https://bugs.launchpad.net/tripleo/+bug/1624727

Changed in tripleo:
milestone: newton-rc2 → ocata-1
tags: added: newton-backport-potential
Changed in tripleo:
milestone: ocata-1 → newton-rc3
Steven Hardy (shardy) wrote :

So, there's two possible solutions to this:

First one is to use relative paths, not absolute, then the rendered file should be found in the swift container:

cat > ~/swap.yaml <<EOF
resource_registry:
  OS::TripleO::AllNodesExtraConfig: extraconfig/all_nodes/swap-partition.yaml
EOF

The other solution (which I've not yet tested) is matbu's patch for https://bugs.launchpad.net/tripleo/+bug/1624727 which I think should allow local resolution of the absolute path. (see https://review.openstack.org/#/c/379547/)

Alex Schultz (alex-schultz) wrote :

just FYI, the relative path does not work as it uses the current working directory to resolve the full path.

[centos@undercloud ~]$ openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e ~/swap.yaml
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
Removing the current plan files
Uploading new plan files
Started Mistral Workflow. Execution ID: 0883e60e-aff5-4fde-9a3e-ccd42e627c70
Plan updated
Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates
Could not fetch contents for file:///home/centos/extraconfig/all_nodes/swap-partition.yaml

Steven Hardy (shardy) wrote :

Ack, further testing today confirms the finding from comment #5 so I've been working on a refined version of the proposed fix for bug #1624727 which should fix both bugs.

Basically we're going to have to work around this by downloading the rendered templates inside tripleoclient to a temporary directory, then rewriting the user provided environment.

Not ideal, but the only workable short term solution (during ocata we need to discuss refining the interfaces here to reduce the client-side workarounds).

Changed in tripleo:
status: Triaged → Confirmed
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/383829
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=4abd0cd791a295f4fcfd0a5bf4ded6d3757f2d30
Submitter: Jenkins
Branch: master

commit 4abd0cd791a295f4fcfd0a5bf4ded6d3757f2d30
Author: Steven Hardy <email address hidden>
Date: Fri Oct 7 18:08:07 2016 +0100

    Allow referencing rendered yaml files in resource_registry

    For backwards compatibility we need a fallback when an environment
    file references a file that's *.j2.yaml in the user --templates dir
    but it's rendered via the plan generation and is then downloaded
    to the local temporary tree. In this case we've got to rewrite
    the paths in the resource_registry values to point to the new tree
    where the rendered templates exist.

    Without this logic, some environment files (such as those enabling
    swap for all roles) no longer work, so it's a backwards compatibility
    workaround.

    Change-Id: I560784dc49842113563f617b343702bb045fca76
    Closes-Bug: #1625783

Changed in tripleo:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/385325
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=c1868d2f560ee96a46b127af9ca86e1849c14f7d
Submitter: Jenkins
Branch: stable/newton

commit c1868d2f560ee96a46b127af9ca86e1849c14f7d
Author: Steven Hardy <email address hidden>
Date: Fri Oct 7 18:08:07 2016 +0100

    Allow referencing rendered yaml files in resource_registry

    For backwards compatibility we need a fallback when an environment
    file references a file that's *.j2.yaml in the user --templates dir
    but it's rendered via the plan generation and is then downloaded
    to the local temporary tree. In this case we've got to rewrite
    the paths in the resource_registry values to point to the new tree
    where the rendered templates exist.

    Without this logic, some environment files (such as those enabling
    swap for all roles) no longer work, so it's a backwards compatibility
    workaround.

    Change-Id: I560784dc49842113563f617b343702bb045fca76
    Closes-Bug: #1625783
    (cherry picked from commit 4abd0cd791a295f4fcfd0a5bf4ded6d3757f2d30)

tags: added: in-stable-newton

This issue was fixed in the openstack/python-tripleoclient 5.3.0 release.

This issue was fixed in the openstack/python-tripleoclient 5.5.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers