Upgrading overcloud from MItaka to Newton fails, can't find templated file

[Graeme Gillies]

Hi,

I am just in the process of upgrading an RDO mitaka overcloud to an RDO Newton installation.

I have successfully upgrade my undercloud to newton.

I am at the first step

"And run overcloud deploy, passing in full set of environment files plus major-upgrade-pacemaker-init.yaml and overcloud-repos.yaml:

openstack overcloud deploy --templates \
    -e <full environment> \
    -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-init.yaml \
    -e overcloud-repos.yaml"

I have all my work in /home/stack/rhosops-test-tripleo/openstack-deployment (environment files, etc).

I have copied the new templates from

/usr/share/openstack-tripleo-heat-templates

into

/home/stack/rhosops-test-tripleo/openstack-deployment/templates

I have created a script
/home/stack/rhosops-test-tripleo/openstack-deployment/upgrades/mitaka-to-newton/01-major-upgrade-pacemaker-init.sh

with the following

#!/bin/bash
. /home/stack/stackrc
cd /home/stack/rhosops-test-tripleo/openstack-deployment
time openstack overcloud deploy \
--templates templates \
-t 180 \
--ntp-server clock.corp.redhat.com \
--validation-warnings-fatal \
--overcloud-ssh-user rdo-admins \
-e rhosops-test-node-data.yaml \
-e rhosops-test-scheduler-hints.yaml \
-e rhosops-test-storage-environment.yaml \
-e templates/environments/puppet-pacemaker.yaml \
-e templates/environments/network-isolation.yaml \
-e templates/environments/ips-from-pool-all.yaml \
-e rhosops-test-network-environment.yaml \
-e rhosops-test-customisations.yaml \
-e templates/environments/major-upgrade-pacemaker-init.yaml \
-e upgrades/mitaka-to-newton/overcloud-repos.yaml

my contents of upgrades/mitaka-to-newton/overcloud-repos.yaml is

parameter_defaults:
  UpgradeInitCommand: |
    set -e
    yum --enablerepo='*' clean all
    yum remove -y centos-release-openstack-mitaka
    yum install -y centos-release-openstack-newton

The error I get when I attempt to run the file is

$ ./01-major-upgrade-pacemaker-init.sh
Removing the current plan files
Uploading new plan files
Started Mistral Workflow. Execution ID: 6678e67b-c1e4-40fe-b47c-1fd539832676
Plan updated
Deploying templates in the directory /home/stack/rhosops-test-tripleo/openstack-deployment/templates
Could not fetch contents for file:///home/stack/rhosops-test-tripleo/openstack-deployment/templates/extraconfig/tasks/major_upgrade_pacemaker_init.yaml

real 0m27.140s
user 0m2.281s
sys 0m0.195s

I notice that

/home/stack/rhosops-test-tripleo/openstack-deployment/templates/extraconfig/tasks/major_upgrade_pacemaker_init.j2.yaml

exists, so I assume that somewhere along the line the file gets templated by mistral, put into swift, before being invoked by heat.

I'm not sure if this is because I prefer to use a local copy of templates (stored in git) and relative pathing of environment files vs absolute paths, or something else.

Regards,

Graeme

[Graeme Gillies (graeme-r-gillies)]

Weird thing is if I do

swift list overcloud

I can see

extraconfig/tasks/major_upgrade_pacemaker_init.yaml

In there. If I download it from swift it looks completely valid to me

[Michele Baldessari (michele)]

Do you have change If95825e7df5d2c0e6cbe3575d06d57db1e8182da in your tripleoclient? https://review.openstack.org/#/c/383825/ was merged a couple of days ago and I believe should have taken care of this

[Michele Baldessari (michele)]

Also please take a look at https://review.openstack.org/#/c/384717/ where we are trying to finalize the upgrade steps for M->N (there are at least two mandatory additional steps beyond the usual init, major-upgrade, converge steps.

[Graeme Gillies (graeme-r-gillies)]

Hi,

I am using

python-tripleoclient-5.2.1-0.1.34590ccgit.el7.noarch

Which is from RDO Stable CBS (CentOS Build System) repos at

http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/

It looks like we need to get those packages upgraded.

Thanks for the links to the (WIP) upgrade notes as well.

Regards,

Graeme

[Steven Hardy (shardy)]

Thanks for the report Graeme, not there are three closely related bugs (I think as Michele mentions that this is a dupe of bug #1624727) to be aware of:

https://bugs.launchpad.net/tripleo/+bug/1624727

https://bugs.launchpad.net/tripleo/+bug/1631426

https://bugs.launchpad.net/tripleo/+bug/1625783

I just approved the last of the fixes for these, and will ensure we've got stable backports proposed where not already done:

https://review.openstack.org/#/q/I560784dc49842113563f617b343702bb045fca76,n,z

https://review.openstack.org/#/c/383813/

https://review.openstack.org/#/q/If95825e7df5d2c0e6cbe3575d06d57db1e8182da,n,z

If you're happy those resolve this issue, please close the bug as a duplicate, thanks!

[Graeme Gillies (graeme-r-gillies)]

Hi Steven,

Thanks for the heads up about that, it's likely this is a duplicate of

https://bugs.launchpad.net/tripleo/+bug/1624727

Though I am not entirely sure then why it's not fixed in RDO Stable CBS. Hopefully that will get updated soon.

Regards,

Graeme