OC upgrade O->P containers fails with error: su: unrecognized option '--sacks-number=128'

[Jose Luis Franco]

Description
===========
During the upgrade from Ocata to Pike (containerized) using latest master it was returned an error in one of the steps and the upgrade didn't succeed:

http://paste.openstack.org/show/617787/

The root cause seems to be a wrong argument in the command : /usr/bin/bootstrap_host_exec gnocchi_api su gnocchi -s /bin/bash -c /usr/bin/gnocchi-upgrade --sacks-number=128

"Usage:",
            " su [options] [-] [USER [arg]...]",
            "Change the effective user id and group id to that of USER.",
            "A mere - implies -l. If USER not given, assume root.",
            "Options:",
            " -m, -p, --preserve-environment do not reset environment variables",
            " -g, --group <group> specify the primary group",
            " -G, --supp-group <group> specify a supplemental group",
            " -, -l, --login make the shell a login shell",
            " -c, --command <command> pass a single command to the shell with -c",
            " --session-command <command> pass a single command to the shell with -c",
            " and do not create a new session",
            " -f, --fast pass -f to the shell (for csh or tcsh)",
            " -s, --shell <shell> run shell if /etc/shells allows it",
            " -h, --help display this help and exit",
            " -V, --version output version information and exit",
            "For more details see su(1).",
            "stderr: su: unrecognized option '--sacks-number=128'",

Steps to reproduce
==================

Follow this instructions: http://www.anstack.com/blog/2016/11/28/testing-composable-upgrades.html

1. Deploy Pike undercloud and overcloud using TripleO-quickstart:
bash ./tripleo-quickstart/quickstart.sh --install-deps
export VIRTHOST=127.0.0.2
export CONFIG=~/deploy-config.yaml
bash ./tripleo-quickstart/quickstart.sh --clean --release master --teardown all --tags all -e @$CONFIG $VIRTHOST

2. Delete overcloud:
source ~/stackrc
openstack stack delete overcloud

3. Delete images and load new ones

4. Download tht for ocata and deploy Ocata overcloud (non-HA)
git clone -b stable/ocata https://github.com/openstack/tripleo-heat-templates tht-ocata
openstack overcloud deploy \
  --libvirt-type qemu \
  --ntp-server pool.ntp.org \
  --templates /home/stack/tht-ocata/ \
  -e /home/stack/tht-ocata/overcloud-resource-registry-puppet.yaml

5. Once deployed, download tht-master and load docker images:
git clone https://github.com/openstack/tripleo-heat-templates tht-master
openstack overcloud container image upload --config-file /usr/share/openstack-tripleo-common/container-images/overcloud_containers.yaml

6. Perform overcloud upgrade:
source ~/stackrc
export THT=/home/stack/tht-master
openstack overcloud deploy
--templates $THT
--libvirt-type qemu
--ntp-server pool.ntp.org
-e $THT/overcloud-resource-registry-puppet.yaml
-e $THT/environments/major-upgrade-composable-steps.yaml
-e upgrade_repos.yaml
-e $THT/environments/docker.yaml
-e $THT/environments/major-upgrade-composable-steps-docker.yaml
-e $THT/environments/docker-centos-tripleoupstream.yaml
-e docker_registry.yaml

Expected result
===============
Overcloud upgrade from Ocata to Pike successfully performed

Actual result
=============
(undercloud) [stack@undercloud ~]$ openstack stack list
+--------------------------------------+------------+----------------------------------+---------------+----------------------+----------------------+
| ID | Stack Name | Project | Stack Status | Creation Time | Updated Time |
+--------------------------------------+------------+----------------------------------+---------------+----------------------+----------------------+
| 4accb024-7125-421e-9bde-37f669da042b | overcloud | 2137162f9649432289f959f0881f3a35 | UPDATE_FAILED | 2017-08-07T13:47:51Z | 2017-08-08T10:49:43Z |
+--------------------------------------+------------+----------------------------------+---------------+----------------------+----------------------+

Full failure output:
http://paste.openstack.org/show/617787/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/491826

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/491826
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9b79a9bc7424dc5de71d0024117a768f8d1f3bed
Submitter: Jenkins
Branch: master

commit 9b79a9bc7424dc5de71d0024117a768f8d1f3bed
Author: Jose Luis Franco Arza <email address hidden>
Date: Tue Aug 8 16:19:20 2017 +0200

    Correct gnocchi-upgrade command quotes

    After merging commit 488796, single quotation marks
    were missed. This causes the upgrade to fail as the
    flag --sacks-number is considered a su command flag.

    Also mounts Ceph config data into the container which
    seems needed for the gnocchi-upgrade command when
    configured to use Ceph.

    Also move the gnocchi db sync to step 4, so ceph is
    ready. Add a retry loop to ceilometer-upgrade cmd so
    it doesnt fail while apache is restarted.

    Closes-Bug: #1709322
    Change-Id: I62f3a5fa2d43a2cd579f72286661d503e9f08b90

[Jose Luis Franco (jfrancoa)]

Jenkins jobs are now passing and the error is not appearing any more:
http://logs.openstack.org/26/491826/9/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/b26a86b/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.0.0rc1 release candidate.