heat-api-cloudwatch service running after upgrade

[Jose Luis Franco]

(also discussed in https://bugzilla.redhat.com/show_bug.cgi?id=1496485)

Description
===========
When upgrading from Ocata to Pike containerized the environment ends up with the process heat-api-cloudwatch up and running under httpd in Pike upgraded overcloud intead of being containarized.

Steps to reproduce
==================

1. Deploy an overcloud:

openstack overcloud deploy --libvirt-type qemu --ntp-server pool.ntp.org --templates /home/stack/tht-ocata/ -e /home/stack/tht-ocata/overcloud-resource-registry-puppet.yaml

2. Check that the service is not started:

[heat-admin@overcloud-controller-0 ~]$ sudo systemctl list-units | grep heat
  session-5.scope loaded active running Session 5 of user heat-admin
  openstack-heat-api-cfn.service loaded active running Openstack Heat CFN-compatible API Service
  openstack-heat-api-cloudwatch.service loaded active running OpenStack Heat CloudWatch API Service
  openstack-heat-api.service loaded active running OpenStack Heat API Service
  openstack-heat-engine.service loaded active running Openstack Heat Engine Service
  user-1000.slice loaded active active User Slice of heat-admin

HTTP Heat Services: None
[heat-admin@overcloud-controller-0 ~]$ sudo httpd -t -D DUMP_VHOSTS | grep heat
[heat-admin@overcloud-controller-0 ~]$

3. Upgrade to Pike (containerized):
3.1. Download master tht to tht-master

3.2. Specify docker registry in docker_registry.yaml file:
parameter_defaults:
  DockerNamespace: 192.168.24.1:8787/tripleoupstream
  DockerNamespaceIsRegistry: true
EOF

3.3. Download container images:
openstack overcloud container image upload --config-file /usr/share/openstack-tripleo-common/container-images/overcloud_containers.yaml

3.4 Prepara container image definition yaml file:
openstack overcloud container image prepare \
--namespace tripleoupstream \
--tag latest \
--env-file docker-centos-tripleoupstream.yaml

3.5 Upgrade via command:
export THT=/home/stack/tht-master
(undercloud) [stack@undercloud ~]$ openstack overcloud deploy --templates $THT \
 --libvirt-type qemu \
 --ntp-server pool.ntp.org \
 -e $THT/environments/docker.yaml \
 -e $THT/environments/major-upgrade-composable-steps-docker.yaml \
 -e docker-centos-tripleoupstream.yaml \
 -e docker_registry.yaml \
 -e upgrade_repos.yaml

4. Service heat_api_cloudwatch is running under apache:

[heat-admin@overcloud-controller-0 ~]$ sudo httpd -t -D DUMP_VHOSTS | grep heat
192.168.24.7:8003 overcloud-controller-0.internalapi.localdomain (/etc/httpd/conf.d/10-heat_api_cloudwatch_wsgi.conf:6)

[heat-admin@overcloud-controller-0 ~]$ sudo systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/httpd.service.d
           └─openstack-dashboard.conf
   Active: active (running) since Tue 2017-09-26 13:46:07 UTC; 1h 53min ago
     Docs: man:httpd(8)
           man:apachectl(8)
 Main PID: 115195 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec"
   Memory: 355.0M
   CGroup: /system.slice/httpd.service
           ├─115195 /usr/sbin/httpd -DFOREGROUND
           ├─115243 cinder_wsgi -DFOREGROUND
           ├─115244 cinder_wsgi -DFOREGROUND
           ├─115245 heat_api_cloudw -DFOREGROUND <<<<<<<<<<
           ├─115247 /usr/sbin/httpd -DFOREGROUND
           ├─115248 /usr/sbin/httpd -DFOREGROUND
           ├─115249 /usr/sbin/httpd -DFOREGROUND
           ├─115250 /usr/sbin/httpd -DFOREGROUND
           ├─115251 /usr/sbin/httpd -DFOREGROUND
           ├─115253 /usr/sbin/httpd -DFOREGROUND
           ├─115263 /usr/sbin/httpd -DFOREGROUND
           └─115264 /usr/sbin/httpd -DFOREGROUND

5. But no heat service is running under systemd:
[heat-admin@overcloud-controller-0 ~]$ sudo systemctl list-units | grep heat
  session-27.scope loaded active running Session 27 of user heat-admin
  user-1000.slice loaded active active User Slice of heat-admin

6. All heat services are running under containers except heat-api-cloudwatch:

[heat-admin@overcloud-controller-0 ~]$ sudo docker ps | grep heat
085051b5dfbb tripleoupstream/centos-binary-heat-api:latest "kolla_start" About an hour ago Up About an hour (healthy) heat_api_cron
39e2b83a8bfc tripleoupstream/centos-binary-heat-api-cfn:latest "kolla_start" About an hour ago Up About an hour (healthy) heat_api_cfn
b80786761582 tripleoupstream/centos-binary-heat-engine:latest "kolla_start" About an hour ago Up About an hour (healthy) heat_engine
e9b2ddafcd7b tripleoupstream/centos-binary-heat-api:latest "kolla_start" About an hour ago Up About an hour (healthy) heat_api

Expected result
===============

Actual result
=============

Environment
===========

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/510101

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/510101
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=4d21451666f2dd7a8935da3a7166a9afc2ccd6bd
Submitter: Jenkins
Branch: master

commit 4d21451666f2dd7a8935da3a7166a9afc2ccd6bd
Author: marios <email address hidden>
Date: Fri Oct 6 15:47:32 2017 +0300

    Remove Heat Cloudwatch API during upgrade and disable by default

    This adds a heat-api-cloudwatch-disabled.yaml and wires it up in
    the resource registry. During the Ocata to Pike upgrade this service
    will thus be stopped and disabled by default.

    If you wish to keep the Heat Cloudwatch API then you should instead
    use the provided heat-api-cloudwatch.yaml environment file.

    Change-Id: I3f90a9799b90ca365f675f593371c1d3701fede6
    Related-Bug: 1713531

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/511155

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/511155
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=5a1297b02bfe51dd7707dce39eaf40b78f80f28e
Submitter: Zuul
Branch: stable/pike

commit 5a1297b02bfe51dd7707dce39eaf40b78f80f28e
Author: marios <email address hidden>
Date: Fri Oct 6 15:47:32 2017 +0300

    Remove Heat Cloudwatch API during upgrade and disable by default

    This adds a heat-api-cloudwatch-disabled.yaml and wires it up in
    the resource registry. During the Ocata to Pike upgrade this service
    will thus be stopped and disabled by default.

    If you wish to keep the Heat Cloudwatch API then you should instead
    use the provided heat-api-cloudwatch.yaml environment file.

    Change-Id: I3f90a9799b90ca365f675f593371c1d3701fede6
    Related-Bug: 1713531
    (cherry picked from commit 4d21451666f2dd7a8935da3a7166a9afc2ccd6bd)