Newton->Ocata upgrade fails during 'Check service openstack-ceilometer-api is running' task

[Marius Cornea]

Newton->Ocata upgrade fails during 'Check service openstack-ceilometer-api is running' task.

On the overcloud controllers openstack-ceilometer-api service is not running:

[root@overcloud-controller-0 ~]# /usr/bin/systemctl show 'openstack-ceilometer-api' --property ActiveState | grep '\bactive\b'; echo $?
1

[root@overcloud-controller-0 ~]# /usr/bin/systemctl status 'openstack-ceilometer-api'
● openstack-ceilometer-api.service - OpenStack ceilometer API service
   Loaded: loaded (/usr/lib/systemd/system/openstack-ceilometer-api.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

[stack@undercloud-0 tht]$ openstack stack failures list overcloud --long

overcloud.AllNodesUpgradeSteps.ControllerUpgrade_Step0.2:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: 80a192c1-9c34-43d7-83f8-4e2a23db49a0
  status: CREATE_FAILED
  status_reason: |
    Error: resources[2]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |

    PLAY [localhost] ***************************************************************

    TASK [setup] *******************************************************************
    ok: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-cinder-api is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-cinder-scheduler is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-glance-api is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-heat-api is running]
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-heat-api-cfn is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-heat-api-cloudwatch is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-heat-engine is running] ***
    changed: [localhost]

    TASK [Check for galera root password] ******************************************
    ok: [localhost]

    TASK [PreUpgrade step0,validation: Check service neutron-dhcp-agent is running]
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service neutron-l3-agent is running] **
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service neutron-metadata-agent is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service neutron-server is running] ****
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service neutron-openvswitch-agent is running] ***
    changed: [localhost]

    TASK [PreUpgrade step0,validation: Check service openstack-ceilometer-api is running] ***
    fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/bin/systemctl show 'openstack-ceilometer-api' --property ActiveState | grep '\\bactive\\b'", "delta": "0:00:00.012859", "end": "2017-02-02 13:38:39.878724", "failed": true, "rc": 1, "start": "2017-02-02 13:38:39.865865", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}
     to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/c9141e19-89e3-4af6-aaad-ee127f64191d_playbook.retry

    PLAY RECAP *********************************************************************
    localhost : ok=14 changed=12 unreachable=0 failed=1

  deploy_stderr: |

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/429749

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/429749
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=353c2dede0089149a2ac2d46634d1df4ab0d2441
Submitter: Jenkins
Branch: master

commit 353c2dede0089149a2ac2d46634d1df4ab0d2441
Author: Marius Cornea <email address hidden>
Date: Mon Feb 6 16:26:56 2017 +0100

    Remove openstack-ceilometer-api pre upgrade check

    This change removes the pre upgrade check for a running
    openstack-ceilometer-api service as this service doesn't exists in
    Newton. Ceilometer API runs under httpd:
    [root@overcloud-controller-0 ~]# httpd -t -D DUMP_VHOSTS | grep ceilo
    10.0.0.23:8777 overcloud-controller-0.internalapi.localdomain
    (/etc/httpd/conf.d/10-ceilometer_wsgi.conf:6)

    Change-Id: I5cbf8ccf72f9071e328f52d373cf9e8edf5793f4
    Closes-Bug: 1661251

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 6.0.0.0rc1 release candidate.