Netwon to Ocata upgrade failure because of ceilometer-upgrade
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
Critical
|
Pradeep Kilambi |
Bug Description
Description
===========
While upgrading from Newton to Ocata, the upgrade fails reporting an error running ceilometer-upgrade
Error: /Stage[
Before upgrading to Ocata, Newton was updated to the latest Newton bits available on rdo-trunk-
Seems similar to https:/
Steps to reproduce
==================
* Deploy Newton openstack
* Update overcloud to latest Newton bits available
* Upgrade to Ocata
The upgrade script is the same than the deploy one but adding
-e templates/
-e overcloud-
-e skip-validation
Expected result
===============
UPDATE_COMPLETE
Actual result
=============
UPDATE_FAILED
Re-running ceilometer-upgrade on one of the controller fails.
The logs show ERROR 503 is reported while trying to reach keystone-admin.
All the resources in the controller pcs cluster are reported as being unmanaged
openstack endpoint list fails with error
Failed to contact the endpoint at http://
Unable to establish connection to http://
Setting the cluster out of maintenance mode does not bring keystone-admin back to test ceilometer-upgrade
Environment
===========
Three HA controllers (
Tripleo-related RPM:
puppet-
openstack-
openstack-
python-
openstack-
openstack-
openstack-
openstack-
openstack-
Logs & Configs
==============
ceilometer-
openstack stack failure list --long http://
Changed in tripleo: | |
importance: | Undecided → High |
milestone: | none → queens-2 |
status: | New → Triaged |
tags: | added: upgrade |
Changed in tripleo: | |
importance: | High → Critical |
Changed in tripleo: | |
assignee: | nobody → mathieu bultel (mat-bultel) |
Changed in tripleo: | |
assignee: | mathieu bultel (mat-bultel) → Pradeep Kilambi (pkilambi) |
status: | Triaged → In Progress |
In case it helps, find some responses I am getting when using openstackclient:
$ openstack endpoint list A.B.C.D: 35357 for discovery. Fallback to using that endpoint as the base url. A.B.C.D: 35357/endpoints: ('Connection aborted.', BadStatusLine( "''",))
Failed to contact the endpoint at http://
Unable to establish connection to http://
----
$ openstack server list
The server is currently unavailable. Please try again at a later time.<br /><br />
(HTTP 503) (Request-ID: req-6b0f1bc8- 457c-4ed5- 8978-b50e47df61 64) ------+ ------- ------- ------- ------- ------+ ------+ ------- ------- ------- ------- ------+ 18T14:02: 19+0000 | aa47b3b062bd9c2 cc | b873422092b923e af | 3aa2ce1eb03ffb4 c2 | ------+ ------- ------- ------- ------- ------+
----
$ openstack token issue
+------
| Field | Value |
+------
| expires | 2017-10-
| id | 447a16ef8bf6412
| project_id | 86dcd37905ec440
| user_id | 72f88c9866d14fc
+------