tripleo

  1. Bug #1634823
  2. Comment #4

Comment 4 for bug 1634823

Revision history for this message
Michele Baldessari (michele) wrote :

There are two separate issues we will need to address:
1) Today in Liberty and Mitaka when we create the endpoints after the heat stack completed the deployment, we actually have no guarantee whatsoever that keystone is really up and running and ready to serve. We do not have this issue in Newton because the endpoint creation is done via puppet. What we probably want to do here for L/M is to add a reasonable retry logic in the tripleoclient if we get a 503 error from keystone.

2) The reason we seem to get a 503 at least on Mitaka where keystone runs under httpd under two virtual hosts and the main one listening to port 5000 starts 4-5 *minutes*. For example:
http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-ha-mitaka/8b298bc/logs/overcloud-controller-0/var/log/keystone/keystone.txt.gz#_2016-10-24_07_24_46_294 has the following:

2016-10-24 07:24:46.294 19600 WARNING oslo_config.cfg [req-dd11202b-309d-448c-b8dd-25dbc140dfe2 - - - - -] Option "admin_bind_host" from group "DEFAULT" is deprecated. Use option "admin_bind_host" from group "eventlet_server".
2016-10-24 07:24:46.295 19600 WARNING oslo_config.cfg [req-dd11202b-309d-448c-b8dd-25dbc140dfe2 - - - - -] Option "admin_bind_host" from group "eventlet_server" is deprecated for removal. Its value may be silently ignored in the future.
....
2016-10-24 07:30:19.383 19601 WARNING oslo_config.cfg [req-60a32b5e-86da-40f2-bb35-15fe759876d9 - - - - -] Option "admin_bind_host" from group "DEFAULT" is deprecated. Use option "admin_bind_host" from group "eventlet_server".
2016-10-24 07:30:19.383 19601 WARNING oslo_config.cfg [req-60a32b5e-86da-40f2-bb35-15fe759876d9 - - - - -] Option "admin_bind_host" from group "eventlet_server" is deprecated for removal. Its value may be silently ignored in the future.

The first one is the admin instance on 35357 and the second one is on port 5000. This took almost *five* minutes.

I still need to investigate what has changed with liberty recently because we do not have keystone running under httpd there. We for sure do have the problem that when the heat stack is completed we have no guarantee whatsoever that it is ready to serve (see prob 1) above)