tripleo

  1. Bug #1634823
  2. Comment #2

Comment 2 for bug 1634823

Revision history for this message
Michele Baldessari (michele) wrote :

One thing I noticed is the following on controller-0:
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core [req-a910cef1-4202-41dd-a895-d5d04b3076ae - - - - -] Could not find user: ceilometer
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core Traceback (most recent call last):
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/auth/plugins/core.py", line 173, in _validate_and_normalize_auth_data
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core user_name, domain_ref['id'])
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 124, in wrapped
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core __ret_val = __f(*args, **kwargs)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 433, in wrapper
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core return f(self, *args, **kwargs)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 443, in wrapper
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core return f(self, *args, **kwargs)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1053, in decorate
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core should_cache_fn)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 657, in get_or_create
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core async_creator) as value:
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 158, in __enter__
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core return self._enter()
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 98, in _enter
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core generated = self._enter_create(createdtime)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/core/dogpile.py", line 149, in _enter_create
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core created = self.creator()
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 625, in gen_value
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core created_value = creator()
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1049, in creator
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core return fn(*arg, **kw)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 902, in get_user_by_name
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core ref = driver.get_user_by_name(user_name, domain_id)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core File "/usr/lib/python2.7/site-packages/keystone/identity/backends/sql.py", line 253, in get_user_by_name
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core raise exception.UserNotFound(user_id=user_name)
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core UserNotFound: Could not find user: ceilometer
keystone/keystone.log:2016-10-19 10:49:03.523 16145 ERROR keystone.auth.plugins.core
keystone/keystone.log:2016-10-19 10:49:03.527 16145 WARNING keystone.common.wsgi [req-a910cef1-4202-41dd-a895-d5d04b3076ae - - - - -] Authorization failed. The request you have made requires authentication. from 172.17.0.13

Although in the CI job http://logs.openstack.org/68/388568/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/b8e1308/ we have the following:
2016-10-19 10:47:14.651913 | 2016-10-19 10:47:02 [ExtraConfig]: CREATE_COMPLETE state changed
2016-10-19 10:47:14.651994 | 2016-10-19 10:47:02 [overcloud-BlockStorageNodesPostDeployment-5cqyvvmdpldh]: CREATE_COMPLETE Stack CREATE completed successfully
2016-10-19 10:47:15.342118 | Stack overcloud CREATE_COMPLETE
2016-10-19 10:48:23.482117 | Service Unavailable (HTTP 503)
2016-10-19 10:48:23.544456 | #################

But the 503 gets returned *before* the ceilometer messages (unless there is a clock skew undercloud/overcloud). Not 100% sure that error is relevant to this failure, but the ceilometer exception is probably worth investigating anyway