On subnode-2, manila and glance api died to resulted in TImed out due to DB connection error https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/308d696/logs/subnode-2/var/log/containers/glance/api.log.txt.gz#_2018-09-13_01_56_30_564
2018-09-13 01:56:30.564 20 ERROR glance Traceback (most recent call last): 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/bin/glance-manage", line 10, in <module> 2018-09-13 01:56:30.564 20 ERROR glance sys.exit(main()) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 563, in main 2018-09-13 01:56:30.564 20 ERROR glance return CONF.command.action_fn() 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 395, in sync 2018-09-13 01:56:30.564 20 ERROR glance self.command_object.sync(CONF.command.version) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 154, in sync 2018-09-13 01:56:30.564 20 ERROR glance curr_heads = alembic_migrations.get_current_alembic_heads() 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/alembic_migrations/__init__.py", line 52, in get_current_alembic_heads 2018-09-13 01:56:30.564 20 ERROR glance engine = db_api.get_engine() 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line 97, in get_engine 2018-09-13 01:56:30.564 20 ERROR glance facade = _create_facade_lazily() 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line 87, in _create_facade_lazily 2018-09-13 01:56:30.564 20 ERROR glance _FACADE = session.EngineFacade.from_config(CONF) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1352, in from_config 2018-09-13 01:56:30.564 20 ERROR glance expire_on_commit=expire_on_commit, _conf=conf) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1273, in __init__ 2018-09-13 01:56:30.564 20 ERROR glance slave_connection=slave_connection) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 496, in _start 2018-09-13 01:56:30.564 20 ERROR glance engine_args, maker_args) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 520, in _setup_for_connection 2018-09-13 01:56:30.564 20 ERROR glance sql_connection=sql_connection, **engine_kwargs) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/debtcollector/renames.py", line 43, in decorator 2018-09-13 01:56:30.564 20 ERROR glance return wrapped(*args, **kwargs) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 202, in create_engine 2018-09-13 01:56:30.564 20 ERROR glance test_conn = _test_connection(engine, max_retries, retry_interval) 2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 388, in _test_connection 2018-09-13 01:56:30.564 20 ERROR glance six.reraise(type(de_ref), de_ref) 2018-09-13 01:56:30.564 20 ERROR glance File "<string>", line 3, in reraise 2018-09-13 01:56:30.564 20 ERROR glance DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.23' ([Errno 113] No route to host)") (Background on this error at: http://sqlalche.me/e/e3q8) 2018-09-13 01:56:30.564 20 ERROR glance
and https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/308d696/logs/subnode-2/var/log/containers/manila/manila-manage.log.txt.gz#_2018-09-13_03_01_16_681
It might be a transitory issue.
On subnode-2, manila and glance api died to resulted in TImed out due to DB connection error /logs.rdoprojec t.org/openstack -periodic/ git.openstack. org/openstack- infra/tripleo- ci/master/ legacy- periodic- tripleo- ci-centos- 7-multinode- 1ctlr-featurese t019-master/ 308d696/ logs/subnode- 2/var/log/ containers/ glance/ api.log. txt.gz# _2018-09- 13_01_56_ 30_564
https:/
2018-09-13 01:56:30.564 20 ERROR glance Traceback (most recent call last): glance- manage" , line 10, in <module> python2. 7/site- packages/ glance/ cmd/manage. py", line 563, in main action_ fn() python2. 7/site- packages/ glance/ cmd/manage. py", line 395, in sync object. sync(CONF. command. version) python2. 7/site- packages/ glance/ cmd/manage. py", line 154, in sync migrations. get_current_ alembic_ heads() python2. 7/site- packages/ glance/ db/sqlalchemy/ alembic_ migrations/ __init_ _.py", line 52, in get_current_ alembic_ heads python2. 7/site- packages/ glance/ db/sqlalchemy/ api.py" , line 97, in get_engine facade_ lazily( ) python2. 7/site- packages/ glance/ db/sqlalchemy/ api.py" , line 87, in _create_ facade_ lazily EngineFacade. from_config( CONF) python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py", line 1352, in from_config on_commit= expire_ on_commit, _conf=conf) python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py", line 1273, in __init__ n=slave_ connection) python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py", line 496, in _start python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py", line 520, in _setup_ for_connection sql_connection, **engine_kwargs) python2. 7/site- packages/ debtcollector/ renames. py", line 43, in decorator python2. 7/site- packages/ oslo_db/ sqlalchemy/ engines. py", line 202, in create_engine n(engine, max_retries, retry_interval) python2. 7/site- packages/ oslo_db/ sqlalchemy/ engines. py", line 388, in _test_connection type(de_ ref), de_ref) err.Operational Error) (2003, "Can't connect to MySQL server on '192.168.24.23' ([Errno 113] No route to host)") (Background on this error at: http:// sqlalche. me/e/e3q8)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/bin/
2018-09-13 01:56:30.564 20 ERROR glance sys.exit(main())
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance return CONF.command.
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance self.command_
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance curr_heads = alembic_
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance engine = db_api.get_engine()
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance facade = _create_
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance _FACADE = session.
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance expire_
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance slave_connectio
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance engine_args, maker_args)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance sql_connection=
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance return wrapped(*args, **kwargs)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance test_conn = _test_connectio
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/
2018-09-13 01:56:30.564 20 ERROR glance six.reraise(
2018-09-13 01:56:30.564 20 ERROR glance File "<string>", line 3, in reraise
2018-09-13 01:56:30.564 20 ERROR glance DBConnectionError: (pymysql.
2018-09-13 01:56:30.564 20 ERROR glance
and /logs.rdoprojec t.org/openstack -periodic/ git.openstack. org/openstack- infra/tripleo- ci/master/ legacy- periodic- tripleo- ci-centos- 7-multinode- 1ctlr-featurese t019-master/ 308d696/ logs/subnode- 2/var/log/ containers/ manila/ manila- manage. log.txt. gz#_2018- 09-13_03_ 01_16_681
https:/
It might be a transitory issue.