Comment 1 for bug 1792296

Revision history for this message
chandan kumar (chkumar246) wrote :

On subnode-2, manila and glance api died to resulted in TImed out due to DB connection error
https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/308d696/logs/subnode-2/var/log/containers/glance/api.log.txt.gz#_2018-09-13_01_56_30_564

2018-09-13 01:56:30.564 20 ERROR glance Traceback (most recent call last):
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/bin/glance-manage", line 10, in <module>
2018-09-13 01:56:30.564 20 ERROR glance sys.exit(main())
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 563, in main
2018-09-13 01:56:30.564 20 ERROR glance return CONF.command.action_fn()
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 395, in sync
2018-09-13 01:56:30.564 20 ERROR glance self.command_object.sync(CONF.command.version)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 154, in sync
2018-09-13 01:56:30.564 20 ERROR glance curr_heads = alembic_migrations.get_current_alembic_heads()
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/alembic_migrations/__init__.py", line 52, in get_current_alembic_heads
2018-09-13 01:56:30.564 20 ERROR glance engine = db_api.get_engine()
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line 97, in get_engine
2018-09-13 01:56:30.564 20 ERROR glance facade = _create_facade_lazily()
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/glance/db/sqlalchemy/api.py", line 87, in _create_facade_lazily
2018-09-13 01:56:30.564 20 ERROR glance _FACADE = session.EngineFacade.from_config(CONF)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1352, in from_config
2018-09-13 01:56:30.564 20 ERROR glance expire_on_commit=expire_on_commit, _conf=conf)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1273, in __init__
2018-09-13 01:56:30.564 20 ERROR glance slave_connection=slave_connection)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 496, in _start
2018-09-13 01:56:30.564 20 ERROR glance engine_args, maker_args)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 520, in _setup_for_connection
2018-09-13 01:56:30.564 20 ERROR glance sql_connection=sql_connection, **engine_kwargs)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/debtcollector/renames.py", line 43, in decorator
2018-09-13 01:56:30.564 20 ERROR glance return wrapped(*args, **kwargs)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 202, in create_engine
2018-09-13 01:56:30.564 20 ERROR glance test_conn = _test_connection(engine, max_retries, retry_interval)
2018-09-13 01:56:30.564 20 ERROR glance File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 388, in _test_connection
2018-09-13 01:56:30.564 20 ERROR glance six.reraise(type(de_ref), de_ref)
2018-09-13 01:56:30.564 20 ERROR glance File "<string>", line 3, in reraise
2018-09-13 01:56:30.564 20 ERROR glance DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '192.168.24.23' ([Errno 113] No route to host)") (Background on this error at: http://sqlalche.me/e/e3q8)
2018-09-13 01:56:30.564 20 ERROR glance

and
https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/308d696/logs/subnode-2/var/log/containers/manila/manila-manage.log.txt.gz#_2018-09-13_03_01_16_681

It might be a transitory issue.