nova.servicegroup.drivers.db [-] model server went away

[Anastasia Palkina]

"build_id": "2014-08-21_02-01-17",
"ostf_sha": "c6ecd0137b5d7c1576fa65baef0fc70f9a150daa",
"build_number": "464",
"auth_required": true,
"api": "1.0",
"nailgun_sha": "25eba6fbb2047f26d9da4d27ffdb742c9c27832a",
"production": "docker",
"fuelmain_sha": "25a0c228d998707f90e90877559f17817a749d2f",
"astute_sha": "efe3cb3668b9079e68fb1534fd4649ac45a344e1",
"feature_groups": ["mirantis"],
"release": "5.1",
"fuellib_sha": "52f3ebfa968f0338e0584edf47cff10911109de5"

1. Create new environment (Ubuntu, HA mode)
2. Choose neutron, vlan
3. Choose both Ceph
4. Add 3 controller+ceph, 1 compute
5. Start deployment. Deployment has failed. Timeout of deployment is exceeded.

There are similar errors in nova-cert and cinder-volume logs (node-5):

nova.servicegroup.drivers.db [-] model server went away
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db Traceback (most recent call last):
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/servicegroup/drivers/db.py", line 95, in _report_state
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db service.service_ref, state_catalog)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 218, in service_update
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db return self._manager.service_update(context, service, values)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 967, in wrapper
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139, in inner
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db return func(*args, **kwargs)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 491, in service_update
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db svc = self.db.service_update(context, service['id'], values)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 148, in service_update
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 146, in wrapper
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db return f(*args, **kwargs)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 533, in service_update
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db service_ref.update(values)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 447, in __exit__
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self.rollback()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db compat.reraise(exc_type, exc_value, exc_tb)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 444, in __exit__
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self.commit()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 358, in commit
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db t[1].commit()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1195, in commit
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self._do_commit()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1226, in _do_commit
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self.connection._commit_impl()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 491, in _commit_impl
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self._handle_dbapi_exception(e, None, None, None, None)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db exc_info
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 195, in raise_from_cause
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 488, in _commit_impl
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db self.engine.dialect.do_commit(self.connection)
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db File "/usr/lib/python2.7/dist-packages/sqlalchemy/dialects/mysql/base.py", line 1961, in do_commit
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db dbapi_connection.commit()
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db OperationalError: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') None None
2014-08-21 11:00:52.018 31393 TRACE nova.servicegroup.drivers.db

nova.servicegroup.drivers.db [-] Recovered model server connection!

[Bogdan Dobrelya (bogdando)]

2014-08-21T12:07:41.884342 node-7 ./node-7.domain.tld/puppet-apply.log:2014-08-21T12:07:41.884342+01:00 debug: (/Stage[main]/Galera/Exec[wait-initial-sync]/returns) Exec try 60/60
2014-08-21T12:13:22.823529 node-7 ./node-7.domain.tld/puppet-apply.log:2014-08-21T12:13:22.823529+01:00 debug: (/Stage[main]/Galera/Exec[wait-for-synced-state]/returns) Exec try 60/60
2014-08-21T13:05:47.994647 node-7 ./node-7.domain.tld/puppet-apply.log:2014-08-21T13:05:47.994647+01:00 debug: (/Stage[main]/Galera/Exec[wait-for-synced-state]/returns) Exec try 60/60

Galera sync issue once again