Nova cells can die unexpectedly on boot due to db failure
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Medium
|
Christopher Lefelhocz |
Bug Description
We have seen a crash in the cells booting process with the following traceback:
2014-07-15 01:00:07.688 3070 CRITICAL nova [req-badc12a2-
2014-07-15 01:00:07.688 3070 TRACE nova Traceback (most recent call last):
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova sys.exit(main())
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova manager=
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova db_allowed=
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self.manager = manager_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self.state_manager = cell_state_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova return CellStateManage
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self._cell_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova return f(*args, **kwargs)
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova db_cells = self.db.
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova return IMPL.cell_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova return getattr(
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova backend_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self._load_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self._backend = backend_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova return API()
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self._launch_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova self._check_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova result = f(*args, **kwargs)
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova schema = conn.get_schema()
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova tables = self._get_tables()
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova columns = self._get_
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova cursor = self.execute(
2014-07-15 01:00:07.688 3070 TRACE nova File "/opt/rackstack
2014-07-15 01:00:07.688 3070 TRACE nova raise db_exc.DBError(e)
2014-07-15 01:00:07.688 3070 TRACE nova DBError: (1030, 'Got error 28 from storage engine')
2014-07-15 01:00:07.688 3070 TRACE nova
Since this is a DB issue it seems the process should at the very least retry.
description: | updated |
Changed in nova: | |
importance: | Undecided → Medium |
Changed in nova: | |
milestone: | none → juno-3 |
status: | Fix Committed → Fix Released |
Changed in nova: | |
milestone: | juno-3 → 2014.2 |
Fix proposed to branch: master /review. openstack. org/107168
Review: https:/