nova-manage db_sync should run before restarting nova conductor

Bug #1331125 reported by Jerry Zhao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Compass
New
Undecided
Sam Su

Bug Description

http://12.234.32.44/logs/91/99991/1/check/compass-bare-e2e-7vm-ubuntu/38e66fe8aa7f438a99367bea68004c56/cobbler_logs/host1.1/nova-conductor.log

http://12.234.32.44/logs/91/99991/1/check/compass-bare-e2e-7vm-ubuntu/38e66fe8aa7f438a99367bea68004c56/cobbler_logs/host1.1/chef-client.log

Jun 16 13:48:10 host1.1 [2014-06-16T13:48:08-07:00] INFO: Processing service[nova-consoleauth] action start (openstack-compute::vncproxy line 64)
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:08-07:00] INFO: Processing package[nova-conductor] action upgrade (openstack-compute::conductor line 26)
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:09-07:00] INFO: Processing service[nova-conductor] action enable (openstack-compute::conductor line 32)
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:09-07:00] INFO: service[nova-conductor] enabled
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:09-07:00] INFO: Processing service[nova-conductor] action restart (openstack-compute::conductor line 32)
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:09-07:00] INFO: service[nova-conductor] restarted
Jun 16 13:48:10 host1.1 [2014-06-16T13:48:09-07:00] INFO: Processing execute[nova-manage db sync] action run (openstack-compute::nova-setup line 26)
Jun 16 13:49:12 host1.1 [2014-06-16T13:49:10-07:00] INFO: execute[nova-manage db sync] ran successfully

 2014-06-16 13:48:10.186 5646 TRACE nova Traceback (most recent call last):
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/bin/nova-conductor", line 53, in <module>
 2014-06-16 13:48:10.186 5646 TRACE nova service.wait()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 689, in wait
 2014-06-16 13:48:10.186 5646 TRACE nova _launcher.wait()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 209, in wait
 2014-06-16 13:48:10.186 5646 TRACE nova super(ServiceLauncher, self).wait()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 179, in wait
 2014-06-16 13:48:10.186 5646 TRACE nova service.wait()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
 2014-06-16 13:48:10.186 5646 TRACE nova return self._exit_event.wait()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
 2014-06-16 13:48:10.186 5646 TRACE nova return hubs.get_hub().switch()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
 2014-06-16 13:48:10.186 5646 TRACE nova return self.greenlet.switch()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
 2014-06-16 13:48:10.186 5646 TRACE nova result = function(*args, **kwargs)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 147, in run_server
 2014-06-16 13:48:10.186 5646 TRACE nova server.start()
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 434, in start
 2014-06-16 13:48:10.186 5646 TRACE nova self.host, self.binary)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 261, in service_get_by_args
 2014-06-16 13:48:10.186 5646 TRACE nova binary=binary)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 1352, in wrapper
 2014-06-16 13:48:10.186 5646 TRACE nova return func(*args, **kwargs)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py", line 424, in inner
 2014-06-16 13:48:10.186 5646 TRACE nova return catch_client_exception(exceptions, func, *args, **kwargs)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py", line 407, in catch_client_exception
 2014-06-16 13:48:10.186 5646 TRACE nova return func(*args, **kwargs)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 325, in service_get_all_by
 2014-06-16 13:48:10.186 5646 TRACE nova result = self.db.service_get_by_args(context, host, binary)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 155, in service_get_by_args
 2014-06-16 13:48:10.186 5646 TRACE nova return IMPL.service_get_by_args(context, host, binary)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 97, in wrapper
 2014-06-16 13:48:10.186 5646 TRACE nova return f(*args, **kwargs)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 412, in service_get_by_args
 2014-06-16 13:48:10.186 5646 TRACE nova filter_by(binary=binary).\
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2156, in first
 2014-06-16 13:48:10.186 5646 TRACE nova ret = list(self[0:1])
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2023, in __getitem__
 2014-06-16 13:48:10.186 5646 TRACE nova return list(res)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2227, in __iter__
 2014-06-16 13:48:10.186 5646 TRACE nova return self._execute_and_instances(context)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2242, in _execute_and_instances
 2014-06-16 13:48:10.186 5646 TRACE nova result = conn.execute(querycontext.statement, self._params)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
 2014-06-16 13:48:10.186 5646 TRACE nova params)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement
 2014-06-16 13:48:10.186 5646 TRACE nova compiled_sql, distilled_params
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
 2014-06-16 13:48:10.186 5646 TRACE nova context)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
 2014-06-16 13:48:10.186 5646 TRACE nova context)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
 2014-06-16 13:48:10.186 5646 TRACE nova cursor.execute(statement, parameters)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
 2014-06-16 13:48:10.186 5646 TRACE nova self.errorhandler(self, exc, value)
 2014-06-16 13:48:10.186 5646 TRACE nova File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
 2014-06-16 13:48:10.186 5646 TRACE nova raise errorclass, errorvalue
 2014-06-16 13:48:10.186 5646 TRACE nova ProgrammingError: (ProgrammingError) (1146, "Table 'nova.services' doesn't exist") 'SELECT services.created_at AS services_created_at, services.updated_at AS services_updated_at, services.deleted_at AS services_deleted_at, services.deleted AS services_deleted, services.id AS services_id, services.host AS services_host, services.`binary` AS services_binary, services.topic AS services_topic, services.report_count AS services_report_count, services.disabled AS services_disabled \nFROM services \nWHERE services.deleted = %s AND services.host = %s AND services.`binary` = %s \n LIMIT %s' (0, 'host1', 'nova-conductor', 1)
 2014-06-16 13:48:10.186 5646 TRACE nova

Jerry Zhao (zhaoxinyu)
Changed in compass:
assignee: nobody → Sam Su (sam-su)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.