| 2016-11-28 12:22:26 |
levon |
description |
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
The full error log from /var/log/nova/nova-conductor.log is looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html
Not sure, but looks like the problem is in this function under nova/service.py
def _update_service_ref(service):
if service.version != service_obj.SERVICE_VERSION:
LOG.info(_LI('Updating service version for %(binary)s on '
'%(host)s from %(old)i to %(new)i'),
{'binary': service.binary,
'host': service.host,
'old': service.version,
'new': service_obj.SERVICE_VERSION})
service.version = service_obj.SERVICE_VERSION
service.save() |
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
The full error log from /var/log/nova/nova-conductor.log is looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html |
|
| 2016-12-08 19:15:05 |
levon |
description |
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
The full error log from /var/log/nova/nova-conductor.log is looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html |
We are running openstack mitaka out of clean installation (no upgrade).
All was functioning correctly until recently nova started to perform updates like this
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2 - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
The full error log from /var/log/nova/nova-conductor.log looks like
2016-11-23 10:48:08.967 17499 INFO nova.service [-] Starting conductor node (version 13.1.2)
2016-11-23 10:48:09.065 17498 INFO nova.service [req-c91ec40f-2dc2- - - - - -] Updating service version for nova-conductor on nova-tlc01.specs.de from 15 to 9
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service [req-c91ec40f-2dc2-4ab0-ad61-1ab1f9d7bbc9 - - - - -] Error starting thread.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service Traceback (most recent call last):
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 680, in run_service
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.start()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 195, in start
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service _update_service_ref(self.service_ref)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 151, in _update_service_ref
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service service.save()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 223, in wrapper
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service return fn(self, *args, **kwargs)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 291, in save
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service self._check_minimum_version()
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service File "/usr/lib/python2.7/dist-packages/nova/objects/service.py", line 275, in _check_minimum_version
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service minver=minver)
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service ServiceTooOld: This service is older (v9) than the minimum (v15) version of the rest of the deployment. Unable to continue.
2016-11-23 10:48:09.068 17498 ERROR oslo_service.service
I checked the changes in recent versions of nova/service.py
https://fossies.org/diffs/nova/13.1.1_vs_13.1.2/nova/service.py-diff.html |
|
