heat-engine fails to start after upgrade of oslo.config

[Roman Podoliaka]

Last message in heat-engine log:

2015-02-26 13:45:36.242 10972 CRITICAL heat.engine [-] Could not load AWSTemplateFormatVersion.2010-09-09: (stevedore 1.0.0.0-a2 (/usr/lib/python2.7/dist-packages), Requirement.parse('stevedore>=1.1.0'))
ERROR: No template format plugins registered

Upstart won't restart heat-engine after failure, but our OCF script will try to do that indefinitely (/usr/lib/ocf/resource.d/mirantis/heat-engine). Not only this breaks Heat itself, it also increases load on the controller nodes, which leads to 'slowness' of other OpenStack services (e.g. nova-api or nova-conductor).

Dependency check shows it's oslo.config who requires a newer stevedore version:

root@node-1:/var/log/upstart# grep stevedore /usr/lib/python2.7/dist-packages -R | grep requires
/usr/lib/python2.7/dist-packages/heat-2014.2.2.egg-info/requires.txt:stevedore>=1.0.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/oslo.vmware-0.6.0.egg-info/requires.txt:stevedore>=0.14
grep: /usr/lib/python2.7/dist-packages/python-support.pth: No such file or directory
/usr/lib/python2.7/dist-packages/python_keystoneclient-0.11.1.egg-info/requires.txt:stevedore>=1.0.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/cliff-1.6.1.egg-info/requires.txt:stevedore>=0.12
grep: /usr/lib/python2.7/dist-packages/paste/evalexception/media/MochiKit.packed.js: No such file or directory
/usr/lib/python2.7/dist-packages/cinder-2014.2.2.egg-info/requires.txt:stevedore>=1.0.0,<=1.2.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/glance_store-0.1.7.egg-info/requires.txt:stevedore>=0.12
/usr/lib/python2.7/dist-packages/python_ceilometerclient-1.0.11.egg-info/requires.txt:stevedore>=0.14
grep: /usr/lib/python2.7/dist-packages/nova/CA/reqs/.gitignore: No such file or directory
grep: /usr/lib/python2.7/dist-packages/nova/CA/projects/.gitignore: No such file or directory
grep: /usr/lib/python2.7/dist-packages/nova/CA/.gitignore: No such file or directory
/usr/lib/python2.7/dist-packages/nova-2014.2.2.egg-info/requires.txt:stevedore>=1.0.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/oslo.config-1.6.0.egg-info/requires.txt:stevedore>=1.1.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/oslo.messaging-1.4.1.egg-info/requires.txt:stevedore>=1.0.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/taskflow-0.4.0.egg-info/requires.txt:stevedore>=0.14
/usr/lib/python2.7/dist-packages/stevedore-1.0.0.0_a2.egg-info/SOURCES.txt:stevedore.egg-info/requires.txt
/usr/lib/python2.7/dist-packages/oslo.db-1.0.3.egg-info/requires.txt:stevedore>=1.0.0 # Apache-2.0
/usr/lib/python2.7/dist-packages/glance-2014.2.2.egg-info/requires.txt:stevedore>=1.0.0,<=1.2.0 # Apache-2.0

Failed build info: http://jenkins-product.srt.mirantis.net:8080/job/6.0.1.staging.ubuntu.bvt_2/60/consoleText

[Viktor Serhieiev (vsergeyev)]

Based on bug description, it looks like we should update `stevedore` version

[Vasyl Saienko (vsaienko)]

stevedore has been updated to 1.1.0

[Roman Podoliaka (rpodolyaka)]

Fixed in https://review.fuel-infra.org/#/c/4095/

[Kyrylo Romanenko (kromanenko)]

Build successful after fix:
Build #62 (Feb 27, 2015 2:41:10 PM)
http://jenkins-product.srt.mirantis.net:8080/job/6.0.1.staging.ubuntu.bvt_2/62/

MOS 6.0.1 build.
{"build_id": "2015-03-05_20-49-44", "ostf_sha": "b38332e6741fc4b0ef40a9b4fe9d2804ee6437cc", "build_number": "118", "auth_required": true, "api": "1.0", "nailgun_sha": "a27dcf6a04c12ab6c0bfa7039ffba84ed7574592", "production": "docker", "fuelmain_sha": "bc1a1279509a87de0b9201e6dd2d393e0e122905", "astute_sha": "f7cda2171b0b677dfaeb59693d980a2d3ee4c3e0", "feature_groups": ["mirantis"], "release": "6.0.1", "release_versions": {"2014.2.2-6.0.1": {"VERSION": {"build_id": "2015-03-05_20-49-44", "ostf_sha": "b38332e6741fc4b0ef40a9b4fe9d2804ee6437cc", "build_number": "118", "api": "1.0", "nailgun_sha": "a27dcf6a04c12ab6c0bfa7039ffba84ed7574592", "production": "docker", "fuelmain_sha": "bc1a1279509a87de0b9201e6dd2d393e0e122905", "astute_sha": "f7cda2171b0b677dfaeb59693d980a2d3ee4c3e0", "feature_groups": ["mirantis"], "release": "6.0.1", "fuellib_sha": "b667689e264b0f81073193026fb43d446e7dab3d"}}}, "fuellib_sha": "b667689e264b0f81073193026fb43d446e7dab3d"}

Stevedore Version: 1.1.0 as it should be.