swift-proxy fails to start during deployment due to incorrect version of oslo.utils python package

[Igor Zinovik]

Problem description:

Failed Jenkins job
https://jenkins-tpi.bud.mirantis.net/job/systest.vcenter/BRANCH=master,OPENSTACK_RELEASE=Ubuntu,TEST_GROUP=vcenter_ceilometer,label_exp=vsphere/273/console

Environment:
Fuel ISO 6.1 #352
Ubuntu 14.04 as target operating system
vCenter 5.5update2
Ceilometer enabled

test scenario (see 'vcenter_ceilometer' test in test_vcenter.py in fuel-qa/):
        Scenario:
            1. Create cluster with Ceilometer support
            2. Add 3 nodes with controller+MongoDB roles
            3. Deploy the cluster
            4. Run OSTF

Puppet log snippet: http://paste.openstack.org/show/207959

/var/log/upstart/swift-proxy.log snippet (full log - http://paste.openstack.org/show/207960):
Starting proxy-server...(/etc/swift/proxy-server.conf)
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 23, in <module>
    sys.exit(run_wsgi(conf_file, 'proxy-server', **options))
  File "/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 445, in run_wsgi
    loadapp(conf_path, global_conf=global_conf)
...

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (oslo.utils 1.2.1 (/usr/lib/python2.7/dist-packages), Requirement.parse('oslo.utils==1.4.0'))

Signal proxy-server pid: 14545 signal: 15

Expected result:
Cluster is successfully deployed and running

Actual result:
Primary controller deployment fails

[Ivan Berezovskiy (iberezovskiy)]

https://bugs.launchpad.net/fuel/+bug/1425552/comments/17

[Ivan Berezovskiy (iberezovskiy)]

Steps To Reproduce:
1. Create environment with HA, 3 controllers and Swift + Ceilometer (and 3 mongo nodes).
2. Try to deploy cloud with Ceilometer.

[Ivan Berezovskiy (iberezovskiy)]

Root cause of this problem is patch https://review.fuel-infra.org/gitweb?p=openstack/python-keystoneclient.git;a=commit;h=aa750f928a34f8321eb0d8591a291209d2549f1b that changes requirements.txt file keystoneclient:

+oslo.utils==1.4.0

If we don't need upgrade oslo.utils to 1.4.0 version, we should patch requirements.txt in keystoneclient

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Change author: Alexander Makarov <email address hidden>
Review: https://review.fuel-infra.org/6248

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/6248

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/6248
Submitter: Alexander Makarov <email address hidden>
Branch: openstack-ci/fuel-6.1/2014.2

Commit: aab5f5397ef52f28c6c301b450e864df41df9a45
Author: Alexander Makarov <email address hidden>
Date: Mon Apr 27 14:34:15 2015

Adjust oslo.utils version to 1.2.1

Change-Id: Idbaa1709dc8a6885e703e8c83050814bb5815a4b
Closes-Bug: #1449023

[OSCI Robot (oscirobot)]

Reviewed: https://review.fuel-infra.org/6248
Committed: https://review.fuel-infra.org/gitweb?p=openstack/python-keystoneclient.git;a=commitdiff;h=aab5f5397ef52f28c6c301b450e864df41df9a45
Submitter: Alexander Makarov
Branch: openstack-ci/fuel-6.1/2014.2

commit aab5f5397ef52f28c6c301b450e864df41df9a45
Author: Alexander Makarov <email address hidden>

Adjust oslo.utils version to 1.2.1

Change-Id: Idbaa1709dc8a6885e703e8c83050814bb5815a4b
Closes-Bug: #1449023

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-6.0.1/2014.2
Change author: Alexander Makarov <email address hidden>
Review: https://review.fuel-infra.org/6259

[OSCI Robot (oscirobot)]

Fix proposed to branch: openstack-ci/fuel-6.0.1/2014.2
Review: https://review.fuel-infra.org/6259

[OSCI Robot (oscirobot)]

Reviewed: https://review.fuel-infra.org/6259
Committed: https://review.fuel-infra.org/gitweb?p=openstack/python-keystoneclient.git;a=commitdiff;h=22b201aa45746cdba05c4fc0e95374eb716d16f4
Submitter: mos-infra-ci
Branch: openstack-ci/fuel-6.0.1/2014.2

commit 22b201aa45746cdba05c4fc0e95374eb716d16f4
Author: Alexander Makarov <email address hidden>

Adjust oslo.utils version to 1.2.1

Change-Id: Idbaa1709dc8a6885e703e8c83050814bb5815a4b
Closes-Bug: #1449023
(cherry picked from commit aab5f5397ef52f28c6c301b450e864df41df9a45)

[Alexei Sheplyakov (asheplyakov)]

It looks like 5.1.2 is affected too: http://jenkins-product.srt.mirantis.net:8080/job/5.1.2.staging.ubuntu.bvt_2/125

[Nastya Urlapova (aurlapova)]

@Alexey, there is problem with swift http://pastebin.com/LqzEhbUf

[Nastya Urlapova (aurlapova)]

Oops, sorry , I've attached wrong link.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Change author: Vasyl Saienko <email address hidden>
Review: https://review.fuel-infra.org/6287

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-5.0.3/2014.1.1
Change author: Vasyl Saienko <email address hidden>
Review: https://review.fuel-infra.org/6289

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/6289
Submitter: Alexander Makarov <email address hidden>
Branch: openstack-ci/fuel-5.0.3/2014.1.1

Commit: 519ee4a8c654a783a9a796cbae4f8bf99f4bf869
Author: Vasyl Saienko <email address hidden>
Date: Tue Apr 28 17:11:02 2015

fix import for oslo.utils

 we have oslo.utils 1.0.0 version in 5.1.2 and it doesn't contain new namespace eg: oslo_utils
 change import statement to use correct namespace: oslo.utils

 Closes-Bug: #1449023

Change-Id: Ie994c05e781668fcf855d1fa481df0fd59421fd4

[Timur Nurlygayanov (tnurlygayanov)]

Fix verified on SWARM platform tests for MOS 6.1 ISO #359:
http://jenkins-product.srt.mirantis.net:8080/view/6.1_swarm/job/6.1.system_test.ubuntu.services_ha/

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-5.1.2/2014.1.1
Change author: Ivan Berezovskiy <email address hidden>
Review: https://review.fuel-infra.org/6351

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: openstack-ci/fuel-5.0.3/2014.1.1
Change author: Ivan Berezovskiy <email address hidden>
Review: https://review.fuel-infra.org/6352

[Dmitry Mescheryakov (dmitrymex)]

The issue was not fixed completely, hence Ivan reopened it. The problem is we didn't set oslo.utils as a dependency of python-keystoneclient.

[Ivan Kolodyazhny (e0ne)]

Staging job failed: http://jenkins-product.srt.mirantis.net:8080/view/5.1.2/job/5.1.2.staging.ubuntu.bvt_2/127/

[Alexei Sheplyakov (asheplyakov)]

On this planet BVTs have successfully passed: http://jenkins-product.srt.mirantis.net:8080/view/5.1.2/job/5.1.2.staging.ubuntu.bvt_2/128/

[Alexei Sheplyakov (asheplyakov)]

> The problem is we didn't set oslo.utils as a dependency of python-keystoneclient

Quite a number (most?) of OpenStack components depend on oslo.utils, so missing dependency should not cause any problems. That said adding missing dependencies is a good idea anyway.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/6351
Submitter: Dmitry Mescheryakov <email address hidden>
Branch: openstack-ci/fuel-5.1.2/2014.1.1

Commit: dbcbef4159a5be6e6778e83c6004d33c041f7ad8
Author: Ivan Berezovskiy <email address hidden>
Date: Wed Apr 29 17:55:05 2015

Keystoneclient should depend on oslo.utils

  https://github.com/openstack/python-keystoneclient/blob/stable/icehouse/requirements.txt#L13

  Related-bug: #1449023

Change-Id: I8b157066be95ebc8a735bcc529a9e176a46595a5

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/6352
Submitter: Dmitry Mescheryakov <email address hidden>
Branch: openstack-ci/fuel-5.0.3/2014.1.1

Commit: c3c27b54c444270cb74753b578f341fd0b7694c8
Author: Ivan Berezovskiy <email address hidden>
Date: Wed Apr 29 17:56:00 2015

Keystoneclient should depend on oslo.utils

  https://github.com/openstack/python-keystoneclient/blob/stable/icehouse/requirements.txt#L13

  Related-bug: #1449023

Change-Id: I8b157066be95ebc8a735bcc529a9e176a46595a5

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Alexander Makarov <email address hidden>
Review: https://review.fuel-infra.org/8302

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-5.1.1-updates/2014.1.1
Change author: Alexandr Nevenchannyy <email address hidden>
Review: https://review.fuel-infra.org/9286

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9286
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-5.1.1-updates/2014.1.1

Commit: 620f4d1fe8045fe1e142187268a4145024d45f3d
Author: Alexandr Nevenchannyy <email address hidden>
Date: Thu Jul 16 15:54:06 2015

Fix s3_token middleware parsing insecure option

Added function _bool_from_string from oslo.utils

Closes-Bug: #1449023

Change-Id: Ie994c05e781668fcf855d1fa481df0fd59421fd4
Closes-bug: #1442579
Closes-Bug: #1449023

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-5.1-updates/2014.1.1
Change author: Alexandr Nevenchannyy <email address hidden>
Review: https://review.fuel-infra.org/9441

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexander Makarov <email address hidden> on branch: openstack-ci/fuel-7.0/2015.1.0
Review: https://review.fuel-infra.org/8302

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9441
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-5.1-updates/2014.1.1

Commit: da8749c64263fd5361a99d91e461daab155e9186
Author: Alexandr Nevenchannyy <email address hidden>
Date: Thu Jul 23 15:59:10 2015

Fix s3_token middleware parsing insecure option

Added function _bool_from_string from oslo.utils.

Closes-Bug: #1449023

Change-Id: Ie994c05e781668fcf855d1fa481df0fd59421fd4
Closes-bug: #1442579
Closes-Bug: #1449023
(cherry picked from commit 620f4d1fe8045fe1e142187268a4145024d45f3d)