[stable/liberty] gate-tempest-dsvm-neutron-src-oslo.service fails with "ContextualVersionConflict: (oslo.service 0.9.1.dev6 (/opt/stack/new/oslo.service), Requirement.parse('oslo.service>=0.12.0'), set(['oslo.messaging']))"

[Elena Ezhova]

stable/liberty jobs are constantly failing and in devstack logs the following error can be seen:

2015-12-28 10:10:08.307 | usage: openstack role list [-h] [-f {csv,json,table,value,yaml}] [-c COLUMN]
2015-12-28 10:10:08.307 | [--max-width <integer>] [--noindent]
2015-12-28 10:10:08.307 | [--quote {all,minimal,none,nonnumeric}]
2015-12-28 10:10:08.307 | [--domain <domain> | --project <project>]
2015-12-28 10:10:08.308 | [--user <user> | --group <group>]
2015-12-28 10:10:08.308 | [--group-domain <group-domain>]
2015-12-28 10:10:08.308 | [--project-domain <project-domain>]
2015-12-28 10:10:08.308 | [--user-domain <user-domain>] [--inherited]
2015-12-28 10:10:08.308 | openstack role list: error: argument --user: expected one argument
2015-12-28 10:10:08.333 | + user_role_id=
2015-12-28 10:10:08.333 | + [[ -z '' ]]
2015-12-28 10:10:08.333 | + openstack role add --user --project --os-url=http://127.0.0.1:5000/v3 --os-identity-api-version=3
2015-12-28 10:10:08.848 | usage: openstack role add [-h] [--domain <domain> | --project <project>]
2015-12-28 10:10:08.849 | [--user <user> | --group <group>]
2015-12-28 10:10:08.849 | [--group-domain <group-domain>]
2015-12-28 10:10:08.849 | [--project-domain <project-domain>]
2015-12-28 10:10:08.849 | [--user-domain <user-domain>] [--inherited]
2015-12-28 10:10:08.849 | <role>
2015-12-28 10:10:08.849 | openstack role add: error: argument --user: expected one argument

http://logs.openstack.org/21/261021/1/check/gate-tempest-dsvm-neutron-src-oslo.service/bf24209/logs/devstacklog.txt.gz#_2015-12-28_10_10_08_307

[Davanum Srinivas (DIMS) (dims-v)]

Problem "Internal Server Error (HTTP 500)" occured before in line:
http://logs.openstack.org/21/261021/1/check/gate-tempest-dsvm-neutron-src-oslo.service/bf24209/logs/devstacklog.txt.gz#_2015-12-28_10_10_07_742

Caused by version conflict:
http://logs.openstack.org/21/261021/1/check/gate-tempest-dsvm-neutron-src-oslo.service/bf24209/logs/apache/keystone.txt.gz#_2015-12-28_10_10_02_369721

[Elena Ezhova (eezhova)]

So pip-freeze.txt shows that there was installed oslo.messaging 3.0.0 [1] and if we take a look at it's requirements it can be seen that it requires oslo.service>=0.12.0 while in global-requirements it's pinned at oslo.service>=0.7.0 and oslo.service===1.1.0 in upper-constraints.

At the same time the latest version tag we had in stable/liberty was 0.9.0 so I think that is why there was a version conflict. I don't have enough knowledge of how upper-constraints mechanism works but shouldn't the value in constraints.txt somehow correlate with the latest stable/liberty version?

[1] http://logs.openstack.org/43/256443/1/check/gate-tempest-dsvm-neutron-src-oslo.service/d3c7268/logs/pip-freeze.txt.gz
[2] https://github.com/openstack/oslo.messaging/blob/3.0.0/requirements.txt#L13

[Matt Riedemann (mriedem)]

I'm not sure why it's installing oslo.service 0.9.1.dev but it does look like the job is cloning the master branch of oslo.service rather than stable/liberty:

http://logs.openstack.org/43/256443/1/check/gate-tempest-dsvm-neutron-src-oslo.service/d5a5a68/logs/devstacklog.txt.gz#_2016-02-15_03_07_05_171

+ local branch=master

I'd think that should be stable/liberty.

[Matt Riedemann (mriedem)]

The branch probably doesn't matter since the job should just be pip installing what's been checked out from gerrit.

[Matt Riedemann (mriedem)]

I'd suggest that we should lower the oslo.messaging bound in upper-constraints on stable/liberty to 2.5.0 which would align that version with the stable/liberty g-r version on oslo.service, which is >=0.7.0 (which would also match what's in keystone server requirements).

oslo.messaging 3.0.0 just doesn't really work with older versions of oslo libraries (it would work with the version of oslo.service that's in u-c in stable/liberty, 1.1.0).

[Matt Riedemann (mriedem)]

I'm proposing that we exclude the job from running on stable/liberty oslo.service changes here:

https://review.openstack.org/#/c/280328/

[Matt Riedemann (mriedem)]

Here is a related change to stable/liberty upper-constraints to reset oslo.service to 0.9.0 and oslo.messaging to 2.5.0:

https://review.openstack.org/#/c/280931/

[Matt Riedemann (mriedem)]

This re-enables the job: https://review.openstack.org/#/c/281559/