deb/rpm version conflicts for dask/distributed deps

Bug #1676197 reported by Alexandru Avadanii
24
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Critical
Vladimir Kozhukalov

Bug Description

After artifacts from [1] were pushed to the public mirror, and corresponding code to enable them was merged as part of [2], Fuel Master node can no longer be deployed.
We observed this regression in Fuel@OPNFV (also in Armband@OPNFV), and CI tests confirm my short investigation:
- Nailgun DB sync (/usr/bin/nailgun_syncdb) fails due to missing packages;
- installing required packaged by hand leads to more missing dependecies;
- trying to install said missing deps by hand leads to version mismatch, since distributed==1.16.0 requires newer versions than existing RPMs in our repos;

I tinkered a little with this, and here's how I got DB sync to stop complaining about missing deps:

# nailgun_syncdb requires distributed, which is not installed on master node (?)
$ yum install python-distributed
# distributed requirement, can be solved by installing our RPM
$ yum install python-sortedcollections
# python packages below are available as RPMs, but their version is too old (!)
$ yum install pip
$ pip install zict==0.1.1
$ pip install click==6.6
$ pip install cloudpickle==0.2.2
$ pip install tornado==4.2
# It would have clearly been simpler to just `pip install -Ur distributed==1.16.0`

[1] https://bugs.launchpad.net/fuel/+bug/1673032
[2] https://review.openstack.org/#/q/topic:bp/distributed-serialization+status:merged
[3] https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1348/console

Revision history for this message
Vladimir Khlyunev (vkhlyunev) wrote :

BVT is broken now, traceback from /var/log/puppet/bootstrap_admin_node.log

http://paste.openstack.org/show/604263/

tldr:
pkg_resources.DistributionNotFound: The 'distributed==1.16.0' distribution was not found and is required by nailgun

Changed in fuel:
milestone: none → 10.1
importance: Undecided → Critical
tags: added: bvt-fail swarm-blocker
Revision history for this message
Vladimir Kozhukalov (kozhukalov) wrote :
Changed in fuel:
status: New → Fix Committed
Roman Vyalov (r0mikiam)
Changed in fuel:
assignee: nobody → Vladimir Kozhukalov (kozhukalov)
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to packages/centos7/python-tornado (10.0/newton)

Related fix proposed to branch: 10.0/newton
Change author: Ivan Suzdal <email address hidden>
Review: https://review.fuel-infra.org/32541

Changed in fuel:
status: Fix Committed → In Progress
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to packages/centos7/python-tornado (10.0/newton)

Reviewed: https://review.fuel-infra.org/32541
Submitter: Pkgs Jenkins <email address hidden>
Branch: 10.0/newton

Commit: e697aedeb1b5b33c3ee46ed0c6e2f8f105143583
Author: Ivan Suzdal <email address hidden>
Date: Mon Mar 27 09:58:12 2017

Update python-tornado to 4.4.2

Cherry-pick from master

Change-Id: I3b84b4005e1e4c5231bc5e37b5735f1bb738bda7
Related-Bug: #1676197

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-web (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/450324

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-web (stable/newton)

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/450325

Revision history for this message
Alexey Shtokolov (ashtokolov) wrote :
Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/450324
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=962f009fcecf5f2a64e812febbc989343ad1ccc3
Submitter: Jenkins
Branch: master

commit 962f009fcecf5f2a64e812febbc989343ad1ccc3
Author: Vladimir Kuklin <email address hidden>
Date: Mon Mar 27 19:13:26 2017 +0300

    Fix typo in packages list for distributed serialization

    Change-Id: Iacc987275064609f5c1f3c731073baa8a345e7d3
    Related-bug: #1676197

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-web (stable/newton)

Change abandoned by Alexander Kislitsky (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/450325
Reason: Doesn't required due to reverted https://review.openstack.org/#/c/450320/

tags: added: on-verification
Revision history for this message
TatyanaGladysheva (tgladysheva) wrote :

Verified on 10.0 build #1461.

Latest runs of bvt job with fix are passed (since #1445):
https://product-ci.infra.mirantis.net/view/10.0/job/10.0.main.ubuntu.bvt_2/1461/

tags: removed: on-verification
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.