Every openstack client is built in the repo build

Bug #1745215 reported by Major Hayden
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-ansible
Confirmed
Wishlist
Major Hayden

Bug Description

While investigating some of the repo build slowdowns in CentOS, I found that every single OpenStack client is built (along with dependencies) during the repo build process.

For some clients, like python-swiftclient, this isn't a big deal.

However, some clients like python-monascaclient drag in heavy dependencies, like cassandra-driver. Building that particular wheel often takes 4-6 minutes, which is about half the entire run of the wheel build itself.

All of these client requirements flow in through heat: https://github.com/openstack/heat/blob/master/requirements.txt

Should we go through the list of clients during a selective wheel build and remove any clients that correspond to OpenStack services that are not being deployed. As an example, if monasca is being deployed, why should we build, store, and venv-itize the monasca client and its dependencies?

Revision history for this message
Marc Gariépy (mgariepy) wrote :

I think it would make sense not to inlude all the clients. since they will never be used anyway.

If the deployer want to deploy a new service the repobuild process will need to be re-ran anyway.

Revision history for this message
Jean-Philippe Evrard (jean-philippe-evrard) wrote :

Please be aware we are refactoring things to prevent the need of building wheels, and just installs things in venvs.

On top of that, it's fine for me to respect the requirements of heat, because we install heat.

What's wrong is, like what Doug Hellmann said on the ML post, the way heat is packaging its things.

I think we should have instead:

This is what the "extras" section of setup.cfg is for. It should be
possible to say something like:

[extras]
monasca =
   python-monascaclient>=1.0.0 (or whatever version)

Then users of pip would install a heat that uses monasca with:

   pip install heat[monasca]

Revision history for this message
Jean-Philippe Evrard (jean-philippe-evrard) wrote :

Major, because you're assigned on this, could you have a look at what can be done on heat side? Else I will do it.

Revision history for this message
Major Hayden (rackerhacker) wrote :

I'll gladly unassign myself, JP. I wasn't aware of that work to avoid building wheels anyway.

Changed in openstack-ansible:
assignee: Major Hayden (rackerhacker) → nobody
Changed in openstack-ansible:
status: New → Confirmed
importance: Undecided → Wishlist
Revision history for this message
Major Hayden (rackerhacker) wrote :
Changed in openstack-ansible:
assignee: nobody → Major Hayden (rackerhacker)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to openstack-ansible-repo_build (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/546319

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to openstack-ansible (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/546331

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible-repo_build (master)

Change abandoned by Major Hayden (<email address hidden>) on branch: master
Review: https://review.openstack.org/546319

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible (master)

Change abandoned by Major Hayden (<email address hidden>) on branch: master
Review: https://review.openstack.org/546331

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers