swift is missing a venv-based test run

Bug #909177 reported by Monty Taylor
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Core Infrastructure
Fix Released
High
Monty Taylor
OpenStack Object Storage (swift)
Fix Released
Undecided
Maru Newby

Bug Description

We've been moving all of the jenkins unittests to running in virtualenvs, and I just noticed that we don't have the infrastructure for this at the moment for swift. Additionally, we've been looking at tox as a mechanism for testing against mutliple python versions, which itself would use the above stuff.

ANYWAY - this is mainly just a placeholder note to remind me - I'm going to hack up a patch on the plane for you in just a sec.

Monty Taylor (mordred)
Changed in swift:
assignee: nobody → Monty Taylor (mordred)
Changed in openstack-ci:
assignee: nobody → Monty Taylor (mordred)
importance: Undecided → High
status: New → Triaged
Revision history for this message
Maru Newby (maru) wrote :

Any progress on this? I need to get CI working for swift and would prefer to avoid duplication of effort. I would also be happy to work on this bug if that will speed things along.

Revision history for this message
Maru Newby (maru) wrote :

I have ported nova's test running / venv solution as a starting point. Can I put it up for review in gerrit to resolve this bug, or do you have a different solution (tox) in mind?

Revision history for this message
Monty Taylor (mordred) wrote : Re: [Bug 909177] Re: swift is missing a venv-based test run

On 01/20/2012 04:52 PM, Maru Newby wrote:
> I have ported nova's test running / venv solution as a starting point.
> Can I put it up for review in gerrit to resolve this bug, or do you have
> a different solution (tox) in mind?

AWESOME. That's a great first step. I would like to do tox - and if you
feel like poking at it, python-quantumclient has a fully working
tox-based setup.

But if you don't feel like it, this patch is great - I'll go check it out.

Monty

Revision history for this message
Maru Newby (maru) wrote :

Sorry for the delayed response, I apparently wasn't subscribed for the bug. I'll take a stab at implementing a tox-based solution, if that's the way forward.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to swift (master)

Fix proposed to branch: master
Review: https://review.openstack.org/3602

Changed in swift:
assignee: Monty Taylor (mordred) → Maru Newby (maru)
status: New → In Progress
Revision history for this message
Maru Newby (maru) wrote :

FYI, adding an explicit 'deps = pep8' to the config for [testenv:pep8] makes for much faster execution due to not having to install all dependencies in the venv.

Revision history for this message
Maru Newby (maru) wrote :

I notice the tox-based unittests for nova separates the runtime dependencies from the test dependencies. Should I update the proposed fix for swift to do the same?

Revision history for this message
Monty Taylor (mordred) wrote : Re: [Bug 909177] Re: swift is missing a venv-based test run

Yeah, if you don't mind - also, any chance you can make yours not use
the nova test runner and instead just use nova with the
openstack.nose_plugin?

On 02/07/2012 04:47 PM, Maru Newby wrote:
> I notice the tox-based unittests for nova separates the runtime
> dependencies from the test dependencies. Should I update the proposed
> fix for swift to do the same?
>

Revision history for this message
Maru Newby (maru) wrote :

Can do. I'll update the change with split dependencies and the plugin instead of the nova test runner and upload a new patch.

I see that run_script.sh, with_venv.sh, and install_venv.sh scripts are still in nova. Should they be included in the swift repo too? I had included them in my change previously but removed them when I switched to tox, but I'm wondering if venv support for more than just tests is considered useful.

Revision history for this message
Monty Taylor (mordred) wrote :

On 02/07/2012 05:21 PM, Maru Newby wrote:
> Can do. I'll update the change with split dependencies and the plugin
> instead of the nova test runner and upload a new patch.

Sweet.

> I see that run_script.sh, with_venv.sh, and install_venv.sh scripts are
> still in nova. Should they be included in the swift repo too? I had
> included them in my change previously but removed them when I switched
> to tox, but I'm wondering if venv support for more than just tests is
> considered useful.

Well, I plan on removing install_venv and run_tests _eventually_ ... but
I thought I'd leave them for a while so that we could give devs a chance
to try out the new stuff for a while and make sure we've got the kinks
worked out.

However, for projects that didn't have them in the first place (like
quantum or melange or python-keystoneclient) - I've just been leaving
them out.

So let's not add them.

Thanks!
Monty

Revision history for this message
Maru Newby (maru) wrote :

Please review the updated change: https://review.openstack.org/3602

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to swift (master)

Reviewed: https://review.openstack.org/3602
Committed: http://github.com/openstack/swift/commit/e6ea310751b58be0ad1d06d8acffc2b8fff8e5a3
Submitter: Jenkins
Branch: master

commit e6ea310751b58be0ad1d06d8acffc2b8fff8e5a3
Author: Maru Newby <email address hidden>
Date: Tue Jan 31 15:27:29 2012 -0800

    Add support for venv-based test run with tox.

     * Adds tox config
       - based on the config from python-quantumclient and updated for
         test, pep8 and coverage execution as per nova's run_tests.sh.
     * Adds nosetests defaults in setup.cfg
     * Adds runtime dependencies in tools/pip-requires
       - dependencies were gathered by referencing the packages used in
         creation of a Swift All In One. Versions were determined by
         checking the swift-core/trunk ppa or, failing that, the version
         available in lucid.
     * Adds test dependencies in tools/test-requires
     * Updates swift/common/middleware/formpost.py for pep8 compliance
     * Adds instructions for executing the tests with Tox to the
       developer_guidelines
     * Adds instructions for installing openstack.nose_plugin to
       developer_saio
     * Fixes bug 909177

    Change-Id: I5407924d2181e9ab335aaf76bf30c8d40deccbb4

Changed in swift:
status: In Progress → Fix Committed
Monty Taylor (mordred)
Changed in openstack-ci:
status: Triaged → Fix Released
Thierry Carrez (ttx)
Changed in swift:
milestone: none → 1.4.8
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.