OpenStack Object Storage (Swift)

swift is missing a venv-based test run

Reported by Monty Taylor on 2011-12-27
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Core Infrastructure
High
Monty Taylor
OpenStack Object Storage (swift)
Undecided
Maru Newby

Bug Description

We've been moving all of the jenkins unittests to running in virtualenvs, and I just noticed that we don't have the infrastructure for this at the moment for swift. Additionally, we've been looking at tox as a mechanism for testing against mutliple python versions, which itself would use the above stuff.

ANYWAY - this is mainly just a placeholder note to remind me - I'm going to hack up a patch on the plane for you in just a sec.

Monty Taylor (mordred) on 2011-12-27
Changed in swift:
assignee: nobody → Monty Taylor (mordred)
Changed in openstack-ci:
assignee: nobody → Monty Taylor (mordred)
importance: Undecided → High
status: New → Triaged
Maru Newby (maru) wrote :

Any progress on this? I need to get CI working for swift and would prefer to avoid duplication of effort. I would also be happy to work on this bug if that will speed things along.

Maru Newby (maru) wrote :

I have ported nova's test running / venv solution as a starting point. Can I put it up for review in gerrit to resolve this bug, or do you have a different solution (tox) in mind?

On 01/20/2012 04:52 PM, Maru Newby wrote:
> I have ported nova's test running / venv solution as a starting point.
> Can I put it up for review in gerrit to resolve this bug, or do you have
> a different solution (tox) in mind?

AWESOME. That's a great first step. I would like to do tox - and if you
feel like poking at it, python-quantumclient has a fully working
tox-based setup.

But if you don't feel like it, this patch is great - I'll go check it out.

Monty

Maru Newby (maru) wrote :

Sorry for the delayed response, I apparently wasn't subscribed for the bug. I'll take a stab at implementing a tox-based solution, if that's the way forward.

Fix proposed to branch: master
Review: https://review.openstack.org/3602

Changed in swift:
assignee: Monty Taylor (mordred) → Maru Newby (maru)
status: New → In Progress
Maru Newby (maru) wrote :

FYI, adding an explicit 'deps = pep8' to the config for [testenv:pep8] makes for much faster execution due to not having to install all dependencies in the venv.

Maru Newby (maru) wrote :

I notice the tox-based unittests for nova separates the runtime dependencies from the test dependencies. Should I update the proposed fix for swift to do the same?

Yeah, if you don't mind - also, any chance you can make yours not use
the nova test runner and instead just use nova with the
openstack.nose_plugin?

On 02/07/2012 04:47 PM, Maru Newby wrote:
> I notice the tox-based unittests for nova separates the runtime
> dependencies from the test dependencies. Should I update the proposed
> fix for swift to do the same?
>

Maru Newby (maru) wrote :

Can do. I'll update the change with split dependencies and the plugin instead of the nova test runner and upload a new patch.

I see that run_script.sh, with_venv.sh, and install_venv.sh scripts are still in nova. Should they be included in the swift repo too? I had included them in my change previously but removed them when I switched to tox, but I'm wondering if venv support for more than just tests is considered useful.

Monty Taylor (mordred) wrote :

On 02/07/2012 05:21 PM, Maru Newby wrote:
> Can do. I'll update the change with split dependencies and the plugin
> instead of the nova test runner and upload a new patch.

Sweet.

> I see that run_script.sh, with_venv.sh, and install_venv.sh scripts are
> still in nova. Should they be included in the swift repo too? I had
> included them in my change previously but removed them when I switched
> to tox, but I'm wondering if venv support for more than just tests is
> considered useful.

Well, I plan on removing install_venv and run_tests _eventually_ ... but
I thought I'd leave them for a while so that we could give devs a chance
to try out the new stuff for a while and make sure we've got the kinks
worked out.

However, for projects that didn't have them in the first place (like
quantum or melange or python-keystoneclient) - I've just been leaving
them out.

So let's not add them.

Thanks!
Monty

Maru Newby (maru) wrote :

Please review the updated change: https://review.openstack.org/3602

Reviewed: https://review.openstack.org/3602
Committed: http://github.com/openstack/swift/commit/e6ea310751b58be0ad1d06d8acffc2b8fff8e5a3
Submitter: Jenkins
Branch: master

commit e6ea310751b58be0ad1d06d8acffc2b8fff8e5a3
Author: Maru Newby <email address hidden>
Date: Tue Jan 31 15:27:29 2012 -0800

    Add support for venv-based test run with tox.

     * Adds tox config
       - based on the config from python-quantumclient and updated for
         test, pep8 and coverage execution as per nova's run_tests.sh.
     * Adds nosetests defaults in setup.cfg
     * Adds runtime dependencies in tools/pip-requires
       - dependencies were gathered by referencing the packages used in
         creation of a Swift All In One. Versions were determined by
         checking the swift-core/trunk ppa or, failing that, the version
         available in lucid.
     * Adds test dependencies in tools/test-requires
     * Updates swift/common/middleware/formpost.py for pep8 compliance
     * Adds instructions for executing the tests with Tox to the
       developer_guidelines
     * Adds instructions for installing openstack.nose_plugin to
       developer_saio
     * Fixes bug 909177

    Change-Id: I5407924d2181e9ab335aaf76bf30c8d40deccbb4

Changed in swift:
status: In Progress → Fix Committed
Monty Taylor (mordred) on 2012-03-16
Changed in openstack-ci:
status: Triaged → Fix Released
Thierry Carrez (ttx) on 2012-03-20
Changed in swift:
milestone: none → 1.4.8
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers