integration tests are no longer independent from heat

Bug #1489459 reported by Pavlo Shchelokovskyy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Medium
Angus Salkeld

Bug Description

Our OS::Heat::TestResource residing in heat_integrationtests/common/test_resources/test_resource.py naturally imports some modules from main heat tree and is already used in functional tests. We also hack the heat.conf in order to register this resource in Heat.

This kind of defeats the idea of being able to run integration tests "standalone" against (remote) Heat. Currently the whole thing works only because pwd for integtaion tests run is our top repo dir, so "heat" can be imported and used.

I see two options here:

1) stay committed to "standalone integration tests" idea and move the OS::Heat::TestResource in the main tree;
2) dump the whole "standalone" idea, require whole heat + requirements for integration tests and clearly advertise that the only supported way of running them is installing on the same box heat is installed and updating heat.conf accordingly.

Given gripes we had with requirements for integration tests not being able to sync with global requirements, I'm leaning towards option 2. Also, latest pip is caching the downloaded packages, so one of the purposes of separate requirements for integration tests (decrease gate setup time) is no longer critical.

Changed in heat:
status: New → Triaged
importance: Undecided → High
milestone: none → liberty-3
Revision history for this message
Steve Baker (steve-stevebaker) wrote :

I would like to see OS::Heat::TestResource move in to the main tree and I think there will be valid use cases for users to write templates using them such as:
- operators scale testing their own heat deployment
- users mocking out a real resource with OS::Heat::TestResource for debugging (once we have a way of disabling constraints like nova.server)

If we can't make requirements work by leaving them in heat_integrationtests/requrements.txt or moving them to test_requirements.txt then I'd rather consider we create a new repo for integration tests.

Revision history for this message
Angus Salkeld (asalkeld) wrote :

 I'll post a follow up to https://review.openstack.org/215489 to move it in-tree.

Changed in heat:
assignee: nobody → Angus Salkeld (asalkeld)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to heat (master)

Fix proposed to branch: master
Review: https://review.openstack.org/218695

Changed in heat:
status: Triaged → In Progress
Revision history for this message
Pavlo Shchelokovskyy (pshchelo) wrote :

I'm working on a script that deprecates explicit heat_integrationtests/requirements.txt and will auto-generate it before running the tox integration job by syncing exact required versions from heat's [test-]requirements.txt

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/218929

Changed in heat:
assignee: Angus Salkeld (asalkeld) → Pavlo Shchelokovskyy (pshchelo)
Revision history for this message
Pavlo Shchelokovskyy (pshchelo) wrote :

that's the patch I was referring to in my previous comment.

Should I probably file a separate bug so not to hijack tracking Angus's work on test resource?

Revision history for this message
Steve Baker (steve-stevebaker) wrote : Re: [Bug 1489459] Re: integration tests are no longer independent from heat

On 01/09/15 04:15, Pavlo Shchelokovskyy wrote:
> Should I probably file a separate bug so not to hijack tracking Angus's
> work on test resource?
Yes please :)

Revision history for this message
Pavlo Shchelokovskyy (pshchelo) wrote :
Changed in heat:
assignee: Pavlo Shchelokovskyy (pshchelo) → Angus Salkeld (asalkeld)
Angus Salkeld (asalkeld)
Changed in heat:
importance: High → Medium
milestone: liberty-3 → liberty-rc1
Changed in heat:
assignee: Angus Salkeld (asalkeld) → Pavlo Shchelokovskyy (pshchelo)
Changed in heat:
assignee: Pavlo Shchelokovskyy (pshchelo) → Angus Salkeld (asalkeld)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to heat (master)

Reviewed: https://review.openstack.org/218695
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=78a2bb63e144c85928b71ded0ffcef7efe88c9d1
Submitter: Jenkins
Branch: master

commit 78a2bb63e144c85928b71ded0ffcef7efe88c9d1
Author: Angus Salkeld <email address hidden>
Date: Wed Sep 2 10:33:09 2015 +1000

    Move TestResource in-tree

    Change-Id: I83c52f099621610f75c3b986c359292606978885
    Closes-bug: #1489459

Changed in heat:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in heat:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in heat:
milestone: liberty-rc1 → 5.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.