Flag for tests not applicable to current series

Bug #913812 reported by Max Brustkern on 2012-01-09
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
QA Regression Testing
Medium
Unassigned

Bug Description

Is there any method for flagging tests cases that no longer apply to the current series? Removing the test doesn't make sense as long as it works in some supported series, but some sort of indication that it isn't expected to install or run in the current series could be useful for the automated test runners we're using on the QA team. If nothing else is suggested, I propose
# QRT-Deprecated
This currently appears to apply to test-opie.py (after Natty,) test-zope3.py (after Hardy,) test-python2.4 (after Hardy,) test-python2.5 (after Hardy,) test-python3.1 (after Natty,) and test-ruby1.9 (after Lucid.) I can provide a patch if the idea is accepted.

Changed in qa-regression-testing:
status: New → Opinion
summary: - opie not installable in Oneiric, Precise
+ Flag for tests not applicable to current series
description: updated
tags: added: qa-test-code
description: updated
description: updated
Steve Beattie (sbeattie) wrote :

Hi Max,

As an aside, the Opinion status is for when there's a dispute as to whether the issue is actually a bug. It is treated as a closed bug in launchpad, making it not show up in the default bug listing for a project.

In discussing it with the security team, something like this would probably be reasonable. I assume the QRT-Deprecated flag would take a release version in which the test became deprecated? We may have a weird issue where for some reason a package gets removed for a release or more and then comes back into the archive; netbeans is an example of a package doing this, though we don't have a test script for it.

Thanks!

Changed in qa-regression-testing:
importance: Undecided → Medium
status: Opinion → In Progress
Max Brustkern (nuclearbob) wrote :

In the branch I submitted I did use the first release in which the package was no longer available. I'm not sure of the best way to handle cases like netbeans without just moving to a list of releases in which the package is deprecated since we can't use a programmatic method without ensuring dependencies are in place.

pawciobiel (pawciobiel) wrote :

IMHO adding more logic either directly in test code or to runners as flags/switches/options makes test cases code unmaintainable.
I think it would be better if there was `qa-regression-testing` branch for every release.
Pros:
 * readable code
 * easier to maintain
 * faster to lunch and run
 * history separated but kept
 * easier to fix test cases

mark jacob (mark5151) wrote :

It has to be an open project and a separate configuration has to be done for that. A new symbol will be fruitful for the testing. I have done the testing for the error solution for the project of https://oniton.com/blog/fix-epson-error-code-0x97/

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers