Comment 1 for bug 1025633
Revision history for this message
| Max Brustkern (nuclearbob) wrote Re: Utah reports Success despite some post installation test failures | #1 |
© 2004
Canonical Ltd.
•
Terms of use
•
Data privacy
•
Contact Launchpad Support
•
Blog
•
Careers
•
System status
•
0e1f616
(Get the code!)
Right now, we try to report success if all jobs were run successfully, regardless of their outcome. If we want to put in a check for the job success as part of the test script, I think we should decide what output codes mean "test failure" and what output codes mean "test error" and report those consistently. The other option would be to validate the test results as part of the jenkins job; we could grep all the yaml files for returncode, and fail if a non-zero value comes up.
Relatedly, jobs which threw exceptions when provisioning a machine also reported success earlier today. That should now be fixed in the tree, and I'll be ordering a new build once I've got all the merges done.