Fuel CI job for fuel-web ignores Nailgun test failures

Bug #1300483 reported by Dmitry Borodaenko on 2014-03-31
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Critical
Dmitry Borodaenko

Bug Description

Running the following test case manually like this:
nosetests nailgun/nailgun/test/unit/test_task_handler.py:TestTaskHandlers

on fuel-web master currently gets you the following error:
Traceback (most recent call last):
  File "/home/angdraug/work/mirantis/git/fuel-web/nailgun/nailgun/test/unit/test_task_handler.py", line 54, in test_forced_task_deletion
    self.env.create(
AttributeError: 'TestTaskHandlers' object has no attribute 'env'

Bisecting shows that this was broken by https://review.openstack.org/77983

The problem is, that review has +1 from Fuel CI, so it was reasonable for developers and reviewers to assume that unit tests passed.

However, if you look into console log of that Fuel CI job (https://fuel-jenkins.mirantis.com/job/stackforge-verify-fuel-web/2603/consoleFull), you'll see that it has failed with the same traceback:
test_forced_task_deletion (nailgun.test.unit.test_task_handler.TestTaskHandlers) ... ERROR

And yet, that error ended up being ignored by the test script and it proceeded to run UI and Flake8 tests and ultimately reported success.

Dmitry Borodaenko (angdraug) wrote :

The original root cause is "| tee" in the following line:
https://github.com/stackforge/fuel-web/blob/64b5e4ef552a00c81ec74567df3657a4249ac145/run_tests.sh#L187

This causes the "tee" exit code to overwrite exit code of nosetests, so run_nailgun_tests() never returns error.

Changed in fuel:
status: Confirmed → Triaged

Fix proposed to branch: master
Review: https://review.openstack.org/84280

Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Dmitry Borodaenko (dborodaenko)
status: Triaged → In Progress
Andrew Woodward (xarses) on 2014-04-01
tags: added: backports-4.1.1

Reviewed: https://review.openstack.org/84280
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=b689d18d9fb739c60a144dde61a00d57bac5da16
Submitter: Jenkins
Branch: master

commit b689d18d9fb739c60a144dde61a00d57bac5da16
Author: Dmitry Borodaenko <email address hidden>
Date: Mon Mar 31 16:35:57 2014 -0700

    Do not overwrite nosetests exit code with tee

    When output of nosetests is piped to tee, tee's exit status overwrites
    that of nosetests, and the pipeline returns 0 even on failures and
    errors.

    This reverts part of the changes done in:
    440aded2f5025a8117bf87f74de8c24ef79f68a5

    Change-Id: Ia7fc2ab5719c878c740d9167d31cdaa018404c3c
    Closes-bug: #1300483
    Signed-off-by: Dmitry Borodaenko <email address hidden>

Changed in fuel:
status: In Progress → Fix Committed
Changed in fuel:
status: Fix Committed → In Progress
status: In Progress → Fix Committed
status: Fix Committed → In Progress
milestone: 5.0 → 4.1.1
Mike Scherbakov (mihgen) on 2014-04-02
tags: removed: system-tests

Reviewed: https://review.openstack.org/84509
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=deaf23c0407c6681539ed57325ce34f772772984
Submitter: Jenkins
Branch: stable/4.1

commit deaf23c0407c6681539ed57325ce34f772772984
Author: Dmitry Borodaenko <email address hidden>
Date: Tue Apr 1 10:54:24 2014 -0700

    Do not overwrite nosetests exit code with tee

    When output of nosetests is piped to tee, tee's exit status overwrites
    that of nosetests, and the pipeline returns 0 even on failures and
    errors.

    This reverts part of the changes done in:
    440aded2f5025a8117bf87f74de8c24ef79f68a5

    Change-Id: I37db61be9c9dae105324319f995a7d65a9907531
    Closes-bug: #1300483
    Signed-off-by: Dmitry Borodaenko <email address hidden>

Changed in fuel:
status: In Progress → Fix Committed
Andrew Woodward (xarses) on 2014-04-04
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers