Test suite doesn't clean up its own tmp files

Bug #177721 reported by Tom Haddon on 2007-12-20
6
Affects Status Importance Assigned to Milestone
Launchpad itself
Low
Unassigned

Bug Description

We were getting failures in PQM due to having over 70,000 files/folders in /tmp. This is in a chroot that's only used for Launchpad's PQM operations.

Cleaning out /tmp fixed the issue. So, we need to alter the test suite to clean up after itself better to avoid this issue in the future.

Changed in soyuz:
importance: Undecided → Unknown
milestone: none → 1.2.1
status: New → Confirmed
James Henstridge (jamesh) wrote :

Perhaps using a tool like tmpreaper on the chroot's tmp dirs might be appropriate here. While it would be nice if the test suite never leaked any tmp files, we shouldn't depend on it never leaking tmp files.

Tom Haddon (mthaddon) wrote :

I've created an RT to remind me to discuss this with James Troup. Will comment on this bug once I've had a chance to do that.

I'm pretty certain all the tools we use will honour the TMPDIR environment
variable so we could just create a temporary directory using mktemp -d, set
TMPDIR to point to it, run the tests, and nuke the temporary directory when
complete. This could be done in the Makefile or in test.py.

--
Stuart Bishop <email address hidden> http://www.canonical.com/
Canonical Ltd. http://www.ubuntu.com/

Changed in soyuz:
importance: Unknown → Undecided

Soyuz leaves some data in the test tree from old uploads where it pulls the .orig.tar.gz file from the librarian. I will clean those up.

Changed in soyuz:
assignee: nobody → julian-edwards
Changed in soyuz:
importance: Undecided → Medium
Tom Haddon (mthaddon) wrote :

Spoke with James Troup and he likes the sound of Stuart's idea of using a custom $TMPDIR.

Can someone implement that?

Julian Edwards (julian-edwards) wrote :

Soyuz test fix in RF 5430.

Changed in soyuz:
status: Confirmed → Fix Committed
Changed in soyuz:
status: Fix Committed → Fix Released
Michael Hudson-Doyle (mwhudson) wrote :

This just happened again. We need to do something about it before it bites us again...

Changed in launchpad:
importance: Undecided → High
status: New → Confirmed
Stuart Bishop (stub) on 2008-04-22
Changed in launchpad:
milestone: none → 1.2.4
Stuart Bishop (stub) on 2008-04-22
Changed in launchpad:
assignee: nobody → stub
Stuart Bishop (stub) wrote :

jml - do you have time to look at the last two failing tests using the cvs server? We should land this this cycle, so if there is no time let me know and I'll work around these two.

Changed in launchpad:
assignee: stub → jml
milestone: 1.2.4 → 1.2.5
Jonathan Lange (jml) wrote :

Which tests?

Stuart Bishop (stub) wrote :

The branch implementing the fix, and blocking future leaks, can land once a fix for Bug #224920 is landed.

Changed in launchpad:
assignee: jml → stub
Stuart Bishop (stub) wrote :

Bug #224920 is marked as released, but we still have codehosting tests leaving rubbish around:

Error in test test_import (canonical.codehosting.codeimport.tests.test_worker.TestCVSImport)
Traceback (most recent call last):
  File "/home/stub/lp/testsuite-tempfiles/lib/canonical/testing/layers.py", line 374, in flagTestIsolationFailure
    raise LayerIsolationError(message)
LayerIsolationError: Test left temporary files: set(['cvs-serv10425'])

Can someone have a look to see if fixing this is not too difficult? If it is benign I can update the test to do the cleanup and land this branch, but if it isn't benign and only affecting the test suite this might cause issues on production too.

Stuart Bishop (stub) wrote :

Branch that needs to land is bzr+ssh://devpad.canonical.com/code/stub/launchpad/testsuite-tempfiles

Changed in soyuz:
assignee: julian-edwards → thumper
milestone: 1.2.1 → 1.2.6
status: Fix Released → Confirmed
assignee: thumper → julian-edwards
milestone: 1.2.6 → none
status: Confirmed → Fix Released
Changed in launchpad-bazaar:
assignee: nobody → thumper
milestone: none → 1.2.6
Stuart Bishop (stub) on 2008-06-04
Changed in launchpad:
milestone: 1.2.5 → 1.2.6
Tim Penhey (thumper) wrote :

Michael, can you take a look at the import tests leaving stuff behind please?

Changed in launchpad-bazaar:
assignee: thumper → mwhudson

Stuart, do you need to push some more revisions to that branch? I'm seeing lots of failures from the way bzrlib's test deal with temporary files.

But to answer the question, the cvs-serv$NNN files are created by 'cvs server' processes (which I found out with 'strace', yay), which are launched by cscvs. While this is arguably a cvs bug which should be worked around in cscvs, it's benign -- we only run cvs server processes for tests, not production.

Changed in launchpad-bazaar:
assignee: mwhudson → stub
Stuart Bishop (stub) on 2008-06-16
Changed in launchpad-bazaar:
status: New → In Progress
Jonathan Lange (jml) wrote :

Just updating the priority to make triage easier.

How's this work going? Anything we can do to help?

Changed in launchpad-bazaar:
importance: Undecided → High
Stuart Bishop (stub) wrote :

All done except for the GPG Handler, which was passing locally here but failing under PQM. This seems to need quite a bit of refactoring to fix.

Changed in launchpad-bazaar:
milestone: 1.2.6 → 1.99
status: In Progress → Fix Committed
status: Fix Committed → Fix Released
Stuart Bishop (stub) on 2008-07-10
Changed in launchpad:
milestone: 1.2.6 → 1.99
Stuart Bishop (stub) on 2008-07-31
Changed in launchpad:
milestone: 1.99 → 2.1.10
Stuart Bishop (stub) on 2008-10-16
Changed in launchpad-foundations:
milestone: 2.1.10 → 2.1.11
Changed in launchpad-foundations:
milestone: 2.1.11 → 2.1.12
Stuart Bishop (stub) on 2008-12-19
Changed in launchpad-foundations:
milestone: 2.1.12 → 2.2.1
Changed in launchpad-foundations:
importance: High → Low
milestone: 2.2.1 → none
status: Confirmed → Triaged
Stuart Bishop (stub) on 2011-05-26
Changed in launchpad:
assignee: Stuart Bishop (stub) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers