ValueError in tests.core.test_util.test_diskfree

[BJ Dierkes]

Using amsterdam (current) on Fedora 14 64bit:

[wdierkes@derks-linux holland]$ ./scripts/mkvirtenv.py --no-site-packages
New python executable in /home/wdierkes/holland-test/bin/python
Installing setuptools..........................done.
[INFO] Installed holland-core.
[INFO] Installing holland plugins
[INFO] Installed plugin holland.lib.common
[INFO] Installed plugin holland.lib.mysql
[INFO] Installed plugin holland.lib.lvm
[INFO] Installed plugin holland.backup.mysqldump
[INFO] Installed plugin holland.backup.mysql_lvm
[INFO] Installed plugin holland.backup.xtrabackup
[INFO] Starting shell in virtual environment /bin/bash - use ctrl-d to exit

[holland-test]% nosetests --verbosity 3
nose.config: INFO: Set working dir to /home/wdierkes/devel/holland/tests
nose.config: INFO: Working directory /home/wdierkes/devel/holland/tests is a package; adding to sys.path
nose.plugins.cover: ERROR: Coverage not available: unable to import coverage module
... snip ok tests ...
tests.core.test_util.test_diskfree ... ERROR

======================================================================
ERROR: tests.core.test_util.test_diskfree
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 186, in runTest
    self.test(*self.arg)
  File "/home/wdierkes/devel/holland/tests/core/test_util.py", line 83, in test_diskfree
    actual_bytes = int(commands.getoutput(real_cmd).strip())
ValueError: invalid literal for int() with base 10: '80%'

----------------------------------------------------------------------
Ran 96 tests in 38.849s

FAILED (errors=1)

[BJ Dierkes (derks)]

The offending test is:

#XXX: df output isn't 100% portable
#XXX: race condition between running df and disk_free()
def test_diskfree():
    import commands
    real_cmd = "df -B1 / | tail -n +2 | awk '{ print $4; }'"
    actual_bytes = int(commands.getoutput(real_cmd).strip())
    assert_equals(path.disk_free("/"), actual_bytes)

The output of 'real_cmd' on Fedora linux 14 is:

[holland-test]% df -B1 / | tail -n +2 | awk '{ print $4; }'

80%

The leading line is not the problem though, it is the '%':

>>> commands.getoutput(real_cmd).strip()
'80%'

>>> int('80%')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '80%'

>>> int('80')
80

Adding an additional ".rstrip('%')" would solve this problem... though it isn't exactly elegant:

>>> actual_bytes = int(commands.getoutput(real_cmd).strip().rstrip('%'))
>>> actual_bytes
80

The problem, as noted in the 'XXX' comments is that this test isn't very portable. The same 'real_cmd' on Darwin/OSX is:

derks-macbook [~] $ df -B1 / | tail -n +2 | awk '{ print $4; }'
df: illegal option -- B
usage: df [-b | -H | -h | -k | -m | -g | -P] [-ailn] [-T type] [-t type] [filesystem ...]

[BJ Dierkes (derks)]

Just realized that the percentage (80%) isn't even the number this test is looking for...

[Andrew Garner (muzazzi)]

That test is just verifying that holland's disk_free (pure python) method matches what we might expect from 'df' output - basically "how much available space is on this path?" This works with bytes so it's trying to coax df to give available bytes for some filesystem path and compare it to how holland is performing it (via os.statvfs). I think the holland method is portable to OS X, but obviously the test is not.

On Fedora14 (my main development platform), this is probably broken due to say, an LVM root volume or something where the output is shifted down by one line. This was a quick and dirty test that I haven't got around to making more robust. There are likely a other issues with some of the (portable, pure python) functions that try to ensure those methods match various shell command output (as a sanity check). Be sure to send a pull request if you come up with any good fixes.