fusecompress eats up memory when reading a file

[Andreas Hilboll]

Binary package hint: fusecompress

Ubuntu Lucid Lynx, AMD64.
fusecompress 2.6-2

I have one compressed foler, which I mount like this:

## MOUNT THE ASP DATA STORE (COMPRESSED)
if [ `mount | grep fusecompress | grep /mnt/data/ascii | wc -l` -eq 0 ]
    then
        fusecompress -o fc_c:lzo /mnt/data/.ascii /mnt/data/ascii
fi

I then filled this folder with a bunch (~2500) of ASCII files, about 29MB each. The copying *to* this folder went smoothly, and actually surprisingly quick.

However, when I read a file (just by copying it *from* the compressed folder to my home directory), fusecompress quickly eats up all the RAM i have (4G) and even more, until swap is also completely taken by fusecompress and my machine basically crashes because there's no memory left. The copy process never finishes.

[Andreas Hilboll (andreas-h)]

The compressed folder lies on a xfs filesystem on a software raid-0.

[Andreas Hilboll (andreas-h)]

some more info:

in general, the read performance seems to be extremely low.

copying a 3.2M ASCII file from the compressed folder to my home directory takes 1.37s, whereas copying it within my homedirectory only takes .006s.

is this to be expected? I would have expected a much higher performance ...

[Ritesh Raj Sarraf (rrs)]

On Wednesday 19 May 2010 23:07:48 you wrote:
> copying a 3.2M ASCII file from the compressed folder to my home
> directory takes 1.37s, whereas copying it within my homedirectory only
> takes .006s.
>
> is this to be expected? I would have expected a much higher performance

Certainly not. Can you educate me on your setup ?
You mentioned that the box has 4G of memory. So I am assuming you must be
having a powerful CPU too. What is the kernel version you are using ?
Can you also attach the dmesg output when the bug is triggered ?

--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
"Necessity is the mother of invention."

[Andreas Hilboll (andreas-h)]

Here's some specs:

Ubuntu 10.04 Lucid AMD64
Kernel 2.6.32-22-generic
libfuse 2.8.1-1
Fusecompress 2.6-2
Filesystem: XFS
4G RAM, 8G swap
CPU Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz

dmesg doesn't show anything. i'll do some more testing and post results later ...

[Andreas Hilboll (andreas-h)]

okay, did some more testing.

I created 3 test ascii files test_{1,2,3}.txt, having sizes of approx. 3M, 28M, 56M.

Then I ran this script:

#!/bin/sh
for ALG in zlib bzip2 lzo
do
    echo $ALG
    for TEST in test_*.txt
    do
        ls -l -h $TEST
        fusecompress -o fc_c:$ALG ../compressed/$ALG $ALG
        time cp $TEST $ALG/
        fusermount -u $ALG

        fusecompress -o fc_c:$ALG ../compressed/$ALG $ALG
        time cp $ALG/$TEST .
        fusermount -u $ALG

    done
done

write performance for all three algorithms was okay, even though admittedly the 28s for 56M with bzip2 surprised me a bit.

read performance for bzip2 and zlib was fine, no anomalies found.

however, reading for lzo produced the same behaviour as I described earlier. Memory gets filled up quickly, even for the 3M file ... again, dmesg doesn't show anything.

[Ritesh Raj Sarraf (rrs)]

On Thursday 20 May 2010 14:09:20 you wrote:
> Here's some specs:
>
> Ubuntu 10.04 Lucid AMD64
> Kernel 2.6.32-22-generic
> libfuse 2.8.1-1
> Fusecompress 2.6-2
> Filesystem: XFS
> 4G RAM, 8G swap
> CPU Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz
>
> dmesg doesn't show anything. i'll do some more testing and post results
> later ...

2.6.32 suffer a memory related bug. I am not sure if ubuntu backported the fix
for it. If not, you might be suffering the same bug. But the symptoms for that
are easily seen in dmesg. Do you have an intel wifi card (using the iwlagn
driver) ?

For fusecompress, there was a memory leak some time back but that was fixed
before 2.6 version. You might want to try running fusecompress in debug mode
and see if that can produce some more information.

There's also a change that you might have hit this bug:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=551973

Regards,
Ritesh
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."

[Andreas Hilboll (andreas-h)]

Thanks for your suggestion; I had somehow missed there's a debug option.

Attached is the log for reading with lzo compression. Please note that I cancelled the read operation at some point.

Hope that helps ...

[Ritesh Raj Sarraf (rrs)]

On Friday 21 May 2010 20:42:39 andreas wrote:
> Thanks for your suggestion; I had somehow missed there's a debug option.
>
> Attached is the log for reading with lzo compression. Please note that I
> cancelled the read operation at some point.
>
> Hope that helps ...

Not much help from the logs.

The only things I can ask you are these:
* Run `vmstat 1` and trigger your bug.
* Profile the application. Maybe use sysprof or valgrind.

Regards,
Ritesh
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
"Necessity is the mother of invention."

[Andreas Hilboll (andreas-h)]

okay, here's the vmstat output. i dont't understand what it tries to tell me ...

probably i'll just go for zlib then, but would rather have the supposedly higher performance of lzo ...

anyways, thanks for your help!