gigantic memory leak

[Stéphane Lesimple]

Binary package hint: curlftpfs

I already posted this to the upstream tracker, but it doesn't seem to be very active as all the bugs reported the last 1-2 years are still marked as "NEW", so as I have a fix (which may require reviewing, because I discovered the curlftpfs source code as I was trying to fix this bug), I'm posting this here too.

$ dpkg -l curlftpfs | tail -n 1
ii curlftpfs 0.9.2-1build1 filesystem to access FTP hosts based on FUSE and cURL

After downloading several files of several megabytes each, my Ubuntu Karmic-packaged curlftpfs 0.9.2 eats up all available RAM, and is killed by the kernel OOM_killer.
I then compiled my own unstripped version for debugging purposes, A short run under valgrind says (among other less-important leaks) :

==11954== 4,039,584 bytes in 10 blocks are possibly lost in loss record 39 of 39
==11954== at 0x4024D12: realloc (vg_replace_malloc.c:476)
==11954== by 0x804A5E9: buf_add_mem (ftpfs.c:73)
==11954== by 0x804A677: read_data (ftpfs.c:238)
==11954== by 0x41247B7: Curl_client_write (in /usr/lib/libcurl-gnutls.so.4.1.1)
==11954== by 0x4137B79: Curl_readwrite (in /usr/lib/libcurl-gnutls.so.4.1.1)
==11954== by 0x413EFD3: multi_runsingle (in /usr/lib/libcurl-gnutls.so.4.1.1)
==11954== by 0x413F578: curl_multi_perform (in /usr/lib/libcurl-gnutls.so.4.1.1)
==11954== by 0x804BF13: ftpfs_read_chunk (ftpfs.c:406)
==11954== by 0x804C359: ftpfs_read (ftpfs.c:837)
==11954== by 0x40F065D: fuse_fs_read (in /lib/libfuse.so.2.7.4)
==11954== by 0x40F6543: ??? (in /lib/libfuse.so.2.7.4)
==11954== by 0x40FB83B: ??? (in /lib/libfuse.so.2.7.4)

I tracked down the problem to be in the free_ftpfs_file() function, where the passed ftpfs_file structure is not correctly freed. More specifically the two uint8_t *p of the 2 struct buffer contained in the ftpfs_file structure.
This means - if I am right - that all the files read on the FTP are kept into curlftpfs memory space for ever... which will undoubtedly lead to big problems when a lot of files are read from the FTP server.

Attached is a patch that fixes it. I'm running the patched version since several hours, with intensive downloading of big backup files from an FTP server, and top is now reporting a stable memory occupation for curlftpfs.

[Adam Guthrie (therigu)]

Thanks for the bug report and the patch. This looks to be your upstream report:

http://sourceforge.net/tracker/?func=detail&aid=2924678&group_id=160565&atid=816357

If it doesn't look like upstream is maintaining this any longer, it would be worth opening a bug in Debian with at attached patch. If it gets accepted there then it will get synced into Ubuntu also.

[Stéphane Lesimple (speed47)]

Thanks for your answer, I opened a bug in Debian as suggested.
I hope they'll be more reactive than upstream :)
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=587250

[Micah Gersten (micahg)]

Was fixed several years ago:
curlftpfs (0.9.2-3) unstable; urgency=low

  * Fix a memory leak, thanks to a patch from Stéphane Lesimple.
    Closes: #587250.

 -- Vincent Bernat <email address hidden> Sun, 27 Jun 2010 22:39:30 +0200