Copying files to webdav broken (first completely copied to RAM and then transfered)

[Sebastian Dröge]

Hi,
when copying files to a webdav share with nautilus or gnomevfs-copy the following happens currently:

1. a 0 byte file is created
2. the file is completely copied to RAM, nautilus shows this as progress from 0% to 100%
3. the file is finally transfered, nautilus stalls at 100% until it's finished

while copying the file to RAM you could obviously run out of memory if the file is too large

(to get an easy to setup webdav share you could use gnome-user-share)

[Olaf Lüke (borg)]

same problem here.

[J.D. (fungs)]

I don't know whether this is the same, but copying large files (~2 GB) make the progress bar to advance way to fast until 1 GB and then stop there forever. If you try to unmount the virtual file system, it gives you a DBUS error message:

"DBus error org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken."

To use the FS again, you have to reboot (at least this is the only way I know of).

I counterchecked with cadaver which uploads these files without problems.

[Fahmy (fahmi)]

Same too. When I transferring file more than 2GB my RAM (even though I've 4GB) runs out and desktop freezed.

[JaSauders (jasauders)]

Just ran into this myself. I have 8GB in my system, and my system locks up @ 4.3GB of RAM each and every single time. This is happening when I'm sending a single 5.9GB compressed folder to a WebDAV server I have on my LAN. I'm doing this through Nautilus.

[JaSauders (jasauders)]

I've done some testing around in this area as well as discovered some information that may be relevant to this bug, so I wanted to update it here. I'm doing all of this on my LAN, as my server resides on my LAN. I'm also working on a gigabit connection. I was able to push 8,000 files amounting to 30GB to my WebDAV server with 0 issues from my desktop, which is the system I mentioned above with 8GB of RAM. I saw no memory spike whatsoever. Likewise, if I sent a single 5.9GB file to my WebDAV server, it spiked and when the file had sent roughly 4GB of the 5.9GB size, it tanked the connection with the RAM nearly 100% maxed.

It seemed to play significantly nicer with smaller files, such as the 8,000 above (30GB) that it had zero issues with. I have to assume that it's caching the files prior to sending them, and the system is able to let go of the cached files once they're already sent, thereby resulting in no visible memory spike. LIkewise, the singular 5.9GB file is one solid large file would have nothing to "let go" since it's one continuous file. That's probably where the 8,000 file scenario has an advantage.

When I ran into this "bug" I decided to try out alternatives. I found davfs2 which mounts WebDAV shares via terminal. Davfs2 had the same issue, however I realized it was more reliant on hard disk space versus RAM. A quick read of the man page of davfs showed:

##################################
Caching

mount.davfs tries to reduce HTTP-trafic by caching and reusing data. Information about directories and files are held in memory, while downloaded files are cached on disk.

mount.davfs will consider cached information about directories and file attributes valid for a configurable time and look up this information on the server only after this time has expired (or there is other evidence that this information is stale). So if somebody else creates or deletes files on the server it may take some time before the local file system reflects this.
##################################

I can't help but to wonder... if davfs caches by default, is that to suggest Nautilus/.gvfs is actually handling WebDAV services as intended? Can any devs comment on this info?

[Sebastien Bacher (seb128)]

@jasauders: thanks for your comments, gnome-vfs has been deprecated and replace by gvfs for years though so you likely comment at the wrong place or do you still have a software using gnome-vfs somewhere and that's what you are using?

if you want to talk to devs you should probably open a bug on https://bugzilla.gnome.org/enter_bug.cgi?product=gvfs (or check the open bugs, there is likely a similar one to this issue for gvfs)