Excessive memory usage

[mc0e]

When downloading files, Rapid Photo Downloader loads the whole of each file into RAM as it goes. This is problematic when it hits a multi-gigabyte movie file.

Can things be changed such that RPD streams the content to a file in the target directory, or possibly does that just for files over some threshold size (say 50MB)?

Using mmap for the file is not as good, as it will still cause swapping.

Creating the file in the target directory minimises the work required to rename the file to the target file name.

There should be some cleanup mechanism for any left over files that might exist if a previous run has been interrupted.

[Damon Lynch (dlynch3)]

When writing the program, it's true I didn't account for situations where video files wold be gigabytes in size. Patches to fix it are welcome.

One thing that cannot be changed writing the file directly into the target directory, avoiding a temporary directory. The program is designed to copy from multiple sources simultaneously into one destination. The only way I know of to do that safely is to write to a temporary file, then rename it.

A cleanup mechanism would be nice, but there are other higher priority features. However patches are welcome for this too.