Bulk export files to local directory

[Paul Everitt]

OSF has a need to download a LOT of files (2,800) to investigate them. They'd like to give us a text file with filenames, one per line, and we then give them a ZIP file (or manually upload to Box). They can't give us the community and sub-folder...they only know the "filename".

- We can do this as a one-off console script

- Read the lines in the text file

- Iterate through each File in the catalog, making a mapping of filename -> docid

- Iterate through each line in the text file

- If the entry matches a File filename, "dump it" (see below)

- If not, write to a log file about entries that weren't found

Dumping
===========

- Write the file contents to the current working directory

- Use a filename such as communityname-foldername-subfoldername-filename

- Also in the directory, keep an errors.log file with entries for missing filenames

[Paul Everitt (paul-agendaless)]

I believe this is going to come back, Oleg asked about it.

[Paul Everitt (paul-agendaless)]

Hi Carlos, let's make this your one task for this week. Hopefully you can use Newt queries for getting to the filename. Two other points: case-insensitive and only for things that are file-like: File, attachments, etc.

[Paul Everitt (paul-agendaless)]

Also, run this on *staging* not production (as they want the state of the database a number of months ago.) But make sure that staging has content from July 2016.

[Carlos de la Guardia (cguardia)]

Done. Waiting for Oleg's comments.