Comment 1 for bug 1599001

Revision history for this message
Stuart Bishop (stub) wrote :

The simplest approach is to add --instance-id and --num-instances arguments, and have feed swift skip files unless (LibraryFileContent.id % num_instances==instance_id). This and updating the script locking to allow parallel runs is the least invasive approach, allowing us to easily schedule n tasks in parallel.

IIRC feed-swift will eventually disappear in favour of direct uploads to Swift, removing a SPOF.