Comment 5 for bug 30415

Christian Reis writes ("[Bug 30415] Re: soyuz upload system pays attention to ftp sessions"):
> The main reason this has never been done is that it would involve
> rearchitecting of the upload service, and few if any other users have
> ever reported this to be an inconvenience.

In practice it's not usually too much of an inconvenience if you know
it's happening.

> It would involve rearchitecting, by the way, because our incoming poppy
> server has no concept of a persistent upload directory; every upload is
> made to its own temporary directory, which is then asynchronously
> processed by a separate process that validates and creates the upload.

That sounds like a perfectly fine beginning approach.

> To provide a persistent upload directory would probably require changing
> the way poppy stores the data, and would require identifying the user
> uploading somehow (perhaps based on the gpg key ID being used?) to be
> able to tell what his directory is. It's not a bad idea, and it's not
> crazy, but there are a lot of other pressing issues that need our
> engineering resources too.

I propose the following approach:

 * Move files out of the incoming directories and rename them into a
   holding directory with a filename which includes their checksum.
   (Several hardlinks if several checksums need to be supported.)
   If two identically named files arrive with the same checksums,
   check that they are identical and if not call for help. (Put
   everything aside and block further processing of any file with this
   checksum.)
 * Periodically look for .changes files all of whose pieces have
   arrived, and then process them. When processed, delete the
   .changes but not the other files.
 * Expire files after some period of time (24h?)

Ian.