Comment 1 for bug 1264423

Revision history for this message
Kevin Bowrin (kbkbkbkbkb) wrote :

I thought the 'junk' segments were kept in swift to allow for 'resuming' uploads when an SLO upload failed. This is not the case. The wasteful behaviour that Zhou Yuan describes is accurate. It is doubly wasteful: It keeps unused data in swift, and it doesn't use already uploaded segments to speed up retries.

A new CLI option should be created for SLO segmented uploads, which controls what happens on partial uploads. The junk should either be deleted, or used by subsequent uploads to speed up retries. If the hash and name of a segment is the same as one that already exists, the segment should not be uploaded again.