check warc digest (Etag) after upload
Bug #661522 reported by
siznax
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Archive Widecrawl |
Fix Committed
|
Medium
|
siznax | ||
Bug Description
we send warc md5 checksum (content-md5) in our IAS3 (S3 hereafter) PUT request. the S3 response includes the computed digest (ETag). we should compare the two, and report an error on mis-match.
this would require a significant change to draintasker internals. probably would want to rewrite this portion of the code in a language with better exception handling than bash.
| Changed in archivewidecrawl: | |
| status: | New → Confirmed |
| importance: | Undecided → Low |
| assignee: | nobody → siznax (siznax) |
| Changed in archivewidecrawl: | |
| milestone: | none → ait-to-petabox |
| importance: | Low → Medium |
| Changed in archivewidecrawl: | |
| milestone: | none → ait-petabox-alpha |
Revision history for this message
| siznax (siznax) wrote | #1 |
| Changed in archivewidecrawl: | |
| status: | Confirmed → Fix Committed |
To post a comment you must log in.
fix committed with
-------
r51 | steve | 2010-11-18 22:00:39 +0000 (Thu, 18 Nov 2010) | 1 line
verify ETag, send actual size from PACKED for size_hint