Code path:
write_multivol(backup_type, tarblock_iter, man_outfp, sig_outfp, backend)
creates a local file tdp and then calls
put(tdp, dest_filename, vol_num)
which first gets the local file's size
putsize = tdp.getsize()
then tells the backend to store it
backend.put(tdp, dest_filename)
then calls this function
validate_block(putsize, dest_filename)
which gets the remote filesize
info = backend.query_info([dest_filename])[dest_filename]
size = info['size']
and compares it with the local one
if size != orig_size:
and throws a fatal error if they don't match
code_extra = "%s %d %d" % (util.escape(dest_filename), orig_size, size)
log.FatalError(_("File %s was corrupted during upload.") % util.ufn(dest_filename), log.ErrorCode.volume_wrong_size, code_extra)
It does not retry transfering the file since the retry mechanism is inside `backend.put()`.
The swift client uses the 'content-length' response header to a head request for the file as the remote size.
So there's 2 possible scenarios happening. Either the objectstore actually failed to write the file completely while still returning an http code >= 200 and < 300, or the objectstore returns an inaccurate 'content-length' header.
Both cases seem to indicate something wrong with the objectstore. Perhaps adding retries to work around that is not the most prudent way forward.
Code path: multivol( backup_ type, tarblock_iter, man_outfp, sig_outfp, backend) block(putsize, dest_filename) query_info( [dest_filename] )[dest_ filename] dest_filename) , orig_size, size) (_("File %s was corrupted during upload.") % util.ufn( dest_filename) ,
log.ErrorCode .volume_ wrong_size, code_extra)
write_
creates a local file tdp and then calls
put(tdp, dest_filename, vol_num)
which first gets the local file's size
putsize = tdp.getsize()
then tells the backend to store it
backend.put(tdp, dest_filename)
then calls this function
validate_
which gets the remote filesize
info = backend.
size = info['size']
and compares it with the local one
if size != orig_size:
and throws a fatal error if they don't match
code_extra = "%s %d %d" % (util.escape(
log.FatalError
It does not retry transfering the file since the retry mechanism is inside `backend.put()`.
The swift client uses the 'content-length' response header to a head request for the file as the remote size.
So there's 2 possible scenarios happening. Either the objectstore actually failed to write the file completely while still returning an http code >= 200 and < 300, or the objectstore returns an inaccurate 'content-length' header.
Both cases seem to indicate something wrong with the objectstore. Perhaps adding retries to work around that is not the most prudent way forward.