commit 270ec44a890f4bd2917310e2fcab9ad2bb9413b7
Author: Alexander Tivelkov <email address hidden>
Date: Tue Jan 20 17:25:07 2015 +0300
Fix for CooperativeReader to process read length
CooperativeReader, being an eventlet-friendly wrapper around the
generator- based reader of image data, actually transforms
chunk-by-chunk iteration into the readable stream. It is used when the
image is being copied from the remote source: some generator-based
image data representing the remote source acts as its underlying
object, and the instance of CooperativeReader is passed as a data
stream to the backend client which uses it to read the data.
Before this patch, the CooperativeReader was ignoring the "length"
parameter of the read method, always returning the whole chunk returned
by the underlying generator (in case of HTTP source the size of this
chunk is 16 M). This was causing problems for the clients attempting to
read data from it, and - under some circumstances - the loss of data.
For chunked storage of files in Swift a special class (ChunkReader,
declared in the swift store driver) is used to reduce the requested
read length so no extra data is read and transferred. However, this was
not working as the CooperativeReader (which was the underlying stream
for the ChunkReader) was ignoring the requested size. This was causing
the data to be lost when reading behind the boundaries of the Chunks.
This patchset introduces a buffer in the CooperativeReader to store the
most recently fetched iterator chunk. The reads are independent from
requests to iterator, so the CooperativeReader is able to return the
exact requested amount of bytes and no data is lost due to extra-reads.
Reviewed: https:/ /review. openstack. org/148574 /git.openstack. org/cgit/ openstack/ glance/ commit/ ?id=270ec44a890 f4bd2917310e2fc ab9ad2bb9413b7
Committed: https:/
Submitter: Jenkins
Branch: master
commit 270ec44a890f4bd 2917310e2fcab9a d2bb9413b7
Author: Alexander Tivelkov <email address hidden>
Date: Tue Jan 20 17:25:07 2015 +0300
Fix for CooperativeReader to process read length
Cooperative Reader, being an eventlet-friendly wrapper around the
generator- based reader of image data, actually transforms
chunk-by-chunk iteration into the readable stream. It is used when the
image is being copied from the remote source: some generator-based
image data representing the remote source acts as its underlying
object, and the instance of CooperativeReader is passed as a data
stream to the backend client which uses it to read the data.
Before this patch, the CooperativeReader was ignoring the "length"
parameter of the read method, always returning the whole chunk returned
by the underlying generator (in case of HTTP source the size of this
chunk is 16 M). This was causing problems for the clients attempting to
read data from it, and - under some circumstances - the loss of data.
For chunked storage of files in Swift a special class (ChunkReader,
declared in the swift store driver) is used to reduce the requested
read length so no extra data is read and transferred. However, this was
not working as the CooperativeReader (which was the underlying stream
for the ChunkReader) was ignoring the requested size. This was causing
the data to be lost when reading behind the boundaries of the Chunks.
This patchset introduces a buffer in the CooperativeReader to store the
most recently fetched iterator chunk. The reads are independent from
requests to iterator, so the CooperativeReader is able to return the
exact requested amount of bytes and no data is lost due to extra-reads.
SecurityImpact
Change-Id: Ief37d1e29487bb 03e612320f5cc06 910cfd1c23a
Closes-bug: #1412802