OpenStack Image Registry and Delivery Service (Glance)

Swift uploads through Glance using ridiculously small chunks

Reported by Ewan Mellor on 2011-12-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
High
Jay Pipes

Bug Description

Ever since 57c4e9b6c686f1303c815698cc68b95bf9b033e7, uploads to Swift via Glance are being performed by default with reads of 200 bytes at a time, and going through a new temporary file on disk for each 200 bytes. This makes them really slow, obviously.

By the looks of it, the code before this would have attempted 200 MB reads, which would have been almost as bad -- you'd get 200 MB allocated for buffering per incoming stream.

Ewan Mellor (ewanmellor) on 2011-12-29
Changed in glance:
status: New → In Progress
assignee: nobody → Ewan Mellor (ewanmellor)
Jay Pipes (jaypipes) wrote :

"By the looks of it, the code before this would have attempted 200 MB reads, which would have been almost as bad -- you'd get 200 MB allocated for buffering per incoming stream."

Firstly, it would be great to leave the snark at the door, Ewan...

Secondly, the original code only used 200MB reads when the image size passed to the storage backend was >5GB. So saying that every incoming stream would allocate 200MB is not true...

It seems that, as you say, the 57c4e9b6c686f1303c815698cc68b95bf9b033e7 patch removed the multiplication of the config variable by (1024 * 1024), resulting in, as you say, "ridiculously small chunks". The solution is to return the multiplication of the megabyte to the config value.

FYI, 200MB is the default segment size for large object uploads in Swift, which is why 200MB was used originally.

Changed in glance:
importance: Undecided → High
milestone: none → essex-3
Ewan Mellor (ewanmellor) wrote :

I got halfway through implementing the simple fix, and then realised that the entire disk buffering process was unnecessary. The branch that I've proposed removes that as well as fixing the config variable.

Changed in glance:
assignee: Ewan Mellor (ewanmellor) → Jay Pipes (jaypipes)

Reviewed: https://review.openstack.org/2728
Committed: http://github.com/openstack/glance/commit/48449171a73e9b470b3caa136f20d56cd1757bdc
Submitter: Jenkins
Branch: master

commit 48449171a73e9b470b3caa136f20d56cd1757bdc
Author: Ewan Mellor <email address hidden>
Date: Wed Dec 28 21:13:55 2011 -0800

    Bug #909533: Swift uploads through Glance using ridiculously small chunks

    Remove the use of a temporary disk buffer when streaming to Swift. This
    was added on the assumption that it was not possible to stream chunks to
    Swift direct from webob.Request.body_file. That's not true -- a simple
    file-like object wrapping body_file does the job perfectly.

    This removes the need for swift_store_object_buffer_dir in the config file.
    It is also significantly cheaper, since there is one copy fewer.

    Fix the parsing of the swift_store_large_object_size and
    swift_store_large_object_chunk_size options. These are
    specified in MB in the config file, but needs to be in bytes internally,
    because they are compared against the image_size parameter given to add().

    Update the unit tests to match. This includes an additional check that
    put_object is called the correct number of times. The unit tests missed the
    latter problem problem because there was no check that a small object is only
    uploaded in one chunk, and despite the comment to the contrary there was no
    check that the right number of chunks were written in the large-object case
    either. Added these in all places where store.add is called.

    Change-Id: Ieb4cf68516b53bd16d2671e49e805b26118b3671

Changed in glance:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-01-25
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in glance:
milestone: essex-3 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers