Downloading large objects from swift containers exhausts web-server memory

[Timur Sufiev]

In case the object to be downloaded from Swift container is very large - several Gigabytes should be enough - the web-server eats up all the memory on Controller node. Line https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/containers/views.py#L212 is the main suspect.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/161204

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/161204
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=46405d456d9b056e648a4e60235b4c1b251f1236
Submitter: Jenkins
Branch: master

commit 46405d456d9b056e648a4e60235b4c1b251f1236
Author: Timur Sufiev <email address hidden>
Date: Wed Mar 4 16:15:04 2015 +0300

    Fix web-server memory overrun when downloading objects from Swift

    To prevent memory overrun when downloading large objects from Swift
    * `resp_chunk_size` keyword should be passed to swiftclient
    * `obj.data` iterator returned from swiftclient is passed to HttpResponse
      (or StreamingHttpResponse for Django>=1.5) as usual since both response
      classes work with iterators/files/byte strings (yet StreamingHttpResponse
      does it better).

    The commit introduces new setting SWIFT_FILE_TRANSFER_CHUNK_SIZE that
    defines the size of chunk in bytes for Swift objects downloading.

    DocImpact

    Change-Id: I18e5809b86bfa24948dc642da2a55dffaa1a4ce1
    Closes-Bug: #1427819

[Ryan Selden (ryanx-seldon)]

You can see this option listed here http://docs.openstack.org/mitaka/config-reference/dashboard/config-options.html#openstack-settings-partial

Does it need to be added anywhere else?