Large object from object storage can not be downloaded via Horizon
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mirantis OpenStack |
Fix Released
|
High
|
Timur Sufiev | ||
5.1.x |
Fix Released
|
High
|
Alex Ermolov | ||
6.0.x |
Fix Released
|
High
|
Alex Ermolov | ||
6.1.x |
Fix Released
|
High
|
Timur Sufiev |
Bug Description
Rados-gateway is used for Object storage. The service radosgw is running on each controller.
We have a container in a project that is ~4.4 GB in size. Each time the object is downloaded via Horizon the "completed" download is only 425 MB.
The same object can be downloaded correctly via CLI.
Log files do not contain any errors (horizon, Apache, radosgw).
Changed in mos: | |
assignee: | nobody → MOS Horizon (mos-horizon) |
Dmitry Sutyagin (dsutyagin) wrote : | #1 |
Dmitry Sutyagin (dsutyagin) wrote : | #2 |
the chart is on second sheet - https:/
tags: | added: ceph |
tags: | added: horizon |
Changed in mos: | |
assignee: | MOS Horizon (mos-horizon) → Vlad Okhrimenko (vokhrimenko) |
Changed in mos: | |
status: | New → Confirmed |
milestone: | none → 6.1 |
Dmitry Borodaenko (angdraug) wrote : | #3 |
Vlad, are you currently working on this? Do you have a plan and an estimate for a fix?
Vlad Okhrimenko (vokhrimenko) wrote : | #4 |
I have been working. I try write new `UploadHandler` and `Storage` for Django
Timur Sufiev (tsufiev-x) wrote : | #5 |
Actually, the Severity is higher than it looked before.
Dmitry, I'm taking this, the ETA will be by 20:00 MSK.
Changed in mos: | |
assignee: | Vlad Okhrimenko (vokhrimenko) → Timur Sufiev (tsufiev-x) |
importance: | Medium → High |
Timur Sufiev (tsufiev-x) wrote : | #6 |
Seems that the root of evil is here: https:/
Timur Sufiev (tsufiev-x) wrote : | #7 |
Unfortunately, just using StreamingHttpRe
Changed in mos: | |
status: | Confirmed → In Progress |
Timur Sufiev (tsufiev-x) wrote : | #8 |
Upstream bug is https:/
Timur Sufiev (tsufiev-x) wrote : | #9 |
Guys, I've made a fix at https:/
OSCI Robot (oscirobot) wrote : | #10 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #11 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #12 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #13 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #14 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #15 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #16 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #17 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #18 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #19 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #20 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #21 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #22 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #23 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #24 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #25 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #26 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
OSCI Robot (oscirobot) wrote : | #27 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #28 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #29 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #30 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /ubuntu-
Timur Sufiev (tsufiev-x) wrote : | #31 |
The second approach to the problem I had tried worked well (4.4GB successfully downloaded), but it's more complicated. Instead of streaming these gigabytes through Django machinery, I'm using usual response with setting 'X-Sendfile' header to the full path to the file after it's extracted from Swift (yes, it needs to be stored on the Controller node before giving it away to the Apache). 'X-Sendfile' header tells Apache that it should handle file serving itself, bypassing the Django. Also for this solution to work, Apache needs to be compiled with mod_xsendfile.c extension. The most significant drawbacks here are a) the necessity to store images from Swift on Controller node and b) manual cleanup of files after they were downloaded (perhaps some Cron job?)
If this solution is unsatisfactory, I will continue investigation on why the connection is closed in case of using Django's StreamingHttpRe
Timur Sufiev (tsufiev-x) wrote : | #32 |
I've found the problem with first approach - there was a typo, instead of using 512KB chunk size I specified 512MB, which lead to encountering the socket timeout while the memory buffer was filled up with 512 Megabytes of data inside Django process. Right now I'm testing several variants of already working solution to:
* ensure that works both in development and production configuration
* to determine the solution that is portable across Django 1.4 - Django 1.7.
I expect the final solution to be ready in 2 days (downloading 4.4 GB takes a lot of time).
Chris Clason (cclason) wrote : Re: [Bug 1423311] Re: Large object from object storage can not be downloaded via Horizon | #33 |
Thanks for the update.
On Mar 12, 2015 5:05 AM, "Timur Sufiev" <email address hidden> wrote:
> I've found the problem with first approach - there was a typo, instead
> of using 512KB chunk size I specified 512MB, which lead to encountering
> the socket timeout while the memory buffer was filled up with 512
> Megabytes of data inside Django process. Right now I'm testing several
> variants of already working solution to:
>
> * ensure that works both in development and production configuration
> * to determine the solution that is portable across Django 1.4 - Django
> 1.7.
>
> I expect the final solution to be ready in 2 days (downloading 4.4 GB
> takes a lot of time).
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> Large object from object storage can not be downloaded via Horizon
>
> Status in Mirantis OpenStack:
> In Progress
>
> Bug description:
> Rados-gateway is used for Object storage. The service radosgw is
> running on each controller.
>
> We have a container in a project that is ~4.4 GB in size. Each time the
> object is downloaded via Horizon the "completed" download is only 425 MB.
> The same object can be downloaded correctly via CLI.
>
> Log files do not contain any errors (horizon, Apache, radosgw).
>
> To manage notifications about this bug go to:
> https:/
>
OSCI Robot (oscirobot) wrote : | #34 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #35 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.1, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /trusty-
OSCI Robot (oscirobot) wrote : | #36 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Igor Gnatenko
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #37 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Igor Gnatenko
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /trusty-
Timur Sufiev (tsufiev-x) wrote : | #38 |
I consider the patchset#10 of CR https:/
Oleksiy Butenko (obutenko) wrote : | #39 |
also reproduced on 5.1.2 (HA, Centos)
Changed in mos: | |
milestone: | 6.1 → 5.1.2 |
status: | In Progress → New |
status: | New → In Progress |
milestone: | 5.1.2 → 6.1 |
Oleksiy Butenko (obutenko) wrote : | #40 |
{u'build_id': u'2015-
OSCI Robot (oscirobot) wrote : | #41 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #42 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #43 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #44 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #45 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #46 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /trusty-
OSCI Robot (oscirobot) wrote : | #47 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #48 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #49 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #50 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /trusty-
OSCI Robot (oscirobot) wrote : | #51 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #52 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #53 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /centos-
OSCI Robot (oscirobot) wrote : | #54 |
package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Timur Sufiev
committer: Timur Sufiev
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
NOTE: Changeset is not merged, created temporary package repository.
repository URL: /trusty-
OSCI Robot (oscirobot) wrote : | #55 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.0.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #56 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2.2, package release == fuel6.0.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #57 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.2, package release == fuel6.0.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #58 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2, package release == fuel6.0~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #59 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.2, package release == fuel6.0.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #60 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.2, package release == fuel6.0~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #61 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #62 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.3, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #63 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.1, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #64 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.1, package release == fuel5.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
OSCI Robot (oscirobot) wrote : | #65 |
RPM package horizon has been built for project openstack/horizon
Package version == 2014.1.1, package release == fuel5.1.
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
RPM repository URL: http://
OSCI Robot (oscirobot) wrote : | #66 |
DEB package horizon has been built for project openstack/horizon
Package version == 2014.1.1, package release == fuel5.1~
Changeset: https:/
project: openstack/horizon
branch: openstack-
author: Alex Ermolov
committer: Alex Ermolov
subject: Fix web-server memory overrun when downloading objects from Swift
status: patchset-created
Files placed on repository:
openstack-
openstack-
python-
python-
NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://
Oleksiy Butenko (obutenko) wrote : | #67 |
on verification
Oleksiy Butenko (obutenko) wrote : | #68 |
verified
{"build_id": "2015-05-
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/horizon (openstack-ci/fuel-5.1-updates/2014.1.1) | #69 |
Reviewed: https:/
Submitter: mos-infra-ci <>
Branch: openstack-
Commit: 53310ecdc020e4e
Author: Alex Ermolov <email address hidden>
Date: Wed Jul 15 12:42:52 2015
Fix web-server memory overrun when downloading objects from Swift
To prevent memory overrun when downloading large objects from Swift
* `resp_chunk_size' keyword should be passed to swiftclient
* `obj.data' iterator returned from swiftclient is passed to HttpResponse
(or StreamingHttpRe
classes work with iterators/
does it better)
Change-Id: I18e5809b86bfa2
Closes-Bug: #1423311
Vitaly Gusev (vgusev) wrote : | #70 |
Verified by me on 5.1.1 version
I can reproduce this in my virtual environment.
Juno - CentOS -HA - Neutron+VLAN.
Controller has 8Gb RAM.
Steps: /docs.google. com/a/mirantis. com/spreadsheet s/d/1ZSV1Qj2skc w506GWH9r03UWeW 42yY-WLUHVZZKxm 65k/edit? usp=sharing
1. Upload a 3.7Gb .iso via horizon into a container.
2. Start downloading.
3. Experience the following memory consumption trend by httpd.event process: https:/
4. Get error 500 after memory is depleted.
I see two problems here:
1. httpd caches loads the whole object into RAM. Means there is no way of downloading an object bigger than node's RAM.
2. memory "explosion" after the object has been cached - takes up all available memory on the node (in my case).