Zope Content Management Framework (CMF)

Reading large files from GS import context hogs RAM

Reported by Tres Seaver on 2008-08-18
2
Affects Status Importance Assigned to Milestone
Zope CMF
Undecided
Tres Seaver

Bug Description

In parallel to the export problem reported in:

  https://bugs.launchpad.net/zope-cmf/+bug/257365

importing large files into the site can also cause excessive
memory use: it requires that the content of the file be in
memory as a string.

Some contexts might be able to support a more efficient
pattern, returning a file handle (or file-like object) which can be
used for chunked reads. The attached patch allows such contexts
to implement a new interface, 'IChunkedImportContext', which
offers a new API, 'openDataFile'.

It also implements that interface for DirectoryImportContext.

Tres Seaver (tseaver) wrote :
Tres Seaver (tseaver) wrote :
Changed in zope-cmf:
assignee: nobody → tseaver
status: New → Fix Committed
Jens Vagelpohl (dataflake) wrote :

No need to wait before calling it "fix released", it will be released eventually by being on the trunk.

Changed in zope-cmf:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers