archive-index uses too much memory

Bug #804144 reported by Jjed
This bug affects 1 person
Affects Status Importance Assigned to Milestone

Bug Description

archive-index currently uses as much RAM as the size of an archive's uncompressed Contents file. It should store Contents paths in a tree data structure to decrease data redundancy.

This is also a roadblock for performance improvements decreasing the amount of regex searching archive-index performs.

Jjed (jjed)
Changed in archive-index:
importance: High → Low
assignee: Jacob Johan Edwards (j-johan-edwards) → nobody
Revision history for this message
Jjed (jjed) wrote :

Unfortunately the memory overhead for Python objects is so high that a tree structure consumes even more memory than the current implementation, even using `___slots__`. Perhaps a C extension would be desirable.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers