loggerhead could be faster if it used statictuple

Bug #567632 reported by Martin Pool
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
loggerhead
Triaged
Low
Unassigned
loggerhead-breezy
Triaged
Medium
Unassigned

Bug Description

mnordhoff started using statictuples in loggerhead in <https://code.edge.launchpad.net/~mnordhoff/loggerhead/statictuples/+merge/13706>: they may use less memory and be faster, because loggerhead allocates a lot of tuples. However there's still some outstanding work needed:

* The disk caching code has to convert between StaticTuples and normal
tuples, which is ugly and quite slow.

* For people without C extensions, or with an old version of bzr,
_static_tuple_py adds some overhead. It's not much, but Loggerhead has a
frightening number of tuples, so it could be a problem.

* I'm not 100% sure StaticTuple's intern cache (a SimpleSet or dict) is
entirely thread-safe. IIRC the answer was "the GIL should handle it",
but I _did_ once have a crash from some sort of bizarre corruption.

* The interning. Honestly, interning hurts my brain. I'm not sure what
all of the consequences are. In particular, _static_tuple_py's cache
never, ever removes things, so it could get quite large. Plus, some
other things could probably be interned (e.g. revno strings).

* There are some TODO comments about minor things that should either be
removed or actually fixed.

It would be worth finishing it off.

Tags: performance
Changed in loggerhead:
status: In Progress → Triaged
importance: Medium → High
Changed in loggerhead:
importance: High → Low
Jelmer Vernooij (jelmer)
Changed in loggerhead-breezy:
status: New → Triaged
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.