Slim down Variable when cextensions are used

Bug #242314 reported by James Henstridge
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Storm
Won't Fix
Undecided
Unassigned

Bug Description

The cextensions work has helped improve speed of Storm a lot, but has left memory usage almost the same as before (well, the same if we use slots in the Python version).

Given that Variable objects will be some of the most common objects in a large application, it'd make sense to try and reduce the size of those objects. Here are a few ideas:

 * _allow_none is stored as a PyObject pointer, while it only needs a single bit. On AMD64, this is almost 8 bytes wasted.
 * if we add a flag to say whether the value is lazy or not, we can collapse _value and _lazy_value to a single pointer.
 * can we determine _validator_attribute from column?
 * can we determine event from _validator_object_factory()?

So this will save a few words of memory per Variable instance. When you multiply that by the number of Storm objects and number of columns on each of those objects, this adds up.

Revision history for this message
James Henstridge (jamesh) wrote :

From discussion on IRC:

20:24 <jamesh> niemeyer: another way to go would be a three-level hierarchy
20:24 <jamesh> e.g. IntVariable -> IntVariable with validator -> instance
20:25 <jamesh> converting some of the variable_factory stuff into classes rather than partial function application
20:29 <niemeyer> jamesh: Interesting. I wonder how much that'd save.. I remember that building the validator stuff required collecting data from a few different places, which is why we ended up like this
20:30 <niemeyer> jamesh: E.g. the partial function is already unique per class/variable, and the object itself can't be made into the IntVariable class with validator
20:30 <jamesh> niemeyer: well, in the existing VariableObject struct, I see _allow_none, _validator, _validator_attribute and column
20:31 <niemeyer> jamesh: Ok
20:31 <jamesh> niemeyer: only _value, _lazy_value, _checkpoint_state, _validator_object_factory and event change per instance
20:31 <jamesh> so that could be a reduction of 12/24 bytes per Variable
20:33 <jamesh> the only variable classes that add extra attributes are DateTimeVariable (_tzinfo) and EnumVariable (_get_map, _set_map)
20:33 <jamesh> both cases where we have per (class,column) data
20:34 <niemeyer> jamesh: Right, makes sense

Benji York (benji)
Changed in storm:
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.