long-term memory leak

[Tom Hoffman]

Welsh reports that their CanDo instances seem to slowly leak memory. Is there a way we can profile that?

[Douglas Cerna (replaceafill)]

I remember Justas advised me to use ZODB savepoints to free some memory after a server crashed when David was importing lots of students, teachers and enrollments. About reports, a problem I've noticed is that they use too many big lists and dictionaries for calculate totals. Ordering the results is also painful because I have to store the values first to compare them later. For instance the completion report:

1. It traverses all the sections in the term. Sections are retrieved using a generator though.
2. For each section it gets a competency dictionary
3. For each student in the section it gets the grade for every competency
4. Then it creates a totals row for that student from the grades
5. And it puts the student's totals row in a BIG list for sorting the result by instructor or by area. I guess this is the cause of the leak, right?

When I test the ACC testing instance which has around 350 sections in a term (year) and around 10K students, it takes a loooot of memory.