Switch spambayes training persistence away from pickle
Bug #1035758 reported by
Jean-Paul Calderone
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Quotient |
Fix Committed
|
Undecided
|
Unassigned | ||
Bug Description
Pickle is an unwieldy format for this data, particularly with the current monolithic approach where the entire training dataset needs to be loaded and saved for any operation. By the time you have a decent amount of training, this gets really slow.
We have SQLite3, we should probably use that.
Related branches
lp:~exarkun/divmod.org/database-classifier-persistence
- Tristan Seligmann: Needs Fixing
-
Diff: 605 lines (+450/-43)6 files modifiedQuotient/README.txt (+1/-1)
Quotient/xquotient/spam.py (+266/-26)
Quotient/xquotient/test/historic/stub_spambayesfilter2to3.py (+36/-0)
Quotient/xquotient/test/historic/test_spambayesfilter1to2.py (+1/-1)
Quotient/xquotient/test/historic/test_spambayesfilter2to3.py (+37/-0)
Quotient/xquotient/test/test_spambayes.py (+109/-15)
| Changed in quotient: | |
| status: | New → Fix Committed |
To post a comment you must log in.
