Did You Mean's search.symspell_dictionary can get significantly bloated
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Evergreen | Status tracked in Main | |||||
3.10 |
Fix Released
|
Medium
|
Unassigned | |||
3.9 |
Fix Released
|
Medium
|
Unassigned | |||
Main |
Fix Released
|
Medium
|
Unassigned |
Bug Description
We observed that search.
- loading ~50K bib records in a large database
- doing bib-to-authority linking using authority_
The combination resulted in a couple hundred GB of disk space getting used unexpectedly.
Focusing on authority_
However, in practice such changes result in the following:
- bib gets reingested (because it's actually different with the $0's)
- all metabib field entries get deleted and recreated. Because the field entry rows don't get updated, just recreated, the attempt by search.
- when search.
The kicker: empirically, autovacuum simply cannot keep up to keep a lid on the size of search.
A patch is forthcoming.
Changed in evergreen: | |
importance: | Undecided → Medium |
tags: | added: database performance pullrequest search |
tags: | added: didyoumean |
Changed in evergreen: | |
status: | New → Confirmed |
A patch is available at user/gmcharlt/ lp1998355_ symspell_ size_control / https:/ /git.evergreen- ils.org/ ?p=working/ Evergreen. git;a=shortlog; h=refs/ heads/user/ gmcharlt/ lp1998355_ symspell_ size_control