Search result rendering can crush the system
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Evergreen |
Fix Released
|
High
|
Unassigned | ||
2.3 |
Fix Released
|
Undecided
|
Unassigned | ||
2.4 |
Fix Released
|
High
|
Unassigned |
Bug Description
We recently addressed an issue where multiple, identical, broad searches would cause extreme load on the db and cause a cascade of other issues, up to and including a system down state from the user's point of view. This can be triggered by entering a search term in the TPAC and then holding down the Enter key. This was addressed by detecting concurrent, identical searches and only performing the first, causing the remaining searches to wait for and then use the results of the first.
However, once those concurrent searches retrieve their cached results, they all individually render the result page directly from the database via CPU-expensive in-db unAPI calls. This commit uses a very similar technique to add a short-term caching layer (10 seconds) per identical unAPI call to relieve the load of this second wave of load coming from concurrent, identical searches.
Changed in evergreen: | |
status: | Fix Committed → Fix Released |
Perhaps we could also disable the form submit once the form's been submitted? (Although that goes against the "minimal javascript" thing)