Search result rendering can crush the system

[Mike Rylander]

We recently addressed an issue where multiple, identical, broad searches would cause extreme load on the db and cause a cascade of other issues, up to and including a system down state from the user's point of view. This can be triggered by entering a search term in the TPAC and then holding down the Enter key. This was addressed by detecting concurrent, identical searches and only performing the first, causing the remaining searches to wait for and then use the results of the first.

However, once those concurrent searches retrieve their cached results, they all individually render the result page directly from the database via CPU-expensive in-db unAPI calls. This commit uses a very similar technique to add a short-term caching layer (10 seconds) per identical unAPI call to relieve the load of this second wave of load coming from concurrent, identical searches.

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/unapi-spam-reduction

[Pasi Kallinen (paxed)]

Perhaps we could also disable the form submit once the form's been submitted? (Although that goes against the "minimal javascript" thing)

[Mike Rylander (mrylander)]

That would address the other bug, but only when JS is available (IOW, it closes an accidental DoS hole for "nice" clients, but not the root issue).

[Dan Wells (dbw2)]

Works as advertised, pushed back through 2.3. Thanks, Mike!