Attempting to validate an empty field causes scan of most/all of authority.record_entry

Bug #1745462 reported by Galen Charlton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Fix Released
Medium
Unassigned
3.1
Fix Released
Medium
Unassigned

Bug Description

Clicking the "Validate" button in the MARC editor when an authority-controlled field is present but empty (i.e., all subfield values are empty) can result in the following query being run in the database:

SELECT "are".id AS "id" FROM authority.record_entry AS "are" WHERE "are".control_set = '1' AND "are".deleted = 'f' AND "are".simple_heading IS NOT NULL;

This is at best inefficient, as it can cause a sequential scan of authority.record_entry and the return of the IDs of most/all of the authority records. It can have worse consequences, since the underlying cstore request is invoked from an atomic method, open-ils.search.authority.simple_heading.from_xml.batch.atomic, meaning that the cstore drone ends up storing the entire result set in memory, which can be sizable if the database contains a lot of authority records.

I have reproduced the problem in the web staff client's MARC editor and suspect it also affects the XUL MARC editor.

Evergreen master

Revision history for this message
Galen Charlton (gmc) wrote :

A patch is available at the tip of the user/gmcharlt/lp1745462_we_will_not_bow_to_all_the_authorities branch:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmcharlt/lp1745462_we_will_not_bow_to_all_the_authorities

tags: added: authority cataloging
Revision history for this message
Galen Charlton (gmc) wrote :

^^ Rather, a two-patch series.

Changed in evergreen:
milestone: 3.0.4 → 3.05
Galen Charlton (gmc)
tags: added: pullrequest
Changed in evergreen:
milestone: 3.0.5 → 3.0.6
Changed in evergreen:
milestone: 3.0.6 → 3.0.7
Changed in evergreen:
milestone: 3.0.7 → 3.0.8
Changed in evergreen:
milestone: 3.0.8 → 3.0.9
Changed in evergreen:
milestone: 3.0.9 → 3.0.10
Revision history for this message
Chris Sharp (chrissharp123) wrote :

Works as expected. Pushed to master, rel_3_0 and rel_3_1. Thanks, Galen!

Changed in evergreen:
status: New → Fix Committed
Changed in evergreen:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.