tpac: search filter groups don't play nicely with facets

[Kathy Lussier]

Evergreen version 2.3

We have a system that is using search filter groups to limit by format on the basic search page. When using one of these filters, search results return as expected with facets along the sidebar showing the number of records that match each facet. However. if you click on one of the facets, no results are returned.

Sample search results page at http://bark.cwmars.org/eg/opac/results?query=kronos&fg%3Aformat_filters=12&qtype=keyword&locg=1&sort=

If you click on the Glass, Philip facet, you'll get no results.

I saw the same behavior in two other catalogs using search filter groups.

[Mike Rylander (mrylander)]

To narrow the scope of investigation down a bit, this looks to be squarely in the tpac's query string construction code. If you add the facet directly to the search box, the search works as intended. The facet doesn't get highlighted for later deactivation, of course, but properly constructed searches do work. See: http://bark.cwmars.org/eg/opac/results?query=kronos+author%7Cpersonal%5BGlass%2C+Philip.%5D&fg%3Aformat_filters=12&qtype=keyword&locg=1&sort=

[Dan Wells (dbw2)]

The problem seems to be query order dependent.

This works:

http://bark.cwmars.org/eg/opac/results?query=kronos+author|personal[Glass%2C+Philip.]+filter_group_entry(12)

This doesn't:

http://bark.cwmars.org/eg/opac/results?query=kronos+filter_group_entry(12)+author|personal[Glass%2C+Philip.]

[Kathy Lussier (klussier)]

I just tested this on master and was not able to replicate the problem. Is it possible that the 2.4 query parser work addressed the underlying problem?

[Kathy Lussier (klussier)]

I don't know if this is related, but I've also discovered issues with applying facets for nested Boolean searches.

Using a 2.3 catalog, I conduct the following search: teenagers && (drugs || alcohol). When I select a facet, I get no results. See the search results at http://bark.cwmars.org/eg/opac/results?query=teenagers+%26%26+%28drugs+%7C%7C+alcohol%29&fg%3Aformat_filters=&qtype=keyword&locg=1.

I was able to replicate this problem in Bibliomation's catalog and Conifer's catalog.

I have no problem with simple Boolean searches like "cats && dogs" or "teenagers && drugs || alcohol". Maybe it's the parentheses that are causing problems?

Trying the same search in a catalog using master (with the most query parser branches), I see a different problem. See http://catalog.mvlc.org/eg/opac/results?fi%3Aitem_type=&query=teenagers+%26%26+%28drugs+%7C%7C+alcohol%29&qtype=keyword&locg=1.

When I click on a facet here, the catalog doesn't refine the search. It delivers the same results that were retrieved before the facet was selected.

The facets work as expected when doing the same search in a 2.2 catalog.
http://evergreen.noblenet.org/eg/opac/results?fi%3Aitem_type=&query=teenagers+%26%26+%28drugs+%7C%7C+alcohol%29&qtype=keyword&locg=1

If this is a separate issue from the original bug report, let me know and I'll file a new bug report.

[Mike Rylander (mrylander)]

Kathy,

I tested on 2.4RC, and the master/2.4 issue you see is indeed order-based, and placing the facets and filters before the paren-grouped nested bools makes things happy.

I've seen this before with other search syntax components -- the #available modifier, for instance -- and Dan Wells noted a similar effect above (though on 2.3). For master, it looks like we need to have the tpac query constructor anything complex at the end. This may very well be caused by implicit or explicit nesting of boolean expressions (the tests we've all tried suggest that), but a short-term fix would be to adjust the tpac helper code until we can get to the root of the QP issue.

For 2.3, the same could be helpful, and I suspect that in 2.2 and before the tpac was just building the query in a different way that got around this problem. I don't have evidence to support that ATM, though.

[Jim Keenan (jkeenan)]

This bug is affecting us in C/W MARS.

Jim Keenan

[Kathy Lussier (klussier)]

I just took another look at this since we have another consortium that wants to start using search filter groups in 2.3 and has actually considered disabling facets so that they can get the added flexibility of defining their formats with search filter groups. I would love to see them move forward with both search filter groups and facets, so I'm hoping there might be some way to resolve this in 2.3. Although it appears to work properly in 2.4, our preference is to get the search filter groups working properly sooner rather than later.

I don't have much to add, but did want to mention something that I hadn't noticed during the initial discussion.

Although it appeared as if the:

http://bark.cwmars.org/eg/opac/results?query=kronos+author|personal[Glass%2C+Philip.]+filter_group_entry(12)

URL worked above, it really didn't. There are results, but they are no longer being limited by format filter. If I do what Mike suggested and replace the "facet=author%7Cpersonal%5BGlass%2C%20Philip.%5D" portion of the query with "author%7Cpersonal%5BGlass%2C+Philip.", then I always successfully return the results no matter what the order of the parameters. Actually, I didn't test every conceivable order, but I was able to insert the search filter group parameter into any part of the URL.

[Mike Rylander (mrylander)]

I have a branch(*) that addresses this bug and several other symptoms of the underlying disease (order of advanced query components matters in complex (aka, nested) searches) available at http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/search_user-input-last .

This is not the final solution, and while I think this needs to be committed to at least master and 2.4, I plan to open a new bug against the query parser regarding the underlying problem. IOW, this is a broad and workable bandaid, but it treats only the symptoms, and only one interaction point. It just so happens that this one interaction point is the only one in use today for interactive search by a user, but old full-query bookmarks, for instance, that suffer from the effects of the underlying problem will not be fixed by this.

[Jason Etheridge (phasefx)]

I can say that Mike's branch is working for at least one library.

[Jason Etheridge (phasefx)]

I pushed my sign-off to user/phasefx/search_user-input-last @ working/Evergreen.git
http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/phasefx/search_user-input-last

[Dan Wells (dbw2)]

Passes my eyeball testing; pushing in on the strength of that plus Jason's testing and sign-off, and backporting through rel_2_3. Thanks, all!