search fields in z39.50 sort in random order

[Chris Sharp]

Somewhere between 2.7.2 and 2.9.1 (up to current master), something changed to cause the fields in the z39.50 search interface to sort randomly when loaded. This was found initially in PINES 2.9.1 production, but I have since reproduced the problem in a test server running master. Our test server running 2.7.2 does not exhibit the problem.

Steps to reproduce:

1) In the staff client, go to Cataloging -> Import Record From Z39.50

2) In the Service and Credentials pane, click one of the checkboxes beside the available sources (it is not necessary to log in to see the problem).

3) Open a few new tabs, and repeat the above steps and move between tabs to see the random sorting.

While this does not prevent catalogers from doing work, it does introduce a nuisance that disorients the user. We have received multiple helpdesk tickets about it.

Evergreen 2.9 through current master (I have not determined whether 2.8 is affected)
OpenSRF 2.4.1
PostgreSQL 9.4
Ubuntu 14.04 LTS

[Kathy Lussier (klussier)]

I just tested this on master and an can confirm this behavior. Also noting that we've seen the random ordering of Z39.50 fields in the web client interface. Not sure if the web client work might have spilled over here.

[Galen Charlton (gmc)]

The browser client's Z39.50 code did not make any changes to underlying APIs used to fetch the list of Z39.50 targets and search attributes; I don't think there's any spillover going on here.

Rather, other than the item type being fixed at the top, the order of search attributes in the XUL interface is not currently defined, and can vary based on:

- subtle differences in how the XUL JavaScript engine iterates over JavaScript hashes
- the order in which attributes are returned by the open-ils.search.z3950.retrieve_services call, if the client's JavaScript engine is pulling out attributes in insertion order (and again, that API is not sorting

In srfsh, you mind try running the following repeatedly and see if you get any different results at time:

 request open-ils.search open-ils.search.z3950.retrieve_services "$AUTHKEY"

In any event, the for the moment would be to provide an explicit ordering, e.g., alphabetical by label.

[Chris Sharp (chrissharp123)]

Galen,

I can confirm that the order changes with each srfsh request on 2.9.1, but on 2.7.2, the order is static. That would indicate something different on the server side, right?

Thanks,

Chris

[Jennifer Pringle (jpringle-u)]

We saw this issue start when we upgraded from 2.6 to 2.8.1.

[Chris Sharp (chrissharp123)]

To add hopefully useful information to my test between 2.7.2 and 2.9.1, my 2.7.2 environment is on Ubuntu 12.04/Perl 5.14.2 and my 2.9.1 environment is on Ubuntu 14.04/Perl 5.18.2. I'm wondering if something changed in Perl between versions?

[Jason Stephenson (jstephenson)]

What I see is I get consistent, but non-alphabetical ordering of the fields depending upon whether I choose the Local Catalog or a remote Ztarget, first. (See screenshots.)

I also noticed that two labels change depending upon which I choose first: Pub Date => Publication Date and TCN => Title Control Number.

[Jason Stephenson (jstephenson)]

This is the screenshot with 1 remote chosen. I get the same field order regardless of the remote, YMMV.

[Dan Scott (denials)]

Could perhaps be associated with an upgraded PostgreSQL version resulting in a different implicit ordering of unordered results?

[Kathy Lussier (klussier)]

I saw this issue on a MassLNC VM that has:

* Ubuntu Trusty
* A fairly recent master (no older than a week)
* Postgres 9.3.10
* Perl 5.18.2

I checked some of our production systems, running Evergreen 2.8.4 and 2.9.1 respectively. Neither system showed the sorting problem. I don't know what Postgres or Perl versions are running on those servers, but it seems to point to something other than Evergreen causing the problem.

[Galen Charlton (gmc)]

I have removed the notation that this is a duplicate of bug 1542297; while I think it likely that Perl's new hash randomization is a likely culprit, I don't think it's a good idea to treat this as a global problem. It suffices to just adjust interfaces that will now care about the order of hash keys.

So, to that end: a minimal way to at least achieve a consistent ordering is to sort the search fields alphabetically by label (with the exception of putting the item format drop-down first). Are folks amenable to that?

[Chris Sharp (chrissharp123)]

Galen,

I agree with removing the duplicate status. PINES is good with alphabetical order.

Thanks!

Chris

[Rogan Hamby (rogan-hamby)]

I'm good with alphabetical order as well.

---Sent from iPhone

Galen,

I agree with removing the duplicate status. PINES is good with

alphabetical order.

Thanks!

Chris

--
You received this bug notification because you are subscribed to

Evergreen.

Matching subscriptions: evergreenbugs

https://bugs.launchpad.net/bugs/1541801

Title:

  search fields in z39.50 sort in random order

Status in Evergreen:

  Confirmed

Bug description:

  Somewhere between 2.7.2 and 2.9.1 (up to current master), something

  changed to cause the fields in the z39.50 search interface to sort

  randomly when loaded. This was found initially in PINES 2.9.1

  production, but I have since reproduced the problem in a test server

  running master. Our test server running 2.7.2 does not exhibit the

  problem.

  Steps to reproduce:

  1) In the staff client, go to Cataloging -> Import Record From Z39.50

  2) In the Service and Credentials pane, click one of the checkboxes

  beside the available sources (it is not necessary to log in to see the

  problem).

  3) Open a few new tabs, and repeat the above steps and move between

  tabs to see the random sorting.

  While this does not prevent catalogers from doing work, it does

  introduce a nuisance that disorients the user. We have received

  multiple helpdesk tickets about it.

  Evergreen 2.9 through current master (I have not determined whether 2.8 is affected)

  OpenSRF 2.4.1

  PostgreSQL 9.4

  Ubuntu 14.04 LTS

To manage notifications about this bug go to:

https://bugs.launchpad.net/evergreen/+bug/1541801/+subscriptions

[Chris Sharp (chrissharp123)]

Actually... Our cataloger had a change of heart. Since we're talking about preferences, here is PINES' preferred order (based on OCLC as a z39.50 source, but probably applicable to other sources):

Tcn
Isbn
Issn
Lccn
Author
Title
Publisher
Pubdate

But if that is a "development killer" by being too complex to implement, alpha order is fine.

Thanks!

Chris

[Janet Schrader (jschrader)]

CWMARS prefers the most commonly used fields at the top. Alphabetically Item Type is in the middle which isn't the most logical place for a filter. UPC is most frequently used after ISBN, LCC is really a distant last.

Item type ( the bib record format for limiting searches)
Keyword (first or next to last, it's not available for OCLC searching)
TCN
ISBN
ISSN
Standard number
LCCN
Author
Title
Publisher
Pub Date
Keyword (first or last?)
Subject (last?, not available for OCLC searching)

In the current client (2.8) we have to scroll down to see UPC or make the search results area quite small. In the web client the results display is a lot nicer than what we now have, much easier to read and navigate. However there's more scrolling because so much space is given to the search boxes. Any way to tighten those up with less wasted white space?

[Kathy Lussier (klussier)]

I'm just adding a note that I don't think UPC is a default field that shows up in the Z39.50 interface, which would explain why it's listed at the bottom of the list in your system, Janet. It might not be a bad default field to add, but I'm guessing that would need be considered part of the scope of this bug?

The PINES and C/W MARS list are fairly similar. Is there any chance something close to this sort could be done with little effort?

[Elaine Hardy (ehardy)]

PINES Z39.50 for OCLC only has the list I provided for the individual input wells for the search fields:

Tcn
Isbn
Issn
Lccn
Author
Title
Publisher
Pubdate

UPC and other standard numbers are done as a raw search under our configuration, which is from the original setup eons ago. We have to click on a raw search button and enter the search syntax for that index.

We don't have a keyword search box either.

Since we have not revisited our configuration for OCLC as a Z39.50 source, I don't know if the other fields are available for OCLC as source or not. Since Galen had indicated he was leaving the Item type dropdown menu at the top (which is stationary and doesn't move as a new tab is opened as do the the search fields), I only considered the labelled input wells.

So, if the other indexes and search fields are available in the OCLC Z39.50 interface, I think the CW/MARS order is good. If they are only available for other sources, I still think the order is good, it just won't affect what we see in PINES.

I am hoping this means we should revisit our configuration and be able to add keyword and UPC/standard identifiers and have them appear as individual search boxes!

[Mike Rylander (mrylander)]

IMO, getting alpha sort in, so there is at least a predictable order, would be a boon to 2.10. Then, a separate feature to allow relative ordering of search fields (akin to custom org tree ordering) would be a flexible way for each site to get what they want, along with a compromise default ordering on the seed-data fields that ship with stock.

Thoughts?

[Chris Sharp (chrissharp123)]

For my part, +1 to Mike's suggestion.

[Jennifer Pringle (jpringle-u)]

+1 to Mike's suggestion as well. At this point our cataloguers are just looking for a consistent order.

[Kathy Lussier (klussier)]

Belated +1 to Mike's suggestion. It looks like we missed the window for 2.10. Maybe 2.11?

[Galen Charlton (gmc)]

A two-patch series is available in the user/gmcharlt/lp1541801_alphabetize_z3950_search_fields branch in the working/Evergreen repository:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmcharlt/lp1541801_alphabetize_z3950_search_fields

Note that there may be other places that could make use of the orderObjectBy filter.

[Kathy Lussier (klussier)]

Thanks Galen! The Z39.50 are now consistently sorting alphabetically, but I noticed that the Item Type dropdown menu has been replaced by a text field.

[Kathy Lussier (klussier)]

Also noting that the fix works in the web client, but not in the xul client, where the fields continue to sort randomly and where the dropdown menu is still available.

Are we looking at just fixing the web client at this point and leaving the xul client as is?

[Galen Charlton (gmc)]

Good catch, Kathy. I've pushed a follow-up to the tip of the working branch that fixes the problem.

I myself am not planning on touching the XUL code for this.

[Kathy Lussier (klussier)]

Thanks Galen! That did the trick. Picked to master for inclusion in 2.12.