Evergreen on Perl 5.18+: Hash Randomization

Bug #1542297 reported by Chris Sharp on 2016-02-05
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Undecided
Unassigned

Bug Description

Thanks to Jeff Godin's research, we are able to track the problem reported in bug 1541801 to a change to the way Perl handles hash ordering (described here: http://search.cpan.org/dist/perl-5.18.0/pod/perldelta.pod#Hash_overhaul). I encountered the same behavior in a file being generated for a third-party vendor for PINES that uses the call 'open-ils.collections.users_of_interest.retrieve' and is returning its results in random order.

I'll leave approaches to more knowledgeable and experienced minds, but Evergreen definitely needs to be updated to accommodate these changes.

Affects any Evergreen version running on Ubuntu 14.04 LTS (Perl 5.18.2) or Debian jessie (5.20.2). Maybe others.

Bill Erickson (berick) wrote :

Chris, can you clarify what problem you are seeing with open-ils.collections.users_of_interest.retrieve ? Is the API caller expecting the keys in each user hash to be in a certain/consistent order?

Chris Sharp (chrissharp123) wrote :

Bill,

We're using a bash wrapper (that I believe you may have authored long ago) http://git.evergreen-ils.org/?p=contrib/pines/genasys.git;a=blob;f=templates/utility/run-collections.sh;h=d5f39646e90d537f55ba4e89ecb43daa9dd2ef91;hb=HEAD to call a perl wrapper (again, probably yours) http://git.evergreen-ils.org/?p=contrib/pines/genasys.git;a=blob;f=templates/utility/ums/run-calls.pl;h=37e6699bebcd4216222f34eb7a74d11f73e78427;hb=HEAD that outputs a large file of JSON. Before the upgrade to Evergreen 2.9.2/Ubuntu 14.04/Perl 5.18.2, the fields generated were in a consistent order, and I believe the vendor's processes were written around that static order. After the upgrade, the order was jumbled, causing the vendor to reach out to us for what the problem is. I was told by the vendor that they have developed a workaround, and I'm waiting for this weekend's run of the script to see if they are able to process the files.

Let me know if you need for from me to clarify the issue.

Bill Erickson (berick) wrote :

Thanks, Chris. That answers my question.

There's no indication that we are relying on the hash key ordering within EG for this process, which is something we would have to fix. Since key order assumption is on the vendor side in this instance and since the vendor is developing a workaround, we probably don't need to change anything on the EG side to address this particular manifestation of the Perl change.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers