I signed off on the commit but had to resolve the conflict again, and pushed it to user/dbs/LP888572_clean_marc_fix_corruption - along with another commit, below, to add some unit tests and resolve a potentially severe problem:
*IMPORTANT* In the course of adding some unit tests to ensure that we were generating what we expected, I found that clean_marc() (and very likely the functionality as it exists within Vandelay) introduces subtle but significant corruption to upper case diacritic characters. Check Open-ILS/src/perlmods/t/14-OpenILS-Utils.t to see an example; in the same commit that I added the test, I changed the order in which entityize() was called to resolve the corruption.
To run the Perl unit tests:
cd Open-ILS/src/perlmods
prove -l t
(or "prove -l t/<specific_file_of_tests>").
This may warrant backporting the same fix back to rel_2_1 and rel_2_0 if testing Vandelay directly bears out the same results.
Also note that I'm getting the warning: "UNIVERSAL->import is deprecated and will be removed in a future perl" (for the use of "isa" as a function). This was introduced in OpenILS/Utils/Normalize by your changes, but it's a problem throughout our code base and deserves a bug of its own to have someone address it by the use of Scalar::Util instead.
I signed off on the commit but had to resolve the conflict again, and pushed it to user/dbs/ LP888572_ clean_marc_ fix_corruption - along with another commit, below, to add some unit tests and resolve a potentially severe problem:
*IMPORTANT* In the course of adding some unit tests to ensure that we were generating what we expected, I found that clean_marc() (and very likely the functionality as it exists within Vandelay) introduces subtle but significant corruption to upper case diacritic characters. Check Open-ILS/ src/perlmods/ t/14-OpenILS- Utils.t to see an example; in the same commit that I added the test, I changed the order in which entityize() was called to resolve the corruption.
To run the Perl unit tests: src/perlmods
cd Open-ILS/
prove -l t
(or "prove -l t/<specific_ file_of_ tests>" ).
This may warrant backporting the same fix back to rel_2_1 and rel_2_0 if testing Vandelay directly bears out the same results.
Also note that I'm getting the warning: "UNIVERSAL->import is deprecated and will be removed in a future perl" (for the use of "isa" as a function). This was introduced in OpenILS/ Utils/Normalize by your changes, but it's a problem throughout our code base and deserves a bug of its own to have someone address it by the use of Scalar::Util instead.