marc_export -i gives incorrect record length in the leader when call numbers include UTF8 characters

[Jane Sandberg]

Some of our call numbers include UTF8 characters. When I attempt to export those records using marc_export, I get a totally correct MARC file, with the correct record length in the LDR field. However, when I add the -i flag (to include items and volumes), it returns a MARC record with an invalid length in the LDR field. I have attached an example of an invalid record generated with the -i flag.

I am using EG 2.9.1.

[Jane Sandberg (sandbergja)]

Here is the original record as exported without the -i flag.

[Mike Rylander (mrylander)]

Jane,

The length value in bad_record.mrc is 02736, which matches the output of `wc -c bad_record.mrc`. The LoC documentation[1] specifies that lengths are counted in octets, not characters, so I don't think there's a bug here. Am I misunderstanding what you mean or is there some 3rd party tool that's having problems with the record?

[1] https://www.loc.gov/marc/specifications/specrecstruc.html#define ... specifically "length" and "logical record length".

[Jane Sandberg (sandbergja)]

Sorry, looks like I accidentally corrected bad_record.mrc! Here's the original, which reports 02734 as the length. Here is the real bad_record.mrc.

[Jane Sandberg (sandbergja)]

And the incorrect length is reported when I run the record through MarcEdit or PyMARC.

[Dan Scott (denials)]

I'm seeing this problem on 2.10.10 with copy locations that contain UTF8 characters. I think it's because when we generate the 852 fields in marc_export we need to utf8-encode the values of the individual subfields that we've received from the database.

[Dan Scott (denials)]

Quick fix branch at http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dbs/lp1584891_marc_export_unicode_items resolved the problems for us.

[Chris Sharp (chrissharp123)]

I can confirm that it works using Jane's sample record with a unicode character added to the call number. Signoff here:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/csharp/lp1584891_marc_export_unicode_items

[Dan Scott (denials)]

Thanks Chris! I've pushed the fixes to the master, 2.11, and 2.10 branches.

[Jane Sandberg (sandbergja)]

Thanks so much, Dan and Chris!