Acquisitions: Order record loads fail when there is a null value in a holdings subfield

[Kathy Lussier]

When loading order records, if one of the subfields has a null value, the order fails to import.

In our system, we have a provider configured with a 962 holdings tag and the following holdings subfields:
call_number: i
circ_modifier: g
copy_location: t
estimated_price: r
fund_code: u
note: p
owning_lib: b
quanitity: o

I have no trouble importing a record with the 962 field as follows:
=962 \\$gVideo$tJuvenile$pThis is a note$r19.99$uOR_FIC$bORANGE-NO$o1

However, if there is no note for this particular item, the vendor sends the record with a null subfield p, as follows:
=962 \\$gVideo$tJuvenile$p$r19.99$uOR_FIC$bORANGE-NO$o1

When I try uploading this record, I receive a "ACQ_IMPORT_ERROR" message. The odd thing is that when I look in the logs, I've found that there are problems with several subfields, not just subfield p:

2012-02-01 11:05:34 bd1-bh2 open-ils.acq: [ERR :2040:Order.pm:1371:13280967341859164] Item import extraction error: invlalid circ_modifier 1
2012-02-01 11:05:34 bd1-bh2 open-ils.acq: [ERR :2040:Order.pm:1372:13280967341859164] Holdings Data: {"fund":1238,"quantity":"1","copy_location":"Video","note":"Juvenile","estimated_price":"19.99","owning_lib":251,"circ_modifier":"1","fund_code":"OR_FIC"}

As you can see, the circ modifier value was thrown into the copy location field, the copy location value was thrown into the note field, and the quantity was thrown into the circ modifier field.

[Jason Boyer (jboyer)]

Some additional detail from the 2015 hack-a-way:

The issue is currently because of the way that acq.extract_holdings_attr_table pulls the holding data. It uses XPath to do two things: 1, get a count of holdings tags, and 2, select the code and value via the position() function. This allows the values to be shifted by the number of codes without values, causing values to be misattributed to codes. The fast fix is tell vendors to stop that, but a proper fix would require more complex XPath or rewriting the function in plperl using the MARC::Record module and friends.

[Jennifer Pringle (jpringle-u)]

Confirmed this is still an issue in 3.1

[Tiffany Little (tslittle)]

Still an issue in 3.8.

[Alto Tops (pnwfiddler)]

Look into subfields K and Q. Using a Snapcore app util could help organize things, especially if it came from a multi-core amd64 device. Good luck.