Web Client: Transactions Do Not Always Close When Bill Fully Paid

[John Amundson]

Evergreen 3.0.8 - Web Client

We are noticing that sometimes when a bill is fully paid, (balance_owed=0), the transaction stays open and attached LOST items are not updated to Lost & Paid.

I am not sure if this also occurs on overdue fines.

This only happens occasionally, and I have not been able to reliably duplicate it. More details follow.

First a little background since the issue is related to billing and the LOST status, and I know there is variation in how this status is used. We use LOST to represent an item that was either a) marked "lost (by patron)", or b) has been overdue for 28 days or more. In both cases, when an item becomes lost, the item price is billed and a lost item block is placed on the patron record.

Normally when a lost item is paid, the following happens:
-The balance owed on the bill is reduced to 0, and the bill is moved to the History tab.
-The bill is updated with a transaction finish time of the last payment.
-The item moves to status Lost & Paid.
-The item no longer appears attached to the patron's account.
-The lost item block is lifted from the account.

However, on rare occasions in the web client, we are seeing something different. In these instances, when a lost item is paid, the following happens:
-The balance owed on the bill is reduced to 0, and the bill is moved to the History tab as expected.
-The bill is NOT updated with a transaction finish time, (the field is blank).
-The item continues to stay in status Lost.
-The item continues to appear in the patron's Items Out list.
-The lost item block stays on the patron's account.

The items do not seem share common histories, (i.e. one was marked lost manually, the others organically, one of the transactions was renewed 2 times, the others no renewals, one of the items was deleted, the others not deleted). The only commonality I've found is that in each report, the bill was part of a larger group of bills paid. In the most recent report, 5+ billed items were paid for with the same payment; all but one updated correctly. However, I have tested this on my side with a variety of cases, (all lost, mixed billing types, etc), and every single time the payment/bill is processed correctly, so I am not sure if the batch payment is just correlation or causation.

[Dawn Dale (ddale)]

I can confirm this issue though we cannot reproduce it on command. I am unsure of the trigger for the bug but occasionally we have a bill that does not close when payment is made in full. The item will remain on the patron account as lost. I add $.01 and pay $.01 to the bill to force it to close.

[John Amundson (jamundson)]

I want to update this bug and say that I've noticed that this isn't just limited to LOST bills, but to others, as well, (i.e. overdue fines). I'm editing the title to reflect this. We still haven't figured out why, but one thing seemingly remains constant. The transaction that fails to close is part of a single payment that applied to multiple transactions

[Lindsay Stratton (lstratton)]

I just encountered the same behavior described by John (2019-03-18) where 1 Lost item payment out of 3 paid in the same transaction did not close appropriately, while the others did.

Can also confirm Dawn's (2018-07-11) strategy of adding a bill to force the transaction to close.

[Jason Stephenson (jstephenson)]

I just saw it with 8 overdue transactions. Looking at the data, it was the last circulation paid that did not close.

[Michele Morgan (mmorgan)]

There appears to be a floating point number problem that is at least partially responsible for this bug.

I found activity.log entries for applied payments that look like this:

2019-08-12 18:16:10 brick3 osrf_websocket_translator: [ACT:142032:./osrf_websocket_translator.c:785:15656423361420321156] [10.108.10.77] [] open-ils.circ open-ils.circ.money.payment "60fc9b26b9fdc7c0c0ccdcda0eb660fe", {"userid":"1983487","note":"","payment_type":"forgive_payment","check_number":null,"payments":[[15447046,1.3],[4909522,1],[5048999,0.1],[4909625,0.55],[4909684,0.2],[4909685,0.1999999999999998]],"patron_credit":0,"cc_args":null}, "1473201970.2223321125.3967675516"

Bill 4909685 was the one that did not close, the balance owed was $.20.

The bug can be reproduced using grocery bills as follows.

Retrieve a patron that owes no money.

Add a billing of $.90
Add a billing of $.10
Select both billings
Enter $1 in payment received and Apply the payment

Click on History in the billings screen. Turn on the Total Billed and Transaction Finish Time column and note that the $.10 bill has zero balance owed, but does not have a transaction finish time.

Patch is forthcoming.

[Jason Stephenson (jstephenson)]

Just for the sake of confirmation, I see something similar on my test server, for a set of transactions where 1 did not close.

Dec 17 14:08:41 jasontest gateway: [ACT:8848:osrf-websocket-stdio.c:559:1576609687884811] [127.0.0.1] [] open-ils.circ open-ils.circ.money.payment "dd5dca573955c2122984690a3df04e90", {"userid":"681166","note":"","payment_type":"cash_payment","check_number":null,"payments":[[91988672,0.1],[91988674,0.1],[91989090,0.1],[91989092,0.1],[91989095,0.1],[91989098,0.1],[91989257,1],[91989259,0.9999999999999996]],"patron_credit":0,"cc_args":null}, "1575935128.494926180.04070332971"

The transaction 91989259 is the one that did not close.

[Michele Morgan (mmorgan)]

Branch as promised is at:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/mmorgan/lp1781274_limit_pending_payments_to_2_decimal_places

To test:

Use the steps above to reproduce, noting the lack of Transaction Finish Time.
Apply the patch.
Use the same steps above and note that the post patch payments resulted in both transactions getting a Transaction Finish Time.

[Terran McCanna (tmccanna)]

Works great! Thanks, Michelle!

My sign-off is at:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/mccanna/lp1781274_limit_pending_payments_to_2_decimal_places_signoff

[Bill Erickson (berick)]

Thanks, All.

I have pushed an alternate fix:

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp1781274-webstaff-billpay-rounding

This approach resolves the rounding issue with the standard floating point / money math way of multiplying / dividing by 100, consistent with other math in the bills.js file (and elsewhere).

[Michele Morgan (mmorgan)]

Thanks for looking at this, Bill. Unfortunately, this isn't working for me in all cases. As an example, paying two billings in the amounts of $1.30 and $.75 results in the transaction with $.75 owed not closing. Perhaps there's another calculation in that file that needs the multiply/divide by 100 approach, but it's not jumping out at me. Here is the gateway.log entry for the payment detail

gateway 2020-01-31 16:52:56 [INFO:38292:osrf-websocket-stdio.c:559:15805075323829220] [172.29.120.14] [] open-ils.circ open-ils.circ.money.payment "7f24b7501200b900065bed7701bce5fa", {"userid":"2561821","note":"","payment_type":"cash_payment","check_number":null,"payments":[[24453683,1.3],[24453684,0.7499999999999997]],"patron_credit":0,"cc_args":null}, "1580491315.450369058.3374452733"

Leaving the pullrequest on as the initial branch in comment #7 does work for this example.

[Bill Erickson (berick)]

Thanks for the example, Michele. I'll take another look. I think you're right that I'm missing a calculation.

[Bill Erickson (berick)]

Well that's fun, it looks like ye olde *100 trick does not always work in JS:

> 2.05 * 100
204.99999999999997

I'll resist the urge to start using *1000 and retract my patch.

I see no reason why .toFixed(2) won't work for us here. Will revisit bug soon and merge, barring alternate feedback.

[Michele Morgan (mmorgan)]

Thanks for your work on this, Bill! Restoring the signedoff tag.

[Jason Stephenson (jstephenson)]

It's working for me, but I'm waiting on John Amundson to test it, too.

[John Amundson (jamundson)]

Using Michele's test, I was able to confirm the bug. After installing the patch, the same tests had appropriate finish dates.

I was just about to add my sign off when I performed another test that didn't close.

There were three bills on my test patron's record:
Lost charge balance: 7.84
Lost charge balance: 25.25
Overdue charge: 3.00

I submitted a payment for the full amount, 36.09. The two lost transactions closed. The overdue transaction stayed open.

I'll note that the transaction with a remaining balance of 7.84 had previous payments applied, so it's possible a previous rounding error was contributing.

I then added three grocery bills to the account for the same amounts, applied a payment, and all the transactions closed.

I'm thinking my issue was with the previous payments. This is an observation to be aware of post-patch. I think the patch itself works, though.

[Jason Stephenson (jstephenson)]

John, thanks for testing. Since I could not find any signs of rounding errors in the transaction logs for the payments that you made today, I have pushed this branch without your signoff.

Thanks everyone for testing, commenting, signing off, etc.!