Autorenewal Events Get Stuck in collected and reacting states

Bug #1935971 reported by Jason Stephenson
This bug affects 5 people
Affects Status Importance Assigned to Milestone

Bug Description

Evergreen 3.5.3 (and 3.2.10)
OpenSRF 3.2.1
PostgreSQL 9.6

We occasionally have autorenewal events get stuck in collected or reacting state. It seems intermittent and apparently random. I'm attaching results of a query to count the number of autorenewal events by date and there state where the state was not complete, error, or invalid for the past 6 months. You can see that it doesn't happen all that often.

I have encountered issues with Ejabberd max_stanza_size and these events in the past. Our max_stanza_size is set to 10MB on the utility server that runs the action trigger runner. (I think I've mentioned a/t in the chunking/bundling bug.)

Here is some IRC discussion about this issue that was the impetus for me opening this bug:

Revision history for this message
Jason Stephenson (jstephenson) wrote :
Revision history for this message
Ruth Frasur (rfrasur) wrote :

We see this issue in 3.4.4. Currently, we have a workaround to check for these failures and address. It would be nice to not have to have a work around.

Changed in evergreen:
status: New → Confirmed
Revision history for this message
Jason Stephenson (jstephenson) wrote :

An interesting thing happened this morning/last night. We had 33 autorenewal events get left in the reacting state, but the circulations were renewed, and the autorenewal email notices were generated, i.e. the number of complete email notice events matches the total of complete and reacting autorenewal events.

I used the following query to determine that the reacting events had actually renewed the ciruclations:

select circulation.*
from action.circulation
join action_trigger.event
on circulation.parent_circ =
and event_def = 124
and event.state = 'reacting'
and event.add_time::date = now()::date;

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers