Some cron jobs cause elasticsearch queue to populate when it doesn't need to

Bug #1731067 reported by Robert Lyon on 2017-11-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mahara
High
Cecilia Vela Gurovic

Bug Description

We only record certain information in elasticsearch but we queue things even if unrecorded information changes because we use triggers/procedures on the tables

Example, when the recalculate_quota cron runs it works out what the user quota should be and then updates all the users in usr table that need fixing. This table has a trigger on it so all users needing updating are added to queue to be indexed.

On a big site that can mean thousands of users being indexed. But we don't record quota in elasticsearch so we don't need to index this change.

It would be good to be able to drop/add elasticsearch triggers during cron process for certain cron jobs like we do in upgrade.php script

eg:
  drop_elasticsearch_triggers();
  ... do cron job ...
  create_elasticsearch_triggers();

Robert Lyon (robertl-9) on 2018-03-06
Changed in mahara:
assignee: nobody → Cecilia Vela Gurovic (ceciliavg)

These cron jobs update tables with ES triggers, but don't need to trigger them:

core cron jobs
recalculate_quota - updates tables 'usr' and 'group'
cron_site_data_daily - updates 'site_data', 'view_visit', 'view'
user_login_tries_to_zero - updates 'usr'

interaction plugin cron
 interaction_forum_new_post - 'interaction_forum_post' sets sent=1

Changed in mahara:
status: Confirmed → In Progress

Reviewed: https://reviews.mahara.org/8591
Committed: https://git.mahara.org/mahara/mahara/commit/358070bdf49a73472064a179a68f7b1c004eac1b
Submitter: Robert Lyon (<email address hidden>)
Branch: master

commit 358070bdf49a73472064a179a68f7b1c004eac1b
Author: Cecilia Vela Gurovic <email address hidden>
Date: Thu Mar 8 09:14:09 2018 +1300

Bug 1731067: drop elasticsearch triggers for some cron jobs

behatnotneeded

Change-Id: I63b953b3a31f38c96f66d602e67622b62cdb0b01

Robert Lyon (robertl-9) on 2018-03-08
Changed in mahara:
status: In Progress → Fix Committed
Robert Lyon (robertl-9) on 2018-04-05
Changed in mahara:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers