Memory exhausted on cron import_process_queue

[David Ligne]

Hi,
We're on Mahara 1.10, running on Linux and MySQL.

We got this error :
[INF] c2 (lib/cron.php:30) ---------- cron running Wed, 09 Sep 2015 15:10:01 -0400 ---------- [INF] c2 (lib/cron.php:75) Running PluginSearchElasticsearch::cron [INF] c2 (lib/cron.php:137) Running core cron activity_process_queue [INF] c2 (lib/cron.php:137) Running core cron import_process_queue PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20777539 bytes) in /app/apache2/htdocs/mahara/lib/errors.php on line 148

We don't know exactly what happened, but a simple solution would be to add :
"raise_memory_limit('512M');"
in /import/lib.php, function import_process_queue().

[Mahara Bot (dev-mahara)]

Patch for "master" branch: https://reviews.mahara.org/5276

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/5276
Committed: https://git.nzoss.org.nz/mahara/mahara/commit/0d6147b0c9a831c311d36dda522b8b805a7c8c93
Submitter: Aaron Wells (<email address hidden>)
Branch: master

commit 0d6147b0c9a831c311d36dda522b8b805a7c8c93
Author: David Ligne <email address hidden>
Date: Fri Sep 11 17:15:00 2015 -0400

Bug 1494908 : Memory exhausted on cron import_process_queue
Raised memory limit.
behatnotneeded

Change-Id: Ie3d89c916323b2bbfa381fd059667e1fc9e2de94

[Mahara Bot (dev-mahara)]

Patch for "15.04_STABLE" branch: https://reviews.mahara.org/5319

[Aaron Wells (u-aaronw)]

Hi David,

Thanks for the patch! I've merged it into 15.10dev, and I'm backporting it to 15.04.

In the longer run t'd be preferable to see if we could figure out what's causing it to run out memory, and reduce its memory usage. If you're interested, one thing you could try is to replace the call to get_records_select_array() on line 310, with a call to get_recordset_select(). That would make it so that PHP only stores one import_queue record in memory at a time, instead of all of them. That may solve the problem, if the problem is that there are big records in the import_queue.data database field.

On the other hand, if the problem is that we're trying to process a single Leap2a record that is too large, then raising the memory limit is the only possible fix.

Cheers,
Aaron

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/5319
Committed: https://git.nzoss.org.nz/mahara/mahara/commit/2c53ec38c93fe7bbca36987ec73c3326f89a7784
Submitter: Aaron Wells (<email address hidden>)
Branch: 15.04_STABLE

commit 2c53ec38c93fe7bbca36987ec73c3326f89a7784
Author: David Ligne <email address hidden>
Date: Fri Sep 11 17:15:00 2015 -0400

Bug 1494908 : Memory exhausted on cron import_process_queue
Raised memory limit.
behatnotneeded

Change-Id: Ie3d89c916323b2bbfa381fd059667e1fc9e2de94
(cherry picked from commit 0d6147b0c9a831c311d36dda522b8b805a7c8c93)

[David Ligne (david-ligne)]

Hi Aaron,

I've tried to do tests with big files, but it produces other problems, so I couldn't get the "Memory exhausted" error.
Will keep on trying when I have some time.

Cheers

David