When there are lots of subscribers to a forum or topic, some posts are
not being sent out by email because the cron script runs out of
memory. This has become a problem on mahara.org where most users on
the site are subscribed to the news forum.
The cron script sends email out to some subscribers before failing.
Something in dml, adodb, or the postgres drivers eats memory on every
query, so scripts that perform a lot of queries will hit the php
memory limit eventually, even when the list of subscribers to be
notified fits in memory okay.
As a stopgap I'm going to make the following changes to reduce peak
memory usage a bit:
- When retrieving the list of subscribers, don't pre-generate email
content for the whole subscriber list; instead leave generation until
the point at which a single user is notified.
- Remove each user object from the list as it's notified.
- Make email_user skip the mail disabled query whenever the
maildisabled property is set in the user object. In the
activity_get_users query, pull the maildisabled preference from every
user at the same time we get the list of users.
- Remove an unused artefact_internal_profile_email query from
email_user.
- Only do the mailssent update in email_user when bounces_handle is
configured.
Even if we do all that, and fix the memory leak, we'll soon get to the
stage where the subscriber list won't fit in memory, so we'll need to
change the way it all works, possibly pull subscribers out in chunks
that fit in memory, and push individual notifications for each of them
back on to the activity queue to be handled by a later cron run.
Patch to make maildisabled & mailssent queries optional in email_user