Uploading new users via CSV gives generic error message

[Kristina Hoeppner]

When you attempt to upload a lot of users via CSV and the upload is not made, you only receive a generic error message that doesn't give you the reason why the upload failed. When a certain server limit is reached, in some cases 100 users, the upload doesn't proceed so that the process does not take up all the processing power. The admin should be made aware of that and the error message should be more specific.

[Richard Mansfield (richard-mansfield)]

The easy way to do this is with a config setting for the max number of csv lines. Trapping the out of memory error is super-hard I think, and probably not reliable.

[François Marier (fmarier)]

Also, there is no real reason why this has to take heaps of memory. We should ideally make this scale.

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/1043
Committed: http://gitorious.org/mahara/mahara/commit/7f231697437e70371b9ac2767eb23391612c9e2d
Submitter: Hugh Davenport (<email address hidden>)
Branch: master

commit 7f231697437e70371b9ac2767eb23391612c9e2d
Author: Richard Mansfield <email address hidden>
Date: Mon Feb 13 16:30:42 2012 +1300

    Add a maximum number of users for csv files (bug #823048)

    This allows $cfg->maxusercsvlines to be specified in config.php to
    stop admins and institutional admins from uploading large csv files.
    This is potentially useful on large multi-user sites.

    And until such time as we can reduce the amount of memory used by the
    create_user function, setting this option will provide the user with
    some feedback on csv upload failure (because we can't easily trap the
    php out-of-memory error).

    See also bug #548042.

    This commit also adds a call to set_time_limit() after processing each
    user, so that the script doesn't fail due to php's max. execution
    time.

    Change-Id: I662231288c7d0c6e70602c548123f12c63427a97
    Signed-off-by: Richard Mansfield <email address hidden>

[Patrick Pollet (pp-c)]

I also noticed the problem in my contrib trying to sync a Mahara institution users accounts with a LDAP server.
https://github.com/patrickpollet/mahara_ldap_sync

The call to create_user is indeed consuming a big chunk of memory that is not returned to the pool and the process dies with ~1500 new users when allocating 256Mb to PHP.

Something must be wrong somewhere in the user creation , that does indeed a lot of work and memory allocation

Cheers.