Comment 5 for bug 1352028

Revision history for this message
Mahara Bot (dev-mahara) wrote : A change has been merged

Reviewed: https://reviews.mahara.org/3624
Committed: http://gitorious.org/mahara/mahara/commit/55a8deb8cbd2e1455c7af3a35c50b7cad76d63df
Submitter: Robert Lyon (<email address hidden>)
Branch: master

commit 55a8deb8cbd2e1455c7af3a35c50b7cad76d63df
Author: Nigel Cunningham <email address hidden>
Date: Fri Sep 5 08:49:32 2014 +1000

(Bug1352028) Add a JSON progress bar for bulk operations.

This patch adds a JSON progress meter (I'll call it that to avoid confusion
with progress bars) to the bulk uploading of users, groups and group
memberships and the bulk export and import of users (LEAP), so the user can see
the progress of the operation and not just the submit button changed to
'Processing..' and whatever indication their browser gives while waiting for
content.

The bulk export and import are minor rewrites, replacing the old iframe based
progress bar and the associated multiple pages and additional template file in
the case of the bulk export, and the recursive redirect-to-self of the bulk
import.

To accomplish the display of the progress bar during the operation, we make the
PHP session be closed (read only) except when changes need to be made. This is
for the most part a straightforward change in session.php as it's the only
direct accessor. In other places, we replace direct accessing of the session
variable ($_SESSION) with use of the session class ($SESSION) so that it can
reopen the session, make the change and close the session again.

There is one more aspect to all of this: with previous behaviour, multiple
requests for the same session would queue, taking the session lock in turn.
After this patch is applied, they can proceed in parallel, allowing greater
throughput. There is no additional locking requirement because the issues are
the same as those already dealt with in allowing multiple PHP threads to
process requests from different sessions at the same time.

I have sought to make the progress meter nice and generic, so it can be used in
the other bulk imports and exports too.

Paradoxically, these changes don't just make the import seem to be faster, it
actually is.. at least in the case of users and groups.

Times for importing 1000 users, groups and memberships, averaged over 3 runs
each (Wall time, not CPU time - but the relationship is the same).

Without Progress With Progress
Users 166s 155s
Groups 85s 78s
Memberships 20s 19s

Change-Id: Iec15c57db32c77994edb80c71d65591de51a95e4
Signed-off-by: Nigel Cunningham <email address hidden>