Upgrading from 1.7 to 1.10 failed in big databases

Bug #1400524 reported by Son Nguyen on 2014-12-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mahara
High
Son Nguyen
1.10
High
Son Nguyen

Bug Description

Version: 1.10, and master(15.04)
Platform: apache, postgres

In big database, the upgrading has been failed when updating artefact table
htdocs/lib/db/upgrade.php: 3255-3284

This process should be done in chunks as the the table 'artefact' may be big which will causes exceed execution time.

See also: Bug #1298646

Son Nguyen (ngson2000) on 2014-12-08
description: updated

Reviewed: https://reviews.mahara.org/4069
Committed: http://gitorious.org/mahara/mahara/commit/3fc71a298c400a2e760255b7ac7b74f8e0042444
Submitter: Son Nguyen (<email address hidden>)
Branch: master

commit 3fc71a298c400a2e760255b7ac7b74f8e0042444
Author: Aaron Wells <email address hidden>
Date: Tue Dec 9 14:37:33 2014 +1300

Performance improvements for the code that fills in artefact.path

Bug 1400524: On a large site, the database queries to retrieve
all records from the artefact table can take too long to execute
and cause the PHP script to time out.

This optimization attempts to fix that by separately handling
the simpler case of parentless artefacts, and by removing
a redundant "select * from artefact"

Change-Id: Iddcfea3161fbfe804ec808119fe771757e7cdd5e

Reviewed: https://reviews.mahara.org/4070
Committed: http://gitorious.org/mahara/mahara/commit/2656482ac99a7e51ce4b2a5e4f23f6fb92425e36
Submitter: Son Nguyen (<email address hidden>)
Branch: 1.10_STABLE

commit 2656482ac99a7e51ce4b2a5e4f23f6fb92425e36
Author: Aaron Wells <email address hidden>
Date: Tue Dec 9 14:37:33 2014 +1300

Performance improvements for the code that fills in artefact.path

Bug 1400524: On a large site, the database queries to retrieve
all records from the artefact table can take too long to execute
and cause the PHP script to time out.

This optimization attempts to fix that by separately handling
the simpler case of parentless artefacts, and by removing
a redundant "select * from artefact"

Change-Id: Iddcfea3161fbfe804ec808119fe771757e7cdd5e

Aaron Wells (u-aaronw) wrote :

Note: Patch 4072 is a follow-up patch which is yet further optimized.

tags: added: performance
tags: added: upgrade
Changed in mahara:
status: In Progress → Fix Committed
status: Fix Committed → In Progress

Reviewed: https://reviews.mahara.org/4072
Committed: http://gitorious.org/mahara/mahara/commit/31a1931435e4cd1e9489c1814a42e9ae96c6490d
Submitter: Aaron Wells (<email address hidden>)
Branch: master

commit 31a1931435e4cd1e9489c1814a42e9ae96c6490d
Author: Aaron Wells <email address hidden>
Date: Tue Dec 9 14:37:33 2014 +1300

Even quicker way to fill in artefact paths

Bug 1400524: On a large site, the database queries to retrieve
all records from the artefact table can take too long to execute
and cause the PHP script to time out.

This optimization attempts to fix that by filling in the paths
with an iterative series of UPDATE queries instead.

Change-Id: I46c13cc97bac8a6d2bc626e3b2dc8e4cdeecd2bb

Aaron Wells (u-aaronw) on 2015-01-12
Changed in mahara:
status: In Progress → Fix Committed
Aaron Wells (u-aaronw) wrote :

Merged in patch https://reviews.mahara.org/#/c/4145/ for 1.10_STABLE, although the MaharaBot failed to post a comment about it for some reason.

Mahara Bot (dev-mahara) wrote :

Reviewed: https://reviews.mahara.org/4145
Committed: http://gitorious.org/mahara/mahara/commit/dc478caf2b219efd93b06a9bf0ed8c2d2fc40a92
Submitter: Son Nguyen (<email address hidden>)
Branch: 1.10_STABLE

commit dc478caf2b219efd93b06a9bf0ed8c2d2fc40a92
Author: Aaron Wells <email address hidden>
Date: Tue Dec 9 14:37:33 2014 +1300

Even quicker way to fill in artefact paths

Bug 1400524: On a large site, the database queries to retrieve
all records from the artefact table can take too long to execute
and cause the PHP script to time out.

This optimization attempts to fix that by filling in the paths
with an iterative series of UPDATE queries instead.

Change-Id: I46c13cc97bac8a6d2bc626e3b2dc8e4cdeecd2bb

Robert Lyon (robertl-9) on 2015-04-17
Changed in mahara:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers