Leap2a: import/leap/lib.php:1117 unserialize: Error at offset 65512 of 65535 bytes"

[Aaron Wells]

This bug was reported by Kwantlen Polytechnic.

The problem they were experiencing was that some leap2a files error out when they're attempted to be imported. With error-reporting turned on, the same message is always present:

[WAR] f7 (import/leap/lib.php:1117) unserialize(): Error at offset 65512 of 65535 bytes
Call stack (most recent first):
log_message("unserialize(): Error at offset 65512 of 65535 byte...", 8, true, true, "/var/www/mahara-15.04.1/import/leap/lib.php", 1117) at /var/www/mahara-15.04.1/lib/errors.php:441
error(8, "unserialize(): Error at offset 65512 of 65535 byte...", "/var/www/mahara-15.04.1/import/leap/lib.php", 1117, array(size 5)) at Unknown:0
unserialize("a:11:{s:5:"title";s:40:"Occupational health and sa...") at /var/www/mahara-15.04.1/import/leap/lib.php:1117
PluginImportLeap->render_import_entry_requests() at /var/www/mahara-15.04.1/import/leap/lib.php:269
PluginImportLeap->build_import_entry_requests_form() at /var/www/mahara-15.04.1/import/index.php:225
print_import_items_form() at /var/www/mahara-15.04.1/import/index.php:73

[Aaron Wells (u-aaronw)]

This problem only occurs if:

1. You are using MySQL

2. You are doing self-import (interactive import), not the admin's "import into a new account" option.

3. Your export includes a page which is more than 65 kilobytes in size, when serialized. (not including file attachments)

The problem occurs because one of the steps during the interactive import process is to serialize the page, and store the serialized page in the database. The serialized page goes in the "import_entry_requests.entrycontent" table in the database. This table is the "text" data type, but because we didn't specify a size, in MySQL it defaults to being a standard TEXT field, which has a limit of 65,536 bytes. Attempts to insert values longer than that, are simply truncated to 65,536 bytes.

So what happens is, the page is serialized into a big long string greater than 65,536 bytes in length. It's inserted into the import_entry_requests.entrycontent column. Then once you make all your import decisions and submit that, we try to pull the page back out of import_entry_requests.entrycontent and unserialize it. We retrieve the truncated value from the database, try to unserialize it, and the call to unserialize() fails because the truncated string lacks a closing quotation mark and bracket.

The fix here is to expand the size of the import_entry_requests column to a LONGTEXT, which can hold up to 4GB of text. That should be enough.

[Aaron Wells (u-aaronw)]

This problem occurs in every version of Mahara since we introduced interactive import (1.8.0). But because the fix requires a database update, and it is not a high-priority bug, we will not be backporting it.

[Mahara Bot (dev-mahara)]

Patch for "master" branch: https://reviews.mahara.org/4963

[Aaron Wells (u-aaronw)]

To test:

1. Install Mahara using MySQL.

2. Create a page that has text content greater than 65 KB. (There is a limit of 65KB of text per text field in Mahara. So the easiest way to do this, is to put two Text blocks on the page, and put 50 KB into each one.)

3. Export the page to leap2a.

4. Try to import the leap2a file back into your portfolio.

Expected result: You should be able to import it

Actual result: The import process will error out once you get past the interactive step.

[Aaron Wells (u-aaronw)]

I'm attaching a Leap2a file I generated, which will trigger this problem. (It contains the first 150,000 characters of Moby Dick by Herman Melville.)

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/4963
Committed: https://git.nzoss.org.nz/mahara/mahara/commit/2b5b8da70b8ba5f257123de26828fe314b800efe
Submitter: Robert Lyon (<email address hidden>)
Branch: master

commit 2b5b8da70b8ba5f257123de26828fe314b800efe
Author: Aaron Wells <email address hidden>
Date: Wed Jul 15 17:41:01 2015 +1200

Increase the size of the Leap2a entrycontent column (Bug 1474659)

In MySQL, the default "text" column size is 65kB. Because this column
can hold the serialized text from several other columns, it needs to
be larger than that. Setting it to "big" generates a "longtext", which
is 4GB.

This change has no effect in Postgres. Postgres has only one text
size (unlimited). The XMLDB generator is smart enough that it
recognizes this and doesn't run any SQL in this case.

Change-Id: Ic26f0eb3d3c3181fdcaf8729373ca87b6ecc9c8d

[Aaron Wells (u-aaronw)]

Thanks to Kwantlen Polytechnic University for reporting this bug & sponsoring this bug fix!

[Aaron Wells (u-aaronw)]

Corrected the SQL query in my previous comment (now hidden):

To anyone out there who is running a Mahara 15.04 or earlier site, and can't yet upgrade to 15.10, you may find it tricky to cherry-pick that patch, because it increments the core version.php database version.

The easiest way to fix an existing 15.04 or earlier site, is to alter the table definition directly, with this query:

 ALTER TABLE import_entry_requests MODIFY COLUMN entrycontent LONGTEXT;