Custom layouts aren't displayed properly and can't be created with more than 4 rows

[Angela Karl]

I just tested the Mahara 1.8rc1, on a linux server with a MySQL DB. To view the site I am using a Mac OS Laptop with Chrome Version 30.0.1599.69.

When using the "create custom layout" feature I have multiple problems:

- The preview only is displayed correctly for 1 - 3 rows, after adding the fourth row the preview doesn't update anymore
- When adding a custom layout with up to 3 rows, the layout gets added to the other layout options. Adding a 4 row layout does work, but there is just a default thumb, not a picture of the layout.
- Even though I can add up to 6 rows the adding of custom layouts with 5 or 6 rows doesn't work at all.

I attached a screenshot of the problems.

Regards,
Angela

[Kristina Hoeppner (kris-hoeppner)]

Hello Angela,

I can't replicate this issue. I can create 4 rows, 5 and 6 rows just fine. And see previews as well.

I think it's a limitation that you can only add up to 6 rows which makes sense to me because you can place multiple blocks in one row after one another. But it's good testing to try those boundaries. :-)

[Kristina Hoeppner (kris-hoeppner)]

And that's on Mac 10.8 in Chrome.

[Szilvia Toth (szilvia)]

Retested on Mac in Chrome.

I added 6 rows, each with 3 columns. Each time a new row was added, the layout preview was displayed correctly.

However, I got an error message when I saved the page.

Please see the screen shot.

[Angela Karl (x-launchpad-u)]

I never got any error messages, I just couldn't create the custom layout. When I clicked the button, nothing happened.

[Aaron Wells (u-aaronw)]

I've spun off a separate bug to address the "invalidnumrows" error: https://bugs.launchpad.net/mahara/+bug/1237721

[Aaron Wells (u-aaronw)]

Switching status back to "Incomplete" because the original bugs that Angela reported, can't be replicated.

Angela, are you still getting this same behavior?

[Angela Karl (x-launchpad-u)]

I still got the same problems.

The next thing I just tried, was dropping the db and installing the 1.8rc2dev from gitorious. I still got the same problems.
Preview doesn't work for more than 4 rows, can't add layout with 5 or 6 rows, thumb for 4 rows isn't displayed correctly.

I made a short screencast of the problem, I hope it helps: http://angela-karl.de/customlayouts/

I used the newest chrome for the screencast, but have the same problem in Firefox 12.0 and Safari 6.0.5.

Thanks for your help.

[Angela Karl (x-launchpad-u)]

I attached a txt with contents of some db tables. Are these are the correct ones?

Hope that helps.

[Aaron Wells (u-aaronw)]

Hi Angela,

Due to an ongoing technical issue, the copy of the code on gitorious may be out of date. To make sure you've got the latest version, I recommend downloading your code directly from our gerrit review system, which you can do like so:

   git clone https://reviews.mahara.org/mahara

Or to switch your current workspace from using the gitorious repo to the gerrit repo:

   git remote set-url origin https://reviews.mahara.org/mahara

Thanks for the screencasts, that should help us figure out what's going on.

Cheers,
Aaron

[Aaron Wells (u-aaronw)]

Hi again Angela,

I took a look at the screencast, and it is indeed just as you described:

 - Clicking "add row" to add a fourth, fifth, or sixth row doesn't update the preview image
 - The "create layout" button is non-responsive when there are 5 or 6 rows
 - When there are four rows, the "Create layout" button seems to create a layout, but it is missing its preview thumbnail.

I was, however, still unable to replicate this locally. It works correctly in my Firefox and Chromium browsers on Ubuntu 12.04 -- I can create layouts of up to 6 rows, with the correct thumbnails. I also tried changing my local site from HTTP to HTTPS to see if that might be the problem, but it still worked, in both browsers

Here are a few more pieces of information that might help us track this down:

1. Does it behave the same if you use a different browser?

2. Do you see any error messages in your PHP error logs when this happens? (Since you're running an Ubuntu server, by default PHP errors will be printed into the Apache error logs at /var/log/apache2/error.log). You could also turn on "$cfg->productionmode=false;" in your config.php file to display warnings and errors in the browser, in which case the warning messages from this would probably show up on the next page after you navigated away.

3. If you don't see any PHP warnings, do you see any Javascript errors? (You may need to activate the Chrome dev tools to see Javascript errors)

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

1. I already tested Firefox 12.0 and Safari 6.0.5. and it shows the exact same behavior.

2. I will test that and report what I found.

3. When I use the Chrome dev tools, no Javascript errors are displayed in the lower right corner, but errors show up in the console. I attached two screenshots of the error messages.

[Angela Karl (x-launchpad-u)]

I forgot to mention that the 'cannot read property 'data'' error occurs whenever I add or delete the 5th or 6th row, the 'cannot read property 'layoutid'' when I click the add layout button.

[Son Nguyen (ngson2000)]

I can not replicate the error you mentioned.

As Aaron recommeded, do you see any error messages in PHP and Apache log files?

[Aaron Wells (u-aaronw)]

Hi Angela,

Thanks for the update. :) Can you elaborate on the "Cannot read property 'data'" and "Cannot read property 'layoutid'" errors? Are these PHP or Javascript errors?

If it's a PHP error, can you please paste the full error stack in here? That should let us know which line of the code it's failing on.

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

It is the error that shows up in the chrome dev tools console. You can see it in the screenshots I attached to my last post.

[Angela Karl (x-launchpad-u)]

I can't access the apache2 folder on the server (rights). I turned the productionmode=false on, and errors are displayed. (see screenshot) It seems to me those errors are displayed because of the missing thumbs. It happens as soon as I open the 'advanced options'.
When I click on 'Create new layout' no new errors are displayed.

[Aaron Wells (u-aaronw)]

Hi Angela,

Thanks for pointing out those screenshots. From those Javascript errors, it looks like the JSON requests to blocks.json.php are not returning the data that the rest of the app expects to be there. I am kind of surprised that you're not seeing PHP error messages generate by blocks.json.php, but perhaps it's doing something that is legal and not error-causing, but just doesn't return the necessary data...

What's the update history of the site you're testing on? Was it a new installation of 1.8rc1, or was it upgraded from a previous Mahara version?

Cheers,
Aaron

[Kristina Hoeppner (kris-hoeppner)]

Aaron, could it be a problem that we were having earlier with a version upgraded just before RC1 (or between RC1 and 2) when a change was made to the layouts but the DB upgrade was not bumped? Bob had to add a table manually into my DB to circumvent the problem as I could not do a fresh install.

Angela: Do you see the same problem when you install RC2 and a fresh database?

[Angela Karl (x-launchpad-u)]

Hello Kristina & Aaron,

both times I made a fresh installation with a new db. First I tried rc1 then rc2.

Regards,
Angela

[Aaron Wells (u-aaronw)]

Hi Angela,

I still can't replicate the issue. I did a clean install of 1.8rc2 just now, and was able to create a custom layout with up to 6 rows, with the thumbnail generated correctly for each one. I've also looked at the database data that you attached a few comments back, and compared that with what is in my database, and they look the same in all the important respects.

Since this issue occurs in multiple different web browsers, it is likely to be caused by something going on in your web server. Specifically, as I mentioned above, it appears that the problem is blocks.json.php not returning data in the format that the Javascript would expect it to.

Here are the next steps I can think of to try to figure out what's going on here:

1. Could you post the contents of your config.php file here? (Removing any sensitive data, of course, like database passwords and password salts). Perhaps this is a problem that only occurs when certain configuration flags are turned on.

2. Have you used a packet sniffer, or HTTP interceptor, such as Wireshark or Burp before? A program such as this would allow us to see exactly what data is returned by blocks.json.php are, which may help.

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

Here is the config.php, I never used one of those programs, but I will try it.

<?php

/**
 * Mahara: Electronic portfolio, weblog, resume builder and social networking
 * Copyright (C) 2006-2009 Catalyst IT Ltd and others; see:
 * http://wiki.mahara.org/Contributors
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program. If not, see <http://www.gnu.org/licenses/>.
 *
 * @package mahara
 * @subpackage core
 * @author Catalyst IT Ltd
 * @license http://www.gnu.org/copyleft/gpl.html GNU GPL
 * @copyright (C) 2006-2009 Catalyst IT Ltd http://catalyst.net.nz
 *
 */

/**
 * MAHARA CONFIGURATION FILE
 *
 * INSTRUCTIONS:
 * 1. Copy this file from config-dist.php to config.php
 * 2. Change the values in it to suit your environment.
 *
 * Information about this file is available on the Mahara wiki:
 * http://wiki.mahara.org/System_Administrator's_Guide/Installing_Mahara#Create_Mahara's_config.php
 *
 * This file includes only the most commonly used Mahara configuration directives. For more options
 * that can be placed in this file, see the Mahara lib file
 * htdocs/lib/config-defaults.php
 */

$cfg = new stdClass();

/**
 * database connection details
 * valid values for dbtype are 'postgres' and 'mysql'
 */
$cfg->dbtype = 'mysql5';
$cfg->dbhost = 'localhost';
$cfg->dbport = null;
$cfg->dbname = 'XXX';
$cfg->dbuser = 'XXX';
$cfg->dbpass = 'XXX';

/**
 * Note: database prefix is NOT required, you don't need to set one except if
 * you're installing Mahara into a database being shared with other
 * applications (this happens most often on shared hosting)
 */
$cfg->dbprefix = '';

/**
 * wwwroot - the web-visible path to your Mahara installation
 * Normally, this is automatically detected - if it doesn't work for you
 * then try specifying it here.
 * This value must end with a /
 *
 * Example: $cfg->wwwroot = 'http://myhost.com/mahara/';
 *
 * If you want to serve all of your Mahara content via HTTPS, just set
 * $cfg->wwwroot to use HTTPS.
 */
// $cfg->wwwroot = 'https://myhost.com/mahara/';

/**
 * If you are using a proxy to force HTTPS connections, you will need to
 * enable the next line. If you have set this to true, ensure your wwwroot
 * is a HTTPS address.
 */
// $cfg->sslproxy = true;

/**
 * dataroot - uploaded files are stored here
 * This is a ABSOLUTE FILESYSTEM PATH. This is NOT a URL.
 * For example, valid paths are:
 * * /home/user/maharadata
 * * /var/lib/mahara
 * * c:\maharadata
 * INVALID paths:
 * * http://yoursite/files
 * * ~/files
 * * ../data
 *
 * This path must be writable by the webserver and outside document root (the
 * p...

[Angela Karl (x-launchpad-u)]

I just tried wireshark but I can't really get much information out of it. I think I'll have to read a tutorial on how it works.
Any suggestions what I should look at specifically?

[Aaron Wells (u-aaronw)]

Hi Angela,

Hm, your config.php file looks perfectly average, no unusual settings in place.

It has occurred to me, rather than using Wireshark, it'd be a lot easier to just use the Chrome Developer Tools, since you're using Chrome anyway. Here is how to activate them: https://developers.google.com/chrome-developer-tools/

Once the Chrome Dev Tools are on, go to the Dev Tools "Network" tab to see the individual HTTP requests and responses. Then replicate the layout error in your web browser. Towards the bottom of the request list you should then see a request for "blocks.json.php" followed by "thumb.php?type=...". Click on the blocks.json.php request, and then click the "Response" subtab, and you should see the raw response returned by Mahara. On success, it'll be a string of json like this:

{"error":false,"message":{"message":"","data":{"data":"http:\/\/vegas.wgtn.cat-it.co.nz\/mahara-clients\/htdocs\/thumb.php?type=customviewlayout&cvl=vl-33-33-33_33-33-33_33-33-33","alttext":"33-33-33 \/ 33-33-33 \/ 33-33-33","newimage":1}},"returnCode":0}

Yours, since it's not working, will hopefully give an informative error message instead.

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

I attached some screenshots.

When I try to add a new layout with a fourth row, the request gets canceled and it says "no response data available" (see screenshot).

Adding a fifth and sixth row and when I click on "create new layout" it says the status of the request is ok and the response is: {"error":false,"message":null,"returnCode":0}

[Aaron Wells (u-aaronw)]

Hi Angela,

Actually all I see in your screenshots is the thumb.php requests. What I'm most interested in is the blocks.json.php requests. (see my screenshot).

However, that JSON string you've described, well, that is the heart of the problem. Rather than "message":null, it should be giving us a "message" with sufficient data to set up the next couple of rows.

Unfortunately, there's no error message!

Hm... I'm running low on things that I can troubleshoot from a distance. :) There are only two more things I can think of right now:

1. In Chrome, open up the Javascript Console (it's in the Tools menu) and replicate the error, to see whether there are any Javascript errors

2. See whether we're sending a properly formed request to blocks.json.php . Again on the Developer Tools "Network" tab, you should see a request to blocks.json.php when you click the "Add row" button. Click on that, go to the "Headers" subtab, and then look for the "Form Data" section. You should see something like this:

id:7
change:1
action_updatecustomlayoutpreview_numrows_3_row1_5_row2_5_row3_5:1

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

There are neither javascript errors nor warnings.

I attached the screenshot of the blocks.json.php request when adding a fifth row and when clicking on 'create new layout';

Since you mentioned the troubleshooting from a distance, I used the release of Mahara 1.8 and freshly installed it on:

https://wwwdid.mathematik.tu-darmstadt.de/mahara_demo/

The errors are the same on this installation. I made a test account, perhaps you may take a look at it:

testuser
test1234

Regards,
Angela

[Aaron Wells (u-aaronw)]

Hi Angela,

Thanks for your perserverance with this issue!

I logged in with the test account and I was indeed able to replicate the issue. In fact, I was able to replicate it on the third row by entering different combinations of rows. For instance, trying to add three rows with 5 columns each, will cause this error to manifest on the third row.

I don't have much more insight into this now, though. From looking at the PHP code that gets called (view/blocks.json.php to lib/view.php::View->process_changes() to lib/view.php::View->updatecustomlayoutpreview() ) I can't even make out any path through the code that would result in the JSON reply that I'm seeing. The "error" field is "false", which limits us on blocks.json.php to the call to json_reply() on line 31 or line 42. It can't be the one on line 42 because there the "message" field is an array, whereas we're seeing a message field that's null. So that leaves the one line 31, where "message" is set to the return value of View->process_changes(). But I can't see any way *that* return value can be null, unless there's something strange going on with your $_POST variable.

Looking back through this thread, I see you described your server environment as "a linux server with a MySQL DB". Can you provide a few more specifics?

1. Which distro and version number of Linux are you using?
2. Which web server and version number are you using?
3. What's your MySQL version number?
4. What's your PHP version number?

At this point in the debugging process, the only thing I can think of to do is line-by-line debugging to see what's going on in the PHP side. On my local machine I'd do this using XDebug, but since yours is running on a remote server, perhaps we could replace some of the files used with ones that have extra calls to log_debug() in order to get more output?

Cheers,
Aaron

[Angela Karl (x-launchpad-u)]

I used php_info() to display the information (see pdf). Do you find everything you need in there?

[Angela Karl (x-launchpad-u)]

If you have files you want to put on the server you can give them to me.

[Son Nguyen (ngson2000)]

Hi Angela;

What did you see in the PHP and Apache error log?

[Son Nguyen (ngson2000)]

Hi Angela;

Can you put these code:

  log_debug('action:' . $action, false);
  log_debug('values:' . var_export($values, true), false);

at the line 1632 of the file lib/view.php

Replicate the error and show me the appache error log file

[Aaron Wells (u-aaronw)]

For the record (from the screenshot of phpinfo()) the vital statistics on the server where this error was encountered:

PHP 5.3.3
MySQL (unclear what version)
Apache 2.2.16
Debian squeeze

[Aaron Wells (u-aaronw)]

And for comparison, my system where it works correctly:

PHP 5.3.10
MySQL 5.5.34
Apache 2.2.22
Ubuntu 12.04 (precise)

I note that in debian squeeze the current version of MySQL is 5.1, so in theory it's possible this could be a MySQL 5.1 issue. But we have no specific evidence to suggest that the MySQL version is the cause.

[Nigel Cunningham (nigelc-g)]

I've managed to reproduce this with an install of Debian Squeeze

PHP 5.3.3-7+squeeze17
mysql 5.1.73
Nginx 0.7.67

I'll see if an upgrade of mysql makes it go away.

[Nigel Cunningham (nigelc-g)]

An update of MySQL to 5.5.31 (the dotdeb version) doesn't make the problem go away.

Will try updating PHP next.

[Nigel Cunningham (nigelc-g)]

Updating PHP to 5.4.30 (dotdeb again) fixes the issue.

If someone says it's worth the time, I'll put the effort into diagnosing the exact cause (but not before Monday).

[Nigel Cunningham (nigelc-g)]

5.3.28 - also dotdeb - also works.

[Nigel Cunningham (nigelc-g)]

On my test system, the issue is caused by the use of Suhosin. In the earlier versions mentioned above, the limit on the length of a POST variable name and request variable in general are both 64 characters.

When the number of rows or the length of the action variable exceeds 64 characters, it is discarded by PHP and the JSON request is unable to do anything.

The solution for my system was to set both

suhosin.request.max_varname_length = 128
suhosin.post.max_name_length = 128

in /etc/php5/conf.d/suhosin.ini.

In later versions of PHP, suhosin is uninstalled by default, causing the problem to go away.

[Nigel Cunningham (nigelc-g)]

I should mention that if variable name length is the issue, you'll also be able to reproduce the issue with 3 rows with a greater number of columns per row.

[Nigel Cunningham (nigelc-g)]

I've just come back to this and noticed Angela's PHP info PDF. That shows that Suhosin was indeed the cause - it has the settings I mentioned above with values of 64.

I'll therefore be bold enough to close this bug report as Invalid - it's not a bug in Mahara, but rather a PHP config issue.