Can't upload 13 mb pdf

[Spencer]

It's just a collection of images. Not sure why it's not uploading. There's no wacky characters in the title.

[Spencer (spencer-fabricatorz)]

I've been sitting on 3 documents that I can't upload. I've tried everything. I'm assuming it has something to do with the images imbedded. Any suggestions? I can't make any progress without being able to upload.

[Brad Phillips (brad-bradphillips)]

Spencer, can you attach these files to this bug, so I can test the files and troubleshoot the upload?

[rejon (rejon)]

please attach...i think need to switch to that plupload...the one file approach is failing on the big files...brad, please check out.

[rejon (rejon)]

i forwarded you the 13mb file brad you asked for...

[Brad Phillips (brad-bradphillips)]

Ok, I have the file & have tested uploading it in various converted formats to no avail, so I can confirm there is something weird & I have a feeling it has to do with the complexity of said file (embedded images & such).

I'll try changing the upload method (to plupload as rejon suggested) to see if we can get this thing on the server & submissions pages.

[Brad Phillips (brad-bradphillips)]

So, I have been testing every possible conversion of this file & nothing successfully uploads. I switched the upload method to the multi-image format that is used on sites like ocal. The multi-upload says it successfully uploads a file & it displays, but no mime-type is stored in the db.

When I try downloading the uploaded file from the site, the file isn't the same size as the original file uploaded. Each time, it's reduced to 1mb & corrupt. I even saved each page of the .doc individually as a .pdf & tried uploading a page with only text & got the same result.

Next, I tried uploading a JPG image that was 2.9mb. It also returned successfully, but, upload navigating to the uploaded file, only 1mb of the image is uploaded. Literally, only the top portion of the image displayed on the web page.

All of this taken into account seems to indicate that the problem isn't with the file itself, but with uploading in general. Can anyone else confirm that nothing over 1mb successfully uploads. Bassel, do you have any ideas as to what might be going on? Is this an Aiki issue?

[Brad Phillips (brad-bradphillips)]

Hmm..so I switched back to single upload form & am successfully able to upload the doc as a pdf, one page at a time, except for the pages with multiple images. In those instances, only 1mb of each gets uploaded & they are corrupt.

Also, I can upload larger images using the single upload form.

I don't know why the multi-upload form has trouble in all of these cases...it seems that the single upload is less buggy to me.

Anyone have any suggestions? I'm not sure what this issue is.

[rejon (rejon)]

On Wed, Feb 9, 2011 at 6:22 AM, Brad Phillips <email address hidden> wrote:
> Hmm..so I switched back to single upload form & am successfully able to
> upload the doc as a pdf, one page at a time, except for the pages with
> multiple images.  In those instances, only 1mb of each gets uploaded &
> they are corrupt.

Bassel, why do you think this is? need support on this one.

>
> Also, I can upload larger images using the single upload form.
>
> I don't know why the multi-upload form has trouble in all of these
> cases...it seems that the single upload is less buggy to me.
>
> Anyone have any suggestions?  I'm not sure what this issue is.

bassel, need the big guns.

> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

[Bassel Safadi (bassel)]

on which browser is that? I replaced the upload.php with latest that has chunking and it should be working now.. could you please re-test on firefox

[rejon (rejon)]

wait wait, you just forked the current code of scale?

On Wed, Feb 9, 2011 at 12:23 PM, Bassel Safadi <email address hidden> wrote:
> on which browser is that? I replaced the upload.php with latest that has
> chunking and it should be working now.. could you please re-test on
> firefox
>
> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Bassel Safadi (bassel)]

so the multi upload javascript is by default HTML5 but it fail back to flash if not enough HTML5 support exists..
which browser are you using and which system. also can you please confirm which version of the uploader is being used on your machine, you can do that by right clicking on it and see if flash menu popup.
that will make it easier to figure out what's wrong there.

Thanks

[Brad Phillips (brad-bradphillips)]

I'm using Latest Firefox to test uploads, Bassel, and it's executing HTML5 version of the uploader. Hope this helps.

[rejon (rejon)]

bassel worked on it last night. any updates???

On Mon, Feb 14, 2011 at 11:45 PM, Brad Phillips
<email address hidden> wrote:
> I'm using Latest Firefox to test uploads, Bassel, and it's executing
> HTML5 version of the uploader.  Hope this helps.
>
> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Bassel Safadi (bassel)]

The system still can't create directories, still trying to find a work
arround and need to look at php.ini
On Feb 17, 2011 9:05 AM, "rejon" <email address hidden> wrote:
> bassel worked on it last night. any updates???
>
> On Mon, Feb 14, 2011 at 11:45 PM, Brad Phillips
> <email address hidden> wrote:
>> I'm using Latest Firefox to test uploads, Bassel, and it's executing
>> HTML5 version of the uploader. Hope this helps.
>>
>> --
>> You received this bug notification because you are a member of Scale
>> Journal Developers, which is subscribed to scalejournal.
>> https://bugs.launchpad.net/bugs/701171
>>
>> Title:
>> Can't upload 13 mb pdf
>>
>
>
> --
> Jon Phillips
> http://rejon.org/ | http://fabricatorz.com/
> chat/skype: kidproto | irc: rejon
> +1.415.830.3884 (global) | [phone] (sf)
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
> Can't upload 13 mb pdf
>
> Status in Scale Journal:
> Confirmed
>
> Bug description:
> It's just a collection of images. Not sure why it's not uploading.
> There's no wacky characters in the title.
>
>

[Bassel Safadi (bassel)]

update: there are two bugs here:

1- aiki bug: multi upload doesn't record mime types
2- other bug ( server configuration): php is not able to create directories

[rejon (rejon)]

Bassel, can you fix or provide more analysis on the problem? spencer
is going to kill us, and scale is not moving at all...have a call wed
night.

On Fri, Feb 18, 2011 at 2:23 PM, Bassel Safadi <email address hidden> wrote:
> update: there are two bugs here:
>
> 1- aiki bug: multi upload doesn't record mime types
> 2- other bug ( server configuration): php is not able to create directories
>
> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Bassel Safadi (bassel)]

rejon the upload is working but mime types are not calculated.

Brad: what is a work around not having mime types? I can create a process to create mime types after the upload. could you please explain how they are used in the page ?

[rejon (rejon)]

ok, so the file is there, but there must be a query that is used to
find mime-type causing problem...but the problem is the file is not
uploading totally...

what is the problem here? is this in other aiki installs?

jon

On Mon, Feb 21, 2011 at 1:01 PM, Bassel Safadi <email address hidden> wrote:
> rejon the upload is working but mime types are not calculated.
>
> Brad: what is a work around not having mime types? I can create a
> process to create mime types after the upload. could you please explain
> how they are used in the page ?
>
> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Bassel Safadi (bassel)]

No Rejon it's only on scale. I'll try to update the software again.. btw do you have ftp on that server? like pureftp or some other services

[rejon (rejon)]

NO FTP ;)

SFTP

On Mon, Feb 21, 2011 at 1:17 PM, Bassel Safadi <email address hidden> wrote:
> No Rejon it's only on scale. I'll try to update the software again.. btw
> do you have ftp on that server? like pureftp or some other services
>
> --
> You received this bug notification because you are a member of Scale
> Journal Developers, which is subscribed to scalejournal.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Bassel Safadi (bassel)]

can connect the uploader with ftp not sftp, need to try though

On Mon, Feb 21, 2011 at 9:40 PM, <email address hidden> <email address hidden> wrote:

> NO FTP ;)
>
> SFTP
>
> On Mon, Feb 21, 2011 at 1:17 PM, Bassel Safadi <email address hidden>
> wrote:
> > No Rejon it's only on scale. I'll try to update the software again.. btw
> > do you have ftp on that server? like pureftp or some other services
> >
> > --
> > You received this bug notification because you are a member of Scale
> > Journal Developers, which is subscribed to scalejournal.
> > https://bugs.launchpad.net/bugs/701171
> >
> > Title:
> > Can't upload 13 mb pdf
> >
>
>
>
> --
> Jon Phillips
> http://rejon.org/ | http://fabricatorz.com/
> chat/skype: kidproto | irc: rejon
> +1.415.830.3884 (global) | [phone] (sf)
>

--
Bassel Safadi
http://bassel.ws
http://aikilab.org
Global [phone]
Singapore [phone]

[rejon (rejon)]

ftp sucks

On Mon, Feb 21, 2011 at 1:42 PM, Bassel Safadi <email address hidden> wrote:
> can connect the uploader with ftp not sftp, need to try though
>
> On Mon, Feb 21, 2011 at 9:40 PM, <email address hidden> <email address hidden> wrote:
>>
>> NO FTP ;)
>>
>> SFTP
>>
>> On Mon, Feb 21, 2011 at 1:17 PM, Bassel Safadi <email address hidden>
>> wrote:
>> > No Rejon it's only on scale. I'll try to update the software again.. btw
>> > do you have ftp on that server? like pureftp or some other services
>> >
>> > --
>> > You received this bug notification because you are a member of Scale
>> > Journal Developers, which is subscribed to scalejournal.
>> > https://bugs.launchpad.net/bugs/701171
>> >
>> > Title:
>> >  Can't upload 13 mb pdf
>> >
>>
>>
>>
>> --
>> Jon Phillips
>> http://rejon.org/ | http://fabricatorz.com/
>> chat/skype: kidproto | irc: rejon
>> +1.415.830.3884 (global) | [phone] (sf)
>
>
>
> --
> Bassel Safadi
> http://bassel.ws
> http://aikilab.org
> Global [phone]
> Singapore [phone]
>
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)

[Brad Phillips (brad-bradphillips)]

Bassel, the mime-type is used to determine if the thumbnail will display a rescaled version of the actual image (if it's something a browser can display, like a png) or a default icon (if it's something like a doc that can't be displayed by a browser).

To determine which to display I use an aiki php code for each mime-type, like <php if mime-type=="png" then <img src="[root]/((full_path))" height="90px" /> php>.

Currently, for certain large files, not only is the mime-type not recorded, but only 1mb of the files are even uploaded. If I switch to the single upload form, in most cases, the file isn't uploaded at all. No errors are returned in either case (the single upload form just redirects back to it's page, after a few minutes, as if nothing ever happened at all).

[rejon (rejon)]

Ok, getting same problem on yumumu, openfontlibrary and other places. If this is a server setting, we need to know what it is. If its plupload, then we need to solve that.

We need this analyzed and repaired ASAP. Its affecting us too often now!

I propose:

* uprade aiki plupload and make sure in aiki core we are checking out the latest code always!
* test scale and solve that first
* apply fix to aiki
* update wiki with recommendation
* apply across the board to other aiki based sites

This also highlights we need sanity in managing our aiki sites.

The best solution will be to use git to checkout our code and other modifications and bring some sanity to managing these installations...CC does this, and many of my friends doing now. Just need to find the best solution and apply.

http://joemaller.com/990/a-web-focused-git-workflow/

I'm sick of this problem and aiki forms problem. Time to solve and stop bitching.

[Christopher Adams (christopheradams)]

Just to get our grounding, can everyone working on an Aiki instance with upload problems find out the following values?

<?php

echo ini_get('post_max_size');
echo ini_get('upload_max_filesize');

?>

In Apache, the default value of post_max_size is 8MB, and upload_max_filesize is 2MB.

This means that no one file can be more than 2MB while the total payload cannot exceed 8MB.

I've been led to believe that plupload can circumvent these limits through chunking but a) I have not tested this claim, b) I have not checked to see if the Aiki sites are using chunking, and c) I would note that chunking is not supported by HTML5 as per http://www.plupload.com/index.php.

[rejon (rejon)]

Good point. I think there is a good chance that plupload tries to get around
this limit, but this is the irregular problem we are seeing with using it.

Thus, I'm trying to upload that 13 MB file now to see if it works.

Then will up the limit on php.ini on the sever at /etc/php5/apache2/php.ini
, restart apache and see if that fixes the problem.

If so, then we've tracked this mothertrucker down!

blat

On Fri, Apr 15, 2011 at 10:22 AM, Christopher Adams <email address hidden>wrote:

> Just to get our grounding, can everyone working on an Aiki instance with
> upload problems find out the following values?
>
> <?php
>
> echo ini_get('post_max_size');
> echo ini_get('upload_max_filesize');
>
> ?>
>
> In Apache, the default value of post_max_size is 8MB, and
> upload_max_filesize is 2MB.
>
> This means that no one file can be more than 2MB while the total payload
> cannot exceed 8MB.
>
> I've been led to believe that plupload can circumvent these limits
> through chunking but a) I have not tested this claim, b) I have not
> checked to see if the Aiki sites are using chunking, and c) I would note
> that chunking is not supported by HTML5 as per
> http://www.plupload.com/index.php.
>
> --
> You received this bug notification because you are a member of Aiki
> Framework Developers, which is subscribed to aikiframework.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
> Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)
[phone] (beijing)

[rejon (rejon)]

Ok, fabricatorz server has 8M post_max_size and 50M upload_max_filesize.

I tried the plupload with a 1.1 mb file and it works fine in both firefox4 and latest chrome.

I then tried a 2.9 mb file. It fails in both browser.

The 10.9 mb file i have too fails.

Thus, there is some other problem.

Brad, what is a link to a single file uploader. Lets try this.

Also realized same named file has problem with scale at present too.

We need to get plupload to latest version.

First we need to make sure the single file uploader works before using plupload.

Then, we need to make sure aiki software is at latest version, to analyze this upload problem.

I thought it might have been that plupload if has a limit of 8M for post_max_size, and you uploaded a 20M file, it would attempt to chunk the file in sizes that are too large for the 8M default and that is where the error would introduce. That does not appear to be the problem, because 2.9 M file is well under 8 M post max size.

Need to try single uploader first and find problem.

This is not a firefox or chrome latest issue either it seems.

[Christopher Adams (christopheradams)]

My understanding is that post_max_size is the upper limit of TOTAL
payload being posted, including headers and everything. I don't think
chunking will help because the server will add up the total of all the
chunks. As I said upload_max_filesize is the size of each individual
file, and it's possible that chunking will get around that limit.

The server has 8M post_max_size and 50M upload_max_filesize?

In general the post_max_size should be at least TWICE
upload_max_filesize. It doesn't make sense for it to be less!

[rejon (rejon)]

bassel, what is the working php.ini you have on other sites, like ocal, etc?

need to see those and figure out this problem real bad!

On Sun, Apr 17, 2011 at 1:31 AM, Christopher Adams <email address hidden>wrote:

> My understanding is that post_max_size is the upper limit of TOTAL payload
> being posted, including headers and everything. I don't think chunking will
> help because the server will add up the total of all the chunks. As I said
> upload_max_filesize is the size of each individual file, and it's possible
> that chunking will get around that limit.
>
> The server has 8M post_max_size and 50M upload_max_filesize?
>
> In general the post_max_size should be at least TWICE upload_max_filesize.
> It doesn't make sense for it to be less!
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)
[phone] (beijing)

[Bassel Safadi (bassel)]

The working one is on OCAL
both
post_max_size and upload_max_filesize should be 50MB or higher limit
post_max_size should be at least 10MB

On Sun, Apr 17, 2011 at 10:00 AM, <email address hidden> <email address hidden> wrote:

> bassel, what is the working php.ini you have on other sites, like ocal,
> etc?
>
> need to see those and figure out this problem real bad!
>
>
> On Sun, Apr 17, 2011 at 1:31 AM, Christopher Adams <email address hidden>wrote:
>
>> My understanding is that post_max_size is the upper limit of TOTAL payload
>> being posted, including headers and everything. I don't think chunking will
>> help because the server will add up the total of all the chunks. As I said
>> upload_max_filesize is the size of each individual file, and it's possible
>> that chunking will get around that limit.
>>
>> The server has 8M post_max_size and 50M upload_max_filesize?
>>
>> In general the post_max_size should be at least TWICE upload_max_filesize.
>> It doesn't make sense for it to be less!
>>
>
>
>
> --
> Jon Phillips
> http://rejon.org/ | http://fabricatorz.com/
> chat/skype: kidproto | irc: rejon
> +1.415.830.3884 (global) | [phone] (sf)
> [phone] (beijing)
>
>

--
Bassel Safadi
http://bassel.ws
http://aikilab.org
Global [phone]
Singapore [phone]

[Christopher Adams (christopheradams)]

File upload is working on OFLB, following the limits imposed by php.ini.

[rejon (rejon)]

brad, lets revisit scale as single upload with changed settings. need
to make clear on that form what is allowed too.

jon

On Mon, Apr 25, 2011 at 12:57 AM, Christopher Adams <email address hidden> wrote:
> File upload is working on OFLB, following the limits imposed by php.ini.
>
> ** Changed in: openfontlibrary
>       Status: Confirmed => Fix Committed
>
> --
> You received this bug notification because you are a member of Aiki
> Framework Developers, which is subscribed to aikiframework.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>

--
Jon Phillips
http://rejon.org/ | http://fabricatorz.com/
chat/skype: kidproto | irc: rejon
+1.415.830.3884 (global) | [phone] (sf)
[phone] (beijing)

[Christopher Adams (christopheradams)]

One difference on OFLB is that the upload is being processed by a
separate PHP script in assets/extensions. So it's pure PHP that
bypasses Aiki.

On Mon, Apr 25, 2011 at 11:08 PM, rejon <email address hidden> wrote:
> brad, lets revisit scale as single upload with changed settings. need
> to make clear on that form what is allowed too.
>
> jon
>
> On Mon, Apr 25, 2011 at 12:57 AM, Christopher Adams <email address hidden> wrote:
>> File upload is working on OFLB, following the limits imposed by php.ini.
>>
>> ** Changed in: openfontlibrary
>>       Status: Confirmed => Fix Committed
>>
>> --
>> You received this bug notification because you are a member of Aiki
>> Framework Developers, which is subscribed to aikiframework.
>> https://bugs.launchpad.net/bugs/701171
>>
>> Title:
>>  Can't upload 13 mb pdf
>>
>
>
> --
> Jon Phillips
> http://rejon.org/ | http://fabricatorz.com/
> chat/skype: kidproto | irc: rejon
> +1.415.830.3884 (global) | [phone] (sf)
> [phone] (beijing)
>
> --
> You received this bug notification because you are a member of Aiki
> Framework Developers, which is subscribed to aikiframework.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
>  Can't upload 13 mb pdf
>
> Status in Aiki Framework:
>  Confirmed
> Status in Open Font Library:
>  Fix Committed
> Status in Scale Journal:
>  Confirmed
> Status in Yumumu:
>  Confirmed
>
> Bug description:
>  It's just a collection of images. Not sure why it's not uploading.
>  There's no wacky characters in the title.
>

--
———
christopher adams
86 186 1172 0021 | 1 646 201 3335
49 15 156 219931 | 886 953 036 630
chris.raysend.com

[Bassel Safadi (bassel)]

tested on latest aiki with latest plupload, no problems upload success on 2Gig file, if it still happen on scale it's server side problem that I'll be happy to look at. also updated the plupload version on scale

[Brad Phillips (brad-bradphillips)]

Just tested plupload form on scalejournal with a 15mb pdf file & same issue still persists. The file appears to successfully upload (% completed goes all the way up to 100%), an 'upload success' message displays. When downloading the file from scalejournal, the file is an unreadable 1mb snippet.

[Bassel Safadi (bassel)]

ok trying to find out what is wrong on server side

On Sat, Apr 30, 2011 at 10:41 PM, Brad Phillips
<email address hidden>wrote:

> Just tested plupload form on scalejournal with a 15mb pdf file & same
> issue still persists. The file appears to successfully upload (%
> completed goes all the way up to 100%), an 'upload success' message
> displays. When downloading the file from scalejournal, the file is an
> unreadable 1mb snippet.
>
> --
> You received this bug notification because you are a member of Aiki
> Framework Developers, which is subscribed to aikiframework.
> https://bugs.launchpad.net/bugs/701171
>
> Title:
> Can't upload 13 mb pdf
>
> Status in Aiki Framework:
> Fix Released
> Status in Open Font Library:
> Fix Committed
> Status in Scale Journal:
> Confirmed
> Status in Yumumu:
> Confirmed
>
> Bug description:
> It's just a collection of images. Not sure why it's not uploading.
> There's no wacky characters in the title.
>

--
Bassel Safadi
http://bassel.ws
http://aikilab.org
Global [phone]
Singapore [phone]

[rejon (rejon)]

must be server-side.