Jump to content

Max file size for uploading a file?


tinacious
 Share

Recommended Posts

Hello.

I have a client who wants the ability to upload a large file (~350 MB) and share it.

I'm wondering if there will be any issues with uploading large files through ProcessWire and what should be done to prepare to make sure this all runs smoothly before I begin development.

Thank you for your help.

Link to comment
Share on other sites

Regardless of whether handling this with processwire you should change your server variables to handle such a request.

Your php.ini should be adjusted for it (at least):

upload_max_filesize = 350M
post_max_size = 350M

And your apache should be configured too:

memory_limit = 400M

Update (thx Wanze)

You should also adjust settings like

max_execution_time
max_input_time

in your php.ini...

I don't know if your clients host has such capabilities (-> memory limitations for such one vhost) but you should consider to convince your client to upload such files with ftp (not with http) and it is less error-prone for you and him with ftp...

  • Like 6
Link to comment
Share on other sites

  • 2 years later...
  • 1 year later...

hi!

ok i'm now also having problems with file uploads between 500 - 1000MB on plesk.

first problem was an error after some time due to MaxRequestLen on php FCGI

that seemed to fix it: https://www.opensource-excellence.com/blog/item/430-cant-upload-files-due-to-fcgi-maxrequestlen-restriction-in-plesk.html

now the upload proceeds until 100% but throws this error in the console:

InputfieldFile.min.js?v=124-1463995380:1 POST https://.../admin/page/edit/?id=xxx&InputfieldFileAjax=1 net::ERR_EMPTY_RESPONSEn @ InputfieldFile.min.js?v=124-1463995380:1w @ InputfieldFile.min.js?v=124-1463995380:1(anonymous function) @ InputfieldFile.min.js?v=124-1463995380:1

The file is around 700MB. After upload i have a 502MB file in the folder. When i save the page with the stuck 100% upload this file is removed completely (thats the part where i know why ^^ )

Any help for this? I have no errors in the apache error log for the second problem. The first Problem showed this (for reference and search engine):

[Tue May 31 15:34:02 2016] [warn] [client xxx] mod_fcgid: HTTP request length 134225920 (so far) exceeds MaxRequestLen (134217728), referer: https://xxx/admin/page/edit/?id=xxx
Link to comment
Share on other sites

Maybe I am looking at this the wrong way, however when I look at the image (screenshot) you posted, I say to myself "Ok, That shouldn't work"

You have a post_max_size that is the exact same size (800M) as your upload_max_filesize setting.  Even though the explanation below it explicitly states "Typically, should be larger than upload_max_filesize and smaller than memory_limit."

It may not make a difference, but I would have at least set the upload_max_filesize to 750M.  That would actually follow the PHP guidance in the notes on the panel.  In other words, having something the same size is not the same as it being larger (at least where I come from).

Also, if I am dealing with files that are that large (700M), I would be using SFTP or SSH to upload the files to the server.  To me, those are gigantic files. 

You may have everything setup correctly, however the configuration of the server (Plesk/Web Host/Security Settings) may not allow you to post a file this large.  You would need to consult with the hosting provider (if online)

Edited by cstevensjr
Added additional notes
Link to comment
Share on other sites

No matter what you are going to setup in your php.ini, with uploading files of that size you are probably going to run into your hoster limitations, internet connection time outs, file corruption, etc.

First you should try to embed such files through dropbox, box, gdrive, etc.

If you want to keep everything in the house you can use html5 FileApi,

http://www.html5rocks.com/en/tutorials/file/dndfiles/

or this one: https://github.com/mailru/FileAPI

Link to comment
Share on other sites

  • 3 weeks later...

hmmm...

Quote

wir konnten das Problem nachvollziehen, es scheint hier ein Problem mit Ihrer Anwendung zu geben.
Bitte beachten Sie jedoch das wir keinen Support für Drittanbieter-Software und Kunden-Skripte anbieten.
Leider können wir Ihnen bei diesem Problem nicht helfen. Wenden Sie sich daher bitte an den Hersteller der Software, oder Ihren Entwickler der die Anwendung geschrieben hat.

Mit freundlichen Grüßen

i did not get any more information about WHAT the problem could be... they do not provide 3rd party software support. strange that i am the only one having this problem if it is really related to processwire...

i'll try another upload tool and see what happens.

Link to comment
Share on other sites

At least it let enough room for speculations.

  • Did they have investigated?
  • Is it a server config that they don't want tell about or change it?
  • Or do they NOT have investigated and this is a boilerplate answer that is sent with enough timedelay? :)

annoying!

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

ok i got another step further: the processwire error logs say the following:

/admin/page/edit/	Error: Allowed memory size of 1048576000 bytes exhausted (tried to allocate 444071936 bytes) (line 144 of /var/www/vhosts/.../httpdocs/wire/core/WireUpload.php)

https://github.com/ryancramerdesign/ProcessWire/blob/master/wire/core/WireUpload.php#L144

what does that mean?? allowed = 1GB, exhausted = 444MB

does anybody have an idea on this? :( i'm really getting frustrated...

2016-07-05 20_33_21-Developer Tools - https___new.mentalestaerke.at_admin_page_edit__id=6114.png

Link to comment
Share on other sites

hmmmm... update: on a fresh install WITHOUT https it works for a 614MB file.

i switched off HTTPS on the other install, but still it does NOT work. debugging is really painful, as every upload needs >10mins :(

any ideas what the problem could be or how i can speed up debugging?

edit: got this now in the apache error log when trying to upload a file without https on the client site (only this time!)

[Tue Jul 05 21:45:55 2016] [warn] [client xx.xx.xx.xx] (70008)Partial results are valid but processing is incomplete: mod_fcgid: can't get data from http client, referer: http://.../admin/page/edit/?id=6114

:(

Link to comment
Share on other sites

10 hours ago, bernhard said:

got this now in the apache error log when trying to upload a file without https on the client site (only this time!)

What are the values of :

  • IdleTimeout
  • ProcessLifeTime
  • FcgidMaxRequestLen
  • FcgidIOTimeout

?

  • Like 1
Link to comment
Share on other sites

  • 4 weeks later...

thank you @flydev i changed my server so i don't have this problem any more...

but i still have a problem i need to solve, but that's another topic. on my research i found this answer: http://stackoverflow.com/a/5828654/6370411

it states that this should work:

 upload_max_filesize = 1G
 post_max_size = 1G
 memory_limit = 32M

that's strange because i'm getting out of memory errors now on uploads via processwire file field...

i will investigate further and hold this here as reference. or does anybody know why memory_limit should not be a problem and IS a problem for processwire?

Link to comment
Share on other sites

I'm not sure what exactly you're building there, but maybe you could use something like flysystem's dropbox adapter to let people upload such big files to dropbox, which is far less error prone, and import from there. Tech. php can handle those files, but some hick-up in your connection (on mobile) or anything will corrupt any upload and you need to start over again.

Edit: Also this is probably not a ProcessWire specific issue, until the file is uploaded 100% it's not even touched by processwire. 

  • Like 4
Link to comment
Share on other sites

  • 6 years later...

Hello,

unfortunately I have to rehash this thread;-)

I have a field of type "file" and I just can't get files larger than 128M uploaded.
The php.ini settings are as follows:

upload_max_filesize: = 256
post_max_size = 256
memory_limit = 512
max_execution_time = local value: 120 / master value: 300
max_input_time = 600

Is there somewhere in the file settings a hidden possibility to limit the upload volume?

Thank you very much for your input!

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...