parallelUploads setting (Dropzone?)

[DrQuincy]

It appears PW uses dropzone.js for file uploads. I have used this many times before in non-PW projects and with my hosting company I run into problems with their firewall (I think) if I upload lots of files at the same time. Basically what happens is (again, I think) their network detects a bunch of large POST requests in a short space of time and temporarily cuts you out — causing some of the uploads to fail.

I get the same thing with PW: if I upload a lot of files together sometimes it just freezes on me with no error message.

In my custom projects the fix was easy — just set parallelUploads to 1.

https://www.dropzonejs.com/#config-parallelUploads

I've searched the source and online and can't find any way of controlling how many file PW will attempts to upload at one time.

Can I override dropzone.js anywhere and/or is there a way to control how many files PW will attempt to upload at one time?

I accept that even with parallelUploads: 1 you could still run into this issue with multiple uploaders on one template, multiple tabs open, etc but in practice the above setting has solved the problem.

Any ideas?

Thanks.

[horst]

Another thing, on PW site, would be to allow upload of zip(ed) files to pagefile pageimage fields. As a workaround.

[BitPoet]

4 hours ago, DrQuincy said:

It appears PW uses dropzone.js for file uploads

I'm not so sure about that. AFAIK, PW has its own Ajax upload implementation in InputfieldFile.js and doesn't queue uploads at all.

[BitPoet]

After looking into the code, adding support for a parallel uploads limit wasn't too hard, I think. If you're willing to try, I've created a fork of the current PW dev branch and added that functionality.

This lets you set a configurable limit for how many uploads happen at once, the rest is queued and processed as soon as others finish.

[DrQuincy]

Thanks to you both. I am flat out with work at the moment so will reply as soon as I can. I am not ignoring you! ?

[DrQuincy]

@BitPoet, thanks so much for this.

You are right, it doesn't use dropzone.js but it does use the term “dropzone” in the code — which confused me!

Thanks for your code. I can see you have amended the following:

• processwire/wire/modules/Inputfield/InputfieldFile/config.php
• processwire/wire/modules/Inputfield/InputfieldFile/InputfieldFile.js

If I use these modified files then I presume they will be overwritten in each PW upgrade. I don't know the process for making changes to Processwire but do you think Ryan would be open to merging this into the core? I think it's a useful feature as, particularly on shared servers, Apache can 503 if it gets too many requests too soon.

Thanks again for your help.

[BitPoet]

I'm not sure he will. Generally, a new feature needs to be used by an estimated 30% of users to be accepted into the core. There are, however, exceptions to the rule - especially where hosting issues are concerned. I'm not happy with my adaptions yet, though, since they work on some installations here but not on others. I'm still on the lookout for the issue there. However, once I work out the kinks, and if the change doesn't have to get more complicated, I think it could have a good chance to be accepted (adding a feature request at processwire-requests and finding others who vote for it would certainly help).

[horst]

3 hours ago, DrQuincy said:

Thanks for your code. I can see you have amended the following:

• processwire/wire/modules/Inputfield/InputfieldFile/config.php
• processwire/wire/modules/Inputfield/InputfieldFile/InputfieldFile.js

Hi, you may try to copy the complete wire/modules InputfieldFile folder with all files into site/modules InputfieldFile. Then PW should ask you in the admin that there are two modules and which one you want to use. But please better first try this in a backup copy. I used this sometimes in the past, but not with newer PW versions.

[DrQuincy]

@BitPoet, I certainly wouldn't expect things to change on my account. Maybe the problem is not that common but I have certainly run into it a few times over the years on shared hosting. If you do work the kinks out and put it forward I would 100% vote for it. ?

@horst, that is a great idea, thanks. I'd forgotten it was possible to do that.

As usual, this forum is exceptional when it comes to helpful replies!

[BitPoet]

3 hours ago, DrQuincy said:

If you do work the kinks out

Looks like I have. Found a classic case of shoot-your-foot in the affected installations where I generously added some JS in a ProcessPageEdit::execute hook, my InputfieldFile change was wholly innocent. What I did find was a race condition between the first few added files, so there was one upload more active than configured in the beginning. I believe I caught that with a setTimeout. I still have to tweak that timeout, run a few more test drives and meditate over reasonable defaults, inline documentation and such, but it's taking shape.

[DrQuincy]

Amazing, thank you!