Support for drag & drop, multiple file selects and instant uploads in file field

[ryan]

Good news, I got all the AjaxUpload stuff out of ProcessPageEdit and everything working with WireUpload, so now the AjaxUpload files get routed through there instead. It basically saves it in a temporary dir like PHP does with regular uploads, then creates a new $_FILES var cloning what would be in the PHP one during a regular upload. That way, not much code had to change in WireUpload and it's backwards compatible with regular uploads. It runs through all the same filename and extension validation and sanitization before getting moved to the destination dir and sent back to InputfieldFile.

One thing I realized though is that we need some way to send a success (or error) response with the actual resulting filename back to InputfieldFile.js AjaxUpload. WireUpload sanitizes filenames, makes them lowercase, removes non ascii characters and so forth, so it may have changed the filename or thrown an error that ultimately needs to be output by InputfieldFile.js. I hope to get time this weekend to experiment with it more, but so far everything seems to be working great. I'm still adding a little more validation (for file size) but will post updated files once I've got that going, hopefully sometime this weekend. Thanks for all your work in getting this going. I can't wait to get this officially committed to the 2.1 source.

[apeisa]

Can you describe more or point me to a good link? I'm not sure I understand about the HTTP_X_FIELDNAME.

Heh, that was just me being stupid. That is something I send on FieldtypeFile.js, adn you were trying to get that working on all places (not just inputfield file), so sorry about that. We can forget it

Great news that you got it working! I agree that we need some kind of error handling. Probably best would be if you get it easily to return small json snippet, something like this.

{ 
"status": "ok",
"filename": "filename.jpg"
}

or

{ 
"status": "error",
"errormsg": "file was corrupted"
}

[ryan]

Actually we do need something like HTTP_X_FIELDNAME. Maybe not for this specific instance with file uploads… but in order for PW to support saving any field via ajax, it needs to know what it is ahead of time. Otherwise there would be no way to detect something like an unchecked checkbox. By that I mean that when you submit a form with nothing but an unchecked checkbox, the $_POST would be an empty array, no reference to the checkbox. So something like HTTP_X_FIELDNAME is actually quite a good idea, and necessary if dealing with individually POSTed fields within a form. The only thing I wonder about is: what is the advantage of using a header as opposed to a regular POST var for this? Might it be simpler for it just to be another POST var rather than a header (for simpler ajax with jQuery)? Something like a POST var called "_fieldName" that's a string with a field name or "_fieldNames" that's an array holding one or more field names. I only mention this because I don't yet know how to send headers like HTTP_X_FIELDNAME with regular jQuery, so am just looking for the most straightforward solution.

Probably best would be if you get it easily to return small json snippet, something like this.

Sounds good to me. That looks similar to what it already does. Currently this is what it returns:

{
"error":false,
"message":"Saved page '1' field 'images'"
}

Whenever PW sees it's an ajax request, it just returns a JSON of any messages that you would usually see in the admin. That's how it gets error alerts to the PageList ajax, for example. What would you think about using the same format that's already there, but just adding a 'filename' field to it, like in your example? Like this:

{
"error":false,
"message":"Saved page '1' field 'images'",
"filename": "filename.jpg"
}

and

{
"error":true,
"message":"Please no more photos of your cats.",
"filename": "filename.jpg"
}

I also noticed when testing that I get this message from Firebug in Firefox 6 whenever the ajax upload code executes. It doesn't seem to matter or affect anything, but just wanted to mention it in case it makes any sense to you:

Use of File.fileName is deprecated. To upgrade your code, use standard properties or use the DOM FileReader object. For more help https://developer.mozilla.org/en/DOM/FileReader

[apeisa]

I didn't realize that there are json responses already, that is great. What you suggested looks great!

For some reason this ajax fileupload is not recognized as ajax request (I always get normal response). That is probably this is XMLHttpRequest Level 2 (http://www.w3.org/TR/XMLHttpRequest2/) or then I am just missing headers. I will study this after I get back to code.

Sending additional headers with jQuery: http://api.jquery.com/jQuery.ajax/ Ajax() supports it, not sure about Post() (probably not). But that probably isn't necessary other than in file uploads (which cannot be send with regular js post). I don't see any benefits using headers instead of post in normal use.

Deprecated fileName - I will look into it and change to more modern way to do it to keep it working in future also.

[ryan]

For some reason this ajax fileupload is not recognized as ajax request (I always get normal response). That is probably this is XMLHttpRequest Level 2 (http://www.w3.org/TR/XMLHttpRequest2/) or then I am just missing headers. I will study this after I get back to code.

I'm curious to know what you find here. I had to add this to your JS in order for PW to recognize it as ajax:

xhr.setRequestHeader("X-REQUESTED-WITH", 'XMLHttpRequest');

Should I be having PW core detect the presence of ajax by some other method? Here's how it currently does it:

$config->ajax = (isset($_SERVER['HTTP_X_REQUESTED_WITH']) && $_SERVER['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest');

For the code I'm working on here, I settled with keeping the headers method you are already using. I don't know the non-jQuery ajax syntax well enough, and it sounds like we can't send separate POST vars anyway, so might as well just stick with the header. Though PW will check POST vars as a secondary option if there is no header. (I'll worry about things like checkboxes when and if the issue ever comes up). For now, stick with the HTTP_X_FIELDNAME.

I'm still working on other updates here, but hope to post soon.

Thanks,

Ryan

[apeisa]

I'm curious to know what you find here. I had to add this to your JS in order for PW to recognize it as ajax:

xhr.setRequestHeader("X-REQUESTED-WITH", 'XMLHttpRequest');

Yep, that is what is required:

http://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Requested-With

I have always thought that it is automatic when using XMLHttpRequest, but as wikipedia tells us, it is more like common practice between different js-frameworks to use that. So your solution is just right here.

Should I be having PW core detect the presence of ajax by some other method? Here's how it currently does it:

$config->ajax = (isset($_SERVER['HTTP_X_REQUESTED_WITH']) && $_SERVER['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest');

This is good!

For the code I'm working on here, I settled with keeping the headers method you are already using. I don't know the non-jQuery ajax syntax well enough, and it sounds like we can't send separate POST vars anyway, so might as well just stick with the header. Though PW will check POST vars as a secondary option if there is no header. (I'll worry about things like checkboxes when and if the issue ever comes up). For now, stick with the HTTP_X_FIELDNAME.

We can send separate POST vars (if I understand you correctly), but I couldn't figure out how to do it at the same request with file upload (not sure if it is possible there, like it is with regular ajax posts). So in file uploads we need to use headers, so your current solutions sounds good and flexible implementation.

[ryan]

Attached is the completed AjaxUpload, though I'm sure there are still a few small tweaks.

The attached zip has all the files that need to be replaced in your PW install (organized by PW's dir structure).

In this version, after you upload, messages and errors are reported back for each file. Also, if PW changed the filename, you'll also see the status indicate the final filename. For an example of an error message, try to upload a non jpg/gif/png file to an image field.

It also works with ZIP files! If your file field and platform support ZIPs, then drag in a ZIP and it'll extract all the files and show them to you. (As far as I know, PW's zip file support doesn't work on Windows… though someone correct me if I'm wrong).

Files uploaded go through the same path as regular uploads, so things like max image resizing (at upload time) also work.

Please give this a try and let me know if you run into any errors. I'm hoping to get this bundled into PW 2.1 this week. Once I get confirmation from a couple people that all works good on their end too, then I'll go ahead and commit to the source.

Antti-- also want to get your eye on all my changes to the JS and make sure all is good. Please make any updates as you see fit.

Thanks,

Ryan

AjaxUpload2.zip

[apeisa]

AjaxUploads works perfectly! Great to see this so polished and working so well.

Only thing that does not work is old school uploading - I couldn't add new files with IE.

[ryan]

Thanks–fixed. Old school uploading should work now too. I also made some additional updates for keeping track of max number of files. Original message with the file AjaxUpload2.zip. If you've already installed the previous one, these are the files that changed in the latest version:

/wire/modules/Inputfield/InputfieldFile.module

/wire/modules/Inputfield/InputfieldFile.js

/wire/core/Upload.php

[Soma]

Just intalled it and it works great so far!

Now if we could only see the thumbs after upload is done.

[ryan]

Thanks for testing it. I was wondering about this too, and I think we can. So far this solution is all part of InputfieldFile, so its behavior is currently going to be the same whether dealing with a regular file(s) field or an image(s) field. I was thinking the JSON response could optionally include some markup that would get shown after a file is uploaded. So InputfieldImage could create a thumbnail and return a little markup with an <img> tag. I think it'll be relatively easy to do. But getting this ajax upload was the last "new" thing I wanted to get committed before putting PW 2.1 out, so will add this thumbnail preview afterwards in an incremental update.

[apeisa]

Just tested and old school works, as well as html5. Great job and thanks again for finishing this!

What comes to preview and maybe instant removing => after 2.1 definitely. And previewing might be best to do using js: we could show the image at the same time than it uploads (if there were something to remove, you could stop uploading before it finishes and never ends up to the site). We wouldn't need to create thumbnail image at all on the backend (although that is needed anyways if you use thumbnails setting on your field).

[Soma]

So far it's not throwing an error when upload max size is exceeded. It just shows 100%.

It would be nice, if it's possible to validate before upload starts, and after it's uploaded showing the preview. I don't know if I like the instant preview before the image is even uploaded or validated, it could suggest it's valid and already "there".  IMO this is one of the problems with oldcool uploading, after 30 second of uploading a file, it says not valid or too big...

[apeisa]

We cannot really validate before upload starts (since we don't have file that we want to validate), but definitely could (and should) do some basic checks there (supported filetype and small enough filesize at least). I'll look into this (should be easily doable).

Good point about that someone might get confused to see preview of image before it is uploaded. Although not sure if that is the case, since there is pretty big progressbar going on also. Maybe some label like "preview" or "currently uploading" would make it clear.

[apeisa]

Ok, I did some pre-validation on front-end (simple check on file extension and filesize). See the attached zip (contains only changes files, .js and .module). Please test at least few times and with "difficult" filenames and sizes. Works fine on my localhost.

Only thing I am not sure is how I get max filesize:

<?php
$max_upload = (int)(ini_get('upload_max_filesize'));
$max_post = (int)(ini_get('post_max_size'));
$memory_limit = (int)(ini_get('memory_limit'));

// comes megabytes from php.ini, we convert to bytes (ie. 1024 * 1024 * 2M = 2097152 bytes)
$this->max_filesize = 1024 * 1024 * min($max_upload, $max_post, $memory_limit);

Is there some $config or field setting already on pw or is this ok?

InputfieldFile.zip

[ryan]

I think we were working at the same time because I've also got some updates, though mine don't yet incorporate yours. I'm not too worried about max filesize. PW doesn't keep track of filesize … I see it as a non-issue for the admin usage that these fields were built for. But for the AjaxUpload, it does seem like a nice convenience. Could be a nice thing for those with servers that have 5 meg limits on their php.ini settings and they are trying to upload a 500 meg file. PHP does keep track of max filesize, so I think that's where we should get it (per the ini_get settings you mentioned).

I couldn't leave the image preview thing alone. Every time I used the upload, I wanted to see it right away. So I went ahead and added it. So now it shows you the image immediately after it's been uploaded. It accounts for thumbnail settings and/or post-image resizes too.

I've also made several other updates so that you can edit the description and sort the images right away too.

Lastly, uploads to a single-file field now support replacement. No longer do you have to delete the image before you can upload another. Now you can just drag in the new one and it'll replace the one that's there.

Attached is the latest version (AjaxUpload3.zip). If you already have the AjaxUpload2.zip installed, then you only need to replace the files in the InputfieldFile and InputfieldImage directories.

Antti: Is it relatively easy for you to paste your code updates into this version? If not, I will update my version with your updates, but may need assistance figuring out all the spots that need to be updated. I may be able to figure it out with some diff action, but just wanted to double check so that I don't miss anything.

Thanks,

Ryan

---

Edit: php.net's ini_get() page Example #1 shows how to convert PHP's size notations (like 5M, 1G, etc.) to bytes. This is probably what we need for the upload_max_filesize? http://php.net/manual/en/function.ini-get.php

AjaxUpload3.zip

[apeisa]

That sounds excellent! My update had only few lines of code, so I can easily merge those. I'll do that tomorrow, now heading to bed. I didn't realize that ini file size settings can be also G and others, so I will refine that conversion script too.

[Soma]

...just one word. Orgasmitastic!

Just installed and it's beautiful to watch, uploading 20 6mg images and see them process....

One minor problem though to be almost perfect: sorting after images are uploaded doesn't work nor doesn't fancybox. After refreshing page it works.

EDIT: Well sorting works for newly uploaded images if there IS already images saved there in first place. But fancybox still doesn't work for the newly added.

EDIT: Sorry for being so picky. But I would expect that when instantly adding images this way that the sorting alos gets saved per ajax instantly. It doesn't though.

While playing around I tried add images in tinymce, and it shows full size images among to select which is hard to see them. After selecting one it disapears and I can't resize it. I'm using chrome latest on osx.

[ryan]

Soma, thanks for your testing here.

One minor problem though to be almost perfect: sorting after images are uploaded doesn't work nor doesn't fancybox. After refreshing page it works.

Both are the same issue. I just need to re-apply the fancybox initialization to new images, and have the sort be able to initialize after document.ready(), so it should be a straightforward solution. Just a matter of applying these initializations after rather than before, and do so more selectively.

But I would expect that when instantly adding images this way that the sorting also gets saved per ajax instantly. It doesn't though.

I like the convenience of the AjaxUploads, but dynamically saving sorts is an entirely different thing. It may be okay forms without a save/submit button. But in our context I think it would be an overuse of ajax. I want to be cautious about blurring the lines too much about when something is saved and when it isn't. When people are changing anything on a page, they need to save it. Ajax uploads do create some blur, but it's a progressive enhancement and worth the compromise. I wouldn't want to blur that further. Making it save the page every time you dragged to sort a file would create an expectation that multi-choice fieldtypes (like with asmSelect) should do the same thing, and the template editor, and so on. You wouldn't know when you needed to save and when you didn't. I don't want people to assume that their changes are committed until they've saved the page. One should be able to experiment with different sorts without their experiments being visible to the public.

I got nailed by this same problem recently when a site was saving the state of every image I dragged (unknown to me). I never hit save, nor was I intending to. My cat ended up as my profile picture for several days before I realized it! 

While playing around I tried add images in tinymce, and it shows full size images among to select which is hard to see them. After selecting one it disapears and I can't resize it. I'm using chrome latest on osx.

When selecting images in TinyMCE (after clicking the image icon) it shows all the images on the page and limits their width to the window size of the modal dialog. Let me know if you are getting a horizontal scrollbar, as that would indicate it's not working. I don't often deal with images as large as you (you mentioned 6mb), but one option is that I could have the image selection specify a max width that's smaller, like 50% window size or something like that. But this is one place where I lean towards scaling over resizing. For image fields that have automatic thumbnails turned on, it may be possible for it to use those though.

As for the image disappearing after you selected it, I'm not sure what's going on there so need more info– does it do that too all images, or just the big ones? I can't duplicate here in my testing (also Chrome latest in OS X), but I don't have any 6mb images either.

[apeisa]

I added my modifications (simple validation for filesize & extension) and also added fancybox & sortable for right away added images.

I agree with Ryan on not to go too far with ajax-features....

EDIT: Small modification to Inputfieldfile.js file to line 200:

if (extensions.indexOf(extension) == -1 && extensions.length > 1 && extension != "zip") {

I was too lazy to zip & upload again. If for some reason extensions array is empty, then we skip pre-validation all together. And it seems that allowing zip extension is not required if you want to use "decompress zip files" settings (and I confirm: does not work on windows).

InputfieldFile.zip

[Soma]

Yes the images fit the fancybox width, but if having really large images it gets slow, would be cool to have like 50% though it's may just because I'm on 13" mbp... aaand also the images max width height setting isn't set on those, that would not be the case in a real site.

It's strange after I select the image it disapears... always, I noticed this already sometimes before, I will check out more to may find the real issue. It could be the large imagesize, I developed similar thing in my admin apps and also experience some strange trikcy behavior with fancybox modals and images vs js css etc.

I understand your point and agree in that it should not blur that too much, though having some king of simple ui response notification that makes it clear, like you already have on page tree list move actions, well it is in a kinda different context. But still I agree that it need attention in the ui interaction design

[Soma]

Sorry for the double post. Ok this gets kinda weird, but It worked for me only one time with as smaller image... but after the second try it didn't even with smaller images. Attached a screenshot, the iframe that loads has the selected image div which is empty ... height:0 ... does it get loaded by ajax into it? and replace those "..."? I can't se any errors or anything, might be some kind of chrome bug, cause just tried and it works in safari, can't test on firefox because he hangs himself up everytime I open him

And forget about the 50%! Stupid idea...  

[ryan]

Soma, thanks for the testing. I keep trying to duplicate in Chrome/OS X but so far no luck. What version do you have? I'm running 13.0.782.112

Also, have you restarted Chrome recently (just in case)?

Antti, thanks for the updates! I have applied them and added a few last minute details and have committed this feature to the 2.1 source!

Add support for HTML5 ajax file uploads thanks to @apeisa - Now you can drag and drop one or more files right into PW file or image fields. With images you get instant previews. With single-file fields you can drag-and-drop to replace. This is the most fun-to-use CMS feature I've seen in awhile. Thanks also to Soma for the testing and suggestions.

[apeisa]

Great stuff. I went totally crazy on Twitter because of this Great job fellows!

[Soma]

Soma, thanks for the testing. I keep trying to duplicate in Chrome/OS X but so far no luck. What version do you have? I'm running 13.0.782.112

Also, have you restarted Chrome recently (just in case)?

Yes I did, but no luck... got 13.0.782.220

Antti, thanks for the updates! I have applied them and added a few last minute details and have committed this feature to the 2.1 source!

Awesome, congrats apeisa! and ryan! Great work guys!