Support for drag & drop, multiple file selects and instant uploads in file field

[apeisa]

I am working on few nice html5 features (see topic). I started these as a separate module, but soon realized that these are something that are probably always welcome. Or that is what I think at least, how about you guys? It would be easier and probably cleaner result if I just modify the core files in this (with keeping it 100% backwards compatible).

I am going with html5 way here, so good support is in Firefox 6+. latest Chrome and upcoming IE10. HTML5 File API looks wonderful and this shouldn't require too much code. Actually there is one liner which makes file fields so much faster to use:

$this->setAttribute('multiple', 'multiple'); 

If you add that to line 32 in your InputfieldFile.module then supported browsers will allow you to choose multiple files at once. It works without any other changes to anywhere. And shouldn't cause any problems for older browsers, since it is just a new attribute in file input field.

I'm on very early steps on this, but if Ryan gives a green light and there is no good reason to keep these features out of the core, I will do these right into core InputfieldFile instead of extending that.

[Soma]

Recently thought about similar things that could be done with new html5 and the like

Now that we have the file api, wouldn't it then be possible to see a preview thumb of a selected file in the page edit before saving the page?

[Pete]

Nice ideas, especially since it sounds like it just degrades gracefully on older browsers.

[apeisa]

Now that we have the file api, wouldn't it then be possible to see a preview thumb of a selected file in the page edit before saving the page?

Yep. Also possible to disallow uploading too big files right there and add cropping to images etc.

[Pete]

Wow!

And it's taken them this long to roll HTML5 out because...?

Seriously, if we'd had the file API about 5 years ago so many of my projects would have been so much simpler

[apeisa]

Yep. There isn't cropping in file api, but because you can read chosen files with javascript, you can display them, add cropping and do resizing etc before uploading the files.

[Pete]

Wow. There's the majority of the need for Flash/Java uploaders gone for good

On sites where I'd be creating forms for other admins to use on one of my sites I would probably speciify that they have to update their browsers to use the CMS. It will obviously be a few years before the majority of folks have updated to a HTML5-compatible browser however I'm looking forward to using some of these features down the line to make life easier.

In fact, I'm sure I saw a jQuery-based upload script a few weeks back (before my laptop died ) that used HTML5, fell back to Flash if your browser didn't support it and fell back to doing things the old-fashioned way if you didn't have either. That one was just for uploads, but one that was just HTML5 that might be good for getting ideas from is mentioned previously here: http://processwire.com/talk/index.php/topic,351.0.html

[ryan]

Definitely green light, as you say. Any improvements in this area are certainly welcome. If it's something that is pretty universal then of course makes sense in the core. I am excited about the possibilities here.

[apeisa]

I started working on this and got pretty good start. Multiple selections and drag & drop already works (first thought that as "nice demo" feature, but it feels very nice in use). Next I want to add instant uploads. But that requires some url where js can post those files. What would be best solution here? Probably something like: /processwire/page/upload/?id=5839 (that would only manage ajax file uploads).

Ryan, any tips on this?

EDIT: I ProcessPageUpload looks like a good and simple solution here, will try that.

[apeisa]

Started developing own ProcessPageEditAjaxUpload, but it is so closely what InputfieldFile does, so started to thinking about better method. There is great way to create formdata on the fly with html5, so I could recreate the post with just the one new image and post that to regular page edit page. So far no luck with that. That would still be the best way, since it would allow all the features like the regular page save, so ajax uploads couldn't mess things up.

What stuff the page save form expects to be posted? All the fields or only few + submit_save? How it checks that it is valid post?

[apeisa]

Hmm.. getting closer. There is post fields that very simple page edit posts:

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="name"

grillipaikka

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="parent_id"

5817

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="template"

50

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="sortfield"

sort

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="title"

Grillipaikka

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="submit_save"

Save Page

------WebKitFormBoundaryttJCZBXTR2dQmoZt

Content-Disposition: form-data; name="id"

5819

------WebKitFormBoundaryttJCZBXTR2dQmoZt--

***

So it must be that I don't have parent_id, template etc? Didn't find the place where to check which of those (if all?) are required? Or are those just the same than when working from api?

(I'll work at the same time and throw questions when those occur, more like brainstorming here - not actual or important questions)

[apeisa]

Posted this on other page, but still no luck:

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="parent_id"

5812

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="template"

41

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="name"

linnanmaki

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="sortfield"

sort

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="title"

Testing testing

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="submit_save"

Save page

------WebKitFormBoundary4pYRGbMxtMXW4kdE

Content-Disposition: form-data; name="id"

5815

------WebKitFormBoundary4pYRGbMxtMXW4kdE--

Respond headers say 200 ok and it seems to render the page normally (I can see that from Chrome dev tools, network tab). So it posts ok and to right page. But when you do regular save, then it does 301 redirect (to avoid back button problems). So somehow saving fails somewhere before that.

[ryan]

I think that the only thing PageEdit needs to save a page is the actual 'id' as a POST var. But it's not specifically designed for saving one field at a time, so I'm thinking that in this scenario ProcessPageEdit be inefficient at best, or unpredictable at worst. So I think we probably need something more designed for this (within ProcessPageEdit), or at least I need to update ProcessPageEdit to know to expect it. I'm going to finish replying to messages here and then going to experiment in the code to see if there is a simple solution (I think there is).

[apeisa]

Ok, sounds good. Good solution might be something like if post comes through XMLHttpRequest, then it saves only the fields that are posted.

[ryan]

That sounds like a perfect solution. I'm working with it now...

[apeisa]

If it is simpler to implement, I think that it is also fine to save just one field at the time (at least on this occasion, not sure about the future)

[ryan]

I think I've got this ready to go if you want to give it a try. Replace your /wire/modules/process/ProcessPageEdit.module with the attached version. Then, when you want to save a field via ajax, POST it to the ./fields/ URL, i.e. /processwire/page/edit/fields/. For example:

var page_id = $("#Inputfield_id").val(); 

var postData = {
   id: page_id, 
   title: 'This is a test of changing the page title', 
}; 

$.post('./fields/', postData, function(jsonResult) {
   console.log(jsonResult); 
});

You can specify as many fields as you want in postData, but they have to be ones that are attached to the page's template in Setup > Templates > Fields. Currently you can't change fields that are built-in to pages, like: id, name, template, parent, etc.

I opted to do this with a "./fields/" URL off the page edit because I didn't want to use the same save code. Ajax stuff needs to react quickly and I thought it would just be faster if the code that saves ajax results didn't have to build and process the entire form. So instead, it just deals with the fields that are posted.

Once we've got it all working, I'll commit the ProcessPageEdit updates to the 2.1 source.

Edit:

Somewhat unrelated, but also wanted to add: whenever you are posting files, the form that posts them needs to have an 'enctype' attribute of 'multipart/form-data'. I mention this only because I always forget it and finally realize it after pulling some hair out.

Edit #2:

Wanted to clarify that you don't need to add a /fields/ page in your admin. ProcessPageEdit knows to handle that URL segment, so you don't need to do anything other than replace your existing ProcessPageEdit.module file and set your JS to post to the same URL as you were before, but with "/fields/" url segment appended to it.

Edit #3: found minor error in my code example above, as well as in the .module file, fixed both.

ProcessPageEdit.module

[apeisa]

Thanks Ryan, will test this right after our lego games end and kids are a sleep.

[apeisa]

This works nicely with text only fields. But we cannot send multipart/form-data posts with jquery/js. So it needs some adjustments on the backend to support this. But this got me leaps forward and I should be able to fork from this. Looking better all the time!

[ryan]

I'm not sure what needs to be done to support the multipart/form-data on PW module side? PW's forms are multipart/form-data already, but I think you are referring to something else. I don't know how the HTML5 upload script works exactly.    But if there is anything I can add, just let me know and I'll do it.

[apeisa]

There are security restrictions and it is impossible to do multipart/form-data posts in javascript. But it is possible to send files in File API and the backend side looks simple enough: http://www.sitepoint.com/html5-ajax-file-upload/

So not 100% sure about implementation yet, but I think we'll get there soon I try to fork myself and ask your help when needed.

[apeisa]

Ok, I got this working. Not sure how you manage saving files on pw when using admin. Do they got saved directly to assets/files/id/ folder or is there tmp somewhere?

I did just small if clause to your executeFields method:

<?php
public function ___executeFields() {

	// We check if it is image upload
	$filename = (isset($_SERVER['HTTP_X_FILENAME']) ? $_SERVER['HTTP_X_FILENAME'] : false);

	if ($filename) {
		$this->message("It is a file coming!");
		file_put_contents(  
			'C:/Temp/' . $filename,  
			file_get_contents('php://input')  
		);  
	} else {

		if(!$this->isAjaxPost) throw new WireException("This functionality may only be accessed from AJAX POST at present");

                        // etc etc...

}

And now I am able to save my files to C:/Temp/. Not sure about the best way to handle this from this situation? Save it to temp folder and use regular pw API and save the file? Or save it right to assets/files/id/ folder and then use API? I can easily send more data (like field name) as a http headers.

If we save it directly to right place (probably safe since this is admin usage), how I get the right path? $config->paths->files->3242 or something like that?

[apeisa]

Ok, I added three more lines for my last code snippet, and it is now working:

<?php
if ($filename) {
		$fieldname = $_SERVER['HTTP_X_FIELDNAME'];
		$this->message("Should be file upload");
		// AJAX call  
		file_put_contents(  
			'C:/Temp/' . $filename,  
			file_get_contents('php://input')  
		);
		$this->page->$fieldname->add('C:/Temp/' . $filename);
		$this->page->save();
	}

Of course we don't wanna keep that C:/Temp/ hardcoded there, but waiting for Ryan's reply on that issue. I will finish details and test this on all browsers that supports these goodies (hehe, there are only two - easy job ). Gotta say that this makes image uploading fun and fast And because TinyMCE image inserting (or file linking) works through iFrame-modal, then images/files added with html5 browser are available right away, without first saving the page.

Some more details need to be considered, like what if I drop 10 files, but field allows only one? There seems to be now backend check, so through API it is possible to add more files than field allows. I'm not sure but it might be wiser to move that logic to backend (if there is no room for new images, just skip)? Not sure tough... it wouldn't be that hard to add restrictions on client side also, like the current solution has (no more input fields appear after you have enough images).

EDIT: And for all who are interested, this is the js-code that sends the image:

var xhr = new XMLHttpRequest();
// This allows us to track progress, nice little progressbar coming!
xhr.upload.addEventListener("progress", this.ProgressHandler); 
xhr.open("POST", "./fields?id=" + page_id, true);
xhr.setRequestHeader("X-FILENAME", f.fileName);
xhr.setRequestHeader("X-FIELDNAME", fieldName);
xhr.send(f);

Nice and simple. And here is the great tutorial which helped so much in the progress: http://www.sitepoint.com/html5-file-drag-and-drop/

[ryan]

Glad that you got this working. Though admittedly, I still have no idea exactly what's happening on the front end or what it looks like. All I know is that it sounds good.

The ProcessPageEdit doesn't actually have any logic for saving custom fields, all that is delegated to the Inputfield modules. For instance, InputfieldFile. That module creates an instance of WireUpload (/wire/core/Upload.php) to retrieve the file and make sure all is good with the filename, file type, etc. I'm thinking that's probably where the logic should go (WireUpload class), because then it'll be available to all file upload fields.

The destination path for a given page's files can be found from the field it's going to, i.e.

$page->your_files_field->path; 

You can also get it without knowing the fieldname like this:

$page->filesManager->path; 

[apeisa]

Glad that you got this working. Though admittedly, I still have no idea exactly what's happening on the front end or what it looks like. All I know is that it sounds good.

I know I have tried my best to keep myself away to screencast this until I have something what I consider solid enough. I don't know why, but for me usually things slow down after you get first amount of feedback. I think that the feelgood makes you think that it is good enough already and soon you lose interest to polish the little details.

The ProcessPageEdit doesn't actually have any logic for saving custom fields, all that is delegated to the Inputfield modules. For instance, InputfieldFile. That module creates an instance of WireUpload (/wire/core/Upload.php) to retrieve the file and make sure all is good with the filename, file type, etc. I'm thinking that's probably where the logic should go (WireUpload class), because then it'll be available to all file upload fields.

Sounds like a right place. I think we need to test and play a little and decide how much validation we need here.

$page->filesManager->path; 

Thanks, that worked just fine!

I am now really finishing this (styling the upload bars etc). It's getting little late, so it might be tomorrow before I have something to show. Can't wait to share this one out.