Discussing: Save-as-draft feature

[marco]

Hi folks!

Despite of ProcessWire being one hell of a cms, we and our clients' editors stumbled upon one issue that is at least annoying and sometimes a major drawback that could lead to customers favoring another software for their projects:

In ProcessWire you cannot update an already public pages without simultanously modifing the page's public appearance. In short, we cannot save drafts, use editing copies ore any similar form of manipluating content that should not be published immediatly.

This is quite a common use case if you like to

- preview your changes before publishing (already addressed by the preview module])
- edit content in teams.
- want to schedule publishing of updated content for a future date.
- have to apply some kind of approval chain before publishing updates.
- just want to quit working for today and continue tomorrow.

Personally I think, this should possible a feature request for a future core release instead of a plugin module discussion. But plugin modules are all we have at hand for the time being.

### Reqirements ###

So let's start collecting a list of requirements:

- New logic is only needed when editing already public pages. No need to mess with the process of editing non-public pages.
- Opening a public page for editing:
- Instead of presenting the current public state of the page some kind of draft state has to be used to initialize the editing form.
- A draft state euqlas the page's state at the time of the last publishing - potentially modifed by preceeding save-as-draft actions of an editor.
- The draft state covers the sum of the field values of the page (existing of database content, files of image fields and maybe other forms of serialized data in case of more exotic custom field types being used).
- Additionally pages consist of a set of "direct" properties, such as templates_id, parent_id, status, and sort being the most important. Whether this kind of data should or must be part of a draft state is still to be discussed.
- Instead of one rather two different saving options must be offered to the editor: save changes and publish changes.
- Save changes
- When saving changes the public state of a page has to left untouched.
- The contents of the editing form must be serialized in a different way updating or replacing any existing draft state or creating a new one if necessary.
- Publish changes
- This option equals the normal publishing action on public pages as it is implemented right now (due to the editing form being initialized with the current draft state instead of the page's public state).
- Delete pages
- When deleting a page (entirely) any existing draft state has to be discarded as well.
- Preview pages
- We need to provide a new preview option (in contrast to the view option) that allows rendering the page's draft state instead of the public page.
- This function should be accessible from the editing view as well as from the page tree.

There might and probably will be more requirements, but I think this list covers the most common aspects of a save-as-draft feature.

As I'm writing this post, I already invested some time trying to implement a plugin module. Unsurprisingly I stumbled upon a bunch of obstacles leading to a halt and seeking help and opinions from the comunity.

I'll try to describe the general strategies I came up with for implementing the draft state feature:

### Cloning the page ###

My first attempt was to implement a cloning mechanism that duplcates a public page storing it in a hidden folder. The I tried hooking into ProcessPageEdit::execute() trying to replace the originl page object handdles by the module with my draft copy.

This approach came to a halt due to the following problems:

- The page object within ProcessPageEdit is stored within a protected property ($this->page) and referenced by all the methods of the module. Their is no direct way to manipualte that via a hook. I came up with some kind of dirty hack to circumvent that but this was far from a satisfying solution.
- Some parts of the editing form still had to show details of the original page (e.g. the breadcrumps, the children, the state), so the page object used to initialize the editing form had to be some kind of mix of the public and the draft state of the page.
- Most of this mixing state issues could be solved for the initializing. But when it comes to saving the draft state (not publishing changes) the need to divide the editing form's content in draft data and public data could not be solved (at least by me).

### Cloning field values ###

The second approach was to not clone the entire page but all the page's fields to represent the draft state. This would solve the issues with special treatments for form values for status and template and therefor should be more elegant and "light-weight" in our efforts to mess with ProcessWire's default behaviour as less as possible.

I tried hooking into Fieldtype::savePageField() and FieldtypeMulti::savePageField() providing special logic for storing field values in case of a public page being saved as draft. Additional the former mentioned hooking into ProcessPageEdit::execute() used to unserialize this draft field values and overwrite the page's public field contents.

The main problem I encountered was that there are more than this two types of FieldtypeXYC::savePageField() implementations (Repeater, I'm looking at you!). Every custom Fieldtype module can and often will come up with its own implementation of serializing values which would have to all be hooked into seperatly (correct my if I'm wrong on this one) and handled accordingly rendering a generic Save-as-draft module useless.

### Cloning the form state ###

So I tried a more direct approach, ignoring the existance of pages and fields completly, just looking at the editing form alone.

A page's draft state can be represented by the pure form state (meaning the list of form elements and their values). So save-as-draft action could possibly be implemented by intercepting the svaing process of the ProcessPageEdit module, grabbing the form state and serializing it instead of the normal routine.

Unfortunatly ProcessPageEdit::processSave() is a protected and therefor unhookable method. But even if that wwasn't the case, you still had to face a mix of the same problems that we encountered in the first two approaches (separating draft form values from public state values and the need of implementing different ways of serializing the form state for a multitude and mostly unknown list of different field types.
Additionally you would have to duplicate a lot of code e.g. form validation and handling file uploads that has been covered deep in the guts of the ProcessPageEdit module.

### Starting the discussion ###

So, this was a looong intro. And I already thank you if you're still with me.

If you have any thoughts regarding

- the core feature vs. module question
- the list of requirements I came up with so far
- my efforts in conecpting and implementing a plugin module
- or the need of this kind feature in general

drop me line (or two).

Regards,

Marco

[adrian]

Just a couple of related links, in case you haven't seen them:

http://processwire.com/talk/topic/2700-new-pw-project-notification-and-approving-page-changes/

http://processwire.com/talk/topic/2024-page-draft-module-useful-to-anyone-please-feed-back/

[Soma]

Thanks for this comprehensive writeup.

I wanted to mention that there's something planned for 2.5 http://processwire.com/about/roadmap/

I think it's challenging and not easy to tell how all this should work, especially if you're not aware of all the inner working of PW. Cloning has been discussed and there was some effort to try to find or implement something. There's, as you mention a lot to consider and walls you will be facing.

- The page object within ProcessPageEdit is stored within a protected property ($this->page) and referenced by all the methods of the module. Their is no direct way to manipualte that via a hook. I came up with some kind of dirty hack to circumvent that but this was far from a satisfying solution.

That's true but there's ProcessPageEdit::getPage() that you can access the currently edited page, or using the GET id.

I tried hooking into Fieldtype::savePageField() and FieldtypeMulti::savePageField() providing special logic for storing field values in case of a public page being saved as draft. Additional the former mentioned hooking into ProcessPageEdit::execute() used to unserialize this draft field values and overwrite the page's public field contents.

I think Fieldtype::sleepValue() would be maybe a way to hook into values of fields before saved to db. Inputfield::processInput being another that is used to process values before going to sleepValue or sanitizeValue (I think).

Not sure what really would be the way to go. PW is flexible and maybe you build content of pages from different approaches than just on one page. I can't think of reliable ways to handle all types of setups, custom fields and it's varying data. Maybe there would be a way to define some interface that fieldtypes or modules can/should implement to handle custom stuff, but I'm not seeing through this and just randomly thinking.

[marco]

Thanks for your replies.

After looking at the other topics mentioned, I think it might be best to wait for the "official" solution in PW 2.5.

For the time being I will spend some additional time thinking about the problems I faced so far. I'll let you know if I've stumble about a solution or at least some good ideas.

[ryan]

ProcessWire 1 had built-in live drafts and versioning. I was particularly proud of these features, but I was always surprised at how little they were used. It turns out that most end-users don't actually write content in the CMS. They tend to go to the CMS when we are ready to publish something. This doesn't define everybody, but I think it defines most CMS end-users. At least that was my experience, repeatedly. As people building sites and applications, I think we tend to assign more importance to these things than they deserve because we ourselves are much more likely to use the CMS as a composition tool than our clients are. This is why live drafts (drafts of already published pages) have not been high priorities as yet in PW2. But there is that small group of clients that these features are actually quite important. That's why this is something that I think belongs on the roadmap. the strategy I'd planned to take was to clone the live page to a new but unpublished page that would become the live draft. It would have an extra field (behind the scenes) indicating that it's a variation of another page. When the editor wants to publish it to be the live page, it would push the unpublished page's data to the live page's data, and then delete the unpublished page. This is important because the original page ID needs to remain in tact. That pushing of data gets a little tricky when it comes to some fieldtypes (like repeaters and files) but that's where the fun will be. 

Unfortunatly ProcessPageEdit::processSave() is a protected and therefor unhookable method. 

Side note, but protected methods are still hookable. The only methods that aren't hookable are those that aren't preceded by 3 underscores. 

[apeisa]

I feel the same way. Many times these features are "required features from cms" when organisations are buying a new site. Then afterwards many of them actually never use these features. It's also countless time I have build many kinds of content aggregators, where "we really need to filter/sensor/schedule content". I say them that I believe you never use these features, are you sure you want to buy custom development here. They always want and after a year when I check the site they have never used these features.

I guess it is mostly for "feeling for control". Also these things create a fabulous demo.

What comes to other way around: retrieving history, restoring removed pages etc. These are rarely used things too, but they actually make updating feel more secure and when build well - they lighten the support burden.

[cmscritic]

I'd have to say the auto saving of drafts is rather huge for me, specifically. My whole crew writes directly within the CMS and I can't count how many full articles I've lost due to not clicking save and keep unpublished. It's a huge pain.

[Soma]

That's why I made a module long ago "FormSaveReminder". It's still beyond me why this feature isn't yet in core.

[ryan]

That's why I made a module long ago "FormSaveReminder". It's still beyond me why this feature isn't yet in core.

When I originally looked at this, it worked with some fields and not others (like rich text fields?). TinyMCE has such a plugin too, but it regularly results in false positives, which get to be a real annoyance. I don't like adding stuff to the core unless I know it's going to work universally across any field, triggering when it should, and not triggering when it shouldn't. Otherwise I know I'll get regular bug reports about this or that field not working with the functionality. Another factor is that I've never personally felt the need for it, nor has it ever come up with a need with clients, until Mike mentioned it above. So I've always seen this as a feature that would serve as a big annoyance for myself and many others. But if it's something that at least 30% of users would use, and it can be done in a way that always works, then I think it would make sense as a core module and I'd be happy to include it. That's so long as it's something people that don't want it can leave uninstalled. But in lieu of these conditions, I think it makes a lot of sense as a 3rd party module and I'm thankful that you've created it. It's been awhile since I've tried it, so I'll have to experiment with it again. 

[MatthewSchenker]

I'd have to say the auto saving of drafts is rather huge for me, specifically. My whole crew writes directly within the CMS and I can't count how many full articles I've lost due to not clicking save and keep unpublished. It's a huge pain.

I'm curious: how are articles being lost? Are people navigating to another page without saving? It seems to me there is a minimum of responsibility for anyone creating content, which would include saving your work. Or is there another problem here?

Thanks,

Matthew

[Joss]

Just picking this up again as someone has just asked me about this and I saw this post:

http://processwire.com/talk/topic/5236-admin-session-logs-out-when-wireless-connection-has-hiccup/

I think the use of CMSs for drafts will increase simply because the idea of "working in the cloud" is now the new "in thing." Several of my clients (music clients, mostly) use Google Sites for a lot of their project work, and they are very used to the fact that is saves every so often, quietly in the background, with only a small notification appearing off the work area.

Also, ProcessWire has the ability (as has been shown by others) of being used as a web application for everything from help desks to sales and CRM to general intranet functionality. In these scenarios, perhaps when you are conducting a live interview on the phone and typing into a PW application, some sort of autosave or draft autosave is essential to save embarrassment or possibly the job.

General autosave would be easy enough - the page does not need to be refreshed as you are still working so it can just save in the background. This is fine for a page that has yet to be published (autosave should respect the current state), or for something like a client record or help desk record where there is never a public viewable version as you would have with a blog, for instance.

Autosave for updating an existing published article is a little more complicated and probably should be part of a page version system.

From my point of view, I would see this as just being saved as simple hidden child pages with some sort of specialised ID that shows that they are directly related to the page for versioning rather than as children for other purposes. Clicking Save as Draft or the Autosave would save to the child. Clicking Publish would save to the main page. 

[teppo]

About a week ago I wrote a blog post while on train and some odd hiccup -- not sure if it was connection issue or just something I did wrong as I was using a tablet, which is far from perfect tool for this job -- but the whole post simply disappeared.

Needless to say, I wasn't very happy. Sure, I should've saved more often, so it was really my mistake, but that was still an example of a situation where autosave would've saved the day (quite literally.)

My needs are quite simple so I'm currently thinking of writing a very simple module for that purpose alone, but it's good to see that others are toying with similar ideas here. My issues are pretty far from a full-featured "save as draft" solution, but still

[Soma]

I'm already on a proof of concept thingy as there's already a saveAjax feature in ProcessPageEdit.

Though I also think what's so hard to write an article in a text-editor, or just hit "save" once in a while when you write an article that takes time.

[MatthewSchenker]

Greetings,

I am also working on several ways to save "drafts." With ProcessWire, there can be numerous ways to do this. But the methods that I am doing require the user to choose to save (i.e., click a "save" button).

Generally, I think web devs are perhaps going a bit too far with everything being "auto-whatever.". As we know, there are entire libraries whose main purpose is to make it so users don't have to hit an "enter" button. Really? Some of this is OK, but I think there's value in creating an atmosphere where users realize their responsibilities.

Thanks,

Matthew

[Joss]

I think it depends what you are writing.

Since part of my job is as a writer, I know that you can get very much into the creativity of the words sometimes and stopping to hit save on a regular basis really interrupts the flow. 

When working in something like Word, a simple Ctrl S can be very easy, but on a form, you need to scroll to the save button, mouse to it and click it, then wait for the form to reload.

Personally, when on a desk top, I prefer working in Word and then copying and pasting into a form where there is a large single field - for instance when working on a blog post. But that does not work so well when you are working on a form with loads of fields, or want to put something straight into a form from your mobile and so on.

I suppose in the end, this is all about making the user experience more streamlined and trouble free, and also reflecting what they are used to using in other instances.

EDIT:

Oh, and getting closer to the wonderful pencil and paper - no save button there!

[Soma]

When working in something like Word, a simple Ctrl S can be very easy, but on a form, you need to scroll to the save button, mouse to it and click it, then wait for the form to reload.

You obviously don't know my AdminHotKeys module.

[teppo]

Though I also think what's so hard to write an article in a text-editor, or just hit "save" once in a while when you write an article that takes time.

Not hard, just inconvenient.

CKEditor in inline mode is comfortable enough to finally make writing with RTE in browser comparable to writing in text editor. Having one go-to tool instead of two feels slightly better. This is very much a question of preference, so there won't be any "correct answers" here -- some users prefer to write at Admin and others will probably always write with Word, Notepad, Notepad++ etc.

AdminHotKeys is only partial answer to saving issues mentioned here, unless I'm missing something: it still requires reloading page, which (especially when working with crappy connections) can a) take ages and b) result in timeout and lost data if you're not careful. Isn't that exactly why we're discussing background save methods here?

One more thing to note is that any autosave / background save feature will also need to handle exceptions (such as being momentarily disconnected) properly, otherwise they could even make things worse; user must know what's happening, when last successful save occurred, whether there are connection issues etc. Basic stuff, but very important nevertheless.

[mackenzie]

Hi there, sorry to resurrect. Brand new to PW and about to start a project where having 'save as draft' ability is necessary for team workflow. Glad to hear that it's on the roadmap for 2.5, but just want to confirm there is no published module that can perform this functionality in the meantime? Perhaps since this discussion was last updated...?