24 replies to this topic

[Rob]

I'm sure this must have come up before in the forums but I couldn't find anything.

It would be nice to have a history of page version and rollback / rollforward ability. If a user edits a page and wants to preview their changes before going "live", unless I am mistaken this isn't currently possible. You could mimic this behaviour but it wouldn't be a clean workflow.

I was thinking about the repeater functionality and this essentially seems to re-utilise existing page behaviour. Perhaps it would be possible to utilise a similar approach by creating "mirror" copies of pages under some /admin/versions location in the page tree and use this as a storage for page versions. Theoretically it would then be possible to preview versions, rollback etc and still use core PW functionality that is already in place rather than having to write a whole mass of new code.

I am happy to have a crack at this myself but I thought if I put the idea out there the community will likely have much more advanced module coders than me who may be interested in the idea.

To recap:

If we could have a save->preview->publish workflow by saving copies of pages under a "hidden" location in the page tree then it would create the ability for users to safely make edits, check them etc before going live, and also revert to old version if necessary.

Any takers?!

[interrobang]

In this thread Ryan said a history module will be part of the next major release:
http://processwire.c...history-module/

[ryan]

Actually it's not going to be in 2.3, but it is a long-term work in progress. Versioning that is respectful of resources is rather complicated, especially when it gets into things like automatically created versions with file-based assets, repeaters, and the like. The system I'm working on keeps the version data outside of the pages system and page tree. But I'm a long way off from figuring out all the details.

[apeisa]

I think it would be great to have some basic versioning, even just for textfields would be very beneficial. I know it goes greatly more complicated with files and repeaters, but most (or at least big) benefit comes from just text versions. Page history module is also first item on roadmap, so many people might have high hopes for it: http://processwire.com/about/roadmap/ (if it's not coming for 2.3 then maybe to strike from list and move to 2.4+ part of that list).

Also - would it make any sense to have field history instead of page history? Would it be any simpler to implement that way?

[ryan]

The page history module may yet make it into 2.3, but not as a full blown versioning system. However, it may be able to do some of what you mentioned with text fields. But the point of the page history module is more about seeing what changed, when, and by whom. Though tracking this info crosses over into the territory of versioning in many respects. I guess I think of full blown versioning as including all page data (files, repeaters, etc.), but it's a difficult prospect without actually making clones of the pages themselves (like Rob mentioned). Maybe I should stop trying to think about these things and focus on the stuff that can be versioned efficiently. Your idea of a field history is compelling, I need to give some more thought to it.

[Pete]

I keep meaning to mention something I've seen in a CMS I used about 8 years ago (feel old now) that was pretty bloated.

They had a cache that basically cached the parsed field contents for a page into a text file. Versions were simply stored as that page name with a date stuck on the end as the filename, and the system fed through the latest version and handled it that way, keeping X versions.

It was simple but it worked and didn't put unnecessary amounts of data in the database that way.

Probably not the route we want to go, but I'm wondering if we might be trying to over-complicate versioning - there's certainly a worry that it could add some serious overhead to the database even for the slightest tweak (probably why Apeisa's idea of versioning fields that have changes rather than the whole page sounds appealing).

[ryan]

That's not far off from what I'd been working on with versioning, which was to store page versions in flat JSON encoded strings in a DB table (though could just as easily be stored as text files). Any ProcessWire page can be reduced to a PHP array of basic types (strings, integers, floats, arrays), which means it can be reduced to a JSON string really easily. This lends itself well to version storage, except for assets where only references are encoded (like files and page references).

But I have to admit that the more I think about Antti's proposal to store this at the field-level, the more Iike it. I would create another interface that could optionally be added to a Fieldtype's implements section in the class definition (FieldtypeWithVersions or something like that), and give the individual fieldtypes responsibility for if, how and where they store their versions. That would enable us to get up and running with versions quickly (for text fields at least). It would also seem to provide a nicer level of control to the user, as you don't have to keep track of what is changing in multiple fields at once. And, it would let you to enable versions on fields that you want it, omitting on field where you don't, making things more efficient. Lastly, it would leave the page-structure out of versions completely (where the page lives in the tree), which struck me as a potentially dangerous problem with versions. It could work something roughly like this:



You hover over the revisions label and get a summary of recent revisions. You click on one, it could pop up a modal showing you the revision (maybe with a compare tool) and a button to revert to it or cancel. It just seems like a really easy-to-use solution for the user.

[Pete]

I really like the sound of that.

I would still suggest putting in an option to revert the whole page to a specific date as well, just because that's what people are used to with version control, but I like this method a lot as there's no other system where you could really do it this way without it being really difficult.

I'm not saying it would be easy this way, but since all fields are tables already it sure makes it a bit simpler than it could be.

[Pete]

In fact, wouldn't a very basic way of doing it be to simply have a version, author and datestamp field added to every field table? Or am I over-simplifying things? Could perhaps handle file/image field (and others) versioning simply that way too.

That would add some overhead though I guess on larger sites by leaving versions in the same tables though now I think about it. Oh well, I'm sure you'll come up with a good solution.

[teppo]

@Pete, I'm afraid that you're absolutely right with that point about larger sites, so I wouldn't suggest using same table for version data either

Clearly Ryan has different path in mind already, but I just wanted to throw in this article about MySQL versioning I read some time ago and found quite interesting: http://www.jasny.net...ing-mysql-data/. The idea there is to utilize MySQL triggers and that way move (most of) the versioning logic to database layer, which IMHO is a great idea. Naturally there are some drawbacks with this method too:

"There are some situations where this solution as a bit to basic. A record might span across multiple table, like an invoice with invoice lines. In that case, we don’t want to revision each individual invoice line, but the invoice as a whole." (etc.)

Anyway, just throwing in some food for thought

[ceberlin]

I am also in a need for a feature like that to convince a customer to do the switch.

 

I got used to it with Drupal 7 ("diff"-module) and did not experience it slowing down things, or to be über-complex.

http://drupal.org/project/diff

Cool stuff. It wasn't too complicated so, a.f.a.i.k. it is limited on node text and title fields. You could easily navigate the versions, see all (text) differences highlighted and could roll back (which means that a new version is created with the content of the selected older version, so you could even revert that change later).

Anyone knowing this module? Maybe this is a route a processwire module could go as well.

[teppo]

Originally inspired by a comment by Pete on Process Changelog thread, I've been playing with (and just pushed to GitHub) an experimental version control module for text based fields, which does some of the things discussed here. Somewhat coincidentally it also bears quite a bit of resemblance with the mockup Ryan posted above UI wise..

 

 

It's not production ready, only supports storing content to database (though adding another mechanism for storing the bulk of content wouldn't require much work and most likely will get added soon) and currently only supports Text and Textarea fields. I'm looking into this subject more closely once I find some fr-e time for it, but in the meanwhile anyone interested can check it out, use it, fork it etc. as long as you're aware of the fact that it's more of a proof-of-concept than anything else and that it's far from a perfect solution in more than one way.

 

Regarding Drupal 7 diff module posted above by @ceberlin, this module doesn't provide that kind of features at the moment, though it does store all necessary data (and a bit more) to enable those at some later point. I was originally planning to only store diff data, but since PHP doesn't have native method for doing that I ended up storing full content on each edit. Not very efficient, but for small-scale use (or proper limits, to be added later..) it should be good enough (for now.) I've also omitted many other features, such as those mentioned by Ryan above (modals etc.), for the sake of simplicity and feasibility.

 

Anyway, if anyone is interested to try it out I'd be very happy to hear any comments on this one

[apeisa]

Mr. Log is on fire!

[ryan]

Teppo, this is really a fantastic proof of concept! Very well put together and seems very much fully-functional to me. Worked great in my testing here. It actually seems like much more than just proof of concept--it's quite stable! 

 

Thanks for your great work here. I look forward to seeing this evolve. Let me know anything I can do to help. This is a great addition and perhaps something that should find it's way into the core. 

 

A couple of minor optimizations to mention, at line 263 of the main module: 

// $page = $this->pages->get((int) $this->input->get->id);
$page = $this->page->process->getPage(); 
// if (!$page || !in_array($page->template->id, $this->enabled_templates)) return;
if(!$page || !$page->id || !in_array($page->template->id, $this->enabled_templates)) return;

[teppo]

Thanks Ryan! Just pushed those optimizations to GitHub.

 

Regarding development of this module in general, to be honest I don't have a very good plan right now. Other than making minor improvements here and there, I've planned adding some basic features such as proper cleanup of old entries, option for saving actual content to files on disk and possibly a JavaScript-based diff feature etc. Anyway, if you have any ideas what should be included or to what direction this module could move in order to benefit more users, I'd be more than happy to hear your opinions.

 

For an example I wasn't originally planning to support anything other than basic text fields, but support for images/files would definitely be nice addition at some point. Problem is that it would also add quite a bit of complexity to the module (perhaps that should be another module entirely?) and in the worst case enabling a feature like that could end up consuming a lot of disk space without user even realizing it. That's one idea I'd love to take further, but it will clearly require proper planning first..

[apeisa]

I can't seem to get it working on my local linux test box. Pretty much default installation. Templates and field selections are empty. "Enable for these fields" field first allowed for selections, but after revisiting the module page those selections were gone and nothing could be selected anymore. "Enable for these templates" was always empty. 

 

Fieldtype selection has these options available and nothing selected:

 

FieldtypeFieldsetClose

FieldtypeFieldsetOpen

FieldtypeFieldsetTabOpen

 

Both database tables remain empty.

 

[teppo]

Thanks for reporting this, Antti. I managed to reproduce this by downgrading my test ProcessWire installation from 2.2.13 (though I've upgraded some files independently, so this might not be 100% correct number) to 2.2.9. It was a selector issue at config method.

 

Could you try if updating the module to latest version available at GitHub (0.0.2) fixes the problem for you?

[ryan]

Regarding development of this module in general, to be honest I don't have a very good plan right now. Other than making minor improvements here and there, I've planned adding some basic features such as proper cleanup of old entries, option for saving actual content to files on disk and possibly a JavaScript-based diff feature etc. Anyway, if you have any ideas what should be included or to what direction this module could move in order to benefit more users, I'd be more than happy to hear your opinions.

 

I think that all the things you mention sound good. A few questions comments though:

• What would be the benefit of saving content to disk (vs database?). I'm not sure that it really matters to the user where it is stored, so wanted to inquire more about your thoughts here. 


• What would define old entries? I'm guessing in some cases, people would like to just let it go forever (disk space is cheap).


• Javascript diff feature sounds awesome. Though also have to admit, just being able to toggle between the different versions and see the immediate change (the way you have it working now) is kind of a nice "diff" effect too. You don't necessarily need anything else for a version 1.0. 


• How does it scale? Meaning, what happens when you've got 100 versions. I haven't tried it yet... and maybe you've already figured this out. But I was thinking maybe it shows the most recent 10 edits when you hover the icon, and ajax/paginates them somehow after that? (or opens a modal to a dedicated Process when you click more?)


• How does one handle deleting versions? I was thinking it doesn't need to be manual or interactive, but just a global time or quantity setting, i.e. "only keep last 50 versions" or "only keep versions for [n] days" or something like that. But having the option to keep them forever is also good… perhaps the behavior when the "[n] days" is left blank. 

For an example I wasn't originally planning to support anything other than basic text fields, but support for images/files would definitely be nice addition at some point. Problem is that it would also add quite a bit of complexity to the module (perhaps that should be another module entirely?) and in the worst case enabling a feature like that could end up consuming a lot of disk space without user even realizing it. That's one idea I'd love to take further, but it will clearly require proper planning first..

 

Just supporting text fields for a 1.0 version seems ideal. This probably covers the vast majority of needs. Versioning of files/images sounds fun, but you are right that it's an entirely different task on the development side, since it has to manage files. And these fields aren't just files, but sort order, description, tags… and more in the future. Probably too much work for too little value here. So if it were me, I would just focus on those text fields. I think that the vast majority of versioning needs for files/images could probably be covered just by a file "trash" (whether global or page specific) where one could retrieve old files if they ever needed to… but that would be a different module. 

 

To summarize my thoughts: you've already got something great here that is already hugely useful. I'm not sure what more you need to take it beyond proof-of-concept (seems quite functional as-is), but the only thing I would consider is just making sure it can scale time and quantity. And then get version 1.0 out when ready. I think a lot of us can't wait to start using this. If there is anything that I can do to help (code, testing, etc.), I'm at your disposal. 

[Joss]

What would define old entries? I'm guessing in some cases, people would like to just let it go forever (disk space is cheap).

 

Just a thought about legalities, on certain sites there may be either a legal requirement or necessity to keep old versions for ever.

 

For instance, records about users may be required to kept so that changes in user details are recorded, and news sites may want to keep old versions in case of legal disputes over the accuracy of published information.

[apeisa]

Teppo: settings screen works now, but when editing page I get this error: 

 

Error    Call to a member function getPage() on a non-object (line 269 of /home/apeisa/Apache/Roskis/site/modules/VersionControlForTextFields/VersionControlForTextFields.module)