Prevent saving a page if it was changed in the meantime

[MoritzLost]

I'm trying to find a solution to a problem with parallel editing. We have a site that contains some datasets (pages) that can be edited manually, but may be changed programmatically as part of some internal processes as well. The problem arises if a programmatic process changes some values of a page while an editor is manually editing the same page (controlling when those automated changes happen is not an option, in this case). Once the editor saves the page, the programmatic changes will be overwritten. Sounds like an edge case, but because of the way the processes are structured, this could occur regularly.

I'm looking for an elegant solution to this problem. So far I have a couple of ideas:

• Build a custom module that saves the current timestamp when a user starts to edit a page, and once the editor saves the page prevents it if the page was changed in the meantime. Not ideal because in this case the manual changes would be lost.
• Use Version Control so that IF this problem occurs, the programmatic changes can at least be restored from a revision. Not ideal because in this case the editor has to (a) notice this has happened and (b) merge the two versions manually.
• Store the programmatic changes in a draft using the ProDraft module. I'm not sure this will work at all (we haven't bought the ProDrafts module yet), and it's not great because any programmatic change would have to be approved manually ...

I'd appreciate any insights or suggestions, either on how to improve the approaches above or a better solution. If there are any modules that can handle this I'd be happy to hear about them as well. All input would be welcome. Thanks!

[rick]

Doesn't that pro dev tool thing have a feature to show user activity. Couldn't that be intercepted to accomplish your goal?

I'm just thinkin' out loud.

[MoritzLost]

The User Activity module of the Pro Dev Tools does this, but as far as I can tell from the documentation this only works if both editors are human. We don't have that module, so I can't check right now. Hm, maybe the module can be hooked, or automated activity can somehow be registered as an active editor ... that would be great. Though getting this to work may be more work than a custom solution ^^

[horst]

In the documentation or somewhere else bound to the User Activity module you can read that the core Notification System already has some bare features available, like detecting multiple users or tabs with the same opened pages. Maybe looking into it can reveal some useful bits. ??

[MoritzLost]

9 minutes ago, horst said:

In the documentation or somewhere else bound to the User Activity module you can read that the core Notification System already has some bare features available, like detecting multiple users or tabs with the same opened pages. Maybe looking into it can reveal some useful bits. ??

I'll look into it, but I'm not sure this applies, because in this case the editing conflict / collisions is not between two users, but one user and one website feature that changes values programmatically, so anything that's based on user sessions or the editing page will not work ...

[horst]

Ok, when you want to hide the merge or collision handling away from the editors, you have to make the auto process aware of opened pages, and maybe the opened one should be excluded or queued for later processing.

Or:

Is it right that the auto process modifies not the same fields that editors do? If so, maybe it is possible to hook into those page saves and then only save content to fields allowed by user roles? 

Foreach allowed field setAndSave

(Sorry, on mobile)?

[kongondo]

2 hours ago, MoritzLost said:

Build a custom module that saves the current timestamp when a user starts to edit a page

Timestamps can a be a tricky source of truth in some cases. What if the editor and the script started at the same time? Speaking of which, which process is your single source of truth? The editor or the script? I mean, when there is a conflict, whose changes do you want to accept?

I was thinking of something along the lines of what @horst suggested, i.e. separate fields for the script vs the editors. It's hard to get away from a manual intervention if there are conflicts. Given the situation you've described, I'd be very surprised if you manage to avoid manual oversight in case of conflicts.

[Robin S]

A few more ideas...

1. If the fields that are updated programmatically are only a subset of the fields in the template then you could give those fields special treatment in Page Edit. So if fields A, B and C may be updated programmatically but fields D, E and F are not then in Page Edit DEF will be rendered normally but ABC could be rendered "Closed + load only when opened (AJAX)". Or (trickier) rendered read-only via renderValue() with a button that loads them on demand via AJAX if the editor needs to change the value. The idea here is that the values for ABC are only present in the POST data if the editor has needed to load and change them.

2. In conjunction with your first idea, when your script sets field values have it also store the changed field names and update time in the $page->meta of updated pages. Hook before processInput and remove any clashing field values from POST. Use $session->warning() to alert the user if any values that they changed manually were discarded (the programmatic changes will always take priority because even if the clash doesn't happen during editing they can potentially occur one second after a user saves a manual value).

3. Store changed field names in $page->meta as per idea 2, but use JavaScript in Page Edit to regularly poll and see if any field values have been changed programmatically. If so, remove the inputfield from Page Edit and alert the user to explain. Not as bulletproof as 2 because the programmatic update could occur between polls.

 

[MoritzLost]

@horst @kongondo @Robin S Thanks for all your input! Sorry my description is kind of vague, this is about business processes for a client project, and I'm not sure how much of those I can disclose ...

Quote

Ok, when you want to hide the merge or collision handling away from the editors, you have to make the auto process aware of opened pages, and maybe the opened one should be excluded or queued for later processing.

I don't need to hide the collisions; in fact, if the editor is notified and can manually merge both versions that would be fine. But I'm not sure how to merge both versions because ...

Quote

Is it right that the auto process modifies not the same fields that editors do? If so, maybe it is possible to hook into those page saves and then only save content to fields allowed by user roles? 

Quote

Timestamps can a be a tricky source of truth in some cases. What if the editor and the script started at the same time? Speaking of which, which process is your single source of truth? The editor or the script? I mean, when there is a conflict, whose changes do you want to accept?

Normally, the automated process fills most fields, and the editor changes some fields that are not written to automatically. But the editor may change ANY field that was changed programmatically to correct errors or add new information, so simply limiting the fields in the edit page or discarding edits for certain fields will not work. In case of a collision, the best result would be an automatic merge between both versions, with the manual changes having precedence if both versions have changed the same field. But that would be really involved, and I'm already not sure how to distinguish fields that were changed by the editor and fields that they didn't touch but ProcessWire regards as "changed" because their value was changed by the automated process in the meantime. I think this line of throught is already too complicated ?

Quote

1. If the fields that are updated programmatically are only a subset of the fields in the template then you could give those fields special treatment in Page Edit. So if fields A, B and C may be updated programmatically but fields D, E and F are not then in Page Edit DEF will be rendered normally but ABC could be rendered "Closed + load only when opened (AJAX)". Or (trickier) rendered read-only via renderValue() with a button that loads them on demand via AJAX if the editor needs to change the value. The idea here is that the values for ABC are only present in the POST data if the editor has needed to load and change them.

Interesting idea, this will probably reduce the amout of collisions a lot. Though it might be more work, and of course having every field AJAX loaded is not very nice from a usability perspective ...

Quote

2. In conjunction with your first idea, when your script sets field values have it also store the changed field names and update time in the $page->meta of updated pages. Hook before processInput and remove any clashing field values from POST. Use $session->warning() to alert the user if any values that they changed manually were discarded (the programmatic changes will always take priority because even if the clash doesn't happen during editing they can potentially occur one second after a user saves a manual value).

That's creative! I think this solution is the one to be. I like that it results in mostly a clean merge, and I could differentiate between "editor-owned" and "process-owned" fields to decide which version to drop in case of a collision. Hmmm, maybe I could even add an extra read-only field that get's populated with all the dropped values for restoration / copy-pasting purposes. Thanks for the suggestion!

Quote

3. Store changed field names in $page->meta as per idea 2, but use JavaScript in Page Edit to regularly poll and see if any field values have been changed programmatically. If so, remove the inputfield from Page Edit and alert the user to explain. Not as bulletproof as 2 because the programmatic update could occur between polls.

Probably confusing to the editor if fields are disappearing left and right ? But maybe I could just use JS polling to check for changed fields and warn the editor not to save the page – this way, they could open the edit page in another tab and manually copy over their edits. Not perfect from a usability perspective, but a good compromise to avoid data loss.

Thanks for all the suggestions from all of you, I'll probably go with some sort of tracking timestamps of field changes to abort / warn about collisions. I'll report back once I've implemented a solution ?

[kongondo]

On 6/22/2020 at 11:15 AM, MoritzLost said:

I'll probably go with some sort of tracking timestamps of field changes to abort / warn about collisions. I'll report back once I've implemented a solution

Or if you really want to annoy the heck out of 'em...you can go with something this silly ?