Module Settings Import / Export

[adrian]

Hi everyone,

This is the new official thread for the module that was previewed some time ago here: https://processwire.com/talk/topic/14117-module-settings-import-export/

Big thanks to @Robin S for help testing and feature suggestions!

http://modules.processwire.com/modules/module-settings-import-export/

https://github.com/adrianbj/ModuleSettingsImportExport

 

Module features

• Ability to copy and paste settings from one PW install to another
• Optional automatic backup of module settings on uninstall and ability to restore settings if you reinstall the module
• Backup current settings at any time
• Restore backed up settings at any time
• Import option checks module name and version number and warns if importing settings from a different version

As always, let me know if you find any problems or have any suggestions!

[gmclelland]

Hi @adrian, thanks for releasing this module.  I really like the concept.  I can see this being installed on every site.  I was able to copy over my TracyDebugger configs and AdminOnSteroids configs successfully.

It may be premature to ask, but is there a way to copy the configs of multiple modules at one time like you can do when you export your field settings?

[adrian]

2 minutes ago, gmclelland said:

Hi @adrian, thanks for releasing this module.  I really like the concept.  I can see this being installed on every site.  I was able to copy over my TracyDebugger configs and AdminOnSteroids configs successfully.

It may be premature to ask, but is there a way to copy the configs of multiple modules at one time like you can do when you export your field settings?

This is really the domain of 

although that module does still need a little attention.

That said, there might also be a place for it in this module - let me think on this a little more.

[gmclelland]

I was just reading about that module when you posted.

I just tried Module Toolkit...Wow what a time saver!  Thanks again adrian.

 

 

[tpr]

@adrian haven't you think about merging the two Module-related modules?

[adrian]

4 minutes ago, tpr said:

@adrian haven't you think about merging the two Module-related modules?

Yeah, I have and yesterday I decided not to because I didn't want to bloat this will all the tools available in Module Toolkit, but now I am second guessing that decision especially given that Module Toolkit is mostly a Process module so if you don't run it, it's not doing anything.

I'll think on this a little more - stay tuned

[adrian]

Some new features just added. Yes, these are starting to get into the functionality provided by ModuleToolkit, but I am still playing with the user interface and workflow trying to find the most efficient scenario. Along with the interface that is added to each module's settings page, this screencap shows the new interface available from this module's settings page. It shows a list of all stored module setting backups and adds a new interactive interface for selecting modules whose settings you want to copy to another site. As you check/uncheck modules, their settings will be added to the Settings textarea ready to copy to another site. If you paste in settings from another site and check "Update All", these settings will be imported into this site. You can choose to be warned of version mismatches, or force the import anyway.

Let me know what you guys think - am I on the right track with this, or is the ModuleToolkit approach better, or is there room for both? I still may combine the functionalities of both modules, or maybe even create a service that lets you manage your modules and their settings across multiple sites from once central dashboard.

[cstevensjr]

3 minutes ago, adrian said:

 or maybe even create a service that lets you manage your modules and their settings across multiple sites from once central dashboard

I would be interested in this service.

[dotnetic]

I think the two modules should be merged, as they both serve module specific tasks.

[adrian]

On 5/9/2017 at 8:27 AM, jmartsch said:

I think the two modules should be merged, as they both serve module specific tasks.

I might do that sometime when I have more time.

On another note, I have just fixed the automatic backup of settings on module uninstall feature. Apparently it wasn't working properly  Anyway, I think it's really useful so you know you can easily restore settings after an uninstall - very useful for some of the more complex modules out there.

[adrianromega]

Hi

Thanks for this nice module, you have created so far very useful modules, thank you again for all your contributions to Processwire.

One thing I noticed, this module does not work (modules not displayed) for modules who are not implementing the ConfigurableModule interfacce but are using the new module configuration described in here. https://processwire.com/blog/posts/new-module-configuration-options/

[adrian]

40 minutes ago, adrianromega said:

Hi

Thanks for this nice module, you have created so far very useful modules, thank you again for all your contributions to Processwire.

One thing I noticed, this module does not work (modules not displayed) for modules who are not implementing the ConfigurableModule interfacce but are using the new module configuration described in here. https://processwire.com/blog/posts/new-module-configuration-options/

Hi @adrianromega - thanks for the kind words and also for reporting this. Please test the latest version and let me know if that works ok for you. I also fixed a bug with the "Backed Up Settings" section on the config settings page for this module - where you can get the settings for all modules.

[adrianromega]

yes, it is working as it should now.

thank you for the quick fix and for your devotion for this superb project called Processwire, 

[Robin S]

Hi @adrian,

Could you please make a small change in ModuleSettingsImportExport.js to avoid an undefined variable error in PW 2.x? On line 11:

var allModuleSettings = typeof ProcessWire !== 'undefined' && ProcessWire.config ? ProcessWire.config.allModuleSettings : config.allModuleSettings;

I think this might be all that's needed for PW 2.x compatibility (although I haven't tested extensively).

Thanks.

[adrian]

44 minutes ago, Robin S said:

Hi @adrian,

Could you please make a small change in ModuleSettingsImportExport.js to avoid an undefined variable error in PW 2.x?

Thanks @Robin S - I have made that change along with some code cleanup and making all the module setting strings translatable.

[MarkE]

Hi @adrian. I just tried to use this module for the first time and hit a snag. If a config field references a page, then the module uses the page id as the value. Fair enough, but it does not work if the target environment has different page ids (which may be the case if your development database is not an exact clone of the production database - in which case you probably don't need this module anyway).

In my ProcessDbMigrate module, I get round this by converting ids to paths (and providing a mapping if the paths have changed - which is far less likely). I was going to add a module migration component to ProcessDbMigrate, but then came across this module which I thought would do the trick. Is it possible to modify ModuleSettingsImportExport, or am I best to go ahead with my original plan and extend ProcessDbMigrate, in which case, can I 'borrow' a bit of code from your module (with accreditation)?

[adrian]

Hi @MarkE - great point - I haven't come across this, but that's probably because at least in Tracy I have the module itself store page references by path rather than ID.

But, I think it would definitely make sense to modify this module to support conversion to paths, but my question is how? I don't think we can realistically check the type of data being stored and know for certain it's a page ID or array of IDs. Sure we could make the assumption that an numbers with 4 or more digits is a page ID and then do a basic check to see if it matches a page. This might be enough, but to make it more robust I think would require parsing the code in the module's files looking for the key in the module settings JSON and determining what type of field it is. I think this could work, but seems a little nuts :)

Any thoughts?

[MarkE]

Hi @adrian. Just coming back to this -

On 4/10/2024 at 10:11 AM, MarkE said:

it does not work if the target environment has different page ids

after a while as it is a particular issue with ListerPro is you have a lot of listers (I do). This is discussed specifically here: 

 I haven't tried @Kiwi Chris's module but it and ModuleSettingsImportExport (for ListerPro) have the same issue (with template ids as well as page ids). My only solution so far is to paste the json into a file and display it in a neatly nested form so that the id references are obvious. It is then a simple matter (for a human) to manually change the ids to match those in the target database and paste the result into the target module.

But, while it may be obvious to a human, it is a bit tricky to devise a reliable algorithm.

The best I can come up with so far is:

1. In the source environment, find all occurrences in the config json of words like xxx=yyy (possibly with leading quote and trailing quote or comma) where yyy is an integer
2. If xxx is 'template' then replace yyy with the template name of template id yyy (similarly with field=yyy)
3. Otherwise assume xxx is a page reference and replace yyy with the page path. (It might be possible to be more sophisticated here and check that xxx is the name of a page reference field - or if xxx is of the form www.zzz that zzz is a page reference field. If not, is it a page reference type of selector, e.g. has_parent? Otherwise assume it is just an ordinary integer field - maybe check that too and give an error message if not).
4. Then in the target environment, perform the above in reverse to create the config json.

This could perhaps be done as a separate module with export and import methods that just reads and writes json files which can be sourced from and entered into the config fields provided by ModuleSettingsImportExport. Or perhaps it could optionally be included via config setting - 'Convert ids to/from names or paths' - in ModuleSettingsImportExport?

What do you think?

[MarkE]

Hi @adrian (and @Robin S if interested). I have forked your module and added a config field to use names/paths rather than ids. Fork is here: https://github.com/MetaTunes/ModuleSettingsImportExport, if you would like to have a look/play. I have tested it with some modules (but not all) and you can easily see the results in the multiple module settings box. Note that it deliberately gives some warning/error messages at the moment. They are not really necessary but do highlight where some modules are not perfectly constructed.

Enjoy 😀

[adrian]

Hi @MarkE - thanks for your hard work on this. It looks like a great start but in my initial testing I am unfortunately seeing some issues. I'd love to see this working reliably and I think #1, #2, and #3 are manageable, but not sure about #4.

1. I think page names need to be page paths because currently children and grandchildren are stored just by name and during an import these could match multiple pages (same name but different branch).

2. If a module stores JSON as one of its config settings, the quote escaping backslashes prevent your regex from matching. An example is my CustomUploadNames module: "ruleData":"[{\"tempDisabled\":0,\"enabledFields\":[\"102\"],\"enabledTemplates\":[\"29\",\"1\"]

3. Single value fields don't seem to work. For example a config settings field like this still returns: "specifiedTemplate":"29"

4. I fear that if you try to support that then you'll also need to be careful about numbers that could be valid template or field IDs. If these are stored as an InputfieldInteger (like I do with Tracy's log line length settings where the default is 100) you'll be OK, but if the module uses an InputfieldText then the quotes will be added to the number and it will possibly match a field and then things will become a mess. Any thoughts?

 

[MarkE]

Hi @adrian. Thanks for the comments. 

1. That was the intention, but I missed it in one important place - hopefully now fixed. 
    
2. I think my new version handles this. 
    
3. Ditto. 
    
4. My code only replaces config items which have page, field, template, role, or user in the key name (case insensitive) and where the value is numeric and is a valid id for the related type. Hopefully this avoids the problem you mention. If someone builds a module with a type name included in the config key with numeric values which are valid ids, but where the values are not intended to refer to ids of that object type, then it won’t work, but that just seems really perverse to me!

I have uploaded the new code to my GitHub and have incremented the version and amended the ReadMe. Obviously it is pretty hard to test it on all modules, certainly as regards importing, but it seems ok so far. Let me know if you find any exceptions. One module that causes problems is (deprecated?) ProcessEmailToPage but it is a problem even without my changes. That is because the configdata is wrong - it self-overwrites - so not something that your module can fix, I think. 

[adrian]

Thanks @MarkE - 1, 2, & 3 all look to be working great now! As for 4 - sorry, I hadn't looked at your code well enough to see those restrictions on the key name - looks like a solution that should work often enough to be OK.

Would you like to send me a PR?

Thanks again for this - should be an awesome improvement.

[MarkE]

1 hour ago, adrian said:

Would you like to send me a PR?

Done.