Importing MODX template variables into a new PWire structure

[creativejay]

Hello everyone!

I've built several Processwire sites now and I just love the framework more and more with each successful launch.

I'm about to tackle what is, to me, a huge site redesign, taking a large catalog-style site and updating it from MODX's evo to PW 2.5 (assuming there isn't a 2.6 by the time I finish..).

The site is modest as far as catalogs go, but while I could copy-paste tV values into PW custom fields, I feel like there has to be a solution I can execute via myPHPAdmin with the right site setup in processwire's admin.

My MODX Install (henceforth known as Current Site) has a resource for each product, and tied to that resource's template are any number of ≤70 template variables.

Ideally, I'd like to plan an approach that lets me export the values for the Current Site's TVs from the database, and import it into fields in the Processwire Install (henceforth New Site).

It appears to me that the Current Site's tables are built with page content and TVs in one table, whereas the New Site's PWire structure is not as one-to-one as the MODX method. So I have to untangle some yarn, first, to figure out what to import and where, if I understand it correctly.

Would it be a pursuit of madness to try what I'm thinking? Does anyone have any experience with this? Is there something else you'd recommend instead?

Thank you!

[teppo]

First of all, you really shouldn't try to do an import via phpMyAdmin. Using ProcessWire's API is always a better (and easier) way. Direct SQL import is going to be a lot of work and might cause all sorts of unexpected issues, some of which you might not notice until it's too late.

Import scripts are easy to write -- usually the hardest part is getting data out of the other system and, of course, planning your new data structure. I'm not familiar enough with MODX to know how easy or hard it's going to get data from there, though, but I've done my fair share of import scripts for other systems. Usually the easiest method is exporting data from original system as CSV, JSON or something else that's "machine readable" and then writing a simple import script using the bootstrap method, but there are other methods too.

You might want to check out this case story -- and perhaps ask some questions in that thread too -- as it seems to have some similarities with your case. Also, if you can get your data out in CSV format and it's not too complex in structure, Import Pages CSV just might do most of the work for you. Migrator tool built by Adrian is somewhat related too, though that's not exactly off-the-shelf solution for this particular situation (there's a WordPress migrator available, but not one for MODX yet, at least as far as I know).

Hope this helps a bit.

[pwired]

maybe this can help in some way

http://processwire.com/talk/topic/3691-tutorial-a-quick-guide-to-processwire-for-those-transitioning-from-modx/

http://processwire.com/talk/topic/3111-a-modx-refugee-questions-on-features-of-processwire/

http://processwire.com/talk/topic/2850-processwire-for-designers/page-2#entry30349

[creativejay]

@teppo thank you so much for your reply! I have never worked with so much data in a site migration before that a manual copy/paste wasn't easier than learning the proper way to get data from one db to another. I'm able to work with an otherwise undeveloped install, so I have room to make some mistakes, this time.

Import Pages CSV looks very promising as well, and I'm excited to see it because it potentially answers the call from another site I'm building where I hope to import a blogger CSV export into the new site structure.

Thanks again!

[adrian]

Given all the ex MODx'ers around here, I would love to see a plugin for Migrator. If you're willing to give it a go, Nico's Worpdress Migrator plugin for Migrator (that teppo mentions) might get you started. Is there an existing MODx to XML/CSV/JSON tool available that could be used as a starting point?

[owzim]

If you go the route of importing your data with the API and an install script it's worth mentioning that you can restore your system easily while developing you install script.

So make a backup before, via

$database->backups()->backup(); 

An in your script at the beginning, add a

$database->backups()->restore("filename");

So each time it runs, you have a fresh version you can hack on, without having to clean up manually each time you mess up something. More details here: https://processwire.com/blog/posts/august-2014-core-updates-4/

I think it's very convenient =)

Of course you have to remove the restore call when you're finished with your script.

Additionally since PW 2.5.4 (yesterday) you can pass arrays to InputfieldWrapper, InputfieldForm, or InputfieldFieldset which makes creating the necessary templates and respective fields for your imported data very easy and fast.

See at the bottom: https://processwire.com/blog/posts/new-module-configuration-options/

https://processwire.com/blog/posts/august-2014-core-updates-4/

[adrian]

Regarding the backup and restore option that owzim mentions. It is indeed very cool, but there's a bit of a problem with it at the moment. 

Any new fields that get created by your import script won't have the DB table removed when you run the restore which will result in table already exists errors when you run the script again after the restore because the field has been removed from the fields table so PW doesn't know about it anymore, but it's field_fieldname table is still there. 

Hopefully Ryan will sort out a fix for this shortly.

Of course if your import script isn't creating new fields (or anything else that adds a DB table to PW), then this won't be an issue.

[creativejay]

Thanks, everyone, for your tips, advice, and experience!

My current plan is to create the table structure in the new site and code and output a CSV file from MODx, and import that into the new site via ImportPagesCSV.

Luckily this is my primary concern with the site so if it foos the bar, I'm happy to wipe it and start over until I get it right, then flesh out the rest.

The next problem to tackle is making sure my URLs (ending in .html from MODx) match, to avoid having to set up some excruciating redirects. But I have yet to look into that, so unless you know the exact thread I need to look at, no worries answer that one here!