Jump to content

Recommended Posts

Run again and find out the id of the assets/files folder where the zip is stored. It should match the id of the Migrator Admin > Setup > Migrator page id, but some reason @NooseLadder is seeing it in one of his normal page id folders. Seems like it's a windows/xampp issue.

Just to clarify, the zip file (files.zip) was correctly created in the /1062/ folder but the error mssg pointed to /1022/ which belongs to another page.

  • Like 1
Link to post
Share on other sites

.....................

@kongondo - thanks for offering. You are right, this thread is a behemoth - I will start a new one as soon as I am ready to release, but for now to test:

  1. Grab the module from: https://github.com/adrianbj/ProcessMigrator and install
  2. Make sure debug mode is on
  3. Run the module (Setup > Migrator)
  4. Choose Export
  5. Choose a Parent Page - to keep things simple, maybe something like the default PW's About page
  6. Export
  7. Check that the downloaded zip file contains a data.json file
  8. Let me know if you get any path related errors like @NooseLadder
  9. If you're feeling energetic, comment out line 335: //unlink($zipFilename);
  10. Run again and find out the id of the assets/files folder where the zip is stored. It should match the id of the Migrator Admin > Setup > Migrator page id, but some reason @NooseLadder is seeing it in one of his normal page id folders. Seems like it's a windows/xampp issue.

Thanks so much!

@Adrian,

Does what it says on the tin. No errors and the id of the assets/files folder where the zip is stored matches id of Migrator page..

  • Like 2
Link to post
Share on other sites
Hi.

First off the work you guys have done with the Wordpress migration tool is amazing, unfortunalety I am not able to get it to work :(

I am running PW 2.4, I have tried to modules both in a WAMP environment and ISP provided environments, both in brand new PW installations and current ones.

I have loaded the blog.zip profile first, this works fine.

Then when I try to import the XML from wordpress (3.9.1) all I am getting is a message saying.

"Missing required ZIP or JSON source"

There is a possibility that I have missed something vital.

This are the basic steps that I take.

Install PW

Install module ProcessMigration

Install module MigratorWordpress

Migrating the blog.zip profile to /home/

Trying to migrate the wordpress.xml to /home/blog (root of blog recently created with blog.zip)

Any Ideas ?

Would really like to get this one working.

Thanks Ronnie
Link to post
Share on other sites

@Ronnie,

Sorry about that - it was a simple fix - please try the latest version on Github and let me know how it goes for you. You might be still be affected by an outstanding multi-language issue but hopefully I can have the fixed soon.

@tobaco

Still have some issues with the language error and also trying to improve the Replace option - need to sort out recursive deletion of existing pages to work better, but I have fixed a bunch of issues around reporting/automatic installation and proper configuring of Inputfield types that are needed by the imported fields - for example, the icon select field now properly detects the need for installing InputfieldSelectMultipleTransfer and then uses it once installed. This should work for all input field types now.

Link to post
Share on other sites

@Ronnie,

I should also mention that you do not need to import the blog.zip file. You can use Kongondo's Blog module (http://modules.processwire.com/modules/process-blog/) as the base, or just import your WP xml and manually create the templates to suit your needs.

If you decide to use the Blog module, here is a screenshot of the MigratorWordpress config settings that are required:

post-985-0-90394500-1404404097_thumb.png

Link to post
Share on other sites

@tobaco - please try the latest version - I have made a lot of fixes and I now get a fully(?) working import of your json file!

Firstly, please run the export on your original site again first as I made a couple of fixes there. Then test on a clean PW install and use the REPLACE option so that your import will completely replace the existing pages from the default PW profile.

Please let me know if all the content appears to be populated as you expect - there are lots of fields with empty values in the json export, but I am not sure if that is correct or not, so I'll need your input on that.

There are still a couple of issues:

  1. Using the replace method a second time results in out of memory errors - I think it's the recursive deletion of the content from that repeater field
  2. I have temporarily disabled importing content for image fields inside repeaters as that is causing errors with your import. I am pretty sure images in repeaters worked fine in my previous testing, so I am not sure where that problem is yet.

Thanks again!

PS The latest version handles installation of all required language modules and gracefully warns of missing language packs and lets you install them as needed.

PPS - if you are doing lots of testing and need to be able to revert to a fresh install regularly, it pays to fire up PHPMyAdmin and create a clean copy of the DB after you have installed PW (or perhaps after you have also installed all your required modules) so that you can simply drop the actual DB and create a new copy from the backed up copy. You might be already aware of this approach, but it's a huge timesaver when testing things like this.

  • Like 1
Link to post
Share on other sites

@Adrian

Thanks for the reply, unfortunately I am still experiencing the same issue.

I have used the latest versions from GitHub that you suggested.

When I turned on debug mode on my PW installation I see something that might be interesting.

I see some references to:
C:/Bitnami/processwire-2.4.0-3/apps/processwire/htdocs/site/assets/files/1043/migratorfiles

and

C:/Bitnami/processwire-2.4.0-3/apps/processwire/htdocs/site/assets/files/1043/migratorfiles\data.json

Attached file: 

post-1232-0-46162200-1404732759_thumb.jp

The only thing that is present in that folder is the WP xml file.

C:\Bitnami\processwire-2.4.0-3\apps\processwire\htdocs\site\assets\files\1043\neossida.xml

No migratorfiles folder / data.json file

Attached file:

post-1232-0-95093300-1404732759_thumb.jp

This file / folder is not present on my machine, I am not completely sure how this works but I quess that the plugin should transform the WP xml to a JSON file that is supposed to be used in the migration.

Maybe that would point you in some direction ? 

Thanks / Ronnie

post-1232-0-46162200-1404732759_thumb.jp

post-1232-0-95093300-1404732759_thumb.jp

Link to post
Share on other sites

Sorry Ronnie,

I have tested more thoroughly this time and the latest version of Migrator should fix the issue.

Thanks again for letting me know how it goes.

Link to post
Share on other sites

@Adrian.

Migrating WP sites like a boss now :)

Thanks for the fix, that seemed to do the trick, I have not tried it thoroughly yet, but I got past the previously issue and content is being created.

  • Like 4
Link to post
Share on other sites

Great to hear :)

Now I just need to hear from @tobaco that everything is working for him with him and I'll be a happy camper - literally :) - I am heading camping for the rest of the week, but I'll follow up with any other issues here when I get back.

  • Like 1
Link to post
Share on other sites
  • 4 weeks later...

Hey Adrian,

I've got a merging of two PWs to deal with and I was wondering if this was possible with Migrator.

PW Site One essentially contains client information (business data - names, addresses, descriptions etc)

PW Site Two contains invoicing information - a site developed to track user billing and payment for the businesses in PW Site One. 

What I would like is to merge the two sites so the Site Two pages would become a branch of the Site One Tree, eg /invoicing/siteTwo

However, it looks like Migrator is developed to migrate dev to live etc, so presumably page ids would be the same etc. What happens if a data page from Site Two has the same id as a invoicing page from Site One for example. Is this magically 'handled' or would a overwrite occur?

Link to post
Share on other sites

muzzer,

Your scenario is exactly what Migrator was designed to do. It does not migrate page IDs at all. Every reference to an id gets converted to a name/path on export and a new page id created on import to the new site.

I would still recommend testing the migration to a test PW install first until you have a full understanding of all the import settings and how they can affect the import. Also when you do it to the live site, please make sure to backup your database first. This module generally works as advertised, but there are some weird scenarios like what Macrura experienced with a page field that points to users that can cause problems. I will be fixing this shortly - I just need some more info from him.

Please let me know how it goes for you and if you think of any possible improvements in the way it works.

  • Like 2
Link to post
Share on other sites
  • 2 weeks later...

Super cool new feature just added - well at least I think it's cool :)

You can now automatically backup your entire site's database and templates directory during the import process. Backups are archived so you can go back to any previously created version whenever you want.

If something goes wrong during the import phase of the migration, or you simply imported the wrong json file, or you changed your mind, you can restore everything with the click of a button!

This database part of this functionality relies on Processwire's brand new WireDatabaseBackup class that Ryan committed today, so to use this functionality, go grab the latest dev version of PW and the latest version of this module.

Please let me know how it goes for you!

post-985-0-07817100-1408509206_thumb.png

post-985-0-78763500-1408509237_thumb.png

post-985-0-67320700-1408509271_thumb.png

  • Like 15
Link to post
Share on other sites

Adrian, do you ever sleep? :P  This is awesome man! Quick question, maybe I missed it, maybe this (especially with the new additions) should now be a superadmin 's only toy? Just wondering....

Link to post
Share on other sites

Adrian, do you ever sleep? :P  This is awesome man! Quick question, maybe I missed it, maybe this (especially with the new additions) should now be a superadmin 's only toy? Just wondering....

The backup/restore stuff was easy thanks to Ryan :)

I haven't given the module a permission setting so I think that limits it to superuser accounts. Are you seeing otherwise?

Something a little weird I just noticed - if you try to enter the url to a Process module, eg: 

http://pwtest.dev/admin/setup/migrator/

when logged in as a user without the appropriate permission, you get this error: TemplateFile: Unrecognized path - maybe that's better than a meaningful error saying you don't have permission or something?

Anyway, I think this is locked down sufficiently to superusers?

Link to post
Share on other sites
  • 1 month later...

@adrian How's further development going on ProcessMigrator?  I was playing around with it yesterday and it seems it's still quite buggy.  I tested it about 20 times on fresh installs and got a few errors which I can discuss with you.

I would like to feature it in a WordPress vs. ProcessWire screencast.  Your plugin solves a huge pain point.

Let me know if I can help.

  • Like 1
Link to post
Share on other sites

Hi @jlahijani,

I haven't done much on it for a while now.

I did think that most bugs were sorted by now though. The biggest remaining issue (I thought) was page fields and how to handle them if their content pages are in a page tree that is not part of the exported parent. At the moment they are exported and recreated on the destination site as a child of home, which obviously may not be ideal, so I think I need to add some additional logic to recreate the entire tree above the page field parent (if it doesn't exist on the destination site), although I can see this being problematic too. Since you've been testing lots, I'd love any thoughts you might have on the best way to handle this.

Also, please send through all the details of the errors you got while testing - it will be a huge help I am sure.

As for Wordpress vs Processwire - are you wanting to feature Migrator's use as an importer of WP content (using Nico's plugin), or just its ability for migration of PW content from one site to another? If the former, then I think that functionality should be pretty robust.

Anyway, please let me know what you have uncovered - it would be great to get this to the next level!

Link to post
Share on other sites

Good to hear.  I will put together some info about the bugs.

I'm planning on featuring the ProcessMigrator plugin as a general exporter/importer of content from one PW site to another.  For example, let's say you have a development and production version of a site, and a new set of content needs to be created (on dev), then eventually launched (on prod).  Rather than re-creating that content manually on production, I want to show how your plugin can achieve that.

WordPress has an export tool, however it's very limited and doesn't work that well for the situation I described.  There are some plugins to enhance it but I haven't had great success with those either.  There's one commercial plugin which I haven't used but seems to be the ultimate solution to do what I described: https://crowdfavorite.com/ramp/

ProcessMigrator, in my eyes, would be the analog to RAMP.

  • Like 1
Link to post
Share on other sites

One thing that would be very helpful is if you can provide the exported JSON (or full zip package if that is relevant) that is causing issues when you import - this will make it much easier to debug.

Thanks again!

Link to post
Share on other sites
  • 2 weeks later...

WordPress has an export tool, however it's very limited and doesn't work that well for the situation I described.  There are some plugins to enhance it but I haven't had great success with those either.  There's one commercial plugin which I haven't used but seems to be the ultimate solution to do what I described: https://crowdfavorite.com/ramp/

 

ProcessMigrator, in my eyes, would be the analog to RAMP.

jlahijani - I finally had a look at the video for RAMP - it looks very cool, especially the automatic pushing from dev to production. Other than that, the big difference I noticed compared to Migrator is that currently Migrator can only export one page tree at a time and it has to be the entire tree of children below it. Sure you have the option to determine which pages get imported to the other site, but I think it would be a great enhancement to offer this selection during export as well. I am thinking the option to choose multiple pages and for each one (if it has children), determine whether to include the children or not. I think this approach would make it much easier to migrate a few newly developed pages, or several new sections, without the need to overwrite everything under an overarching  parent page. You don't actually have to do this with the existing Edit Imported Content option, but it is not as obvious or easy.

Anyway, it has given me some great ideas - thanks! 

  • Like 1
Link to post
Share on other sites

Some more bug fixes - thanks again to muzzer! 

More interestingly though, during import you now have the option to automatically download and install any required missing fieldtypes/inputfields from the PW modules directory.

This is optional and I have noted that if you don't trust the source of the data you are importing, then you might want not want to enable this option.

It is off by default, in which case you'll receive notifications of what's missing so you can install manually before re-starting the import.

  • Like 5
Link to post
Share on other sites
  • 1 month later...

Hi, just thought I would have a go with this. I got this:

Error: Call to a member function size() on a non-object (line 2504 of /home/foodloversdiary/public_html/site/modules/ProcessMigrator/ProcessMigrator.module) 

So, it seems to have imported the categories and a load of tags, but only 5 posts and they have no content.

Any thoughts?

Edit: forgot to mention that it was with a Wordpress import

Link to post
Share on other sites

Hey Joss,

Thanks for testing this out. Looks like you have come across an issue with a very recently added function to scan through RTE fields and automatically generate resized versions of images that have been embedded.

Actually I am wondering - is the WP still still online and working? It needs to be so that the module can download the images from the site and add them to each of the pages in PW.

Any chance you could PM me the WP xml file so I can test?

I am heading out for the rest of the day, but I will look into it in the morning for you - I am sure it will be an easy fix.

PS, Please make sure you are always using the latest version of Migrator and also MigratorWordpress (my fork - Nico's is not up to date)

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By FireWire
      Hello community!

      I want to share a new module I've been working on that I think could be a big boost for multi-language ProcessWire sites.

      Some background, I was looking for a way for our company website to be efficiently translated as working with human translators was pretty laborious and a lack of updating content created a divergence between languages. I, and several other devs here, have talked about translation integrations and have recognized the power that DeepL has. DeepL is an AI deep learning powered service that delivers translation quality beyond any automated service available. After access to the API was opened up to the US, I built Fluency, a DeepL translation integration for ProcessWire.
      Fluency brings automated translation to every multi-language field in the admin, and also provides a translation tool allowing the user to translate their text to any language without it being inside a template's field. With Fluency you can:
      Translate any plain textarea or text input Translate any CKEditor content (yes, with markup) Translate page names for fully localized URLs on every page Translate your in-template translation function wrapped strings Translate modules Fluency is free, and now so is DeepL
      Since this module was first built DeepL has introduced free Developer accounts that allow anyone to start using Fluency at zero cost and beginning with the version 0.3.0 release Fluency now supports free DeepL accounts. As of June 2021 DeepL supports translation to 26 languages and continues to offer more!
      Installation and usage is completely plug and play. Whether you're building a new multi-language site, need to update a site to multi-language, or simply want to stop manually translating a site and make any language a one-click deal, it could not be easier to do it. Fluency works by having you match the languages configured in ProcessWIre to DeepL's. You can have your site translating to any or all of the languages DeepL translates to in minutes (quite literally).
      Let's break out the screenshots...
      When the default language tab is shown, a message is displayed to let users know that translation is available. Clicking on each tab shows a link that says "Translate from English". Clicking it shows an animated overlay with the word "Translating..." cycling through each language and a light gradient shift. Have a CKEditor field? All good. Fluency will translated it and use DeepL's ability to translate text within HTML tags. CKEditor fields can be translated as easily and accurately as text/textarea fields.

      Repeaters and AJAX created fields also have translation enabled thanks to a JavaScript MutationObserver that searches for multi-language fields and adds translation as they're inserted into the DOM. If there's a multi-language field on the page, it will have translation added.

      Same goes for image description fields. Multi-language SEO friendly images are good to go.

      Creating a new page from one of your templates? Translate your title, and also translate your page name for native language URLs. (Not available for Russian, Chinese, or Japanese languages due to URL limitations). These can be changed in the "Settings" tab for any page as well so whether you're translating new pages or existing pages, you control the URLs everywhere.

      Language configuration pages are no different. Translate the names of your languages and search for both Site Translation Files (including all of your modules)

      Translate all of the static text in your templates as well. Notice that the placeholders are retained. DeepL is pretty good at recognizing and keeping non-translatable strings like that. If it is changed, it's easy to fix manually.

      Fluency adds a "Translate" item to the CMS header. When clicked this opens up a modal with a full translation tool that lets the user translate any language to any language. No need to leave the admin if you need to translate content from a secondary language back to the default ProcessWire language. There is also a button to get the current API usage statistics. DeepL account owners can set billing limitations via character count to control costs. This may help larger sites or sites being retrofitted keep an eye on their usage. Fluency can be used by users having roles given the fluency-translate permission.

      It couldn't be easier to add Fluency to your new or existing website. Simply add your API key and you're shown what languages are currently available for translation from/to as provided by DeepL. This list and all configuration options are taken live from the API so when DeepL releases new languages you can add them to your site without any work. No module updates, just an easy configuration. Just match the language you configured in ProcessWire to the DeepL language you want it to be associated with and you're done. Fluency also allows you to create a list of words/phrases that will not be translated which can prevent items such as brands and company names from being translated when they shouldn't

       
      Limitations:
      No "translate page" - Translating multiple fields can be done by clicking multiple translation links on multiple fields at once but engineering a "one click page translate" is not feasible from a user experience standpoint. The time it takes to translate one field can be a second or two, but cumulatively that may take much longer (CKEditor fields are slower than plain text fields). There may be a workaround in the future but it isn't currently on the roadmap. No "translate site" - Same thing goes for translating an entire website at once. It would be great, but it would be a very intense process and take a very (very) long time. There may be a workaround in the future but it isn't on the roadmap. No current support for Inline CKEditor fields - Handling for CKEditor on-demand hasn't been implemented yet, this is planned for a future release though and can be done. I just forgot about it because I've never really used that feature personally.. Alpha release - This module is in alpha. Releases should be stable and usable, but there may be edge case issues. Test the module thoroughly and please report any bugs via a Github issue on the repository or respond here. Please note that the browser plugin for Grammarly conflicts with Fluency (as it does with many web applications). To address this issue it is recommended that you disable Grammarly when using Fluency, or open the admin to edit pages in a private window where Grammarly may not be loaded. This is an issue that may not have a resolution as creating a workaround may not be possible. If you have insight as to how this may be solved please visit the Github page and file a bugfix ticket.
      Requirements:
      ProcessWire  3.0+ UIKit Admin Theme That's Fluency in a nutshell. A core effort in this module is to create it so that there is nothing DeepL related hard-coded in that would require updating it when DeepL offers new languages. I would like this to be a future-friendly module that doesn't require developer work to keep it up-to-date.
      The Module Is Free
      This is my first real module and I want to give it back to the community as thanks. This is the best CMS I've worked with (thank you Ryan & contributors) and a great community (thank you dear reader).
      DeepL Developer Accounts
      In addition to paid Pro Developer accounts, DeepL now offers no-cost free accounts. Now all ProcessWire developers and users can use Fluency at no cost.
      Learn more about free and paid accounts by visiting the DeepL website. Sign up for a Developer account, get an API key, and start using Fluency today.
      Download & Feedback
      Download the latest version here
      https://github.com/SkyLundy/Fluency-Translation/archive/main.zip
      Github repository:
      https://github.com/SkyLundy/Fluency-Translation
      File issues and feature requests here (your feedback and testing is greatly appreciated):
      https://github.com/SkyLundy/Fluency-Translation/issues
       
      Thank you! ¡Gracias! Ich danke Ihnen! Merci! Obrigado! Grazie! Dank u wel! Dziękuję! Спасибо! ありがとうございます! 谢谢你!

    • By monollonom
      (once again I was surprised to see a work of mine pop up in the newsletter, this time without even listing the module on PW modules website 😅. Thx @teppo !)
      Github: https://github.com/romaincazier/FieldtypeQRCode
      Modules directory: https://processwire.com/modules/fieldtype-qrcode/
      This is a simple module I made so a client could quickly grab a QR Code of the page's url in the admin.
      There's not much to it for now, but if need be you can output anything using a hook:
      $wire->addHookAfter("FieldtypeQRCode::getQRText", function($event) { $event->return = "Your custom text"; }) You can also output the QR code on your front-end by calling the field:
      echo $page->qr_code_field; The module uses the PHP library QR Code Generator by Kazuhiko Arase. When looking for a way to generate a QR Code in PW I came across @ryan's integration in his TFA module. I'm not very familiar with fieldtype/inputfield module development so I blindly followed @bernhard (great) tutorial and his BaseFieldtypeRuntime. At some point I'll take a deeper look to make a module on my own.
      Some ideas for improvements :
      add the ability to choose what to ouput : page's url / editUrl / file(s) / image(s) / ... allow to output multiple QR codes ?
    • By Chris Bennett
      https://github.com/chrisbennett-Bene/AdminThemeTweaker
      Inspired by @bernhard's excellent work on the new customisable LESS CSS getting rolled into the core soon, I thought I would offer up the module for beta testing, if it is of interest to anyone.

      It takes a different approach to admin styling, basically using the Cascade part of CSS to over-ride default UiKit values.
      Values are stored in ModuleConfig Module creates a separate AdminThemeTweaker Folder at root, so it can link to AdminThemeTweaker.php as CSS AdminThemeTweaker.php reads the module values, constructs the CSS variables then includes the CSS framework Can be switched on and off with a click. Uninstall removes everything, thanks to bernhard's wonderful remove dir & contents function.
      It won't touch your core. It won't care if stuff is upgraded. You won't need to compile anything and you don't need to touch CSS unless you want to.

      It won't do much at all apart from read some values from your module config, work out the right CSS variables to use (auto contrast based on selected backgrounds) and throw it on your screen.
      You can configure a lot of stuff, leave it as it comes (dark and curvy), change two main colors (background and content background) or delve deep to configure custom margins, height of mastheads, and all manner of silly stuff I never use.

      Have been developing it for somewhere around 2 years now. It has been (and will continue to be) constantly tweaked over that time, as I click on something and find something else to do.
      That said, it is pretty solid and has been in constant use as my sole Admin styling option for all of those 2 years.

      If nothing else, it would be great if it can provide any assistance to @bernhard or other contributor's who may be looking to solve some of the quirkier UiKit behavior.
      Has (in my opinion) more robust and predictable handling of hidden Inputfields, data-colwidths and showIf wrappers.
      I am very keen to help out with that stuff in any way I can, though LESS (and any css frameworks/tools basically) are not my go.
      I love CSS variables and banging-rocks-together, no-dependency CSS you can write with notepad.



       

    • By opalepatrick
      I see old posts saying that repeaters are not the way to go in Custom Process Modules. If that is the case, when using forms (as I am trying to do) how would one tackle things like repeat contact fields where there can be multiple requirements for contact details with different parameters? (Like point of contact, director, etc) or even telephone numbers that have different uses?
      Just for background I am creating a process module that allows me to create types of financial applications in the admin area (no need to publish any of this, pure admin) that require a lot of personal or company information.
      Maybe I am thinking about this incorrectly?
    • By HMCB
      I ran across a reference to IftRunner module. The post was 6 years ago. I cant find it in available modules. Has it been pulled?
×
×
  • Create New...