Jump to content
adrian

Migrator

Recommended Posts

Ok. I installed 2.8.5 with the ProcessMigrator that you suggested + MigratorWordpress. It seemed to surpass the initial error but now after uploading the wordpress XML file is showing me this error:

 Can't save page 1: /: Selected homepage template cannot be used because it does not define access.

After that, going back to the home screen I only see two pages created (Categories & Tags).

I noticed that the error above is showing only when I use replace/overwrite. If I choose Append, it was showing that all 571 posts were processed properly but in the pages I did not see them.

I decided to try and convert the posts to pages (with comments) and this time I was able to see all the pages created. Now the issue is that I do not see the comments so I guess I should redo the import but setup a template with all the fields created in advance and have them added to a template that would be used for the imported pages. On top of that I did not have installed the comments field module so it seems like it might work once I rethink the whole process again.

Do you have any idea how to make replace/overwrite work without the error saving the page?

P.S. It looks like if I use Append, there are no comments attached to the page even though I have the module now installed and created a template with all fields that were initially created during the Append. So I need to figure out how to define the access for the selected template and try again.

Share this post


Link to post
Share on other sites

@MilenKo - I don't think that error has anything to do with Migrator. Please google this for the access issue: 

site:processwire.com "Selected homepage template cannot be used because it does not define access"

Once that is sorted, I think everything else should work fine. I have only ever imported WP posts into a subpage, like "blog", but importing into home should also be fine.

Append vs Replace/Overwrite should both work fine, but if you are importing to a new install, I would go with append.

I would suggest reducing your xml to just a few posts while you try different things - just so each test is much faster.

BTW, I have never needed to change the post type for normal WP blog post type pages - they just worked as is.

Late here, so won't be able to respond again until tomorrow, so good luck!

Share this post


Link to post
Share on other sites

Adrian, thanks for the points. The need to switch the post type is that the themes I used were using custom post types so most of the converters are made to just look into the standard type - posts or else, but not news (for example). Switching the post type is not an issue and is semi automatic (semi, as I had to do around 100 posts at once to avoid the server timeout.)

I will google the error and see where would that lead. Even if the pages are not properly inserted, but the comments are assigned to a title, that would fully satisfy my needs. Again, I have planned the post cleanup no matter of the CMS but PW is now my top prefference.

Share this post


Link to post
Share on other sites

@MilenKo - if you fix that error and posts are still not being imported fully, PM me your XML file (if you can) and I'll take a look - it should work fine.

  • Like 1

Share this post


Link to post
Share on other sites

I will Adrian. The posts are imported correctly but the comments are not there. This is the only issue. I have a feeling that I might have not set up properly the comments field or something else. There are no issues with sending you the XML file  but I would first try to recover the issue myself using your tool and other infos and in case I am failing, will use the Joker ;)

Share this post


Link to post
Share on other sites

@adrian Looks like I had some progress by creating a simple template matching most of the field names etcl with the wordpress one. Installed the FieldTypeComments and imported the XML. For an unknown reason the name/content of my home page changed to one of the pages from the import as well as the comments are imported in repeating one commment several times (ex, Comment1 x10, Comment 2 x10 etc.) I know I need to go over every page and make the fields properly so that won't be an issue even if I have to work over all comments in the page, but in the admin of the comments I just see them with Pending status and I am unable to just delete the repeating ones.

Share this post


Link to post
Share on other sites

@MilenKo - I don't know why this is proving so problematic for you. I haven't ever imported a WP blog into the PW homepage - always into /blog/, but it has always worked perfectly and I have never had to create any fields ahead of time, including the comment fields. The pending status is a known issue. I had been waiting on the response to this: https://github.com/ryancramerdesign/ProcessWire/issues/1034, which ended up here: https://github.com/processwire/processwire-issues/issues/88 - it looks like it can be solved now.

 

  • Like 1

Share this post


Link to post
Share on other sites

@adrian, I did not import the WP blog directly to homepage but created a page called Import and setup the module to point to it. Somehow I ended up with the home page changed to one of the WP Posts, and inside the include folder I got all the pages with the comments. If you feel like having some time, I would send you the XML archive to try and see if it works for you. Thanks in advance ;)

 

Share this post


Link to post
Share on other sites

Btw, thinking of the conversion from WP to PW, it seems like I need to export from the PW profile the pages, images, comments etc. to the theme profile as well so the fun will be on MAX but it worth every effort knowing how many years I have tried to work the things out with WP and how for a really short time I got my first theme (thanks to you and others).

Share this post


Link to post
Share on other sites

Hi @MilenKo - thanks for the XML file - it was very helpful.

I have committed a new version of Migrator to Github that fixes the duplicate comments issue and also now properly sets the approved status of comments.

Note that I didn't test your complete file (because it would take quite a long time I expect). I cut it down to 5 posts, but with those everything is now working as expected. I didn't see the issue with the homepage being renamed, but maybe there is one post in there that is doing something strange. Could you please test with a limited set to make sure everything is working at your end. And then try with all and let me know about the homepage issue. If you could narrow it down to a particular post then I should be able to fix.

It's probably a good idea to start with a completely fresh PW install (after all your previous attempts, something might be messed up by now) - you can even let Migrator install the comments module and set up the field etc. 

Thanks,
Adrian

  • Like 2

Share this post


Link to post
Share on other sites

@adrian Thank you very much for the update and I am glad I helped a bit in the process.

I will try to import a few posts first and see how it goes after which I will try to import the whole database. If the posts are imported properly but only one changes the homepage - I can totally live with this, but I understand your point of having that fixed so that it does not affect somebody else using the module.

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile. After I do that, I will just need to import my existing profile I have almost finished. But let's do this step by step and at the end we will have a nice and smooth approach.

Will let you know as soon as finish both tests.

  • Like 2

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile.

Oh yes, I should mention that I did my testing with 3.0.55, but it should still work in 2.x

Thanks for helping to narrow down what might be causing that homepage change. I think it will be a matter of doing chunks of posts (maybe 20 at a time, or something like that) and then narrowing down from there. There must be something weird :)

  • Like 1

Share this post


Link to post
Share on other sites

Oh, I should be the one thanking you.

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

It would be possible (and probably a good idea) to add this functionality to Migrator, but it's beyond what I have time for at the moment I'm afraid.

  • Like 2

Share this post


Link to post
Share on other sites

Huh, it is true - ideas we could have plenty, but there are only 24 hours in 1 day and they are never enough ;)

Thank you again for being so helpful...

  • Like 2

Share this post


Link to post
Share on other sites

Hello @adrian I tested the importer and got an error. Here are the steps I took and the error to just check if I did everything right or missed something:

1. I cut the whole WordPress file (14-15Mb size)  in 6 pieces of 2Mb to make it much smaller and reduce the load etc. as per your instructions. To reduce the size of the file I used a simple app called wxrsplitter that I know worked for WP before.

2. Installed fresh PW 3.0.55 with blank profile and only added ProcessMigrator & MigratorWordpress (I downloaded latest update for ProcessMigrator from the git source but MigratorWordpress was kept the same as I did not se any updated files).

3. Created a page Wordpress and assigned it as parent at the Import page.

4. Selected to import user details.

5. Selected to Import created/modified dates.

6. Choose to automatically download and install missing filetypes.

7. As data source used WordPress Migrator and pointed to the first chunk.

The error I got after the import is:

Error: Call to a member function eq() on a non-object (line 1704 of C:\OpenServer\domains\recipemagician.pw\site\assets\cache\FileCompiler\site\modules\ProcessMigrator\ProcessMigrator.module) 

This error message was shown because: you are logged in as a Superuser. Error has been logged.

If I refresh the page, then it continued and finished with the informing message of how many pages were imported successfully. The same thing is applied for the rest of the files or if I use the file at once (which by obvious reasons takes more time to import but does not fail other than the error).

So I pushed the whole file for the test and was able to import 559 posts as child pages of Wordpress. I checked a few pages with comments and they were all posted in the proper amount and approved with an ability to change the status if needed!

It is clear now that all posts were imported. I am unable yet to say would all comments be properly imported but I don't see a reason if one works fine why the rest would not. So will consider that also done perfectly.

In regards to the Home page title, it changed again (before it was Home and now is imported a the first page alphabetically added. Because of that it looses its fields and comments (this happened when I imported the XML chunks). Will try now to import the whole file but I doubt that this would happen again. For sure I can preserve the content of the page imported to the Home by creating and adding all the necessary fields to it and reimporting, but just reporting that something is still to be fixed. If a need be, I can provide you with the slices of XML.

  • Like 1

Share this post


Link to post
Share on other sites

Ok. Tried with the whole file and for the testing purposes inside the Home I created a child page WordPress inside of which I created another child calling it Import. Pointed the module to the Import page as parent and thought that this way the Home would not change but would the Wordpress. Well I was wrong - the Home title changed again to the same so for some reason the import grabs a page (probably the first) and imports it to Home.

On top of that, if I try to import the whole file, then some of the pages were not imported (those that were originaly as Wordpress pages). Categories list is empty too.

After the import in both cases I noticed another error:

 SQLSTATE[42S22]: Column not found: 1054 Unknown column 'post' in 'field list' along with the message of successful import. So for now importing the XML in chunks works much better and properly includes categories as well as users etc. Even if I will have to recreate the page that is imported in the Home, I am still totally OK but just reporting to make sure others won't be able to move on.

  • Like 1

Share this post


Link to post
Share on other sites

@MilenKo  - thanks for all those details. I have pushed a new version which fixes that eq() error.

I finally did a full import of your file and I think I now know why you are having troubles. Your XML is for the entire Wordpress site. I have only ever used this to migrate blog posts. I would suggest that you create an XML export of just posts (I think that's an option). I think that will probably take care of the homepage renaming issue, as well as the categories and tags problems. It's all about the pages being relative to the correct parent and when you import pages and posts at once, it messes that up. Obviously this should be fixed, but I think that for the moment if you do this, you'll get a complete import of all posts - fingers crossed :)

  • Like 1

Share this post


Link to post
Share on other sites

Adrian, you are 100% right and how come I did not think of that earlier. I will export now only the posts, but I am not sure if I do that will the export include the comments too. Will test it right away and see what is happening. One thing is sure though - nobody would care if the name of the old website was imported or not as those settings will have to be changed to apply with PW and new theme. 

Crossing my fingers and trying it now with the latest version of the Migrator.

Share this post


Link to post
Share on other sites

Hello @adrian . I installed your updated module and tested to import ONLY THE POSTS export. To be clear, I still needed to convert all custom type posts to the standard Wordpress posts as otherwise it only creates a few empty pages and that is all.

I tried to start the import this time with the entire XML so as expected I got an error:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away  (in other words it means that the import took quite a time and server gives up). 

To help the server "survive" I decided to edit the my.cnf changing the wait_timeout from 60 to 6000 and this time it worked fine.

Unfortunately the issue still exists with the Home page being renamed and unfortunately it is a page that has some comments in it too. It appears that it does not just import to the parent of the page pointed to the module but straight to home.

Another thing I just noticed is that the comments date is not preserved, so most of the comments appear to be posted with a timestamp of the import. It might not be such a big issue, however it won't make much sense to have the page added 3-4 years ago and containing 50-100 comments within the last XX minutes.

In other words if the comments timestamp can be fixed and the page that imports into the home properly inserted to the parent, it will be all perfect!

 

Share this post


Link to post
Share on other sites

Thanks @MilenKo - I'll take a look at the comment timestamp issue tomorrow. Could you please PM me the new XML with just the posts. I might also look at working with the full one - we'll see.

How did you go with Categories with just the posts in the XML?

Share this post


Link to post
Share on other sites

@adrian I did not care much of the categories as I am planning on completely redoing most of the post information, pictures, tags, categories etc. I noticed that after moving the custom type posts to the standard WP post all the posts got to Uncategorized (the default WP category). During the import I do see the category created and pages have the Uncategorized as their category so my guess is that if everything was OK with the posts and categories, they will be imported properly.

One other thing I just noticed is that the tags are also not imported and the field that was automatically created by the module is not allowing to add new tags (need to change a few settings to allow it). Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

I am PM-ing you the XML and thanking you in advance for looking into it.

Share this post


Link to post
Share on other sites
1 minute ago, MilenKo said:

Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

Thanks, but I am certain this used to work fully, so I would like to get it back to that state.

I have the comment date sorted out, but am working on a couple of other things and have to head out shortly, so probably won't post until tomorrow, so will look again at categories and tags.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Pip
      Hi everyone!
      I'm trying out the Login/Register module for my site. Noted that the module assigns the newly registered user to login-register role. 
      Once you modify the login-register role's permissions, particularly adding page-edit, the new member role will be set to guest. 
      Thing is I'd like to grant my new users the power to create their own pages. Any advice? 
      Thanks. 
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-07-03 -- SnipWire 0.8.7 (beta) released! Fixes some small bugs and adds an indicator for TEST mode 2020-04-06 -- SnipWire 0.8.6 (beta) released! Adds support for Snipcart subscriptions and also fixes some problems 2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  

       
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By Sten
      Hello
      Till now I hacked something with the twig template but it works no more with new PW versions so I look forward to create a module. I am working on a site in multiple languages : French, English, Italian, German, Spanish, Portuguese, Hebrew, Russian. The new posts are entered in any language with a field for language. Till now, I got twig files to get the translations with constants defined for each part of the pages.
      So I'd like to create a module to include theses files added according to the url /fr/en/...
      Have you some observations to do before I begin about the direction to take ?
      Thank you
    • By ukyo
      Mystique Module for ProcessWire CMS/CMF
      Github repo : https://github.com/trk/Mystique
      Mystique module allow you to create dynamic fields and store dynamic fields data on database by using a config file.
      Requirements
      ProcessWire 3.0 or newer PHP 7.0 or newer FieldtypeMystique InputfieldMystique Installation
      Install the module from the modules directory:
      Via Composer:
      composer require trk/mystique Via git clone:
      cd your-processwire-project-folder/ cd site/modules/ git clone https://github.com/trk/Mystique.git Module in live reaction with your Mystique config file
      This mean if you remove a field from your config file, field will be removed from edit screen. As you see on youtube video.
      Using Mystique with your module or use different configs path, autoload need to be true for modules
      Default configs path is site/templates/configs/, and your config file name need to start with Mystique. and need to end with .php extension.
      Adding custom path not supporting anymore !
      // Add your custom path inside your module class`init` function, didn't tested outside public function init() { $path = __DIR__ . DIRECTORY_SEPARATOR . 'configs' . DIRECTORY_SEPARATOR; Mystique::add($path); } Mystique module will search site/modules/**/configs/Mystique.*.php and site/templates/Mystique.*.php paths for Mystique config files.
      All config files need to return a PHP ARRAY like examples.
      Usage almost same with ProcessWire Inputfield Api, only difference is set and showIf usage like on example.
      <?php namespace ProcessWire; /** * Resource : testing-mystique */ return [ 'title' => __('Testing Mystique'), 'fields' => [ 'text_field' => [ 'label' => __('You can use short named types'), 'description' => __('In file showIf working like example'), 'notes' => __('Also you can use $input->set() method'), 'type' => 'text', 'showIf' => [ 'another_text' => "=''" ], 'set' => [ 'showCount' => InputfieldText::showCountChars, 'maxlength' => 255 ], 'attr' => [ 'attr-foo' => 'bar', 'attr-bar' => 'foo' ] ], 'another_text' => [ 'label' => __('Another text field (default type is text)') ] ] ]; Example:
      site/templates/configs/Mystique.seo-fields.php <?php namespace ProcessWire; /** * Resource : seo-fields */ return [ 'title' => __('Seo fields'), 'fields' => [ 'window_title' => [ 'label' => __('Window title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'attr' => [ 'placeholder' => __('Enter a window title') ] ], 'navigation_title' => [ 'label' => __('Navigation title'), 'type' => Mystique::TEXT, // or InputfieldText 'useLanguages' => true, 'showIf' => [ 'window_title' => "!=''" ], 'attr' => [ 'placeholder' => __('Enter a navigation title') ] ], 'description' => [ 'label' => __('Description for search engines'), 'type' => Mystique::TEXTAREA, 'useLanguages' => true ], 'page_tpye' => [ 'label' => __('Type'), 'type' => Mystique::SELECT, 'options' => [ 'basic' => __('Basic page'), 'gallery' => __('Gallery'), 'blog' => __('Blog') ] ], 'show_on_nav' => [ 'label' => __('Display this page on navigation'), 'type' => Mystique::CHECKBOX ] ] ]; Searching data on Mystique field is limited. Because, Mystique saving data to database in json format. When you make search for Mystique field, operator not important. Operator will be changed with %= operator.
      Search example
      $navigationPages = pages()->find('my_mystique_field.show_on_nav=1'); $navigationPages = pages()->find('my_mystique_field.page_tpye=gallery');
×
×
  • Create New...