adrian

Migrator

Recommended Posts

Ok. I installed 2.8.5 with the ProcessMigrator that you suggested + MigratorWordpress. It seemed to surpass the initial error but now after uploading the wordpress XML file is showing me this error:

 Can't save page 1: /: Selected homepage template cannot be used because it does not define access.

After that, going back to the home screen I only see two pages created (Categories & Tags).

I noticed that the error above is showing only when I use replace/overwrite. If I choose Append, it was showing that all 571 posts were processed properly but in the pages I did not see them.

I decided to try and convert the posts to pages (with comments) and this time I was able to see all the pages created. Now the issue is that I do not see the comments so I guess I should redo the import but setup a template with all the fields created in advance and have them added to a template that would be used for the imported pages. On top of that I did not have installed the comments field module so it seems like it might work once I rethink the whole process again.

Do you have any idea how to make replace/overwrite work without the error saving the page?

P.S. It looks like if I use Append, there are no comments attached to the page even though I have the module now installed and created a template with all fields that were initially created during the Append. So I need to figure out how to define the access for the selected template and try again.

Share this post


Link to post
Share on other sites

@MilenKo - I don't think that error has anything to do with Migrator. Please google this for the access issue: 

site:processwire.com "Selected homepage template cannot be used because it does not define access"

Once that is sorted, I think everything else should work fine. I have only ever imported WP posts into a subpage, like "blog", but importing into home should also be fine.

Append vs Replace/Overwrite should both work fine, but if you are importing to a new install, I would go with append.

I would suggest reducing your xml to just a few posts while you try different things - just so each test is much faster.

BTW, I have never needed to change the post type for normal WP blog post type pages - they just worked as is.

Late here, so won't be able to respond again until tomorrow, so good luck!

Share this post


Link to post
Share on other sites

Adrian, thanks for the points. The need to switch the post type is that the themes I used were using custom post types so most of the converters are made to just look into the standard type - posts or else, but not news (for example). Switching the post type is not an issue and is semi automatic (semi, as I had to do around 100 posts at once to avoid the server timeout.)

I will google the error and see where would that lead. Even if the pages are not properly inserted, but the comments are assigned to a title, that would fully satisfy my needs. Again, I have planned the post cleanup no matter of the CMS but PW is now my top prefference.

Share this post


Link to post
Share on other sites

@MilenKo - if you fix that error and posts are still not being imported fully, PM me your XML file (if you can) and I'll take a look - it should work fine.

  • Like 1

Share this post


Link to post
Share on other sites

I will Adrian. The posts are imported correctly but the comments are not there. This is the only issue. I have a feeling that I might have not set up properly the comments field or something else. There are no issues with sending you the XML file  but I would first try to recover the issue myself using your tool and other infos and in case I am failing, will use the Joker ;)

Share this post


Link to post
Share on other sites

@adrian Looks like I had some progress by creating a simple template matching most of the field names etcl with the wordpress one. Installed the FieldTypeComments and imported the XML. For an unknown reason the name/content of my home page changed to one of the pages from the import as well as the comments are imported in repeating one commment several times (ex, Comment1 x10, Comment 2 x10 etc.) I know I need to go over every page and make the fields properly so that won't be an issue even if I have to work over all comments in the page, but in the admin of the comments I just see them with Pending status and I am unable to just delete the repeating ones.

Share this post


Link to post
Share on other sites

@MilenKo - I don't know why this is proving so problematic for you. I haven't ever imported a WP blog into the PW homepage - always into /blog/, but it has always worked perfectly and I have never had to create any fields ahead of time, including the comment fields. The pending status is a known issue. I had been waiting on the response to this: https://github.com/ryancramerdesign/ProcessWire/issues/1034, which ended up here: https://github.com/processwire/processwire-issues/issues/88 - it looks like it can be solved now.

 

  • Like 1

Share this post


Link to post
Share on other sites

@adrian, I did not import the WP blog directly to homepage but created a page called Import and setup the module to point to it. Somehow I ended up with the home page changed to one of the WP Posts, and inside the include folder I got all the pages with the comments. If you feel like having some time, I would send you the XML archive to try and see if it works for you. Thanks in advance ;)

 

Share this post


Link to post
Share on other sites

Btw, thinking of the conversion from WP to PW, it seems like I need to export from the PW profile the pages, images, comments etc. to the theme profile as well so the fun will be on MAX but it worth every effort knowing how many years I have tried to work the things out with WP and how for a really short time I got my first theme (thanks to you and others).

Share this post


Link to post
Share on other sites

Hi @MilenKo - thanks for the XML file - it was very helpful.

I have committed a new version of Migrator to Github that fixes the duplicate comments issue and also now properly sets the approved status of comments.

Note that I didn't test your complete file (because it would take quite a long time I expect). I cut it down to 5 posts, but with those everything is now working as expected. I didn't see the issue with the homepage being renamed, but maybe there is one post in there that is doing something strange. Could you please test with a limited set to make sure everything is working at your end. And then try with all and let me know about the homepage issue. If you could narrow it down to a particular post then I should be able to fix.

It's probably a good idea to start with a completely fresh PW install (after all your previous attempts, something might be messed up by now) - you can even let Migrator install the comments module and set up the field etc. 

Thanks,
Adrian

  • Like 2

Share this post


Link to post
Share on other sites

@adrian Thank you very much for the update and I am glad I helped a bit in the process.

I will try to import a few posts first and see how it goes after which I will try to import the whole database. If the posts are imported properly but only one changes the homepage - I can totally live with this, but I understand your point of having that fixed so that it does not affect somebody else using the module.

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile. After I do that, I will just need to import my existing profile I have almost finished. But let's do this step by step and at the end we will have a nice and smooth approach.

Will let you know as soon as finish both tests.

  • Like 2

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile.

Oh yes, I should mention that I did my testing with 3.0.55, but it should still work in 2.x

Thanks for helping to narrow down what might be causing that homepage change. I think it will be a matter of doing chunks of posts (maybe 20 at a time, or something like that) and then narrowing down from there. There must be something weird :)

  • Like 1

Share this post


Link to post
Share on other sites

Oh, I should be the one thanking you.

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

It would be possible (and probably a good idea) to add this functionality to Migrator, but it's beyond what I have time for at the moment I'm afraid.

  • Like 2

Share this post


Link to post
Share on other sites

Huh, it is true - ideas we could have plenty, but there are only 24 hours in 1 day and they are never enough ;)

Thank you again for being so helpful...

  • Like 2

Share this post


Link to post
Share on other sites

Hello @adrian I tested the importer and got an error. Here are the steps I took and the error to just check if I did everything right or missed something:

1. I cut the whole WordPress file (14-15Mb size)  in 6 pieces of 2Mb to make it much smaller and reduce the load etc. as per your instructions. To reduce the size of the file I used a simple app called wxrsplitter that I know worked for WP before.

2. Installed fresh PW 3.0.55 with blank profile and only added ProcessMigrator & MigratorWordpress (I downloaded latest update for ProcessMigrator from the git source but MigratorWordpress was kept the same as I did not se any updated files).

3. Created a page Wordpress and assigned it as parent at the Import page.

4. Selected to import user details.

5. Selected to Import created/modified dates.

6. Choose to automatically download and install missing filetypes.

7. As data source used WordPress Migrator and pointed to the first chunk.

The error I got after the import is:

Error: Call to a member function eq() on a non-object (line 1704 of C:\OpenServer\domains\recipemagician.pw\site\assets\cache\FileCompiler\site\modules\ProcessMigrator\ProcessMigrator.module) 

This error message was shown because: you are logged in as a Superuser. Error has been logged.

If I refresh the page, then it continued and finished with the informing message of how many pages were imported successfully. The same thing is applied for the rest of the files or if I use the file at once (which by obvious reasons takes more time to import but does not fail other than the error).

So I pushed the whole file for the test and was able to import 559 posts as child pages of Wordpress. I checked a few pages with comments and they were all posted in the proper amount and approved with an ability to change the status if needed!

It is clear now that all posts were imported. I am unable yet to say would all comments be properly imported but I don't see a reason if one works fine why the rest would not. So will consider that also done perfectly.

In regards to the Home page title, it changed again (before it was Home and now is imported a the first page alphabetically added. Because of that it looses its fields and comments (this happened when I imported the XML chunks). Will try now to import the whole file but I doubt that this would happen again. For sure I can preserve the content of the page imported to the Home by creating and adding all the necessary fields to it and reimporting, but just reporting that something is still to be fixed. If a need be, I can provide you with the slices of XML.

  • Like 1

Share this post


Link to post
Share on other sites

Ok. Tried with the whole file and for the testing purposes inside the Home I created a child page WordPress inside of which I created another child calling it Import. Pointed the module to the Import page as parent and thought that this way the Home would not change but would the Wordpress. Well I was wrong - the Home title changed again to the same so for some reason the import grabs a page (probably the first) and imports it to Home.

On top of that, if I try to import the whole file, then some of the pages were not imported (those that were originaly as Wordpress pages). Categories list is empty too.

After the import in both cases I noticed another error:

 SQLSTATE[42S22]: Column not found: 1054 Unknown column 'post' in 'field list' along with the message of successful import. So for now importing the XML in chunks works much better and properly includes categories as well as users etc. Even if I will have to recreate the page that is imported in the Home, I am still totally OK but just reporting to make sure others won't be able to move on.

  • Like 1

Share this post


Link to post
Share on other sites

@MilenKo  - thanks for all those details. I have pushed a new version which fixes that eq() error.

I finally did a full import of your file and I think I now know why you are having troubles. Your XML is for the entire Wordpress site. I have only ever used this to migrate blog posts. I would suggest that you create an XML export of just posts (I think that's an option). I think that will probably take care of the homepage renaming issue, as well as the categories and tags problems. It's all about the pages being relative to the correct parent and when you import pages and posts at once, it messes that up. Obviously this should be fixed, but I think that for the moment if you do this, you'll get a complete import of all posts - fingers crossed :)

  • Like 1

Share this post


Link to post
Share on other sites

Adrian, you are 100% right and how come I did not think of that earlier. I will export now only the posts, but I am not sure if I do that will the export include the comments too. Will test it right away and see what is happening. One thing is sure though - nobody would care if the name of the old website was imported or not as those settings will have to be changed to apply with PW and new theme. 

Crossing my fingers and trying it now with the latest version of the Migrator.

Share this post


Link to post
Share on other sites

Hello @adrian . I installed your updated module and tested to import ONLY THE POSTS export. To be clear, I still needed to convert all custom type posts to the standard Wordpress posts as otherwise it only creates a few empty pages and that is all.

I tried to start the import this time with the entire XML so as expected I got an error:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away  (in other words it means that the import took quite a time and server gives up). 

To help the server "survive" I decided to edit the my.cnf changing the wait_timeout from 60 to 6000 and this time it worked fine.

Unfortunately the issue still exists with the Home page being renamed and unfortunately it is a page that has some comments in it too. It appears that it does not just import to the parent of the page pointed to the module but straight to home.

Another thing I just noticed is that the comments date is not preserved, so most of the comments appear to be posted with a timestamp of the import. It might not be such a big issue, however it won't make much sense to have the page added 3-4 years ago and containing 50-100 comments within the last XX minutes.

In other words if the comments timestamp can be fixed and the page that imports into the home properly inserted to the parent, it will be all perfect!

 

Share this post


Link to post
Share on other sites

Thanks @MilenKo - I'll take a look at the comment timestamp issue tomorrow. Could you please PM me the new XML with just the posts. I might also look at working with the full one - we'll see.

How did you go with Categories with just the posts in the XML?

Share this post


Link to post
Share on other sites

@adrian I did not care much of the categories as I am planning on completely redoing most of the post information, pictures, tags, categories etc. I noticed that after moving the custom type posts to the standard WP post all the posts got to Uncategorized (the default WP category). During the import I do see the category created and pages have the Uncategorized as their category so my guess is that if everything was OK with the posts and categories, they will be imported properly.

One other thing I just noticed is that the tags are also not imported and the field that was automatically created by the module is not allowing to add new tags (need to change a few settings to allow it). Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

I am PM-ing you the XML and thanking you in advance for looking into it.

Share this post


Link to post
Share on other sites
1 minute ago, MilenKo said:

Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

Thanks, but I am certain this used to work fully, so I would like to get it back to that state.

I have the comment date sorted out, but am working on a couple of other things and have to head out shortly, so probably won't post until tomorrow, so will look again at categories and tags.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By horst
      Wire Mail SMTP

      An extension to the new WireMail base class that uses SMTP-transport

      This module integrates EmailMessage, SMTP and SASL php-libraries from Manuel Lemos into ProcessWire. I use this continously evolved libraries for about 10 years now and there was never a reason or occasion not to do so. I use it nearly every day in my office for automated composing and sending personalized messages with attachments, requests for Disposition Notifications, etc. Also I have used it for sending personalized Bulkmails many times.

      The WireMailSmtp module extends the new email-related WireMail base class introduced in ProcessWire 2.4.1 (while this writing, the dev-branch only).
       
      Here are Ryans announcement.



      Current Version 0.3.0
      get it from the Modules Directory Install and Configure

      Download the module into your site/modules/ directory and install it.

      In the config page you fill in settings for the SMTP server and optionaly the (default) sender, like email address, name and signature.
      You can test the smtp settings directly there. If it says "SUCCESS! SMTP settings appear to work correctly." you are ready to start using it in templates, modules or bootstrap scripts.


      Usage Examples
      The simplest way to use it:
      $numSent = wireMail($to, $from, $subject, $textBody); $numSent = wireMail($to, '', $subject, $textBody); // or with a default sender emailaddress on config page This will send a plain text message to each recipient.
       
      You may also use the object oriented style:
      $mail = wireMail(); // calling an empty wireMail() returns a wireMail object $mail->to($toEmail, $toName); $mail->from = $yourEmailaddress; // if you don't have set a default sender in config // or if you want to override that $mail->subject($subject); $mail->body($textBody); $numSent = $mail->send(); Or chained, like everywhere in ProcessWire:
      $mail = wireMail(); $numSent = $mail->to($toEmail)->subject($subject)->body($textBody)->send(); Additionaly to the basics there are more options available with WireMailSmtp. The main difference compared to the WireMail BaseClass is the sendSingle option. With it you can set only one To-Recipient but additional CC-Recipients.
      $mail = wireMail(); $mail->sendSingle(true)->to($toEmail, $toName)->cc(array('person1@example.com', 'person2@example.com', 'person3@example.com')); $numSent = $mail->subject($subject)->body($textBody)->send(); The same as function call with options array:
      $options = array( 'sendSingle' => true, 'cc' => array('person1@example.com', 'person2@example.com', 'person3@example.com') ); $numSent = wireMail($to, '', $subject, $textBody, $options); There are methods to your disposal to check if you have the right WireMail-Class and if the SMTP-settings are working:
      $mail = wireMail(); if($mail->className != 'WireMailSmtp') { // Uups, wrong WireMail-Class: do something to inform the user and quit echo "<p>Couldn't get the right WireMail-Module (WireMailSmtp). found: {$mail->className}</p>"; return; } if(!$mail->testConnection()) { // Connection not working: echo "<p>Couldn't connect to the SMTP server. Please check the {$mail->className} modules config settings!</p>"; return; } Following are a ...


      List of all options and features


      testConnection () - returns true on success, false on failures


      sendSingle ( true | false ) - default is false

      sendBulk ( true | false ) - default is false, Set this to true if you have lots of recipients (50+)


      to ($recipients) - one emailaddress or array with multiple emailaddresses

      cc ($recipients) - only available with mode sendSingle, one emailaddress or array with multiple emailaddresses

      bcc ($recipients) - one emailaddress or array with multiple emailaddresses

       
      from = 'person@example.com' - emailaddress, can be set in module config (called Sender Emailaddress) but it can be overwritten here

      fromName = 'Name Surname' - optional, can be set in module config (called Sender Name) but it can be overwritten here


      priority (3) - 1 = Highest | 2 = High | 3 = Normal | 4 = Low | 5 = Lowest

      dispositionNotification () or notification () - request a Disposition Notification


      subject ($subject) - subject of the message

      body ($textBody) - use this one alone to create and send plainText emailmessages

      bodyHTML ($htmlBody) - use this to create a Multipart Alternative Emailmessage (containing a HTML-Part and a Plaintext-Part as fallback)

      addSignature ( true | false ) - the default-behave is selectable in config screen, this can be overridden here
      (only available if a signature is defined in the config screen)

      attachment ($filename, $alternativeBasename = "") - add attachment file, optionally alternative basename


      send () - send the message(s) and return number of successful sent messages


      getResult () - returns a dump (array) with all recipients (to, cc, bcc) and settings you have selected with the message, the message subject and body, and lists of successfull addresses and failed addresses,


      logActivity ($logmessage) - you may log success if you want

      logError ($logmessage) - you may log warnings, too. - Errors are logged automaticaly
       
       
      useSentLog (true | false) - intended for usage with e.g. third party newsletter modules - tells the send() method to make usage of the sentLog-methods - the following three sentLog methods are hookable, e.g. if you don't want log into files you may provide your own storage, or add additional functionality here

      sentLogReset ()  - starts a new LogSession - Best usage would be interactively once when setting up a new Newsletter

      sentLogGet ()  - is called automaticly within the send() method - returns an array containing all previously used emailaddresses

      sentLogAdd ($emailaddress)  - is called automaticly within the send() method
       
      Changelog: https://github.com/horst-n/WireMailSmtp/blob/master/CHANGELOG.md
       
    • By Robin S
      An experimental module for adding images to a field via pasting one or more URLs.
      Add Image URLs
      Allows images to be added to an images field by pasting URLs.

      Usage
      Install the Add Image URLs module.
      A "Paste URLs" button will be added to all image fields. Use the button to show a textarea where image URLs may be pasted, one per line. Images are added when the page is saved.
       
      https://github.com/Toutouwai/AddImageUrls
    • By netcarver
      This module provides a very simple interface to a set of named counters. You simply call a single function, next('name'), to pull the next value out of a counter - or to set it up if it does not yet exist. Next() takes a few extra parameters to allow you to increment by values other than 1 or to start at a certain number.
      This provides some similar functionality to the built-in page naming feature of PW, and to this module recently posted by Stikki but I think it offers a little more flexibility than either. Having said that, I do like the simplicity of Stikki's new auto-increment module.
      Module Availability
      Here is my module on Github.
      Here it is in the module repository.
      Example Usage
      Here's how this module can be used to title and name a new page by adding a couple of simple hooks to site/ready.php. This example applies to new pages using a template called 'invoice' that can be quick-added to the page tree. In order to get the following to work, you must edit the template that will be the parent of the 'invoice' template and setup the template for children to "invoice" and set the "Name Format for Children" field to something other than the default blank value (I use  title  as my value.)
      <?php /** * Function to recognise our special template. */ function isInvoiceTemplate($template) { return ($template == 'invoice'); } /** * Pre-load the page title for invoice pages with a unique value * which includes a counter component. */ $pages->addHookBefore("Pages::setupNew", function($event) { $page = $event->arguments(0); $is_invoice = isInvoiceTemplate($page->template); $no_inv_num = $page->title == ''; if ($is_invoice && $no_inv_num) { $counter_name = 'WR-' . date('Y'); $number = $this->modules->get('DatabaseCounters')->next($counter_name, 10, 5000); $page->title = $counter_name . '-' . sprintf("%06u", $number); } }); /** * Prevent ProcessPageEdit from forcing an edit of the name if we got here * through a quickAdd from ProcessPageAdd. We can do this because we * preset the title field in the Pages::setupNew hook. */ $pages->addHookAfter("ProcessPageEdit::loadPage", function($event) { $page = $event->return; $is_invoice = isInvoiceTemplate($page->template); $is_temp = $page->hasStatus(Page::statusTemp); if ($is_invoice && $is_temp) { $page->removeStatus(Page::statusTemp); $event->return = $page; } }); Note, the above code + module is one direct solution to the problem posted here by RyanJ.


      Version History
      1.0.0 The initial release.
    • By flydev
      PulsewayPush
      Send "push" from ProcessWire to Pulseway.
       
      Description
      PulsewayPush simply send a push to a Pulseway instance. If you are using this module, you probably installed Pulseway on your mobile device: you will receive notification on your mobile.
      To get more information about Pulseway, please visit their website.
      Note
      They have a free plan which include 10 notifications (push) each day.
       
      Usage
      Install the PulsewayPush module.
      Then call the module where you like in your module/template code :
      <?php $modules->get("PulsewayPush")->push("The title", "The notification message.", "elevated"); ?>  
      Hookable function
      ___push() ___notify() (the two function do the same thing)
       
      Download
      Github:  https://github.com/flydev-fr/PulsewayPush Modules Directory: https://modules.processwire.com/modules/pulseway-push/  
      Examples of use case
      I needed for our work a system which send notification to mobile device in case of a client request immediate support. Pulseway was choosen because it is already used to monitor our infrastructure.
      An idea, you could use the free plan to monitor your blog or website regarding the number of failed logins attempts (hooking Login/Register?), the automated tool then block the attacker's IP with firewall rules and send you a notification.
       

       
      - - -
      2017-11-22: added the module to the modules directory
       
    • By cosmicsafari
      Hi all,
      Just wondering if its possible to just add some basic output to my modules config page.
      I was wanting to output a bulleted list of some information which I will be pulling from a third party.
      Retrieving the data is fine i'm just not sure how to output it to the config page?
      From what I can see in the link below, it only seems like you can append form items to the page.
      https://processwire.com/blog/posts/new-module-configuration-options/