adrian

Module
Migrator

294 posts in this topic

Ok. I installed 2.8.5 with the ProcessMigrator that you suggested + MigratorWordpress. It seemed to surpass the initial error but now after uploading the wordpress XML file is showing me this error:

 Can't save page 1: /: Selected homepage template cannot be used because it does not define access.

After that, going back to the home screen I only see two pages created (Categories & Tags).

I noticed that the error above is showing only when I use replace/overwrite. If I choose Append, it was showing that all 571 posts were processed properly but in the pages I did not see them.

I decided to try and convert the posts to pages (with comments) and this time I was able to see all the pages created. Now the issue is that I do not see the comments so I guess I should redo the import but setup a template with all the fields created in advance and have them added to a template that would be used for the imported pages. On top of that I did not have installed the comments field module so it seems like it might work once I rethink the whole process again.

Do you have any idea how to make replace/overwrite work without the error saving the page?

P.S. It looks like if I use Append, there are no comments attached to the page even though I have the module now installed and created a template with all fields that were initially created during the Append. So I need to figure out how to define the access for the selected template and try again.

Share this post


Link to post
Share on other sites

@MilenKo - I don't think that error has anything to do with Migrator. Please google this for the access issue: 

site:processwire.com "Selected homepage template cannot be used because it does not define access"

Once that is sorted, I think everything else should work fine. I have only ever imported WP posts into a subpage, like "blog", but importing into home should also be fine.

Append vs Replace/Overwrite should both work fine, but if you are importing to a new install, I would go with append.

I would suggest reducing your xml to just a few posts while you try different things - just so each test is much faster.

BTW, I have never needed to change the post type for normal WP blog post type pages - they just worked as is.

Late here, so won't be able to respond again until tomorrow, so good luck!

Share this post


Link to post
Share on other sites

Adrian, thanks for the points. The need to switch the post type is that the themes I used were using custom post types so most of the converters are made to just look into the standard type - posts or else, but not news (for example). Switching the post type is not an issue and is semi automatic (semi, as I had to do around 100 posts at once to avoid the server timeout.)

I will google the error and see where would that lead. Even if the pages are not properly inserted, but the comments are assigned to a title, that would fully satisfy my needs. Again, I have planned the post cleanup no matter of the CMS but PW is now my top prefference.

Share this post


Link to post
Share on other sites

@MilenKo - if you fix that error and posts are still not being imported fully, PM me your XML file (if you can) and I'll take a look - it should work fine.

1 person likes this

Share this post


Link to post
Share on other sites

I will Adrian. The posts are imported correctly but the comments are not there. This is the only issue. I have a feeling that I might have not set up properly the comments field or something else. There are no issues with sending you the XML file  but I would first try to recover the issue myself using your tool and other infos and in case I am failing, will use the Joker ;)

Share this post


Link to post
Share on other sites

@adrian Looks like I had some progress by creating a simple template matching most of the field names etcl with the wordpress one. Installed the FieldTypeComments and imported the XML. For an unknown reason the name/content of my home page changed to one of the pages from the import as well as the comments are imported in repeating one commment several times (ex, Comment1 x10, Comment 2 x10 etc.) I know I need to go over every page and make the fields properly so that won't be an issue even if I have to work over all comments in the page, but in the admin of the comments I just see them with Pending status and I am unable to just delete the repeating ones.

Share this post


Link to post
Share on other sites

@MilenKo - I don't know why this is proving so problematic for you. I haven't ever imported a WP blog into the PW homepage - always into /blog/, but it has always worked perfectly and I have never had to create any fields ahead of time, including the comment fields. The pending status is a known issue. I had been waiting on the response to this: https://github.com/ryancramerdesign/ProcessWire/issues/1034, which ended up here: https://github.com/processwire/processwire-issues/issues/88 - it looks like it can be solved now.

 

1 person likes this

Share this post


Link to post
Share on other sites

@adrian, I did not import the WP blog directly to homepage but created a page called Import and setup the module to point to it. Somehow I ended up with the home page changed to one of the WP Posts, and inside the include folder I got all the pages with the comments. If you feel like having some time, I would send you the XML archive to try and see if it works for you. Thanks in advance ;)

 

Share this post


Link to post
Share on other sites

Btw, thinking of the conversion from WP to PW, it seems like I need to export from the PW profile the pages, images, comments etc. to the theme profile as well so the fun will be on MAX but it worth every effort knowing how many years I have tried to work the things out with WP and how for a really short time I got my first theme (thanks to you and others).

Share this post


Link to post
Share on other sites

Hi @MilenKo - thanks for the XML file - it was very helpful.

I have committed a new version of Migrator to Github that fixes the duplicate comments issue and also now properly sets the approved status of comments.

Note that I didn't test your complete file (because it would take quite a long time I expect). I cut it down to 5 posts, but with those everything is now working as expected. I didn't see the issue with the homepage being renamed, but maybe there is one post in there that is doing something strange. Could you please test with a limited set to make sure everything is working at your end. And then try with all and let me know about the homepage issue. If you could narrow it down to a particular post then I should be able to fix.

It's probably a good idea to start with a completely fresh PW install (after all your previous attempts, something might be messed up by now) - you can even let Migrator install the comments module and set up the field etc. 

Thanks,
Adrian

2 people like this

Share this post


Link to post
Share on other sites

@adrian Thank you very much for the update and I am glad I helped a bit in the process.

I will try to import a few posts first and see how it goes after which I will try to import the whole database. If the posts are imported properly but only one changes the homepage - I can totally live with this, but I understand your point of having that fixed so that it does not affect somebody else using the module.

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile. After I do that, I will just need to import my existing profile I have almost finished. But let's do this step by step and at the end we will have a nice and smooth approach.

Will let you know as soon as finish both tests.

2 people like this

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

To import the WP to PW I am definitely using the latest PW 3.X with a blank profile.

Oh yes, I should mention that I did my testing with 3.0.55, but it should still work in 2.x

Thanks for helping to narrow down what might be causing that homepage change. I think it will be a matter of doing chunks of posts (maybe 20 at a time, or something like that) and then narrowing down from there. There must be something weird :)

1 person likes this

Share this post


Link to post
Share on other sites

Oh, I should be the one thanking you.

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

Share this post


Link to post
Share on other sites
Just now, MilenKo said:

I was thinking about the post import - would it be possible to do that limitting via the API and some coding versus cutting the posts manually? Saying the import goes in a step of 20, 50 etc. and keeps on doing it until the last one is processed. This way you might import a bigger DB without the need of messing around with the XML.

It would be possible (and probably a good idea) to add this functionality to Migrator, but it's beyond what I have time for at the moment I'm afraid.

2 people like this

Share this post


Link to post
Share on other sites

Huh, it is true - ideas we could have plenty, but there are only 24 hours in 1 day and they are never enough ;)

Thank you again for being so helpful...

2 people like this

Share this post


Link to post
Share on other sites

Hello @adrian I tested the importer and got an error. Here are the steps I took and the error to just check if I did everything right or missed something:

1. I cut the whole WordPress file (14-15Mb size)  in 6 pieces of 2Mb to make it much smaller and reduce the load etc. as per your instructions. To reduce the size of the file I used a simple app called wxrsplitter that I know worked for WP before.

2. Installed fresh PW 3.0.55 with blank profile and only added ProcessMigrator & MigratorWordpress (I downloaded latest update for ProcessMigrator from the git source but MigratorWordpress was kept the same as I did not se any updated files).

3. Created a page Wordpress and assigned it as parent at the Import page.

4. Selected to import user details.

5. Selected to Import created/modified dates.

6. Choose to automatically download and install missing filetypes.

7. As data source used WordPress Migrator and pointed to the first chunk.

The error I got after the import is:

Error: Call to a member function eq() on a non-object (line 1704 of C:\OpenServer\domains\recipemagician.pw\site\assets\cache\FileCompiler\site\modules\ProcessMigrator\ProcessMigrator.module) 

This error message was shown because: you are logged in as a Superuser. Error has been logged.

If I refresh the page, then it continued and finished with the informing message of how many pages were imported successfully. The same thing is applied for the rest of the files or if I use the file at once (which by obvious reasons takes more time to import but does not fail other than the error).

So I pushed the whole file for the test and was able to import 559 posts as child pages of Wordpress. I checked a few pages with comments and they were all posted in the proper amount and approved with an ability to change the status if needed!

It is clear now that all posts were imported. I am unable yet to say would all comments be properly imported but I don't see a reason if one works fine why the rest would not. So will consider that also done perfectly.

In regards to the Home page title, it changed again (before it was Home and now is imported a the first page alphabetically added. Because of that it looses its fields and comments (this happened when I imported the XML chunks). Will try now to import the whole file but I doubt that this would happen again. For sure I can preserve the content of the page imported to the Home by creating and adding all the necessary fields to it and reimporting, but just reporting that something is still to be fixed. If a need be, I can provide you with the slices of XML.

1 person likes this

Share this post


Link to post
Share on other sites

Ok. Tried with the whole file and for the testing purposes inside the Home I created a child page WordPress inside of which I created another child calling it Import. Pointed the module to the Import page as parent and thought that this way the Home would not change but would the Wordpress. Well I was wrong - the Home title changed again to the same so for some reason the import grabs a page (probably the first) and imports it to Home.

On top of that, if I try to import the whole file, then some of the pages were not imported (those that were originaly as Wordpress pages). Categories list is empty too.

After the import in both cases I noticed another error:

 SQLSTATE[42S22]: Column not found: 1054 Unknown column 'post' in 'field list' along with the message of successful import. So for now importing the XML in chunks works much better and properly includes categories as well as users etc. Even if I will have to recreate the page that is imported in the Home, I am still totally OK but just reporting to make sure others won't be able to move on.

1 person likes this

Share this post


Link to post
Share on other sites

@MilenKo  - thanks for all those details. I have pushed a new version which fixes that eq() error.

I finally did a full import of your file and I think I now know why you are having troubles. Your XML is for the entire Wordpress site. I have only ever used this to migrate blog posts. I would suggest that you create an XML export of just posts (I think that's an option). I think that will probably take care of the homepage renaming issue, as well as the categories and tags problems. It's all about the pages being relative to the correct parent and when you import pages and posts at once, it messes that up. Obviously this should be fixed, but I think that for the moment if you do this, you'll get a complete import of all posts - fingers crossed :)

1 person likes this

Share this post


Link to post
Share on other sites

Adrian, you are 100% right and how come I did not think of that earlier. I will export now only the posts, but I am not sure if I do that will the export include the comments too. Will test it right away and see what is happening. One thing is sure though - nobody would care if the name of the old website was imported or not as those settings will have to be changed to apply with PW and new theme. 

Crossing my fingers and trying it now with the latest version of the Migrator.

Share this post


Link to post
Share on other sites

Hello @adrian . I installed your updated module and tested to import ONLY THE POSTS export. To be clear, I still needed to convert all custom type posts to the standard Wordpress posts as otherwise it only creates a few empty pages and that is all.

I tried to start the import this time with the entire XML so as expected I got an error:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away  (in other words it means that the import took quite a time and server gives up). 

To help the server "survive" I decided to edit the my.cnf changing the wait_timeout from 60 to 6000 and this time it worked fine.

Unfortunately the issue still exists with the Home page being renamed and unfortunately it is a page that has some comments in it too. It appears that it does not just import to the parent of the page pointed to the module but straight to home.

Another thing I just noticed is that the comments date is not preserved, so most of the comments appear to be posted with a timestamp of the import. It might not be such a big issue, however it won't make much sense to have the page added 3-4 years ago and containing 50-100 comments within the last XX minutes.

In other words if the comments timestamp can be fixed and the page that imports into the home properly inserted to the parent, it will be all perfect!

 

Share this post


Link to post
Share on other sites

Thanks @MilenKo - I'll take a look at the comment timestamp issue tomorrow. Could you please PM me the new XML with just the posts. I might also look at working with the full one - we'll see.

How did you go with Categories with just the posts in the XML?

Share this post


Link to post
Share on other sites

@adrian I did not care much of the categories as I am planning on completely redoing most of the post information, pictures, tags, categories etc. I noticed that after moving the custom type posts to the standard WP post all the posts got to Uncategorized (the default WP category). During the import I do see the category created and pages have the Uncategorized as their category so my guess is that if everything was OK with the posts and categories, they will be imported properly.

One other thing I just noticed is that the tags are also not imported and the field that was automatically created by the module is not allowing to add new tags (need to change a few settings to allow it). Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

I am PM-ing you the XML and thanking you in advance for looking into it.

Share this post


Link to post
Share on other sites
1 minute ago, MilenKo said:

Again, reaching to this level is waaay more than I have ever expected so I am sure others would follow switching from WP to PW and enjoying the simplicity and elegance of the code (not to mention the support of the community!)

Thanks, but I am certain this used to work fully, so I would like to get it back to that state.

I have the comment date sorted out, but am working on a couple of other things and have to head out shortly, so probably won't post until tomorrow, so will look again at categories and tags.

1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By psy
      ProcessWire DropboxAPI on GitHub: https://github.com/clipmagic/DropboxAPI
       
      This module is a wrapper for Kunal Varma's Dropbox PHP SDK https://github.com/kunalvarma05/dropbox-php-sdk
      It was created to perform a specific function for a client, ie upload PDF files on a PW website to a specific Dropbox account folder.
      The PDF files, created using @Wanze's excellent PagesToPdf module using the WirePDF functions, are generated from Formbuilder forms completed by front-end site visitors.  Works a treat!
      There's more that could be done to allow ProcessWire to take advantage of all the features of the SDK, eg downloads, multiple Dropbox accounts, etc. You are welcome to request changes and/or fork the GitHub project to extend the feature set.
      Enjoy!
      System requirements
      PHP 5.6.4 or greater Composer The PHP mbstring extension General information
      This module enables you to access a single Dropbox (www.dropbox.com) account to upload files from a ProcessWire website.
      All kudos to (https://github.com/kunalvarma05/dropbox-php-sdk) for the PHP API.
      First steps
      Visit (https://www.dropbox.com/developers) and read the documentation. Log into Dropbox and create a new application. It's recommended to limit ProcessWire App access to a specified folder Make a note of the App key, the App secret and the name of the Dropbox folder Installation
      Download the zip file into your site/modules folder then expand the zip file. Next, login to ProcessWire > go to Modules > click "Refresh". You should see a note that a new module was found. Install the DropboxAPI module. Configure the module with your App key, App secret and your Call Back URL You need to generate a Dropbox access token to enable your site to communicate with the nominated Dropbox account. Dropbox will generate a token for you or you can create a page for the front end of your ProcessWire site with a template to submit the token request to Dropbox, eg:
      <?php namespace ProcessWire; $drop = $modules->get('DropboxAPI'); if ($input->get->code && $input->get->state) { $code = $sanitizer->text($input->get->code); $state = $sanitizer->text($input->get->state); //Fetch the AccessToken $accessToken = $drop->getAccessToken($code, $state); echo "Copy/paste this code into the module configuration: " . $accessToken; } else { echo "<p><a href='" . $drop->getAuthURL() . "'>Log in with Dropbox</a></p>"; } ?> Once you have entered the token in the module configuration, you can unpublish this page.
      Usage
      Read the dropbox-php-sdk documentation!
      An example template for sending a file to a Dropbox App folder from ProcessWire:
      <?php namespace ProcessWire; use Kunnu\Dropbox\Dropbox; use Kunnu\Dropbox\DropboxApp; use Kunnu\Dropbox\DropboxFile; // send pdf to Dropbox $drop = $modules->get('DropboxAPI'); $app = new DropboxApp($drop->app_key, $drop->app_secret, $drop->authorization_code); if ($app) { //Configure Dropbox service $dropbox = new Dropbox($app); $dropboxFile = new DropboxFile('/path/to/myfilename.pdf'); $file = $dropbox->upload($dropboxFile, "/myfilename.pdf", ['autorename' => true]); //Uploaded File meta data if ($file) { $success = $file->getName() . " uploaded to Dropbox"; $drop->log($success); } }  
       
    • By sirhc
      Hi all,
      Im trying to install this core module and got the error of not having the ImageMagick library installed.
      But if im checking for the version of my ImageMagick installed on my server via SSH im getting this response;
      identify -version
      Version: ImageMagick 6.9.4-9 Q16 i386 2016-06-21 http://www.imagemagick.org
      Copyright: Copyright (C) 1999-2016 ImageMagick Studio LLC
      License: http://www.imagemagick.org/script/license.php
      Features: Cipher DPC 
      Delegates (built-in): bzlib freetype gslib jng jpeg png ps tiff xml zlib
      Does this even mean the library is installed? And if yes, why cant i install my module?
      Pleaassee help me
      Greetz.
    • By kixe
      FieldtypeColor is on github

      Fieldtype stores a 32bit integer value reflecting a RGBA value.
      Input
      5 types of Inputfields provided
      Html5 Inputfield of type='color' (if supported by browser)    Inputfield type='text' expecting a 24bit hexcode string (RGB). Input format: '#4496dd'.
      The background color of the input field shows selected color Inputfield of type='text' expecting 32bit hexcode strings (RGB + alpha channel) Input format: '#fa4496dd' Inputfield with Spectrum Color Picker
      (Options modifiable) Inputfield type='text' with custom JavaScript and/or CSS (since version 1.0.3) Output
      Define output format under 'Details' tab in field settings. Select from the following 8 options
      string 6-digit hex color. Example: '#4496dd' string 8-digit hex color (limited browser support).  Example: '#fa4496dd' string CSS color value RGB. Example: 'rgb(68, 100, 221)' string CSS color value RGB. Example: 'rgba(68, 100, 221, 0.98)' string CSS color value RGB. Example: 'hsl(227, 69.2%, 56.7%)' string CSS color value RGB. Example: 'hsla(227, 69.2%, 56.7%, 0.98)' string 32bit raw hex value. Example: 'fa4496dd' int 32bit. Example: '4198799069' (unformatted storage value) The Fieldtype includes
      Spectrum Color Picker by Brian Grinstead

      SCREENSHOTS
      Input type=text with changing background and font color (for better contrast)

      Input type=color (in Firefox)


      Javascript based input (Spectrum Color Picker)


      Settings Output

       
      Settings Input

    • By zlitrox
      Hi!
      I'm relatively new to the world of Processwire, but so far I'm really impressed by this CMS and its ease, power and speed.
      I've been looking at the different field types and also existing CKEditor modules (pwimage and pwlink).
      Here's what i want to achieve:
      I would like to use or create a own module which allows my admin users to upload pdf files inline in a CKEditor field (page content) the same way images are handled. It should be possible to upload a pdf file and specify its name and from the file uploaded and name i want to show an icon or image inline in editor and the output for that upload should be customized based on the module. Lets make it simple and say that i upload datasheet-1.pdf and want it to be named "My product datasheet", i want the output to be:
      <a href="{ link to uploaded pdf file }">My product datasheet</a>  
      Also the uploaded pdf files should be related to the page the user is editing.
       
      Is this doable? 
       
      Any guidance would be greatly appreciated! Thanks in advance.
      PS. I'm a PHP programmer so i know it would require some custom code, but the real question is where do i start, what should it take and is it even possible to do?
    • By flydev
      Auth2Login for ProcessWire
      A Module which give you ability to login an existing user using your favorite thrid-party OAuth2 provider (i.e. Facebook, GitHub, Google, LinkedIn, etc.)..
      You can login from the backend to the backend directly or render a form on the frontend and redirect the user to a choosen page.
      Built on top of ThePhpLeague OAuth2-Client lib.
      Registration is not handled by this module but planned.
       
      Howto Install
      Install the module following this procedure:
       - http://modules.processwire.com/modules/oauth2-login/
       - https://github.com/flydev-fr/OAuth2Login
      Next step, in order to use a provider, you need to use Composer to install each provider
      ie: to install Google, open a terminal, go to your root directory of pw and type the following command-line: composer require league/oauth2-google
      Tested providers/packages :
          Google :  league/oauth2-google     Facebook: league/oauth2-facebook     Github: league/oauth2-github     LinkedIn: league/oauth2-linkedin
      More third-party providers are available there. You should be able to add a provider by simply adding it to the JSON config file.

      Howto Use It
      First (and for testing purpose), you should create a new user in ProcessWire that reflect your real OAuth2 account information. The important informations are, Last Name, First Name and Email. The module will compare existing users by firstname, lastname and email; If the user match the informations, then he is logged in.
      ie, if my Google fullname is John Wick, then in ProcessWire, I create a new user  Wick-John  with email  johnwick@mydomain.com
      Next step, go to your favorite provider and create an app in order to get the ClientId and ClientSecret keys. Ask on the forum if you have difficulties getting there.
      Once you got the keys for a provider, just paste it into the module settings and save it. One or more button should appear bellow the standard login form.
      The final step is to make your JSON configuration file.
      In this sample, the JSON config include all tested providers, you can of course edit it to suit your needs :
      { "providers": { "google": { "className": "Google", "packageName": "league/oauth2-google", "helpUrl": "https://console.developers.google.com/apis/credentials" }, "facebook": { "className": "Facebook", "packageName": "league/oauth2-facebook", "helpUrl": "https://developers.facebook.com/apps/", "options": { "graphApiVersion": "v2.10", "scope": "email" } }, "github": { "className": "Github", "packageName": "league/oauth2-github", "helpUrl": "https://github.com/settings/developers", "options": { "scope": "user:email" } }, "linkedin": { "className": "LinkedIn", "packageName": "league/oauth2-linkedin", "helpUrl": "https://www.linkedin.com/secure/developer" } } }  
      Backend Usage
      In ready.php, call the module :
      if($page->template == 'admin') { $oauth2mod = $modules->get('Oauth2Login'); if($oauth2mod) $oauth2mod->hookBackend(); }  
      Frontend Usage
      Small note: At this moment the render method is pretty simple. It output a InputfieldForm with InputfieldSubmit(s) into wrapped in a ul:li tag. Feedbacks and ideas welcome!
      For the following example, I created a page login and a template login which contain the following code :
      <?php namespace ProcessWire; if(!$user->isLoggedin()) { $options = array( 'buttonClass' => 'my_button_class', 'buttonValue' => 'Login with {provider}', // {{provider}} keyword 'prependMarkup' => '<div class="wrapper">', 'appendMarkup' => '</div>' ); $redirectUri = str_lreplace('//', '/', $config->urls->httpRoot . $page->url); $content = $modules->get('Oauth2Login')->config( array( 'redirect_uri' => $redirectUri, 'success_uri' => $page->url ) )->render($options); }

       
      Screenshot