Module Migrator

[adrian]

I think Nico is working on a few more tweaks to this as we speak.

Firstly, MigratorWordpress is not designed to give you a fully functioning blog in that it doesn't create a template to render your posts. You could use the template from Ryan's blog profile though and depending on names of fields it will work, although I haven't compared these to be certain.

I just tested Nico's new version with a recently exported XML from the current version of WP and it works perfectly. All the tags and categories were created, the page fields created, and they were assigned to each post correctly. The only current issue I had was some issues with html and body tags making it into the body field of the post (which affect the viewing in edit mode), which I think Nico is working on now.

One thing to be aware of is the "Parent Page" setting when importing. If you are testing and trashed a previous import, this will load up that original page from the trash by default. I might need to consider doing a check to make sure this can't happen.

Also, out of curiosity, are you selecting your Home page as the parent, or a child of this? My successful test this morning was to a child page - eg: Home > Blog

[totoff]

Hi Adrian,

thanks for your reply.

Firstly, MigratorWordpress is not designed to give you a fully functioning blog in that it doesn't create a template to render your posts.

I assumed so but was unsure. Thanks for clarifying.

This is what I tried out in the meantime:

• clean, untouched PW install
• import the blog profile from the zip you posted earlier in this thread. Choose home as parent -> perfect working blog, script added /blog/ and the appropriate children to the page tree
• import WP-XML, parent = /blog/, import everything, append, import dates = true, edit content = false -> all posts, categories, tags are there but categories are not assigned to posts and all post dates are today. Page tree as before.

For me this is perfectly good - more than I ever expected. I'll give this a go now with the site I'm working on (keep my fingers crossed ...) and adjust the template styling to my needs.

Thanks again!

[adrian]

Thanks for the report.

Not sure why the categories are not being assigned to your posts. I am getting mixed success with this - on a clean install it always works, so I am wondering if it's an issue with the fields already existing. Have you played with the Import Type options? I wonder if maybe using "Replace" might work?

As for the dates being today - is that the modified date or the created date? The created date works for me, but I still need to fix the modified date - unfortunately when PW saves the page, the modified date is instantly changed. Quiet mode saving works for the created date, but not the modified date. I might need to do this using SQL.

Thanks for helping us to debug this.

[adrian]

Ok, I just tested the "Replace" option and it doesn't solve the tags/categories issue, but if you manually remove those two fields from the post template and then delete them, the next import works perfectly with tags and templates assigned to posts as they should be. I will try to figure out why and see what I can do to avoid the need to do this.

[Nico Knoll]

Thats what i meant in my pm, Adrian. The fields have got a "hard coded" parent after creating. And when importing a second time without removing the fields it keeps the parent because fields aren't overwritten (are they?)

But at the end you only should have to use it one time so that won't be the problem but not really satisfying for me :/

[adrian]

Nico - yeah, I get what you meant now - I thought you were referring to the Parent Page setting during import.

I was actually just looking at the category and tag field input tabs and seeing the parent set to:

/trash/1297_blog-test-3/ Categories 

Which as you say, is exactly why they aren't working when you run an import, then trash the entire blog page tree and import again.

I should be able to to add a check in though to replace the parent if it has a status that flags it as being in the trash. I think that should work fine.

[adrian]

I have just committed a couple of updates that take care of an image filename issue when using the Wordpress import, and more importantly it now completely honors the modified user and modified date values from the original source, whether from Wordpress or another PW installation.

[totoff]

Hi,

I'll continue testing tonight. Will be occupied with other tasks during the day unfortunately. Sent via mobile.

[adrian]

Thanks @totoff,

We have both made a few enhancements in the last couple of hours, so be sure to grab the latest versions of both modules. There are still a few things to iron out, but it's starting to look really good - hopefully in the next couple of days we can get the final tweaks sorted.

Back to real work now

[adrian]

Minor, but important fix just committed to prevent domdocument from adding docytype, html, and body tags to RTE fields.

[totoff]

Hi Adrian, Nico,

sorry that it took a while to get back. Unfortunately I can't report that this is working completely fine for me. Here is what I did:

1st attempt

• Parent = Home
• Everything = true
• Append = true
• Import Dates = true
• File = blogprofile.zip
• Edit content = false
• Result = perfect

Trying to import WP-Data

• Everything = true
• Parent = Blog
• Append = true
• Import Dates = true
• Edit content = false
• Warning: Unknown Selector operator: '=%' -- was your selector value properly escaped?
• Result = Import successful, no tags/categories assigned, post dates as of today. Page tree as before

2nd Attempt

• Remove fields tags and categories from template post
• deleted the fields
• Parent = Blog
• Import everything = true
• Replace = true
• Import Dates = true
• Import-file = WP File
• Edit content = false
• Warning: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'c2-a719-1108' for key ‚name_parent_id'
• Result = Blog posts are present but can’t be edited in the admin. Warning : Unknown Selector operator: '' -- was your selector value properly escaped?

3rd Attempt

• Rollback installation to clean state from 1st attempt
• Repeat import blogprofile.zip from attempt 1
• Result = perfect
  ​

Import WP-Data

• Everything = true
• Parent = Blog
• Replace = true
• Import Dates = true
• Edit content = false
• Warning: Unknown Selector operator: '=%' -- was your selector value properly escaped?
• Result = Import successful, no tags/categories assigned, post dates as of today (like with 1st attempt)

Before testing I installed the last versions of your modules.

[adrian]

Thank you for the thorough testing and detailed reporting - really appreciate it.

I had never tried the combo of importing that blog.zip I posted and then importing some WP posts, so I just gave it a go and found it worked flawlessly, except for assigning tags and categories to the posts - there must be a conflict with those fields already being created from the import of Ryan's blog.zip

This sounds mostly like your first attempt, except that for me, I didn't get the selector operator warning and the post dates were imported correctly. 

Can I ask what you mean by: "Page tree as before"?

I am going to see what I can do about the tag/category assignment to posts problem when using Ryan's blog profile and on subsequent install attempts. Once I get that sorted, it would be great if you can test again and make sure debug mode is on to see if we can get some more info on that selector error.

I am not sure why the post dates aren't importing for you, but I'll take a look at that too.

Stay tuned

[adrian]

Ok, I think we are sorted

I made some changes to the module so get the latest version - there were some session variable issues regarding the Import Type not being used as selected.

This is exactly what I did and I got Ryan's blog profile installed and a test install of 40 WP blog posts, with tags and categories assigned and users and dates imported.

Started with a clean PW install.

Import the blog.zip with the following settings

1. Parent Page: Home
2. Components to Import: Everything
3. Import Type: Append
4. Import User Details: Checked
5. Import Created/Modified Dates: Checked
6. Data Source: blog.zip

Import wordpress xml file with the following settings

1. Parent Page: Blog (this parent page was created by the blog.zip import)
2. Components to Import: Everything
3. Import Type: Replace
4. Import User Details: Checked
5. Import Created/Modified Dates: Checked
6. Data Source: wordpress.xml

Now that kongondo has released his Blog module (https://processwire.com/talk/topic/6465-module-blog/) I think it would be awesome if Nico's MigratorWorpress was made compatible with the page, template, and field names that kongondo is using, because this will be a much better approach than using the blog.zip import I put together. Perhaps a way to configure the names of these in the module config settings!

Anyway totoff, please let me know if it works for you now. I have run this a few times now and it seems perfect!

[diogo]

Now that kongondo has released his Blog module (https://processwire.com/talk/topic/6465-module-blog/) I think it would be awesome if Nico's MigratorWorpress was made compatible with the page, template, and field names that kongondo is using

Maybe it would make sense if PW would have some metadata inputs for all kinds of data (like it has "tags") so we could have some conventions to make it easier this kind of interactions between modules. Like this we could mark the blog parent page on Ryan's blog profile an Kongondo's blog module with the same meta-name, and the blog pages template also with the same meta-name and the same goes for main fields on blog pages (you get the idea). Like this any importing script could easily target all profiles that would comply to conventions. Hope I explained clearly...

[adrian]

I agree diogo!

Just a small issue for anyone who goes to test the WP import. If you are using TinyMCE on the body field, images won't display in the editor. There is a tiny bug in MigratorWordpress that breaks the valid_elements setting. If you want to test before Nico fixes this, after the import, just replace the valid_elements for the body field with the default settings:

@[id|class],a[href|target|name],strong/b,em/i,br,img[src|id|class|width|height|alt],ul,ol,li,p[class],h2,h3,h4,blockquote,-p,-table[border=0|cellspacing|cellpadding|width|frame|rules|height|align|summary|bgcolor|background|bordercolor],-tr[rowspan|width|height|align|valign|bgcolor|background|bordercolor],tbody,thead,tfoot,#td[colspan|rowspan|width|height|align|valign|bgcolor|background|bordercolor|scope],#th[colspan|rowspan|width|height|align|valign|scope],code,pre

[totoff]

Hi adrian,

let me say how much I appreciate the huge effort you put into this. Awesome work!

Unfortunately I dont't have good news regarding my specific case. I get the same results as before with my client's XML-file (I did export it once again to be sure). I again have this misterious warning and categories/tags are not assigned.

However, I suspect something could be wrong with the XML export. If you would like to, please PM me your XML-file an I'll be glad to test it here with my installation in question.

EDIT: Sorry, I forgot the debug mode before importing. Apologies.

[adrian]

You're not making me happy

I'll PM you the XML I have been testing with. I don't suppose you'd be able to send me your one as well?

What version of Wordpress are you running - this could be relevant?

I don't think it should matter, but also what version of PW are you running - I have been testing the latest dev.

[totoff]

Hi adrian,

PW is 2.4.0

I'll PM you the file. It's published content anyway. Nothing top secret.

WP is 3.8.1 - updates come faster than I can keep up with    

[adrian]

Just an update for those following intently

Turns out there was a character encoding issue in totoff's WP xml export. WP was url encoding special characters which is why he was getting the selector error. 

Remove that and it all works because that selector error was responsible for the categories/tags assignment problem and the post date stuff as well.

[horst]

Now that it seems to be that simple to convert from WP to PW, I'm a bit sad that I don't have any WP site.

[adrian]

Yeah, I am sure you are distraught

Now we just need someone with some Drupal skills to write a MigratorDrupal plugin.

Anyone?

[Nico Knoll]

Update: Migrator Wordpress 1.3

A lot of bug fixes are contained in this new update but the main change is that you can now adjust field and template names is the module's settings. That means that if you already created fields or templates or started with the blog profile or started with kongondos BlogMarkup you can import data right into the needed position

In my testing it worked like a charm with adrian's new version. So go - try it yourself

[totoff]

Now that it seems to be that simple to convert from WP to PW, I'm a bit sad that I don't have any WP site.

You could set up one just for fun. WP is a preferred one-click-solution. Go ahead, it's easy.

[Nico Knoll]

@totoff: Not that easy. I tried a friend of mine to set up a blog ant thought WP.con might be a good option because free hosting. And after 20 minutes of struggeling in the backend I told her that we should use Tumblr instead. Now she uses Tumblr

[totoff]

Update: Migrator Wordpress 1.3

Tried it. Worked like a charm.

@totoff: Not that easy.

I was joking.