How import database for wordpress ( user database )

[Marco Ro]

Hi,

Someone has an idea of how to import the user database for wordpress to pw? 

I see the pw database don't have the same structure of wp database. Of course and i'm happy about this, the pw database is big 10 time sell and work better! But about import the user data, some one have an idea? I use LoginRegister module by @flydev (hi flydev).The worpdress data use this structure:

INSERT INTO `pw_users` (`ID`, `user_login`, `user_pass`, `user_nicename`, `user_email`, `user_url`, `user_registered`, `user_activation_key`, `user_status`, `display_name`) VALUES

I see in the pw all the data are split in different tables.  Is there an away to import the data or I have to make the profile one by one in the backend? 

Maybe I can use http://modules.processwire.com/modules/import-pages-xml/ but not really sure how to use it.

Thank you!

[pwired]

 

https://processwire.com/talk/topic/6655-migration-from-wordpress-to-processwire/

 

[Marco Ro]

Thank you @pwired, I try to use this module, I have made the Json file try to upload  but give me this error,

Anyway I don't think can work. because this module don't ask me where I want store the data. 

--------

I think something like the modulo http://modules.processwire.com/modules/import-pages-xml/ of @justb3a could work. The only problem it's this fill the template, and the registration filed aren't in a template but inside a module. I can not connect the xml file to the field for let go the info in the right database tables. 

[Sergio]

You may find great insights on this thread below, on how to do it by hand. A bit old, but probably everything works even today.

 

[adrian]

@MarcoPLY - I haven't used it in a while and unfortunately I don't have time to offer any support, but take a look at: https://github.com/nicoknoll/MigratorWordpress

[Marco Ro]

Thank you @Sergio 

I have the wordpress database, I can use xml, json or any other export from SQL database. I have to import all the user. I use LoginRegister module that write the data in different tables, depends on the field used. But I don't have idea how do this.  

I read the article and yes Ryan do something to similar, but honestly I'm not able to do such a thing.

---

@adrian thank you, at moment I don't have the wordpress installation, I will try to make a copy in local and try to export the new xml file. I try with ProcessMigrator but the xml form the just database not work.

[Autofahrn]

If you can export the user as JSON, it should be simple enough to write a small importer in PHP.

When I did similar, I created a template with a simple file field holding the JSON containing the user database. The associated template php starts like this:
 

<?php namespace ProcessWire;
if(isset($page->json_file))
    $UserData = json_decode(file_get_contents($page->json_file->path . $page->json_file->first()->name), true);
else
    $UserData = [ ];	// ensure always set, yet empty.

depending on your JSON structure you'll iterate through your users somewhat like this:

foreach($UserData as $v)
	{
	// Fetch whatever fields should be imported
	$UserLoginName = $v["LoginName"];
	$UserRealName = $v["RealName"];
	$UserMail = $v["EMail"];

In my implementation I'm building a pagename from the original name and check if the user actually exists:

	$UserPwName = $sanitizer->pageName($UserLoginName, true);
	$usr = $users->get("name={$UserPwName}");

Of course you only want to add a user which does not exist yet (you may do other things like update for existing users):

	if($usr->id == 0)	// User does not exist
		{
		$usr = $users->add($UserPwName);	// Create user
		if($usr->id)	// Success?
			{
			$usr->of(false);	// Prepare for update
			$usr->email = $UserMail;
			$usr->addRole('imported-user');	// add some special role here
			// add more fields depending on your use case
			// $usr->user_name = $UserRealName;
			$usr->save();
			}
		else
			{
			// may $log->error("ERROR creating user {$UserRealName}");
			}
		}

Shouldn't be too difficult to adapt for other scenarios. Take care of the passwords, which likely can't be migrated that way!

[bernhard]

Yep, as @Autofahrn said it's easy to do that via the API and a foreach. But I would suggest using Tracy Console instead of a template. Just create your code, debug it instantly using d() and CTRL/ALT+Enter and then finally do the import when everything works as expected.

You can use the Password class to generate new passwords:

$pw = new Password();
$pw = $pw->randomPass([
  'minLength' => 10,
  'maxLength' => 10,
  'maxSymbols' => -1,
  'minUpper' => 0,
  'maxUpper' => -1,
]);

See https://github.com/processwire/processwire/blob/master/wire/core/Password.php#L419

[Autofahrn]

Sure, using the debugger for such one-shot solutions may be faster to go. In my case the template contains much more functionality, since it generates a regular visible page. There I could pick the users to import or update information after changing fields in the user template etc.

[bernhard]

11 hours ago, Autofahrn said:

Sure, using the debugger for such one-shot solutions may be faster to go. In my case the template contains much more functionality, since it generates a regular visible page. There I could pick the users to import or update information after changing fields in the user template etc.

In this case I'd create a processmodule ? ?  Don't want to be besserwisserisch, really just saying it's very easy to create them and you get a lot of flexibility and features with very little effort. For example a little import site could be as simple as this (available at yourmodulepath/import in the admin):

public function executeImport() {
  if(input()->post->submit) {
    // do the import
    $this->import();
  }

  $form = modules('InputfieldForm');
  $form->add([
    'type' => 'text',
    'name' => 'myinputfieldname'
  ]);
  $form->add([
    'type' => 'submit',
    'name' => 'submit',
    'value' => 'import my data!',
  ]);
  return $form->render();
}

private function import() {
  // import logic
}

 

[Marco Ro]

Thank you @Autofahrn and @bernhard for your help! 

I was write you just new! I don't know what mean besserwisserisch but sure you are so welcome!! 

so, this is what I was writing, but now I will study your code and try to use it!!

 

A few things:

I used Tracy Console but I think this is wrong, How can set where thank the file?

$UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);

 

For the password  if i take the one is in the file, the encrypted password, this will work? or I have anyway to setup a new one?

I think to have make some more error but I can not understand where, the filed are the right.

<?php namespace ProcessWire;
if(isset($page->json_file))
    $UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
else
    $UserData = [ ];	// ensure always set, yet empty.
    
    foreach($UserData as $v)
	{
	// Fetch whatever fields should be imported
	$UserLoginName = $v["user_nicename"];
	$UserMail = $v["user_email"];
	
	$UserPwName = $sanitizer->pageName($UserLoginName, true);
	$usr = $users->get("name={$UserPwName}");
	
	if($usr->id == 0)	// User does not exist
		{
		$usr = $users->add($UserPwName);	// Create user
		if($usr->id)	// Success?
			{
			$usr->of(false);	// Prepare for update
			$usr->email = $UserMail;
			$usr->pad_name = $UserLoginName;
			$usr->addRole('customer');	
			$usr->save();
			}
		else
			{
			// may $log->error("ERROR creating user {$UserRealName}");
			}
		}

in the debug i see only: Exception: syntax error, unexpected end of file on line: 31

also, can I run all the process directly inside the Tracy Console? 

 

 

[bernhard]

@MarcoPLY my comment was for @autofahrn and since he is german he will understand besserwisserisch ? a processmodule might be overkill in your situation. it's more advanced but once you get the concept it makes you more efficient in lots of ways.

Regarding your problem: Start with this code and then continue step by step until you get your result. Use d() = dump() to chech the content of your variables. Then you see instantly wheter the variable contains some data or is NULL:

$UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
d($UserData);

edit:

$http will not work, you can just try d($http); which will show you this error:

Error: Undefined variable: http on line: 1
null

So you need to do this first:

$http = new WireHttp();
d($http); // shows ProcessWire\WireHttp

 

[Marco Ro]

yes there are a lot of things that are to advance for me, so I not really know what I do!

yes now the file is recognized! the problem is that not take the data because tell me the Error: Undefined index: user_login on line: 8 and line 9  that are 
 

$UserLoginName = $v["user_login"];
$UserMail = $v["user_email"];

 

Could be the problem came for the json file. This is an example of a row:

{"ID":"1","user_login":"myname","user_pass":"$P$BKVS\/30bpzejiORa9nGupsAur7g0P\/","user_nicename":"mynicename","user_email":"email@email.it","user_url":"","user_registered":"2016-07-28 16:13:39","user_activation_key":"","user_status":"0","display_name":"myname"},

 

This is all the code:

$http = new WireHttp();
d($http); // shows ProcessWire\WireHttp
$UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
d($UserData);
    
    foreach($UserData as $v) {
    	// Fetch whatever fields should be imported
    	$UserLoginName = $v["user_login"];
    	$UserMail = $v["user_email"];
	
	$UserPwName = $sanitizer->pageName($UserLoginName, true);
	$usr = $users->get("name={$UserPwName}");
	
	if($usr->id == 0)	// User does not exist
		{
		$usr = $users->add($UserPwName);	// Create user
		if($usr->id)	// Success?
			{
			$usr->of(false);	// Prepare for update
			$usr->email = $UserMail;
			$usr->pad_name = $UserLoginName;
			$usr->addRole('customer');	
			$usr->save();
			}
		else
			{
			// may $log->error("ERROR creating user {$UserRealName}");
			}
		}
    }

If I can do let this work I promise not to bore you again! load users one by one ...

 

[zoeck]

Your code looks good... i think the problem is your json file, i tested it with this json Data:

[
{"ID":"1","user_login":"myname","user_pass":"$P$BKVS\/30bpzejiORa9nGupsAur7g0P\/","user_nicename":"mynicename","user_email":"email@email.it","user_url":"","user_registered":"2016-07-28 16:13:39","user_activation_key":"","user_status":"0","display_name":"myname"},
{"ID":"2","user_login":"myname2","user_pass":"$P$BKVS\/30bpzejiORa9nGupsAur7g0P\/","user_nicename":"mynicename","user_email":"email@email.it","user_url":"","user_registered":"2016-07-28 16:13:39","user_activation_key":"","user_status":"0","display_name":"myname"}
]

You need the Start "[" and End "]" (and no comma after the last entry! ? )

[Autofahrn]

2 hours ago, bernhard said:

In this case I'd create a processmodule ? ?  Don't want to be besserwisserisch, really just saying it's very easy to create them and you get a lot of flexibility and features with very little effort.

We are here to exchange ideas, so please try to stay besserwisserisch as I understand that I needed to explain my approach a little better ?. My JSON data not only contained user login information but other stuff like avatar images, signatures various settings (from the origin) etc. My template is more a viewer for that data which allows to import or update contents on a per-user base. Of course I could also create that as a module, but for that one-time use going with a template was more straight forward for me. I prefer making a module for the reusable stuff..,

2 hours ago, MarcoPLY said:

For the password  if i take the one is in the file, the encrypted password, this will work? or I have anyway to setup a new one?

That's what I meant that migrating the password (probably) does not work, unless you are able to decrypt the hash code stored in the database. I'd set some randomized password and integrate the LoginRegister module along with ForgetPassword so users are able to restore their accounts on their own.

And, yes, I've omitted the closing brace in my snippets (thought it was too obvious).

2 hours ago, MarcoPLY said:

Could be the problem came for the json file. This is an example of a row: 

The row looks ok, but maybe your JSON includes some additional headers which need to be referenced first (the part before the first row of user data) like this:

{
	"Type" : "UserData",
	"Users" : [
		{ "ID": ....

Then your foreach would have to start on $UserData["Users"] and you could verify $UserData["Type"] before starting the import.

[Marco Ro]

yes @Autofahrn I have an additional headers.

[
{"type":"header","version":"4.7.3","comment":"Export to JSON plugin for PHPMyAdmin"},
{"type":"database","name":"stef4341_playwood"},
{"type":"table","name":"name_users","database":"nome_data","data":
[
{... row ...}
{... row ...}
]
}
]

In this case what I need use?

Anyway i try just to delate the header and keep only the data in 

[
{data}
{data}
]

This more or less work! give me this error: 

ERROR: Maximum execution time of 30 seconds exceeded on line: 1587 in /Applications/MAMP/htdocs/playwood/wire/core/Page.php

But I think if I split the data in tow file I will fix it.

The second thing is that yes make the users in users page with the correct rules, but not take me the correct data, instead of taking me $v["user_login"] and $v["user_email"] but make all the name with the day and hour when it's created.

This is all the file

$http = new WireHttp();
d($http); // shows ProcessWire\WireHttp
$UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
d($UserData);
    
    foreach($UserData as $v) {
    	// Fetch whatever fields should be imported
    	$UserLoginName = $v["user_login"];
    	$UserMail = $v["user_email"];
	
	$UserPwName = $sanitizer->pageName($UserLoginName, true);
	$usr = $users->get("name={$UserPwName}");
	
      $pw = new Password();
      $pw = $pw->randomPass([
        'minLength' => 10,
        'maxLength' => 10,
        'maxSymbols' => -1,
        'minUpper' => 0,
        'maxUpper' => -1,
      ]);
	
	if($usr->id == 0)	// User does not exist
		{
		$usr = $users->add($UserPwName);	// Create user
		if($usr->id)	// Success?
			{
			$usr->of(false);	// Prepare for update
			$usr->email = $UserMail;
			$usr->pad_firstname = $UserLoginName;
			$user->pass = $pw;
			$usr->CheckBoxPrivacy = true;
			$usr->subscribe_newsletter = true;
			$usr->addRole('customer');	
			$usr->save();
			}
		else
			{
			// may $log->error("ERROR creating user {$UserRealName}");
			}
		}
    }

 

[adrian]

3 hours ago, bernhard said:

a processmodule might be overkill in your situation

This is when AdminActions (https://processwire.com/talk/topic/14921-admin-actions/) comes in handy:

I will get around to providing this as a proof of concept action at some point, but this is what my import page looks like:

This has been working great for me because I get to tweak the SQL queries (that I use to generate the CSV files that this action uses) and easily re-import. Just name the CSV files to match the template and this list is automatically generated:

 

It also makes it easy to test with importing a few, many, all, or a selected one:

I am going to use this for all my future site migrations.

[Autofahrn]

6 minutes ago, MarcoPLY said:

In this case what I need use? 

Well, either you strip those headers manually (which you did) or enumerate over all containers, search for the user table and continue to process that one (code only to get an idea, probably contains some typos):

$AllData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
foreach($AllData as $container)
	{
	if(($container['type'] == 'table') && ($container['name'] ==  'name_users'))
		{
		$UserData = $container['data'];
		foreach($UserData as $v)
			{
			:
			}
		break;
		}
	}

If the generated users are named from current timestamp then I suppose that $UserPwName is not set correctly. Did you check that $UserLoginName is read correctly from your data (maybe adding d($UserLoginName); and d($UserPwName); helps).

[Marco Ro]

Yes, the $UserPwName was not set. This one now work so well!!

It's very nice, need just call the variable from json file and match them to the right filed and everything is as popular as it should be!

Thank you so mach @Autofahrn!

$http = new WireHttp();
d($http); // shows ProcessWire\WireHttp
$UserData = json_decode($http->get('localhost:8888/playwood/pw_users.json'), true);
d($UserData);
    
    foreach($UserData as $v) {
    	// Fetch whatever fields should be imported
    	$UserLoginName = $v["user_login"];
    	$UserMail = $v["user_email"];
    	$UserPwName = $v["user_login"];
	
	$UserPwName = $sanitizer->pageName($UserLoginName, true);
	$usr = $users->get("name={$UserPwName}");
	
	if($usr->id == 0)	// User does not exist
		{
		$usr = $users->add($UserPwName);	// Create user
		if($usr->id)	// Success?
			{
			$usr->of(false);	// Prepare for update
			$usr->email = $UserMail;
			$usr->pad_firstname = $UserLoginName;
			$usr->CheckBoxPrivacy = true;
			$usr->subscribe_newsletter = true;
			$usr->addRole('customer');	
			$usr->save();
			}
		else
			{
			// may $log->error("ERROR creating user {$UserRealName}");
			}
		}
    }