Module: Import Pages from CSV file

[apeisa]

Yep, this is great module. I also want to encourage people to write their own import scripts. It is simpler than you can believe and you can re-import easily.

[nfil]

I've imported pages from an old modx site and this module is great, but I have some doubts about users, I've 500 users to import with the the following values:

"name", "password","member_phone","member_email","member_website_link","member_address","member_post_code"

At the moment I'am just doing simple tests to recreate the users data on pWire but when I try import the csv, just to create the user's name and email I get the error "title missing" using the user template and the users parentPage. 

I don't see a field for name or title in the user templates, just pass, email, roles, and language.

Can I also add the fields to the user system template like member_website_link ?

I don't want to take the wrong route on this one , so I appreciate some advise on how to do this. Thanks. 

[Martijn Geerts]

The user template can be extended with the custom fields you want.

[nfil]

Thank you Martijn! pWire flexibility and principles are amazing and I see they stay coherent throughout the CMS !

Yet to create users using this module I will have to add a name field to the user system template, apart from other my custom fields?

I just find this weird cause there is a name field when you create a user manually, yet this field is not on the template. Thank you again.

iupii I import all the users:

"title","pass","email","member_phone","member_mobile","member_fax","member_long_name","member_website_link","member_address","member_post_code"

[SiNNuT]

Technically 'name' isn't a field but a value stored in the name column of the pages table. It has to follow certain rules (Any combination of letters (a-z), numbers (0-9), dashes or underscores (no spaces).), because name will be (part of) the URL.

Normally, upon first manual entry 'name' is automatically converted from the input of the 'title'. In case of the CSV import module the title field is required, name is not and will be set based on the title value.

Just did a quick test:

Added 'title' field to the user template, and then also added a title column in a CSV. No need to have 'name' in your csv. This csv data imported correctly:

title
Tester 2
Tester 3

which results in users with names tester-2 and tester-3, and titles as in the csv. The problem i see is that there is no way to set the roles, because FieldtypePage isn't supported by ImportPagesCSV. The weird thing is, when importing the users the module doesn't complain and from the overview screen on the Users page it would seem they got guest role, but when you go edit an imported user, and/or look in the datebase no role has been set.

[nfil]

Thank you SiNNuT. It makes sense when you say the available fields are the must common ones, The users were set up as guests. On previous posts I think I read something about setting up new types of fields. I will have a look at the mySql tables to see if the role was assigned there.
I would like to try this with the Ryan's Google Geo IP module. I have to check if the FieldtypeMapMarker only uses simple text fields, or I will try to get the values from members_address adding lat and longitude to generate maps, if possible.

[SiNNuT]

(I'm not familiar with a Google Geo IP module and can't seem to find it on the modules section.)

To conclude; bear in mind that in any real scenario you would probably want to give your imported users 1 or more specific roles (something other than guest) and atm this is not possible with ImportPagesCSV. I think maybe you would be better off making a user import script with PW api, unless i'm missing something obvious in the the capabilities of ImportPagesCSV.

[kongondo]

Have a read at this earlier post on page #4 of this thread. It talks about roles. Seems you can import roles actually.

  Quote

One other thing to note, be sure to have a roles column in your CSV with roles for each user.

I forgot that during my first test import and all the users were set to guest.

http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/?p=10160

[nfil]

I can't edit the name of members I will add a custom role, like you said and change the assigned role on Mysql, if possible.

But the users login is working, yet you need to reset the password. 

Thank you SiNNuT

2 SiNNuT Module Based on Geoip Google maps used on site Ryan's profile skyscrapers:

http://modules.processwire.com/modules/fieldtype-map-marker/

p.s. oh well I was writing this and kongondo wisdom and detective work beats my newbie pWire skills for sure.

 

^ yes I read it now eheh ^

[kongondo]

  On 7/31/2013 at 11:45 AM, nfil said:

I can't edit the name of members I will add a custom role, like you said and change the assigned role on Mysql, if possible.

But the users login is working, yet you need to reset the password.

Thank you SiNNuT

nfil, see my post above yours

[nfil]

Sorry to go back to this again, just to be sure on this.

  On 7/31/2013 at 11:43 AM, kongondo said:

Have a read at this earlier post on page #4 of this thread. It talks about roles. Seems you can import roles actually.

http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/?p=10160

I imported another database, this time I tried to insert the roles field. Although renobird mentioned a way to import roles I tried that but it didn't work.

No problems importing the other fields, but the roles field is missing. 

https://www.dropbox.com/s/btteytow6w2hpwx/last-screen.png

My csv.

"title","pass","email","roles","headline","member_district"
"261","userspassword","afcob@net.com","associado","AGÊNCIA","Lisboa" 

On the roles column I have the name, maybe I need to insert a number associated with that checkbox, also I don't want to mess with the system fields yet (cause I'm a noob) but probably I have to change something in the roles field checkbox single to be compatible with the module.

https://www.dropbox.com/s/mzkggp7hvacu7to/roles-field.png

[SiNNuT]

The 'roles' field is of type 'page' and isn't supported by ImportPagesCSV out of the box. If you want it to show up you need to add something to the modules fieldtype array:

'FieldtypePage', // add this line

I haven't tried it myself but nik has, so courtesy nik.

For more info you should read his comment in this thread: http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/?p=21476

Also pay attention to his words on how to reference the roles in your csv file. In this case i think it would be best to look up the id's of the roles you've defined and put them in your csv (find and replace, you know the drill). In case you don't know: you can look up the page id's of the roles in a couple of ways but hovering the edit link and looking at the status bar of your browser is easiest;

[nfil]

Everyday I'm amazed by the PW member's support. Its amazing! I was over-complicating.

So after nik's great post that I missed   I added the to the module 'FieldtypePage', // add this line

http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/page-4#entry21476

Now I have the numbers of the roles I want to import separated by pipes, and it works great, never thought it would be so straightforward:

"title","pass","email","roles","headline","member_district"
"261","userspassword","afcob@net.com","99|3899","AGÊNCIA","Lisboa"

I only read renobird's post, have framed the posts so I don't forget to read carefully next time

https://www.dropbox.com/s/xyalrrel6pw2mp1/frame.jpg

And it works great! Thanks again SiNNUT

[renobird]

Every time I use this module I laugh when it's complete. I just imported 200+ pages flawlessly — Beer:30.

[neildaemond]

I'm seeing an issue with importing Chinese.

I have a column which is importing Chinese text. However, only the rows which contained an English letter in it will import. The imported field will omit the initial Chinese, then start with the English segment, then the rest of the Chinese will follow as expected.

[ryan]

Chances are that the CSV file is not UTF-8? If you can confirm that it is in fact UTF-8, no-BOM, PM me a copy of the CSV file and I can give it a try here to isolate the issue. 

[adrian]

I think I have found a small bug with this. It only affects sites that have $config->pagefileSecure = true; and if you are using the paste option, instead of uploading the csv file.

This line is the problem:

$csvFilename = $this->config->paths->files . $this->page->id . '/data.csv';

Obviously it is not accounting for the "-" that is prepended to id of the page in the admin /assets/files/xxxx folders.

This seems to fix it:

$csvFilename = $this->config->paths->files . ($this->config->pagefileSecure ? $this->config->pagefileSecurePathPrefix : null) . $this->page->id . '/data.csv';

[FL0RIAN]

Is it possible to include the page name or URL for getting the same website structure?

Thanks for your support!

[adrian]

Hi flod - welcome to PW!

If you mean what I think you mean, stayed tuned over here:

http://processwire.com/talk/topic/4420-page-list-migrator/

[kongondo]

  On 9/6/2013 at 8:14 AM, flod said:

Is it possible to include the page name or URL for getting the same website structure?

Thanks for your support!

Hi flod. Welcome to PW and the forums...

Since PW builds the URL based on the name of the page, and the name is built based on the title, theoretically, if you imported titles, you should get the same name and URL "structure" (depending on what you mean by structure), I should think. Then again, you may be looking for something different as Adrian has suggested

Edited September 6, 2013 by kongondo

[FL0RIAN]

Sorry for my confusing explanation:

i.e. one page is this one: /parent/test1/ and I want this to be exactly the same in PW. With this module I only get /test1/ in the tree.

[kongondo]

I'm still not sure I get the full picture (i.e. whether you will have pages with different parents or not). This module allows you to select a single parent to import into. If you don't select one, all your pages will be imported into the root path (home). On the other hand, it is not currently possible to import into different parents simultaneously. You would have to do the imports for each parent. Adrian is working on a module that would allow to replicate your pages tree structure and import that into a different install whilst maintaining parent-child relationships, etc in one go...Other than that, depending on your skill level, you can also use PHPMyAdmin to export your db and import that into a different install and perfectly maintain multiple parent-child relationships of your pages. 

[Soma]

Updating the module to 1.0.3 it doesn't recognize paste in CSV data I get "Missing required CSV file/data", the file upload works. Could it be that the folder for that page is /-1006/ and it doesn't allow to use file_put_contents() as it's protected by htaccess? It tries to put in "/home/.../site/assets/files/1006/data.csv"

Also the duplicate option doesn't rename names to "name-n", it just throws error and names them "-n" 

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'ilnau-5870' for key 'name_parent_id'

Quick look at 

#404 $page->name = $this->getUniquePageName($pageName); 

$pageName isn't set, it should be $page->name

[adrian]

Hey Soma,

Is this the same as my issue at: http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/?p=43443

I submitted a fix and pull request for that here: https://github.com/adrianbj/ImportPagesCSV/commit/ad4d9e71f844fe2e4c9162152177e119a403ef38

Although it sounds like you have something else going on with the $pageName as well.

[ryan]

Thanks guys, I have updated the module to 1.0.4 with these fixes in place.