97 replies to this topic

[Michael Murphy]

I have found the import CSV module to be really useful, and has become a big part of my site building workflow with processwire. It's much easier to collect and manage lots of data in excel / numbers and then directly import into processwire. This is also great for updating content with a re-import.

I think data journalists and data hackers will find this sort of functionality very useful for doing quick data visualisation mashups (especially when you add on the friendly pw api).

I think content strategists will like this workflow as most of them use excel for collecting and organising site content at the start of projects.

And of course the clients will like it - this will be a nice way to start a new project - once basic site structure is agreed, we can give the client an excel template (or google doc) and tell them to start adding content. Wouldn't it be great to start a project with actual real content!

I hope you continue adding support for other pw fields like page-references and dates (I use pages extensively for category management).

Could this module also work in the opposite direction and export the data to CSV?

Then we just need to work out a way to import / export fields and templates   as much as I admire the processwire interface, I still think managing data / settings / fields is much quicker and easier in text files, at least at the beginning  of a project.

Processwire - a CMS that gets out of the way.

[ffub]

This module looks like exactly what I need for migrating a couple of larger sites to PW. Would it be possible to hack this for adding users as well?

Thanks,
Stephen

[ryan]

Thanks for the message mjmurphy–glad that you like this module. I use it quite a bit myself too. I will definitely be adding support for more fieldtypes to it. Actually, I think dates and page references are the only two that we don't support yet. Adding dates will be easy, but the page references a little more complex. However, I need the page references functional in the near future so will likely be adding both of those types soon.

An ExportCSV module is also planned for sure.

Then we just need to work out a way to import / export fields and templates

This won't be a module, it is already planned for the core (likely in 2.3/2.4). This feature was in PW1, but just hasn't made it into PW2 yet. But it's a very useful thing to have I agree.

This module looks like exactly what I need for migrating a couple of larger sites to PW. Would it be possible to hack this for adding users as well?

I think it should work now, though I haven't tried. Although it looks like I need to add support for FieldtypePassword (another one I missed) if you want to import passwords. Should be easy to add though–added to my list.

[Nico Knoll]

I want to import some users. So I chose "user" as "Template to use for imported pages" and "Users" as "parent page". But if I'm uploading the .csv and have to choose the right fields I can't choose "title". The only option is "email". And if I submit without a title there will be an error saying "Unable to import page because it has no required 'title' field or it is blank."

So how can I import users?

Greets,
Nico

[ryan]

I haven't actually tried to import users yet. But it should be possible. Go ahead and add the 'title' field to your user template and populate it with something (email address?) to see if that makes the import happy. I'll plain to make the import a little smarter in this regard on the next update.

[Nico Knoll]

Ehm, how can I change the user template - it's not in the templates list in "setup"?

[apeisa]

There is filter options above template list. Choose system templates and you are good to go.

[slkwrm]

There is filter option at the top of the list of templates.
Expand it and change "Show system templates?" to "yes".

[apeisa]

I will need to import bunch of news items soon, and planning to use this for the job. Any clues how to get date field working Ryan?

It would be pretty trivial to write custom import script also, but I have looking for opportunity to test this module.

[ryan]

I think this will depend somewhat on what format those dates will be in from your spreadsheet (CSV). If it's a unix timestamp, or any format consistent with what PHP's strtotime() will accept (which includes quite a lot of formats), then you should be able to import dates pretty easily. I think all you'd need to do is add FieldtypeDate to the list of allowed fieldtypes at the top of the modules:

<?php
        protected $fieldtypes = array(
                'FieldtypePageTitle',
                'FieldtypeText',
                'FieldtypeTextarea',
                'FieldtypeInteger',
                'FieldtypeFloat',
                'FieldtypeEmail',
                'FieldtypeURL',
                'FieldtypeCheckbox',
                'FieldtypeFile',
                'FieldtypeDate' // add this line
                );

I don't have a good example to test from at the moment though will try to test it tomorrow. But let me know if you get a chance to test, and if that works I'll go ahead and add it to the module permanently. Looking at it now, I'm not sure why I didn't add it before. But it's the end of the day here so my mind is tired and I may be missing something.

[apeisa]

Thanks Ryan. I don't have csv yet, but I am telling the client in which format the date's should be. I will test this at beginning of January, but it seems like it should be work or with very trivial work.

[apeisa]

I went ahead and tested this. I didn't modify anything and it imported dates just fine. This was my test CSV (I just pasted it):

"title","body","date","images"
"Uusi uutinen","Tämä on leipäteksti, jossa on "lainaus".","12-06-2011 12:45:11","http://www.avoine.fi/@Bin/213898/ulkona1.jpeg"

Only strange this was that body text was in this format in tinyMCE field:

Tämä on leipäteksti, jossa on lainaus"."

Not sure if they should be escaped in csv?

[apeisa]

Forget to say: absolutely brilliant module Ryan. Very friendly UI and it was breeze to use!

[Nico Knoll]

You have to escape '"'s with "\".

[ryan]

Thanks Antti, glad that it's working alright and that the dates imported how they should. That's interesting that your CSV file had unescaped quotes in it. That will confuse any CSV parser. What did you use to export the CSV? Nico's right that quotes should be escaped with a backslash.

[apeisa]

I didn't export that CSV from anywhere - I just wrote that myself as test case. I wasn't sure should the content be escaped so tested it without escaping - I was little bit surprised with the result (it worked, sorta)

[Nico Knoll]

If you would export a .csv from your database it should escapes '"'s automatically

[raydale]

I'm having problems using this module in PW 2.2. I have installed the module and then under 'Setup' and 'Import Pages From CSV' - I get the following message: "This page has no Process assigned."

[diogo]

@raydale I didn't use the module yet, but i would say that you have to look for that page in the tree, and edit it to assign to it the module process.

[raydale]

Thanks for the nudge in the right direction diogo - that's me being a complete noob with PW.

In case anyone else comes across this: under 'Admin' -> 'Setup' -> 'Import Pages From CSV' choose to edit the page and under 'Process' in the 'Content' tab select 'importPagesCSV' from the drop down.