Module: Import Pages from CSV file

[ryan]

Nik has the best posts!

Another thing I wanted to add is that it's also very easy just to use API code to do these types of imports. 90% of the time I use API code to script imports with shell scripts, versus using the import pages CSV module. Though part of that is that I use the API every day and there's not a lot of figuring out to do. But if you want to post a sample of the data you are importing, I'd be happy to respond with a sample of how you might import it from the API.

[nik]

Thanks Ryan, although that's quite an overstatement what you're saying. But I'll take it.

It's always nice to get feedback, especially positive one coming from the Guru Himself.

[sakkoulas]

Thanks Nik

[neildaemond]

It works great. I hoped to be able to add one image, but suddenly I can add even more.

I'm not sure of the format in which to enter the 'image' column... I saw apeisa put a full url to his image column. Would this module then download the image from the link and put it in into the assets folder? or would the system always load that image from the mentioned url (ie, setting the url field equal to any, possibly external, link.)

If it does set to url field to the specified URL, then it may be a good opportunity to use a cdn for images, although I doubt it will work well that way if using the width() or height() resizers..

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Am I way off?

[teppo]

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Your assumption is correct. Images are downloaded during import process.

[Pete]

Yup, that part of the process does make any conversion simpler

As long as that image is directly accessible via a browser then the importer can fetch it and put it into ProcessWire. Same with files and the file field.

The only thing you would have to watch in either case is that there is always the slim possibility you could get an unexpected file type coming from the source site. I don't think it's a problem as PW file fields only accept the file types you tell them to, but I just wasn't sure whether it would throw an error with the API so it might be worth putting in an additional check to see if it is a valid type.

[djpark]

Hello! Is there any way to modify the "Created" field with this module? 

I have a CSV file where I store the dates of my old blog in a field, and the new entries to the blog are sorted with the "-created" selector in processwire. What I want to do is import the old articles of my non-processwire site to my new processwire website but using the date from the CSV to overwrite the "created" field so that it appears that it was created in that date and time.

Is there any way to do this?

[Soma]

It's "not possible". http://processwire.com/talk/topic/651-set-created-by-creating-a-page-via-api/

Some technique would be to import to a date field you add to templates and use that, or copy that date later via SQL to the pages table 'created' field.

[digitex]

I have a situation right now, I'm not sure how unique it is or how easy it would be to solve. I have a group that runs a carshow that maintains their registration list on a custom built database application. They currently have the ability to export a .dbf file and import that directly into their website so registrants can confirm their paperwork is done.

I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? Certainly there's software that allows for a .dbf to be converted to .csv but that's extra steps I'd prefer not to have to impose on them. I could also convince them to try to get the original database application changed to export .csv instead of .dbf. That's a last resort though.

Any suggestions would be appreciated.

[horst]

 I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? ...

I'm not totally sure but it seems that you can read the dBase file directly with PHP, so you will be able to read the records from the file and use the API to import them.

http://php.net/manual/en/ref.dbase.php

EDIT: here is a guy who has converted dBase to CSV: http://www.dev-zero.de/php/mit-php-dbase-zu-csv-konvertieren.html

He uses a dbase class from another guy: http://www.dev-zero.de/downloads/dbf_class.php

and the code to do the convert is short:

<?php
include("dbf_class.php");          // include the class
 
$dbaseDB = ‘DB.dbf’;         
$dbf = new dbf_class($dbaseDB);    // open the dBase file
 
$csvFile = ‘file.csv’;             // create a CSV file
$csv = fopen($csvFile,"w");
 
                                   // get record count, loop through and convert to csv
$num_rec=$dbf->dbf_num_rec;
fputcsv($csv, array_keys($dbf->getRowAssoc(0)));
 
for($i=0; $i<$num_rec; $i++) {
    fputcsv($csv, $dbf->getRow($i));
}
 
fclose($csv);          // ready! now use Ryans CSV-Importer 

But want to note that I only have googled that and have not tested / used this by myself, so- no warranties ;-)

Edited May 18, 2013 by horst

[digitex]

Thanks Horst. I'll see if I can make this work.

[k07n]

Hi! Great module, thanks for it.

I think it can help me to keep catalog up to date. But I need to generate page name from another field than Title (Many titles are similar, but "article number" field is unique). Is it possible somehow?

[doolak]

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

[SiNNuT]

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

I'm not sure what you want but the module can already be reached via the top navigation by 'Setup->Import Pages From CSV'

You could probably also make some easy adjustments to the module file before installing it to make the page go wherever you want.

https://github.com/ryancramerdesign/ImportPagesCSV/blob/master/ImportPagesCSV.module#L548 and further, i guess change the $parent variable.

[doolak]

Yes, I know it is accessible through the "Setup" - but this part of the navigation is not allowed for a normal editor, just for a superuser.

I would like to make it available for a non-superuser in the top navigation ;-)

[SiNNuT]

Of course, stupid me..i will try to come up with a solution. Anyways i was wondering how you were able to move the page out of the admin tree, because this shouldn't be possible; the Import Pages From CSV page has the template 'admin' and this template only allows parents with admin template.

[doolak]

Yes, you have to change the children settings for that template once to move the page, then set it back to the standard settings.

And I tried a second possibility: I copied the admin template and gave that other permissions. But somehow I couldn't manage it to make the page viewable in the top navigation for a non-superuser...

[doolak]

Ok, got it - and I have to say once again: "Thanks, Soma!"

Here the thread and the post with the solution:

http://processwire.com/talk/topic/1272-new-page-nav-in-admin/?p=11276

The important part is the setting of the moduls permisson:

public static function getModuleInfo() {
		return array(
			'title' => 'Import Tabelle als CSV', 
			'version' => 103, 
			'summary' => 'Import CSV files to create ProcessWire pages.',
			'singular' => true, 
			'autoload' => false, 
			'permission' => 'page-edit'
			);

In this way the modul is editable by a non-superuser and can be accessed through the top navigation.

[SiNNuT]

I would have never figured that out.... i think it would be a good addition to http://wiki.processwire.com/index.php/Module_Creation

Maybe also a mention in Module.php in core?

- edit - it's in Process.php, should learn about process modules

[onjegolders]

Just a quick question, is it possible that this could work with a password field (for uploading users). Thanks.

[arjen]

Did you search?

[onjegolders]

I've been a naughty boy...

[RyanJ]

Nik, your FieldtypePage tip with the id's as the reference worked perfectly. Add one more day that you are my hero .

[owzim]

Just wanted to say Thank You for this great module. I am working on a quite large project at the moment for which this module already has helped a lot.

[MatthewSchenker]

Greetings,

I'll jump in as well...

Not too long ago, I used this module to run the initial import for a TV station project, where I needed to get 550+ pages from an old Joomla site.  After getting the data into the right csv format from the database, it was unbelievably smooth and easy to populate the pages.  I kept thinking, "Something will certainly go wrong with this," but it didn't!  I got all the existing Joomla pages into ProcessWire perfectly.  After that, the client used ProcessWire to create new pages.

My next goal with this -- build a front-end way for clients to do the imports themselves.

By the way, I was prompted to post this after reading and responding to this post: http://processwire.com/talk/topic/3917-importing-users-and-content-from-joomla/

Thanks Ryan!

Matthew