Jump to content

Module: Import Pages from CSV file


ryan

Recommended Posts

Hey Ryan,

Nice trick with using: $this->page->filesManager()->path();

I'll make use of that instead of the pagefileSecure check I was using in one of my modules.

Should this be listed in the cheatsheet? Seems pretty useful to me!

  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...

Since the title field creates the page name automatically and represents the url structure of the page, is there a way to define both when importing? When importing, you must declare a title field. My title fields may contain a word with an apostrophe in it.  For example, "it's" when converted to the url would be it-s. The apostrophe is replaced with a hyphen. Is there a way to just trim the apostrophe so the url would be "its"?

Thank in advance for any suggestions. 

Link to comment
Share on other sites

Not currently with this module. Though you could write a quick file to do it for you. Place the following in your web root where ProcessWire's index.php file is, perhaps in a file called fix-names.php. Then load it in your browser. 

<?php
include("./index.php"); // bootstrap PW
$items = wire("pages")->find("template=your-template"); 
foreach($items as $item) {
  $name = preg_replace('/-s(-|$)/', '', $item->name);
  if($name != $item->name) {
    echo "<p>Changing $item->name to $name</p>";
    $item->name = $name;
    // $item->save(); 
  }
}

Uncomment that $item->save(); after you run it the first time to make sure it's doing what you want. I wrote the above here in the forum and haven't tested, so it may need tweaks. 

  • Like 1
Link to comment
Share on other sites

This is awesome Ryan! I actually used your snippet below and modified it to do what I needed and it worked perfectly, Just took me longer to create the temp field :)..

foreach($mypages as $p) {
    $p->of(false);
    $p->image->description = $p->temp;
    $p->save();
}

Your above snippet is definitely a time saver and has been added to my library. It works perfectly when I tested it. I just had to modify it slightly to meet my needs.

I would just like to say your willingness to bend over backwards to assist others in this forum is a true judgement of your character. You and several others go above and beyond to help others learn with examples. One in particular I would like to mention who helped me way beyond my expectations was Nik. Just like you, he took the time to break things down with examples and even explanations. How you both manage to find the time, I do not know. To you, Nik and all the others who help us learn everyday, I thank you sincerely for helping us. I am confident I am speaking for others as well and say that ya'll truly make a difference to those of us following in your paths.

  • Like 8
  • Thanks 1
Link to comment
Share on other sites

  • 2 weeks later...

Thank You for helping me with my problem  :)

It was so simple to get it work... I also found Niks post ;)

Imported all data in two steps. First simple imported the categories and ignored all other fields. After the categories added a second import works fine! Thanks.

Link to comment
Share on other sites

  • 1 month later...
  • 1 month later...

Hi!

I'm reading Nik's post (http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/page-4#entry21476) but I cannot get my Pages to 'link'.

I'm using this line in my csv:

Ocky Docky,425,22-09-2012,,Zaterdagrit,André Ockerman,90,,,,,,,,,1306|1041|1037

The pages that I'm trying to link are 'user' templates. Selecting is absolutely ok (even works with the page's name!)

So the 'pipe' character and multi-select doesn't work :-(

Please help me out here.

Link to comment
Share on other sites

Goodmorning,

Yes I added the 'FieldtypePage' correctly.

After extensive testing I found out that piping id's works with the mulitple field, but not with piping the 'name' or 'title'.

The last two things work when importing a single item in this field.

So my question becomes:

How can I alter the module to support the 'title' field when multiple values are seperated by a 'pipe' (|)?

Edit:

I looked into it, and I think the core fieldtype is not adapted to my use case. Some code should be there where it's currently missing:

https://github.com/ryancramerdesign/ProcessWire/blob/master/wire/modules/Fieldtype/FieldtypePage.module#L319

Maybe there is a quick solution to this?

Link to comment
Share on other sites

Hi Ryan,

I'm trying to import a list of retailer locations into processwire using your module.

The retailer stores all got the same name which contains an "umlaut" (ö). 

In the options I chose "Make the name unique and import new page". The Records are imported correctly but the title is cut after the umlaut (every page title is "xxx" instead of "xxxöyyy").

Do you have any suggestions how to fix this?

  • Like 1
Link to comment
Share on other sites

  • 4 weeks later...

So finally, with help of others including Joss, I managed to get a select to work...

Other people were also mentioning this module. And it's probably what I need considering I need to have select options for at least 30plus values.

But how to set up the cvs file since I only need 1 e.g. title?

Link to comment
Share on other sites

that did only one insert: the last one.

Do I need to specify each line like this:

text1,

text2,

etc?

Yes. CSV (comma-separated values) needs a delimiter. This can be a comma, tab, semicolon or more. I haven't used this module for a while, but i think you can (or may need to) add a very first record as a header containing a list of field names. Furthermore, read the options the module gives. So the CSV something like this:

title,myfield1,myfield2
titleA,myfield1_value,myfield2_value
titleB,myfield1_value,myfield2_value
titleC,myfield1_value,myfield2_value
Link to comment
Share on other sites

  • 5 months later...

Hi Ryan (and others), thank you for this great module. I'm looking to transfer an old ModX Revo site that is currently using latin-1 encoding. My understanding from the thread here:

https://processwire.com/talk/topic/1735-get-data-from-a-latin-1-encoded-db-and-insert-it-into-pw/

is that the following code:

$newPage->title = iconv("ISO-8859-1", "UTF-8", $row['HEADLINE']);

should convert to utf-8 first. Does your import module take this in to account or will I have to somehow add this in to your module code? Or should I just somehow create a separate php file that will cycle through my CSV and do this first, then use your module to import?

I have approximately 12,000 pages of varying types to import, including user accounts.

Thanks,

K.

Link to comment
Share on other sites

Hi @formulate, welcome to the forums!

If I understand right, you first create csv-files from the iso-8859-1 site, then you import them into PW with the csv-importer module?

Why not simply convert the whole iso-files to utf-8 files before using them for the import?

------------------

There is a case study from a (lets call it 'more customized') import scenario with a W**dPr*ss site. I have done something similar with a DB from another system (Gallery2). Maybe this is of interest for you? If you do know the DB structure of the old system good enough, so that you can query the right things in the right order, you may be able to setup a single script that does the whole import in greater chunks.

Importer scripts are easy but powerfull. If you are interested we can point you to other examples or assist in building yours (if you know enough of the old system structure and how to query it).

  • Like 1
Link to comment
Share on other sites

Or if you have the skills, perhaps you'd be interested in creating an plug-in for Migrator, like Nico's WP migrator. We have a lot of ModX refugees here, so I am sure it would get lots of use. I don't know anything about ModX, but I'd be happy to help in any way I can.

While my understanding of some aspects of ModX is top-shelf, my database knowledge and experience is woeful. I can get around in phpMyAdmin and understand ModX's table structure, but writing custom SQL queries, etc. is something I've never done. Trying to create a plug-in for Migrator seems like an unachievable task for me. I think I'm going to have enough trouble as it is just trying to get my database converted properly.

Thanks for the responses.

Link to comment
Share on other sites

  • 2 months later...
This module was written before multi-language support in PW, and hasn't yet been updated to support multi language title/text/textarea fields

Would it be very difficult to create a new version supporting multi-language support?

Or are there any other tools I've missed? (i.e. I know how to create pages / populate fields via API, and also how to edit alternative languages, but it would be nice to have an automated import CSV tool for multilang sites). Perhaps a slight variation: An import tool for already existing pages, that inserts / updates one or multiple alternative language versions? If we'd declare the page ID as mandatory in the CSV, I guess this wouldn't be rocket science?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...