Jump to content
ryan

Module: Import Pages from CSV file

Recommended Posts

Hey Ryan,

Nice trick with using: $this->page->filesManager()->path();

I'll make use of that instead of the pagefileSecure check I was using in one of my modules.

Should this be listed in the cheatsheet? Seems pretty useful to me!

  • Like 1

Share this post


Link to post
Share on other sites

Since the title field creates the page name automatically and represents the url structure of the page, is there a way to define both when importing? When importing, you must declare a title field. My title fields may contain a word with an apostrophe in it.  For example, "it's" when converted to the url would be it-s. The apostrophe is replaced with a hyphen. Is there a way to just trim the apostrophe so the url would be "its"?

Thank in advance for any suggestions. 

Share this post


Link to post
Share on other sites

Not currently with this module. Though you could write a quick file to do it for you. Place the following in your web root where ProcessWire's index.php file is, perhaps in a file called fix-names.php. Then load it in your browser. 

<?php
include("./index.php"); // bootstrap PW
$items = wire("pages")->find("template=your-template"); 
foreach($items as $item) {
  $name = preg_replace('/-s(-|$)/', '', $item->name);
  if($name != $item->name) {
    echo "<p>Changing $item->name to $name</p>";
    $item->name = $name;
    // $item->save(); 
  }
}

Uncomment that $item->save(); after you run it the first time to make sure it's doing what you want. I wrote the above here in the forum and haven't tested, so it may need tweaks. 

  • Like 1

Share this post


Link to post
Share on other sites

This is awesome Ryan! I actually used your snippet below and modified it to do what I needed and it worked perfectly, Just took me longer to create the temp field :)..

foreach($mypages as $p) {
    $p->of(false);
    $p->image->description = $p->temp;
    $p->save();
}

Your above snippet is definitely a time saver and has been added to my library. It works perfectly when I tested it. I just had to modify it slightly to meet my needs.

I would just like to say your willingness to bend over backwards to assist others in this forum is a true judgement of your character. You and several others go above and beyond to help others learn with examples. One in particular I would like to mention who helped me way beyond my expectations was Nik. Just like you, he took the time to break things down with examples and even explanations. How you both manage to find the time, I do not know. To you, Nik and all the others who help us learn everyday, I thank you sincerely for helping us. I am confident I am speaking for others as well and say that ya'll truly make a difference to those of us following in your paths.

  • Like 8

Share this post


Link to post
Share on other sites

Any progress with import page (reference) fields via this module?

I need to import categories to a page reference field.

Great module! :)

Works fine for other fields.

Share this post


Link to post
Share on other sites

Thank You for helping me with my problem  :)

It was so simple to get it work... I also found Niks post ;)

Imported all data in two steps. First simple imported the categories and ignored all other fields. After the categories added a second import works fine! Thanks.

Share this post


Link to post
Share on other sites

Hi!

I'm reading Nik's post (http://processwire.com/talk/topic/383-module-import-pages-from-csv-file/page-4#entry21476) but I cannot get my Pages to 'link'.

I'm using this line in my csv:

Ocky Docky,425,22-09-2012,,Zaterdagrit,André Ockerman,90,,,,,,,,,1306|1041|1037

The pages that I'm trying to link are 'user' templates. Selecting is absolutely ok (even works with the page's name!)

So the 'pipe' character and multi-select doesn't work :-(

Please help me out here.

Share this post


Link to post
Share on other sites

You added 'FieldtypePage' to the $fieldtypes array correct? What field type does the 'user' template have? Have you tried to connect a single page id?

  • Like 1

Share this post


Link to post
Share on other sites

Goodmorning,

Yes I added the 'FieldtypePage' correctly.

After extensive testing I found out that piping id's works with the mulitple field, but not with piping the 'name' or 'title'.

The last two things work when importing a single item in this field.

So my question becomes:

How can I alter the module to support the 'title' field when multiple values are seperated by a 'pipe' (|)?

Edit:

I looked into it, and I think the core fieldtype is not adapted to my use case. Some code should be there where it's currently missing:

https://github.com/ryancramerdesign/ProcessWire/blob/master/wire/modules/Fieldtype/FieldtypePage.module#L319

Maybe there is a quick solution to this?

Share this post


Link to post
Share on other sites

Hi Ryan,

I'm trying to import a list of retailer locations into processwire using your module.

The retailer stores all got the same name which contains an "umlaut" (ö). 

In the options I chose "Make the name unique and import new page". The Records are imported correctly but the title is cut after the umlaut (every page title is "xxx" instead of "xxxöyyy").

Do you have any suggestions how to fix this?

  • Like 1

Share this post


Link to post
Share on other sites

Hi Felix, be sure utf-8 is selected when saving your csv sheet. That should do the trick for you. 

  • Like 5

Share this post


Link to post
Share on other sites

Doh': I thought the files already utf8 encoded. I was wrong. Shame on me! :)

Thanks a lot!

Share this post


Link to post
Share on other sites

So finally, with help of others including Joss, I managed to get a select to work...

Other people were also mentioning this module. And it's probably what I need considering I need to have select options for at least 30plus values.

But how to set up the cvs file since I only need 1 e.g. title?

Share this post


Link to post
Share on other sites

Every page you want to create on a new line like this:

this is a page

and this

and an other

the fourth

we continue

let's make

a few more

and again

a page

this we can use as title

etc.

...

..

.

Save this and import it.

Share this post


Link to post
Share on other sites

that did only one insert: the last one.

Do I need to specify each line like this:

text1,

text2,

etc?

Solved:

I had to specify the header first, like:

title

text1

text2

text3

etc

  • Like 1

Share this post


Link to post
Share on other sites

that did only one insert: the last one.

Do I need to specify each line like this:

text1,

text2,

etc?

Yes. CSV (comma-separated values) needs a delimiter. This can be a comma, tab, semicolon or more. I haven't used this module for a while, but i think you can (or may need to) add a very first record as a header containing a list of field names. Furthermore, read the options the module gives. So the CSV something like this:

title,myfield1,myfield2
titleA,myfield1_value,myfield2_value
titleB,myfield1_value,myfield2_value
titleC,myfield1_value,myfield2_value

Share this post


Link to post
Share on other sites

Hi Ryan (and others), thank you for this great module. I'm looking to transfer an old ModX Revo site that is currently using latin-1 encoding. My understanding from the thread here:

https://processwire.com/talk/topic/1735-get-data-from-a-latin-1-encoded-db-and-insert-it-into-pw/

is that the following code:

$newPage->title = iconv("ISO-8859-1", "UTF-8", $row['HEADLINE']);

should convert to utf-8 first. Does your import module take this in to account or will I have to somehow add this in to your module code? Or should I just somehow create a separate php file that will cycle through my CSV and do this first, then use your module to import?

I have approximately 12,000 pages of varying types to import, including user accounts.

Thanks,

K.

Share this post


Link to post
Share on other sites

Hi @formulate, welcome to the forums!

If I understand right, you first create csv-files from the iso-8859-1 site, then you import them into PW with the csv-importer module?

Why not simply convert the whole iso-files to utf-8 files before using them for the import?

------------------

There is a case study from a (lets call it 'more customized') import scenario with a W**dPr*ss site. I have done something similar with a DB from another system (Gallery2). Maybe this is of interest for you? If you do know the DB structure of the old system good enough, so that you can query the right things in the right order, you may be able to setup a single script that does the whole import in greater chunks.

Importer scripts are easy but powerfull. If you are interested we can point you to other examples or assist in building yours (if you know enough of the old system structure and how to query it).

  • Like 1

Share this post


Link to post
Share on other sites

Or if you have the skills, perhaps you'd be interested in creating an plug-in for Migrator, like Nico's WP migrator. We have a lot of ModX refugees here, so I am sure it would get lots of use. I don't know anything about ModX, but I'd be happy to help in any way I can.

  • Like 2

Share this post


Link to post
Share on other sites

Or if you have the skills, perhaps you'd be interested in creating an plug-in for Migrator, like Nico's WP migrator. We have a lot of ModX refugees here, so I am sure it would get lots of use. I don't know anything about ModX, but I'd be happy to help in any way I can.

While my understanding of some aspects of ModX is top-shelf, my database knowledge and experience is woeful. I can get around in phpMyAdmin and understand ModX's table structure, but writing custom SQL queries, etc. is something I've never done. Trying to create a plug-in for Migrator seems like an unachievable task for me. I think I'm going to have enough trouble as it is just trying to get my database converted properly.

Thanks for the responses.

Share this post


Link to post
Share on other sites
This module was written before multi-language support in PW, and hasn't yet been updated to support multi language title/text/textarea fields

Would it be very difficult to create a new version supporting multi-language support?

Or are there any other tools I've missed? (i.e. I know how to create pages / populate fields via API, and also how to edit alternative languages, but it would be nice to have an automated import CSV tool for multilang sites). Perhaps a slight variation: An import tool for already existing pages, that inserts / updates one or multiple alternative language versions? If we'd declare the page ID as mandatory in the CSV, I guess this wouldn't be rocket science?

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By eelkenet
      Hi! I've created a small Inputfield module called InputfieldFloatRange which allows you to use an HTML5 <input type="range" ../> slider as an InputField. I needed something like this for a project where the client needs to be able to tweak this value more based on 'a feeling' than just entering a boring old number. Maybe more people can use this so I'm hereby releasing it into the wild. 
      EDIT: You can now install it directly from the Modules directory: http://modules.processwire.com/modules/inputfield-float-range/
       
      What is it?
      The missing range slider Inputfield for Processwire. 
      What does it do?
      This module extends InputfieldFloat and allows you to use HTML5 range sliders for number fields in your templates.
      It includes a visible and editable value field, to override/tweak the value if required.  
      Features
      Min/max values Precision (number of decimals) Optional step value (Read more) Optional manual override of the selected value (will still adhere to the rules above) Configurable rounding of manually entered values (floor, round, ceil, disable) Usage
      Clone / zip repo Install FieldtypeFloatRange, this automatically installs the Inputfield Create new field of type `Float (range)` or convert an existing `Float`, `Integer` or `Text` field. To render the field's value simply echo `$page->field` Demo
      A field with Min=0, Max=1, Step=0.2, Precision=2

      Field with settings Min=0, Max=200, Step=0.25, Precision=2

       
      Todo
      Make the display-field's size configurable (will use the Input Size field setting)  Hopefully become redundant  
      Changelog
      005 (current version)
      - Fix bug where the Inputfield would not work properly within repeaters / repeater matrices
      004
      - Make rounding of manually entered values configurable (floor, round, ceil or disable)
      - Fix small JS bug when the value-display field was not displayed
      - Update README
      003
      - Code cleanup, add some ModuleInfo data & LICENSE
      - Submit to PW Modules directory (http://modules.processwire.com/modules/inputfield-float-range/)
      002
      - Fix issue where setting the step value to an empty value created problem with validation
      - Make the display-field optional
      001
      - Initial release
       
      Thanks!
       
       
    • By Gadgetto
      Status update links (inside this thread) for SnipWire development will be always posted here:
      2019-10-18
      2019-08-08
      2019-06-15
      2019-06-02
      2019-05-25
      If you are interested, you can test the current state of development:
      https://github.com/gadgetto/SnipWire
      Please note that the software is not yet intended for use in a production system (alpha version).
      If you like, you can also submit feature requests and suggestions for improvement. I also accept pull requests.
      ---- INITIAL POST FROM 2019-05-25 ----
      I wanted to let you know that I am currently working on a new ProcessWire module that fully integrates the Snipcart Shopping Cart System into ProcessWire. (this is a customer project, so I had to postpone the development of my other module GroupMailer).
      The new module SnipWire offers full integration of the Snipcart Shopping Cart System into ProcessWire.
      Here are some highlights:
      simple setup with (optional) pre-installed templates, product fields, sample products (quasi a complete shop system to get started immediately) store dashboard with all data from the snipcart system (no change to the snipcart dashboard itself required) Integrated REST API for controlling and querying snipcart data webhooks to trigger events from Snipcart (new order, new customer, etc.) multi currency support self-defined/configurable tax rates etc. Development is already well advanced and I plan to release the module in the next 2-3 months.
      I'm not sure yet if this will be a "Pro" module or if it will be made available for free.
      I would be grateful for suggestions and hints!
      (please have a look at the screenshots to get an idea what I'm talking about)
       




    • By Robin S
      Another little admin helper module...
      Template Field Widths
      Adds a "Field widths" field to Edit Template that allows you to quickly set the widths of inputfields in the template.

      Why?
      When setting up a new template or trying out different field layouts I find it a bit slow and tedious to have to open each field individually in a modal just to set the width. This module speeds up the process.
      Installation
      Install the Template Field Widths module.
      Config options
      You can set the default presentation of the "Field widths" field to collapsed or open. Field widths entered into the Template Field Widths inputfield are only applied if the Edit Template form is submitted with the Template Field Widths inputfield in an opened state. "Collapsed" is the recommended setting if you think you might also use core inputs for setting field widths in a template context. You can choose Name or Label as the primary identifier shown for the field. The unchosen alternative will become the title attribute shown on hover. You can choose to show the original field width next to the template context field width.  
      https://github.com/Toutouwai/TemplateFieldWidths
      https://modules.processwire.com/modules/template-field-widths/
    • By adrian
      Tracy Debugger for ProcessWire
      The ultimate “swiss army knife” debugging and development tool for the ProcessWire CMF/CMS

       
      Integrates and extends Nette's Tracy debugging tool and adds 35+ custom tools designed for effective ProcessWire debugging and lightning fast development
      The most comprehensive set of instructions and examples is available at: https://adrianbj.github.io/TracyDebugger
      Modules Directory: http://modules.processwire.com/modules/tracy-debugger/
      Github: https://github.com/adrianbj/TracyDebugger
      A big thanks to @tpr for introducing me to Tracy and for the idea for this module and for significant feedback, testing, and feature suggestions.
×
×
  • Create New...