85 replies to this topic

[ryan]

Here is a new module for ProcessWire 2.1 that imports pages from a CSV file. By default it will create new pages from data in the CSV file, but you can also configure it to modify existing pages too (existing pages that have the same title).

Please give it a try and let me know how it works for you and if you run into any issues with it. This module is something I've had in the works for awhile, and regularly use on various projects, so figured I should clean it up a bit and release it. Also attached are a couple screenshots from it.

How to Install:

1. Download from: https://github.com/r.../ImportPagesCSV
2. Place the file ImportPagesCSV.module in your /site/modules/ directory.
3. In ProcessWire admin, click to 'Modules' and 'Check for new modules'.
4. Click 'install' next to the 'Import Pages CSV' module (under heading 'Import').

Following that, you'll see a new menu option for this module on your Admin > Setup menu.

Supported field types for importing:*

• PageTitle
• Text
• Textarea (including normal or TinyMCE)
• Integer
• Float
• Email
• URL
• Checkbox (single)

*I'll be adding support for multi-value, page-reference and file-based Fieldtypes in a future version.

Attached Thumbnails

• 
• 

[Soma]

Very cool module! Thanks a lot Ryan! I'll have a look and try out soon.

[apeisa]

Looks great and will definitely come in use! Will test and learn from the code later on.

[SiNNuT]

Looking great! Seems like a very useful module. Maybe even something to include by default, because most if not all projects i can think of require you to do some importing.

I have made a reference to this thread/module over here: http://processwire.c...topic,24.0.html

Because my code reading skills suck i do have a question: In 'Step 2', how do you come up with the list of fields found in the csv? I'm guessing you need to have a header line in your csv file?

[apeisa]

Yep, it requires header row: "The list of field names must be provided as the first row in the CSV file. "

[SiNNuT]

Yep, it requires header row: "The list of field names must be provided as the first row in the CSV file. "

stupid, i totally missed that 

[ryan]

I could modify it to not require a header row. I guess I just figured every CSV file I've ever come across had one.

[apeisa]

Usually they have. And since that works by uploading a file then anyone can add those titles to the file.

[SiNNuT]

I could modify it to not require a header row. I guess I just figured every CSV file I've ever come across had one.

I think it's fine to require the header row, without it there will be no way to do Stap 2 in it's current form.
If a header row is missing you can easily add it, because you have control over the file you choose to use.

[formmailer]

Ryan,

How did you know I would need something like this today?
I came to this forum to ask a question about creating pages from a feed and here it is... (ok, I was focused on an xml feed, but CSV will work as well I guess.)

Thanks!

/Jasper

[formmailer]

Hi!

I just started testing this module and it works great to import a product feed. I love it!

Only one minor thing I noticed is images: my productfeed contains URLs to an image, but I can't select the imagefields in the import module.
Is there a reason for this? I tried to manually add a URL into the field_image and it worked great, it saved a copy (or multiple copies in different sizes) of the image on my server.

/Jasper

[SiNNuT]

Hi!

I just started testing this module and it works great to import a product feed. I love it!

Only one minor thing I noticed is images: my productfeed contains URLs to an image, but I can't select the imagefields in the import module.
Is there a reason for this? I tried to manually add a URL into the field_image and it worked great, it saved a copy (or multiple copies in different sizes) of the image on my server.

/Jasper

The following is written in Ryan's topicstart:

Supported field types for importing:*

    * PageTitle
    * Text
    * Textarea (including normal or TinyMCE)
    * Integer
    * Float
    * Email
    * URL
    * Checkbox (single)

*I'll be adding support for multi-value, page-reference and file-based Fieldtypes in a future version.

So i guess you'll have to wait till a future version (dunno if Ryan has this planned anytime soon), or you could have a look at the code and possibly add something yourself.

[formmailer]

Oops, I missed that the imagefield wasn't listed.  Sorry about that. :-\
I am not sure if my coding skills are sufficient to add the image field import myself. But it's of course worth a try.  

A workaround could of course be using the URL field type until Ryan releases a future version.

/Jasper

[ryan]

I don't think a multi-image field will work, but a single image field may very well work if you want to try it.

To try it, backup your PW database and site first. If it doesn't work, you want to be able to restore to where you were. Though, chances are you won't have to do any kind of restoring files, but you can never be too safe.

Next, make sure you are dealing with a single image field. Edit your field (in Setup > Fields) and make sure it's "max number of files" is set to "1".

Next edit the ImportPagesCSV.module file and locate this (near the top, and add the lines indicated at the bottom).

<?php
        protected $fieldtypes = array(
                'FieldtypePageTitle',
                'FieldtypeText',
                'FieldtypeTextarea',
                'FieldtypeInteger',
                'FieldtypeFloat',
                'FieldtypeEmail',
                'FieldtypeURL',
                'FieldtypeCheckbox',
                'FieldtypeFile',  // add this line
                'FieldtypeImage', // add this line
                );

Save, and try it out. Let us know if it worked?

[formmailer]

Save, and try it out. Let us know if it worked?

It didn't really work. It looks like the page need to be created before an image can be added.
I received the following error:

TemplateFile: New page '/shs/verblijf-in-stockholm/0/' must be saved before files can be accessed from it
#0 C:\xampp\htdocs\shs\wire\core\PagefilesManager.php(133): PagefilesManager->path()
#1 C:\xampp\htdocs\shs\wire\core\PagefilesManager.php(43): PagefilesManager->createPath()
#2 C:\xampp\htdocs\shs\wire\core\PagefilesManager.php(32): PagefilesManager->init(Object(Page))
#3 C:\xampp\htdocs\shs\wire\core\Page.php(1132): PagefilesManager->__construct(Object(Page))
#4 C:\xampp\htdocs\shs\wire\core\Pagefiles.php(63): Page->filesManager()
#5 C:\xampp\htdocs\shs\wire\core\Pagefiles.php(47): Pagefiles->setPage(Object(Page))
#6 C:\xampp\htdocs\shs\wire\modules\Fieldtype\FieldtypeImage.module(33): Pagefiles->__construct(Object(Page))
#7 C:\xampp\htdocs\shs\wire\core\Fieldtype.php(289): FieldtypeImage->getBlankValue(Object(Page), Object(Field))
#8 C:\xampp\htdocs\shs\wire\core\Page.php(523): Fieldtype->getDefaultValue(Object(Page), Object(Field))
#9 C:\xampp\htdocs\shs\wire\core\Page.php(467): Page->getFieldValue('hotel_image')
#10 C:\xampp\htdocs\shs\wire\core\Page.php(364): Page->get('hotel_image')
#11 C:\xampp\htdocs\shs\wire\core\Page.php(308): Page->setFieldValue('hotel_image', 'http://images.t...', true)
#12 C:\xampp\htdocs\shs\site\modules\ImportPagesCSV.module(385): Page->set('hotel_image', 'http://images.t...')
#13 C:\xampp\htdocs\shs\site\modules\ImportPagesCSV.module(346): ImportPagesCSV->importPage(Array, Object(InputfieldForm))
#14 C:\xampp\htdocs\shs\site\modules\ImportPagesCSV.module(126): ImportPagesCSV->processForm2(Object(InputfieldForm))
#15 [internal function]: ImportPagesCSV->___executeFields()
#16 C:\xampp\htdocs\shs\wire\core\Wire.php(267): call_user_func_array(Array, Array)
#17 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('executeFields', Array)
#18 C:\xampp\htdocs\shs\wire\core\ProcessController.php(194): Wire->__call('executeFields', Array)
#19 C:\xampp\htdocs\shs\wire\core\ProcessController.php(194): ImportPagesCSV->executeFields()
#20 [internal function]: ProcessController->___execute()
#21 C:\xampp\htdocs\shs\wire\core\Wire.php(267): call_user_func_array(Array, Array)
#22 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('execute', Array)
#23 C:\xampp\htdocs\shs\wire\core\admin.php(42): Wire->__call('execute', Array)
#24 C:\xampp\htdocs\shs\wire\core\admin.php(42): ProcessController->execute()
#25 C:\xampp\htdocs\shs\wire\templates-admin\controller.php(13): require('C:\xampp\htdocs...')
#26 C:\xampp\htdocs\shs\site\templates\admin.php(13): require('C:\xampp\htdocs...')
#27 C:\xampp\htdocs\shs\wire\core\TemplateFile.php(88): require('C:\xampp\htdocs...')
#28 [internal function]: TemplateFile->___render()
#29 C:\xampp\htdocs\shs\wire\core\Wire.php(267): call_user_func_array(Array, Array)
#30 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('render', Array)
#31 C:\xampp\htdocs\shs\wire\modules\PageRender.module(194): Wire->__call('render', Array)
#32 C:\xampp\htdocs\shs\wire\modules\PageRender.module(194): TemplateFile->render()
#33 [internal function]: PageRender->___renderPage(Object(HookEvent))
#34 C:\xampp\htdocs\shs\wire\core\Wire.php(267): call_user_func_array(Array, Array)
#35 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('renderPage', Array)
#36 C:\xampp\htdocs\shs\wire\core\Wire.php(289): Wire->__call('renderPage', Array)
#37 C:\xampp\htdocs\shs\wire\core\Wire.php(289): PageRender->renderPage(Object(HookEvent))
#38 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('render', Array)
#39 C:\xampp\htdocs\shs\wire\modules\Process\ProcessPageView.module(73): Wire->__call('render', Array)
#40 C:\xampp\htdocs\shs\wire\modules\Process\ProcessPageView.module(73): Page->render()
#41 [internal function]: ProcessPageView->___execute()
#42 C:\xampp\htdocs\shs\wire\core\Wire.php(267): call_user_func_array(Array, Array)
#43 C:\xampp\htdocs\shs\wire\core\Wire.php(229): Wire->runHooks('execute', Array)
#44 C:\xampp\htdocs\shs\index.php(170): Wire->__call('execute', Array)
#45 C:\xampp\htdocs\shs\index.php(170): ProcessPageView->execute()
#46 {main}

/Jasper

[ryan]

Sorry, I neglected to think about that before (how pages have to be created before you can add images to them). One possible way around it is to try to first import without the image field, then import the same spreadsheet again but with the image field (updating the existing pages that were created). You'd need to choose the option to "modify existing page." But it's possible that might work.

[formmailer]

I tried that as well, but it didn't work. 
Even when the pages exist, the import module will not add the images.
The error/warning is exactly the same some in my previous reply.

/Jasper

[ryan]

Jasper, I didn't want to leave you empty handed, especially after you've tried this a few times and my suggestions didn't work. Here's an updated version of the ImportPagesCSV module that supports file and image filed importing. It supports both single and multi-file fields, so there aren't any limitations in that area.

https://github.com/r.../ImportPagesCSV

To import a multi-file field, place each filename or URL on it's own line in your spreadsheet, OR separate each by a pipe "|", OR separate each by a tab (you decide) – basically, you are delimiting the filenames/URLs within the field. In my own tests, I used the pipe "|" to separate the URLs and all seemed to work well. Of course, if there is only one image/file, you don't need anything other than the filename or URL (no delimiter necessary).

I ended up changing quite a bit of code, so please let me know if you run into any error messages or anything – it may not be perfect yet, but hopefully close.

[formmailer]

Wow, that's great Ryan. Thank you!

I am going to try it later today and will tell you how it worked.

/Jasper

[formmailer]

It works great. I hoped to be able to add one image, but suddenly I can add even more.

Thanks Ryan
(I owe you a beer  )