Jump to content
ryan

Module: Import Pages from CSV file

Recommended Posts

Nik has the best posts!

Another thing I wanted to add is that it's also very easy just to use API code to do these types of imports. 90% of the time I use API code to script imports with shell scripts, versus using the import pages CSV module. Though part of that is that I use the API every day and there's not a lot of figuring out to do. But if you want to post a sample of the data you are importing, I'd be happy to respond with a sample of how you might import it from the API.

Share this post


Link to post
Share on other sites

Thanks Ryan, although that's quite an overstatement what you're saying. But I'll take it. ;)

It's always nice to get feedback, especially positive one coming from the Guru Himself.

  • Like 1

Share this post


Link to post
Share on other sites

It works great. I hoped to be able to add one image, but suddenly I can add even more.

I'm not sure of the format in which to enter the 'image' column... I saw apeisa put a full url to his image column. Would this module then download the image from the link and put it in into the assets folder? or would the system always load that image from the mentioned url (ie, setting the url field equal to any, possibly external, link.)

If it does set to url field to the specified URL, then it may be a good opportunity to use a cdn for images, although I doubt it will work well that way if using the width() or height() resizers..

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Am I way off?

Share this post


Link to post
Share on other sites

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Your assumption is correct. Images are downloaded during import process.

  • Like 1

Share this post


Link to post
Share on other sites

Yup, that part of the process does make any conversion simpler :)

As long as that image is directly accessible via a browser then the importer can fetch it and put it into ProcessWire. Same with files and the file field.

The only thing you would have to watch in either case is that there is always the slim possibility you could get an unexpected file type coming from the source site. I don't think it's a problem as PW file fields only accept the file types you tell them to, but I just wasn't sure whether it would throw an error with the API so it might be worth putting in an additional check to see if it is a valid type.

Share this post


Link to post
Share on other sites

Hello! Is there any way to modify the "Created" field with this module? 

I have a CSV file where I store the dates of my old blog in a field, and the new entries to the blog are sorted with the "-created" selector in processwire. What I want to do is import the old articles of my non-processwire site to my new processwire website but using the date from the CSV to overwrite the "created" field so that it appears that it was created in that date and time.

Is there any way to do this?

Share this post


Link to post
Share on other sites

I have a situation right now, I'm not sure how unique it is or how easy it would be to solve. I have a group that runs a carshow that maintains their registration list on a custom built database application. They currently have the ability to export a .dbf file and import that directly into their website so registrants can confirm their paperwork is done.

I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? Certainly there's software that allows for a .dbf to be converted to .csv but that's extra steps I'd prefer not to have to impose on them. I could also convince them to try to get the original database application changed to export .csv instead of .dbf. That's a last resort though.

Any suggestions would be appreciated.

Share this post


Link to post
Share on other sites

 I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? ...

I'm not totally sure but it seems that you can read the dBase file directly with PHP, so you will be able to read the records from the file and use the API to import them.

http://php.net/manual/en/ref.dbase.php

EDIT: here is a guy who has converted dBase to CSV: http://www.dev-zero.de/php/mit-php-dbase-zu-csv-konvertieren.html

He uses a dbase class from another guy: http://www.dev-zero.de/downloads/dbf_class.php

and the code to do the convert is short:

<?php
include("dbf_class.php");          // include the class
 
$dbaseDB = ‘DB.dbf’;         
$dbf = new dbf_class($dbaseDB);    // open the dBase file
 
$csvFile = ‘file.csv’;             // create a CSV file
$csv = fopen($csvFile,"w");
 
                                   // get record count, loop through and convert to csv
$num_rec=$dbf->dbf_num_rec;
fputcsv($csv, array_keys($dbf->getRowAssoc(0)));
 
for($i=0; $i<$num_rec; $i++) {
    fputcsv($csv, $dbf->getRow($i));
}
 
fclose($csv);          // ready! now use Ryans CSV-Importer 

But want to note that I only have googled that and have not tested / used this by myself, so- no warranties ;-)

Edited by horst
  • Like 3

Share this post


Link to post
Share on other sites

Hi! Great module, thanks for it.

I think it can help me to keep catalog up to date. But I need to generate page name from another field than Title (Many titles are similar, but "article number" field is unique). Is it possible somehow?

Share this post


Link to post
Share on other sites

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

Share this post


Link to post
Share on other sites

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

I'm not sure what you want but the module can already be reached via the top navigation by 'Setup->Import Pages From CSV'

You could probably also make some easy adjustments to the module file before installing it to make the page go wherever you want.

https://github.com/ryancramerdesign/ImportPagesCSV/blob/master/ImportPagesCSV.module#L548 and further, i guess change the $parent variable.

Share this post


Link to post
Share on other sites

Yes, I know it is accessible through the "Setup" - but this part of the navigation is not allowed for a normal editor, just for a superuser.

I would like to make it available for a non-superuser in the top navigation ;-)

Share this post


Link to post
Share on other sites

Of course, stupid me..i will try to come up with a solution. Anyways i was wondering how you were able to move the page out of the admin tree, because this shouldn't be possible; the Import Pages From CSV page has the template 'admin' and this template only allows parents with admin template.

Share this post


Link to post
Share on other sites

Yes, you have to change the children settings for that template once to move the page, then set it back to the standard settings.

And I tried a second possibility: I copied the admin template and gave that other permissions. But somehow I couldn't manage it to make the page viewable in the top navigation for a non-superuser...

Share this post


Link to post
Share on other sites

Ok, got it - and I have to say once again: "Thanks, Soma!"

Here the thread and the post with the solution:

http://processwire.com/talk/topic/1272-new-page-nav-in-admin/?p=11276

The important part is the setting of the moduls permisson:

public static function getModuleInfo() {
		return array(
			'title' => 'Import Tabelle als CSV', 
			'version' => 103, 
			'summary' => 'Import CSV files to create ProcessWire pages.',
			'singular' => true, 
			'autoload' => false, 
			'permission' => 'page-edit'
			);

In this way the modul is editable by a non-superuser and can be accessed through the top navigation.

  • Like 2

Share this post


Link to post
Share on other sites

Just a quick question, is it possible that this could work with a password field (for uploading users). Thanks.

Share this post


Link to post
Share on other sites

Nik, your FieldtypePage tip with the id's as the reference worked perfectly. Add one more day that you are my hero :).

Share this post


Link to post
Share on other sites

Just wanted to say Thank You for this great module. I am working on a quite large project at the moment for which this module already has helped a lot.

  • Like 1

Share this post


Link to post
Share on other sites

Greetings,

I'll jump in as well...

Not too long ago, I used this module to run the initial import for a TV station project, where I needed to get 550+ pages from an old Joomla site.  After getting the data into the right csv format from the database, it was unbelievably smooth and easy to populate the pages.  I kept thinking, "Something will certainly go wrong with this," but it didn't!  I got all the existing Joomla pages into ProcessWire perfectly.  After that, the client used ProcessWire to create new pages.

My next goal with this -- build a front-end way for clients to do the imports themselves.

By the way, I was prompted to post this after reading and responding to this post: http://processwire.com/talk/topic/3917-importing-users-and-content-from-joomla/

Thanks Ryan!

Matthew

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Robin S
      Repeater Images
      Adds options to modify Repeater fields to make them convenient for "page-per-image" usage. Using a page-per-image approach allows for additional fields to be associated with each image, to record things such as photographer, date, license, links, etc.
      When Repeater Images is enabled for a Repeater field the module changes the appearance of the Repeater inputfield to be similar (but not identical) to an Images field. The collapsed view shows a thumbnail for each Repeater item, and items can be expanded for field editing.
      Screencast

      Installation
      Install the Repeater Images module.
      Setup
      Create an image field to use in the Repeater field. Recommended settings for the image field are "Maximum files allowed" set to 1 and "Formatted value" set to "Single item (null if empty)". Create a Repeater field. Add the image field to the Repeater. If you want additional fields in the Repeater create and add these also. Repeater Images configuration
      Tick the "Activate Repeater Images for this Repeater field" checkbox. In the "Image field within Repeater" dropdown select the single image field. You must save the Repeater field settings to see any newly added Image fields in the dropdown. Adjust the image thumbnail height if you want (unlike the core Images field there is no slider to change thumbnail height within Page Edit). Note: the depth option for Repeater fields is not compatible with the Repeater Images module.
      Image uploads feature
      There is a checkbox to activate image uploads. This feature allows users to quickly and easily add images to the Repeater Images field by uploading them to an adjacent "upload" field.
      To use this feature you must add the image field selected in the Repeater Images config to the template of the page containing the Repeater Images field - immediately above or below the Repeater Images field would be a good position.
      It's recommended to set the label for this field in template context to "Upload images" or similar, and set the visibility of the field to "Closed" so that it takes up less room when it's not being used. Note that when you drag images to a closed Images field it will automatically open. You don't need to worry about the "Maximum files allowed" setting because the Repeater Images module overrides this for the upload field.
      New Repeater items will be created from the images uploaded to the upload field when the page is saved. The user can add descriptions and tags to the images while they are still in the upload field and these will be retained in the Repeater items. Images are automatically deleted from the upload field when the page is saved.
      Tips
      The "Use accordion mode?" option in the Repeater field settings is useful for keeping the inputfield compact, with only one image item open for editing at a time. The "Repeater item labels" setting determines what is shown in the thumbnail overlay on hover. Example for an image field named "image": {image.basename} ({image.width}x{image.height})  
      https://github.com/Toutouwai/RepeaterImages
      https://modules.processwire.com/modules/repeater-images/
    • By EyeDentify
      Hello There Guys.

      I am in the process of getting into making my first modules for PW and i had a question for you PHP and PW gurus in here.

      I was wondering how i could use an external library, lets say TwitterOAuth in my PW module.
      Link to library
      https://twitteroauth.com/

      Would the code below be correct or how would i go about this:
      <?PHP namespace ProcessWire; /* load the TwitterOAuth library from my Module folder */ require "twitteroauth/autoload.php"; use Abraham\TwitterOAuth\TwitterOAuth; class EyeTwitter extends WireData,TwitterOAuth implements Module { /* vars */ protected $twConnection; /* extend parent TwitterOAuth contructor $connection = new TwitterOAuth(CONSUMER_KEY, CONSUMER_SECRET, $access_token, $access_token_secret); */ public function myTwitterConnection ($consumer_key, $consumer_secret, $access_token, $access_token_secret) { /* save the connection for use later */ $this->twConnection = TwitterOAuth::__construct($consumer_key, $consumer_secret, $access_token, $access_token_secret); } } ?> Am i on the right trail here or i am barking up the wrong tree?
      I don´t need a complete solution, i just wonder if i am including the external library the right way.
      If not, then give me a few hint´s and i will figure it out.

      Thanks a bunch.

      /EyeDentify
    • By dimitrios
      Hello,
      this module can publish content of a Processwire page on a Facebook page, triggered by saving the Processwire page.
      To set it up, configure the module with a Facebook app ID, secret and a Page ID. Following is additional configuration on Facebook for developers:
      Minimum Required Facebook App configuration:
      on Settings -> Basics, provide the App Domains, provide the Site URL, on Settings -> Advanced, set the API version (has been tested up to v3.3), add Product: Facebook Login, on Facebook Login -> Settings, set Client OAuth Login: Yes, set Web OAuth Login: Yes, set Enforce HTTPS: Yes, add "http://www.example.com/processwire/page/" to field Valid OAuth Redirect URIs. This module is configurable as follows:
      Templates: posts can take place only for pages with the defined templates. On/Off switch: specify a checkbox field that will not allow the post if checked. Specify a message and/or an image for the post.
      Usage
      edit the desired PW page and save; it will post right after the initial Facebook log in and permission granting. After that, an access token is kept.
       
      Download
      PW module directory: http://modules.processwire.com/modules/auto-fb-post/ Github: https://github.com/kastrind/AutoFbPost   Note: Facebook SDK for PHP is utilized.


    • By kongondo
      FieldtypeRuntimeMarkup and InputfieldRuntimeMarkup
       
      Modules Directory: http://modules.processwire.com/modules/fieldtype-runtime-markup/
      GitHub: https://github.com/kongondo/FieldtypeRuntimeMarkup
      As of 11 May 2019 ProcessWire versions earlier than 3.x are not supported
      This module allows for custom markup to be dynamically (PHP) generated and output within a page's edit screen (in Admin).
       
      The value for the fieldtype is generated at runtime. No data is saved in the database. The accompanying InputfieldRuntimeMarkup is only used to render/display the markup in the page edit screen.
       
      The field's value is accessible from the ProcessWire API in the frontend like any other field, i.e. it has access to $page and $pages.
       
      The module was commissioned/sponsored by @Valan. Although there's certainly other ways to achieve what this module does, it offers a dynamic and flexible alternative to generating your own markup in a page's edit screen whilst also allowing access to that markup in the frontend. Thanks Valan!
       
      Warning/Consideration
      Although access to ProcessWire's Fields' admin pages is only available to Superusers, this Fieldtype will evaluate and run the custom PHP Code entered and saved in the field's settings (Details tab). Utmost care should therefore be taken in making sure your code does not perform any CRUD operations!! (unless of course that's intentional) The value for this fieldtype is generated at runtime and thus no data is stored in the database. This means that you cannot directly query a RuntimeMarkup field from $pages->find(). Usage and API
       
      Backend
      Enter your custom PHP snippet in the Details tab of your field (it is RECOMMENDED though that you use wireRenderFile() instead. See example below). Your code can be as simple or as complicated as you want as long as in the end you return a value that is not an array or an object or anything other than a string/integer.
       
      FieldtypeRuntimeMarkup has access to $page (the current page being edited/viewed) and $pages. 
       
      A very simple example.
      return 'Hello'; Simple example.
      return $page->title; Simple example with markup.
      return '<h2>' . $page->title . '</h2>'; Another simple example with markup.
      $out = '<h1>hello '; $out .= $page->title; $out .= '</h1>'; return $out; A more advanced example.
      $p = $pages->get('/about-us/')->child('sort=random'); return '<p>' . $p->title . '</p>'; An even more complex example.
      $str =''; if($page->name == 'about-us') { $p = $page->children->last(); $str = "<h2><a href='{$p->url}'>{$p->title}</a></h2>"; } else { $str = "<h2><a href='{$page->url}'>{$page->title}</a></h2>"; } return $str; Rather than type your code directly in the Details tab of the field, it is highly recommended that you placed all your code in an external file and call that file using the core wireRenderFile() method. Taking this approach means you will be able to edit your code in your favourite text editor. It also means you will be able to type more text without having to scroll. Editing the file is also easier than editing the field. To use this approach, simply do:
      return wireRenderFile('name-of-file');// file will be in /site/templates/ If using ProcessWire 3.x, you will need to use namespace as follows:
      return ProcessWire\wireRenderFile('name-of-file'); How to access the value of RuntimeMarkup in the frontend (our field is called 'runtime_markup')
       
      Access the field on the current page (just like any other field)
      echo $page->runtime_markup; Access the field on another page
      echo $pages->get('/about-us/')->runtime_markup; Screenshots
       
      Backend
       

       

       
      Frontend
       

×
×
  • Create New...