Jump to content
ryan

Module: Import Pages from CSV file

Recommended Posts

Nik has the best posts!

Another thing I wanted to add is that it's also very easy just to use API code to do these types of imports. 90% of the time I use API code to script imports with shell scripts, versus using the import pages CSV module. Though part of that is that I use the API every day and there's not a lot of figuring out to do. But if you want to post a sample of the data you are importing, I'd be happy to respond with a sample of how you might import it from the API.

Share this post


Link to post
Share on other sites

Thanks Ryan, although that's quite an overstatement what you're saying. But I'll take it. ;)

It's always nice to get feedback, especially positive one coming from the Guru Himself.

  • Like 1

Share this post


Link to post
Share on other sites

It works great. I hoped to be able to add one image, but suddenly I can add even more.

I'm not sure of the format in which to enter the 'image' column... I saw apeisa put a full url to his image column. Would this module then download the image from the link and put it in into the assets folder? or would the system always load that image from the mentioned url (ie, setting the url field equal to any, possibly external, link.)

If it does set to url field to the specified URL, then it may be a good opportunity to use a cdn for images, although I doubt it will work well that way if using the width() or height() resizers..

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Am I way off?

Share this post


Link to post
Share on other sites

otherwise, I'm assuming the image is downloaded and saved to the assets folder.

Your assumption is correct. Images are downloaded during import process.

  • Like 1

Share this post


Link to post
Share on other sites

Yup, that part of the process does make any conversion simpler :)

As long as that image is directly accessible via a browser then the importer can fetch it and put it into ProcessWire. Same with files and the file field.

The only thing you would have to watch in either case is that there is always the slim possibility you could get an unexpected file type coming from the source site. I don't think it's a problem as PW file fields only accept the file types you tell them to, but I just wasn't sure whether it would throw an error with the API so it might be worth putting in an additional check to see if it is a valid type.

Share this post


Link to post
Share on other sites

Hello! Is there any way to modify the "Created" field with this module? 

I have a CSV file where I store the dates of my old blog in a field, and the new entries to the blog are sorted with the "-created" selector in processwire. What I want to do is import the old articles of my non-processwire site to my new processwire website but using the date from the CSV to overwrite the "created" field so that it appears that it was created in that date and time.

Is there any way to do this?

Share this post


Link to post
Share on other sites

I have a situation right now, I'm not sure how unique it is or how easy it would be to solve. I have a group that runs a carshow that maintains their registration list on a custom built database application. They currently have the ability to export a .dbf file and import that directly into their website so registrants can confirm their paperwork is done.

I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? Certainly there's software that allows for a .dbf to be converted to .csv but that's extra steps I'd prefer not to have to impose on them. I could also convince them to try to get the original database application changed to export .csv instead of .dbf. That's a last resort though.

Any suggestions would be appreciated.

Share this post


Link to post
Share on other sites

 I'm redoing the website and now I'm struggling to figure out how I'll be able to give them that same option. I'm not sure how different a .dbf is to a .csv so is it possible to customize the csv import module to allow a .dbf?

Are their any other options? ...

I'm not totally sure but it seems that you can read the dBase file directly with PHP, so you will be able to read the records from the file and use the API to import them.

http://php.net/manual/en/ref.dbase.php

EDIT: here is a guy who has converted dBase to CSV: http://www.dev-zero.de/php/mit-php-dbase-zu-csv-konvertieren.html

He uses a dbase class from another guy: http://www.dev-zero.de/downloads/dbf_class.php

and the code to do the convert is short:

<?php
include("dbf_class.php");          // include the class
 
$dbaseDB = ‘DB.dbf’;         
$dbf = new dbf_class($dbaseDB);    // open the dBase file
 
$csvFile = ‘file.csv’;             // create a CSV file
$csv = fopen($csvFile,"w");
 
                                   // get record count, loop through and convert to csv
$num_rec=$dbf->dbf_num_rec;
fputcsv($csv, array_keys($dbf->getRowAssoc(0)));
 
for($i=0; $i<$num_rec; $i++) {
    fputcsv($csv, $dbf->getRow($i));
}
 
fclose($csv);          // ready! now use Ryans CSV-Importer 

But want to note that I only have googled that and have not tested / used this by myself, so- no warranties ;-)

Edited by horst
  • Like 3

Share this post


Link to post
Share on other sites

Hi! Great module, thanks for it.

I think it can help me to keep catalog up to date. But I need to generate page name from another field than Title (Many titles are similar, but "article number" field is unique). Is it possible somehow?

Share this post


Link to post
Share on other sites

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

Share this post


Link to post
Share on other sites

Is there a possibility to make the module available for an editor? If I move it out of the "admin" tree - it is available, but he has to click "view" to see the module. Would it be possible to leave it in the admin tree and make it visible in the top navigation for him?

I'm not sure what you want but the module can already be reached via the top navigation by 'Setup->Import Pages From CSV'

You could probably also make some easy adjustments to the module file before installing it to make the page go wherever you want.

https://github.com/ryancramerdesign/ImportPagesCSV/blob/master/ImportPagesCSV.module#L548 and further, i guess change the $parent variable.

Share this post


Link to post
Share on other sites

Yes, I know it is accessible through the "Setup" - but this part of the navigation is not allowed for a normal editor, just for a superuser.

I would like to make it available for a non-superuser in the top navigation ;-)

Share this post


Link to post
Share on other sites

Of course, stupid me..i will try to come up with a solution. Anyways i was wondering how you were able to move the page out of the admin tree, because this shouldn't be possible; the Import Pages From CSV page has the template 'admin' and this template only allows parents with admin template.

Share this post


Link to post
Share on other sites

Yes, you have to change the children settings for that template once to move the page, then set it back to the standard settings.

And I tried a second possibility: I copied the admin template and gave that other permissions. But somehow I couldn't manage it to make the page viewable in the top navigation for a non-superuser...

Share this post


Link to post
Share on other sites

Ok, got it - and I have to say once again: "Thanks, Soma!"

Here the thread and the post with the solution:

http://processwire.com/talk/topic/1272-new-page-nav-in-admin/?p=11276

The important part is the setting of the moduls permisson:

public static function getModuleInfo() {
		return array(
			'title' => 'Import Tabelle als CSV', 
			'version' => 103, 
			'summary' => 'Import CSV files to create ProcessWire pages.',
			'singular' => true, 
			'autoload' => false, 
			'permission' => 'page-edit'
			);

In this way the modul is editable by a non-superuser and can be accessed through the top navigation.

  • Like 2

Share this post


Link to post
Share on other sites

Just a quick question, is it possible that this could work with a password field (for uploading users). Thanks.

Share this post


Link to post
Share on other sites

Nik, your FieldtypePage tip with the id's as the reference worked perfectly. Add one more day that you are my hero :).

Share this post


Link to post
Share on other sites

Just wanted to say Thank You for this great module. I am working on a quite large project at the moment for which this module already has helped a lot.

  • Like 1

Share this post


Link to post
Share on other sites

Greetings,

I'll jump in as well...

Not too long ago, I used this module to run the initial import for a TV station project, where I needed to get 550+ pages from an old Joomla site.  After getting the data into the right csv format from the database, it was unbelievably smooth and easy to populate the pages.  I kept thinking, "Something will certainly go wrong with this," but it didn't!  I got all the existing Joomla pages into ProcessWire perfectly.  After that, the client used ProcessWire to create new pages.

My next goal with this -- build a front-end way for clients to do the imports themselves.

By the way, I was prompted to post this after reading and responding to this post: http://processwire.com/talk/topic/3917-importing-users-and-content-from-joomla/

Thanks Ryan!

Matthew

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Mike Rockett
      Jumplinks for ProcessWire
      Release: 1.5.56
      Composer: rockett/jumplinks
      Jumplinks is an enhanced version of the original ProcessRedirects by Antti Peisa.
      The Process module manages your permanent and temporary redirects (we'll call these "jumplinks" from now on, unless in reference to redirects from another module), useful for when you're migrating over to ProcessWire from another system/platform. Each jumplink supports wildcards, shortening the time needed to create them.
      Unlike similar modules for other platforms, wildcards in Jumplinks are much easier to work with, as Regular Expressions are not fully exposed. Instead, parameters wrapped in curly braces are used - these are described in the documentation.
      Under Development: 2.0, to be powered by FastRoute
      As of version 1.5.0, Jumplinks requires at least ProcessWire 2.6.1 to run.
      View on GitLab
      Download via the Modules Directory
      Read the docs
      Features
      The most prominent features include:
      Basic jumplinks (from one fixed route to another) Parameter-based wildcards with "Smart" equivalents Mapping Collections (for converting ID-based routes to their named-equivalents without the need to create multiple jumplinks) Destination Selectors (for finding and redirecting to pages containing legacy location information) Timed Activation (activate and/or deactivate jumplinks at specific times) 404-Monitor (for creating jumplinks based on 404 hits) Additionally, the following features may come in handy:
      Stale jumplink management Legacy domain support for slow migrations An importer (from CSV or ProcessRedirects) Feedback & Feature Requests
      I’d love to know what you think of this module. Please provide some feedback on the module as a whole, or even regarding smaller things that make it whole. Also, please feel free to submit feature requests and their use-cases.
      Note: Features requested so far have been added to the to-do list, and will be added to 2.0, and not the current dev/master branches.
      Open Source

      Jumplinks is an open-source project, and is free to use. In fact, Jumplinks will always be open-source, and will always remain free to use. Forever. If you would like to support the development of Jumplinks, please consider making a small donation via PayPal.
      Enjoy! :)
    • By BitPoet
      As threatened in Ryan's announcement for 3.0.139, I built a little module for sliding toggles as a replacement for checkboxes. Styling of the input is CSS3 only (with all the usual caveats about older browsers), no JS necessary, and may still be a bit "rough around the edges", so to speak, since I didn't have much time for testing on different devices or brushing things up enough so I'd feel comfortable pushing it to the module directory. But here's the link to the GitHub repo for now:
      InputfieldSlideToggle
      Fieldtype and Inputfield that implements smartphone-style toggles as replacement for checkbox inputs. The visualization is CSS-only, no additional JS necessary.
      Status
      Still very alpha, use with caution!
      Features / Field Settings
      Size
      You can render the toggles in four different sizes: small, medium, large and extra large.
      Off Color
      Currently, "unchecked" toggles can be displayed either in grey (default) or red.
      On Color
      "Checked" toggles can be rendered in one of these colors: blue (default), black, green, grey, orange or red.
      Screenshots

      Some examples with checkbox label


      View all Size and Color Combinations
      Small toggles Medium toggles Big toggles Extra big toggles  









    • By Orkun
      Hi Guys
      I needed to add extended functionalities for the InputfieldDatetime Module (module is from processwire version 2.7.3) because of a Request of Customer.
      So I duplicated the module and placed it under /site/modules/.
      I have added 3 new Settings to the InputfieldDatetime Module.
      1. Day Restriction - Restrict different days based on weekdays selection (e.g. saturday, sunday) - WORKING

       
      2. Time Slots - Define Time slots based on custom Integer Value (max is 60 for 1 hour) - WORKING

       
      3. Time Range Rules per Weekday - Define a minTime and MaxTime per Weekday (e.g. Opening Hours of a Restaurant) - NOT WORKING PROPERLY

       
      The Problem
      Time Slots and Day Restriction working fine so far. But the Time Range Rules per Weekday doesn't work right.
      What should happen is, that when you click on a date, it should update the minTime and maxTime of the Time Select.
      But the change on the select only happens if you select a date 2 times or when you select a date 1 time and then close the datepicker and reopen it again.
      The time select doesn't get change when you select a date 1 time and don't close the picker.
      Here is the whole extended InputfieldDatetime Module.
      The Files that I have changed:
      InputfieldDatetime.module InputfieldDatetime.js jquery-ui-timepicker-addon.js (https://trentrichardson.com/examples/timepicker/) - updated it to the newest version, because minTime and maxTime Option was only available in the new version  
      Thats the Part of the JS that is not working correctly:
      if(datetimerules && datetimerules.length){ options.onSelect = function(date, inst) { var day = $(this).datetimepicker("getDate").getDay(); day = day.toString(); var mintime = $(this).attr('data-weekday'+day+'-mintime'); var maxtime = $(this).attr('data-weekday'+day+'-maxtime'); console.log("weekday: "+day); console.log("minTime: "+mintime); console.log("maxTime: "+maxtime); var optionsAll = $(this).datetimepicker( "option", "all" ); optionsAll.minTime = mintime; optionsAll.maxTime = maxtime; $(this).datetimepicker('destroy'); $(this).datetimepicker(optionsAll); $(this).datetimepicker('refresh'); //$.datepicker._selectDate($(this).attr("id"),date); //$.datepicker._base_getDateDatepicker(); // var inst = $.datepicker._getInst($(this)); // $.datepicker._updateDatepicker(inst); /*$(this).datetimepicker('destroy'); InputfieldDatetimeDatepicker($(this), mintime, maxtime); $(this).datetimepicker('refresh'); */ // $(this).datetimepicker('option', {minTime: mintime, maxTime: maxtime}); } } Can you have a look and find out what the Problem is?
      InputfieldDatetime.zip
       
      Kind Regards
      Orkun
    • By teppo
      This module tracks changes, additions, removals etc. of public (as in "not under admin") pages of your site. Like it's name says, it doesn't attempt to be a version control system or anything like that - just a log of what's happened.
      At the moment it's still a work in progress and will most likely be a victim of many ruthless this-won't-work-let's-try-that-instead cycles, but I believe I've nailed basic functionality well enough to post it here.. so, once again, I'll be happy to hear any comments you folks can provide
      https://modules.processwire.com/modules/process-changelog/
      https://github.com/teppokoivula/ProcessChangelog
      How does it work?
      Exactly like it's (sort of) predecessor, Process Changelog actually consists of two modules: Process Changelog and Process Changelog Hooks. Hooks module exists only to serve main module by hooking into various functions within Pages class, collecting data of performed operations, refining it and keeping up a log of events in it's own custom database table (process_changelog.) Visible part is managed by Process Changelog, which provides users a (relatively) pretty view of the contents of said log table.
      How do you use it?
      When installed this module adds new page called Changelog under Admin > Setup which provides you with a table view of collected data and basic filtering tools See attached screenshots to get a general idea about what that page should look like after a while.
      For detailed installation instructions etc. see README.md.
       


    • By Gadgetto
      Status update links (inside this thread) for SnipWire development will be always posted here:
      2019-08-08
      2019-06-15
      2019-06-02
      2019-05-25
      If you are interested, you can test the current state of development:
      https://github.com/gadgetto/SnipWire
      Please note that the software is not yet intended for use in a production system (alpha version).
      If you like, you can also submit feature requests and suggestions for improvement. I also accept pull requests.
      ---- INITIAL POST FROM 2019-05-25 ----
      I wanted to let you know that I am currently working on a new ProcessWire module that fully integrates the Snipcart Shopping Cart System into ProcessWire. (this is a customer project, so I had to postpone the development of my other module GroupMailer).
      The new module SnipWire offers full integration of the Snipcart Shopping Cart System into ProcessWire.
      Here are some highlights:
      simple setup with (optional) pre-installed templates, product fields, sample products (quasi a complete shop system to get started immediately) store dashboard with all data from the snipcart system (no change to the snipcart dashboard itself required) Integrated REST API for controlling and querying snipcart data webhooks to trigger events from Snipcart (new order, new customer, etc.) multi currency support self-defined/configurable tax rates etc. Development is already well advanced and I plan to release the module in the next 2-3 months.
      I'm not sure yet if this will be a "Pro" module or if it will be made available for free.
      I would be grateful for suggestions and hints!
      (please have a look at the screenshots to get an idea what I'm talking about)
       




×
×
  • Create New...