hellomoto

PW equivalent of WP All Import Pro?

Recommended Posts

Would anyone here be able and inspired to develop a ProcessWire equivalent to the WordPress plugin WP All Import Pro? or to help me do so?

This is what I'm envisioning...

  • Upon installation the module creates an admin page titled "Import & Update". On the module config page you can specify allowed templates to run this on, otherwise allowing any.
  • Include the following PHP libraries: hQuery for web scraping, Csv for CSV handling, and Parser for XML.
  • Create template "import-update".
  • On the "Import & Update" page, a list of current import/updaters will be displayed (0 initially), each with corresponding links to "edit" or "run".

When you "Add New", this be the "import-update" template (with all module-specific fields tagged "impupd"):

  • title
  • destination (req.): parent, template
  • source (req.):
    • type (web, csv, xml)
    • location (url, file, text)
      • if web: opt. index URL & link selector, + paginator selector
      • if csv: opt. ignore 1st row
      • if xml: req. individual item node xpath
  • actions (check):
    • import (if none matching UID)
    • update (if matching UID & field values differ)
  • save() [req. here in flow]
  • map (repeater):
    • input (select fields from specified template to affect)
    • intake (corresponding DOM selectors / CSV col. letters/headers / xpath per field) (req.)
  • UID (unique ID; field reference to compare against, from selected input fields) (req.)
  • Lazy Cron interval

Scripts can be run via the import-update template; keep logs; show preview (iframe/ajax) for manual runs. ...

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By jrtderonde
      Hey,
      I'm working around a module that adds pages in my ProcessWire installation. The pages are added based on a JSON array that will be imported through a page save. Everything works fine except for when the pages are added. I get the following error
      Integrity constraint violation: 1062 Duplicate entry '3e215ecd6774fd99c2b0eb5cadf36a07-1269' for key 'name_parent_id' I'm using the following code/loop to generate the pages.
      // Loop through the files foreach ($p->importFile as $file) { // Set the file location $name = $file->data["basename"]; $path = $p->importFile->path; $location = $path . $name; // Get the file $json = file_get_contents($location); $json = json_decode($json); // Loop through the rows of the import foreach ($json->ttEntityDelAddrLink as $client) { // Create new page $new = new Page(); // Create unique hash $unique = md5(date("Y-m-d H:i:s") . "-" . $client->CustomerCode); // Set some variables for the new page $new->setOutputFormatting(false); $new->template = "_client"; $new->parent = $p; // Create hash $new->title = $client->DelAddressName; $new->name = $unique; // Page specific fields $new->company = $client->DelAddressName; $new->companyId = $client->CustomerCode; $new->city = $client->DelAddressCity; $new->address = $client->DelAddressStreet; $new->postcode = $client->DelAddressZipCode; $new->country = $countries[$client->DelAddressCountryCode]; // Save the page $new->save(); } // Exit for debugging exit; }
      Does anybody know what's wrong?
    • By John W.
      SYNOPSIS
      A little guide to generating an sitemap.xml using (I believe) a script Ryan originally wrote with the addition of being able to optionally exclude child pages from being output in the sitemap.xml file.
      I was looking back on a small project today where I was using a php script to generate an xml file, I believe the original was written by Ryan. Anyway, I needed a quick fix for the script to allow me to optionally exclude children of pages from being included in the sitemap.xml output.
      OVERVIEW
      A good example of this is a site where if you visit /minutes/ a page displays a list of board meetings which includes a title,  date, description and link to download the .pdf file.
      I have a template called minutes and a template called minutes-document. The first page, minutes, when loaded via /minutes/ simply grabs all of its child pages and outputs the name, description and actual path of an uploaded .pdf file for a visitor to download.
      In my back-end I have the template MINUTES and MINUTES-DOCUMENT. Thus:


      So, basically, their employee can login, hover over minutes, click new, then create a new (child) record and name it the date of the meeting e.g. June 3rd, 2016 :

       
      ---------------------------
      OPTIONALLY EXCLUDING CHILDREN - SETUP
      Outputting the sitemap.xml and optionally excluding children that belong to a template.
      The setup of the original script is as follows:
      1. Save the file to the templates folder as sitemap.xml.php
      2. Create a template called sitemap-xml and use the sitemap.xml.php file.
      3. Create a page called sitemap.xml using the sitemap-xml template
       
      Now, with that done you will need to make only a couple of slight modifications that will allow the script to exclude children of a template from output to the sitemap.xml
      1. Create a new checkbox field and name it:   sitemap_exclude_children
      2. Add the field to a template that you want to control whether the children are included/excluded from the sitemap. In my example I added it to my "minutes" template.
      3. Next, go to a page that uses a template with the field you added above. In my case, "MINUTES"
      4. Enable the checkbox to exclude children, leave it unchecked to include children.
      For example, in my MINUTES page I enabled the checkbox and now when /sitemap.xml is loaded the children for the MINUTES do not appear in the file.

       
      A SIMPLE CONDITIONAL TO CHECK THE "sitemap_exclude_children" VALUE
      This was a pretty easy modification to an existing script, adding only one line. I just figure there may be others out there using this script with the same needs.
      I simply inserted the if condition as the first line in the function:
      function renderSitemapChildren(Page $page) { if($page->sitemap_exclude_children) return ""; ... ... ...  
      THE FULL SCRIPT WITH MODIFICATION
      <?php /** * ProcessWire Template to power a sitemap.xml * * 1. Copy this file to /site/templates/sitemap-xml.php * 2. Add the new template from the admin. * Under the "URLs" section, set it to NOT use trailing slashes. * 3. Create a new page at the root level, use your sitemap-xml template * and name the page "sitemap.xml". * * Note: hidden pages (and their children) are excluded from the sitemap. * If you have hidden pages that you want to be included, you can do so * by specifying the ID or path to them in an array sent to the * renderSiteMapXML() method at the bottom of this file. For instance: * * echo renderSiteMapXML(array('/hidden/page/', '/another/hidden/page/')); * * patch to prevent pages from including children in the sitemap when a field is checked / johnwarrenllc.com * 1. create a checkbox field named sitemap_exclude_children * 2. add the field to the parent template(s) you plan to use * 3. when a new page is create with this template, checking the field will prevent its children from being included in the sitemap.xml output */ function renderSitemapPage(Page $page) { return "\n<url>" . "\n\t<loc>" . $page->httpUrl . "</loc>" . "\n\t<lastmod>" . date("Y-m-d", $page->modified) . "</lastmod>" . "\n</url>"; } function renderSitemapChildren(Page $page) { if($page->sitemap_exclude_children) return ""; /* Aded to exclude CHILDREN if field is checked */ $out = ''; $newParents = new PageArray(); $children = $page->children; foreach($children as $child) { $out .= renderSitemapPage($child); if($child->numChildren) $newParents->add($child); else wire('pages')->uncache($child); } foreach($newParents as $newParent) { $out .= renderSitemapChildren($newParent); wire('pages')->uncache($newParent); } return $out; } function renderSitemapXML(array $paths = array()) { $out = '<?xml version="1.0" encoding="UTF-8"?>' . "\n" . '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">'; array_unshift($paths, '/'); // prepend homepage foreach($paths as $path) { $page = wire('pages')->get($path); if(!$page->id) continue; $out .= renderSitemapPage($page); if($page->numChildren) { $out .= renderSitemapChildren($page); } } $out .= "\n</urlset>"; return $out; } header("Content-Type: text/xml"); echo renderSitemapXML(); // Example: echo renderSitemapXML(array('/hidden/page/'));  
      In conclusion, I have used a couple different processwire sitemap generating modules. But for my needs, the above script is fast and easy to setup/modify.
      - Thanks
       
    • By cosmicsafari
      Hi all,
      I have noticed that LazyCorn seems to run maybe once or twice and then just stops.
      Has anybody else encountered this, I have the following within 2 different modules.
      public function init(){ $this->addHookAfter('ProcessModule::executeEdit', $this, 'configurableButtons'); wire()->addHook('LazyCron::every15Minutes', $this, 'autoGenerate'); } public function init(){ wire('forms')->addHookAfter('ProcessModule::executeEdit', $this, 'configurableButtons'); wire()->addHook('LazyCron::everyHour', $this, 'getExport'); } Does having multiple LazyCrons setup cause any issues?
    • By nickngqs
      Does anyone know how to do an export of a page entries to CSV using the Processwire way?
      I'm trying to setup a list where the client can see a list of user's form values.

      Example ->
      A front end form -> Upon submit -> Adds to database -> Shows on the admin of Processwire.

      From there, I can see the entries and sort stuff. And lastly I can export them in CSV.

      I'm looking at the Processwire cheatsheet and see the $files, is this the one I should be looking at? 
    • By benbyf
      HELLO!
      Thought of a super interesting module and wondered if anyone had created it or not, and some feedback on the idea.
      New page field defaults - on creating a new child page, you choose a template and page title and click submit -> the resulting page is prepopulated with content dependent on the template you chose.
      Not sure about implmentation yet just thought was an interesting idea.