hellomoto

PW equivalent of WP All Import Pro?

Recommended Posts

Would anyone here be able and inspired to develop a ProcessWire equivalent to the WordPress plugin WP All Import Pro? or to help me do so?

This is what I'm envisioning...

  • Upon installation the module creates an admin page titled "Import & Update". On the module config page you can specify allowed templates to run this on, otherwise allowing any.
  • Include the following PHP libraries: hQuery for web scraping, Csv for CSV handling, and Parser for XML.
  • Create template "import-update".
  • On the "Import & Update" page, a list of current import/updaters will be displayed (0 initially), each with corresponding links to "edit" or "run".

When you "Add New", this be the "import-update" template (with all module-specific fields tagged "impupd"):

  • title
  • destination (req.): parent, template
  • source (req.):
    • type (web, csv, xml)
    • location (url, file, text)
      • if web: opt. index URL & link selector, + paginator selector
      • if csv: opt. ignore 1st row
      • if xml: req. individual item node xpath
  • actions (check):
    • import (if none matching UID)
    • update (if matching UID & field values differ)
  • save() [req. here in flow]
  • map (repeater):
    • input (select fields from specified template to affect)
    • intake (corresponding DOM selectors / CSV col. letters/headers / xpath per field) (req.)
  • UID (unique ID; field reference to compare against, from selected input fields) (req.)
  • Lazy Cron interval

Scripts can be run via the import-update template; keep logs; show preview (iframe/ajax) for manual runs. ...

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By karian
      I don't know why multiple instances (repeater_repeat_columns1, repeater_repeat_columns2, ...) of my repeater field are displayed inside Template field (see image).
      Is there a way to clean/reset it ?
       

    • By psy
      I'm combining two PW sites into one, Site A into Site B.
      At each step, I did it bit by bit as the 'all at once' approach failed.
       
      First, I exported all the fields from Site A and imported into Site B. Any field types not supported by import/export, eg FieldtypeOptions I manually recreated. All good.
      Next I exported all the templates from Site A and imported them into Site B and copied across their associated template files. All good.
      Finally I exported the pages I needed from Site A into Site B - again, bit by bit to ensure it all went smoothly.
      From the admin side, it all looked and worked perfectly.
      Front end was a totally different story. All existing pages in Site B worked as expected. NONE of the pages imported from Site A displayed. They all ended in a redirect loop with no errors in the PW logs or Tracy Debugger.
      After some trial-and-error, I finally got it working with:
      - create a new template in Site B admin with no associated template file and just a title field
      - import the fields from the imported Site A template into the newly created template (both on Site B)
      - copy the Site A php template file into a new file that matched the new PW Site B template name and save in Site B site/templates
      I can deal with the above workaround. Just curious to know if I did something wrong or if the template import/export feature is problematic?
       
      ### Solution:
      While the export/import was a slow process, turned out the front end redirecting issue was unrelated. For reasons unknown, all templates marked as HTTPS only were the ones redirecting, ie all templates from Site A. Finally solved it by changing the $config->https to true in site/config.php
      Now the pages display correctly as https whether the template forces the issue or not.
       
    • By ryanC
      Hi, I have created a new install of Processwire 3.0.98, (local machine) and I am unable to install any modules from within Processwire itself. I go to Modules/Site/Add New/ then under "Module Class Name" I enter the name. The module appears, I hit "Download and Install", but I always get an error. For example: 
      "File could not be downloaded (https://github.com/adrianbj/TracyDebugger/archive/master.zip) 2: fsockopen(): SSL operation failed with code 1. OpenSSL Error messages: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version, 2: fsockopen(): Failed to enable crypto, 2: fsockopen(): unable to connect to ssl://github.com:443 (Unknown error) (tried: curl, fopen, socket)"
      I have been able to track down some of the modules and just upload the zip file manually, but this is happening with every module that I had previously installed in another installation with no problems. Any ideas? 
      Thanks!
    • By gebeer
      Hi,
      just wanted to share something I came across while working on an import module for XML data from a web service. The XML I got was not huge, but still, loading around 3.5 MB of XML with 250+ large child nodes into memory at once with simplexml_load_file() and then looping over it had significant impact on performance.
      I searched for a solution and found this great article about how to parse large XML files.
      It basically explains how to utilize the native XMLReader class together with SimpleXMLElement to handle such situations in a memory efficient way.
      After implementing it I got a significant improval on perceived performance. No comparison in numbers to share here as I'm a bit short on time.
    • By rareyush
      i am receiving and error whenever I try to run my processwire on localhost,

       
       
      sql code
       
      -- -- Table structure for table `field_fieldset_meta_end` -- CREATE TABLE `field_fieldset_meta_end` ( `pages_id` int(10) UNSIGNED NOT NULL, `data` int(11) NOT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; -- -------------------------------------------------------- -- -- Table structure for table `field_fieldset_meta_END` -- CREATE TABLE `field_fieldset_meta_END` ( `pages_id` int(10) UNSIGNED NOT NULL, `data` int(11) NOT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; anyone ?
      whenever I make a new database and upload it there, database get imported without errors.