hellomoto

PW equivalent of WP All Import Pro?

Recommended Posts

Would anyone here be able and inspired to develop a ProcessWire equivalent to the WordPress plugin WP All Import Pro? or to help me do so?

This is what I'm envisioning...

  • Upon installation the module creates an admin page titled "Import & Update". On the module config page you can specify allowed templates to run this on, otherwise allowing any.
  • Include the following PHP libraries: hQuery for web scraping, Csv for CSV handling, and Parser for XML.
  • Create template "import-update".
  • On the "Import & Update" page, a list of current import/updaters will be displayed (0 initially), each with corresponding links to "edit" or "run".

When you "Add New", this be the "import-update" template (with all module-specific fields tagged "impupd"):

  • title
  • destination (req.): parent, template
  • source (req.):
    • type (web, csv, xml)
    • location (url, file, text)
      • if web: opt. index URL & link selector, + paginator selector
      • if csv: opt. ignore 1st row
      • if xml: req. individual item node xpath
  • actions (check):
    • import (if none matching UID)
    • update (if matching UID & field values differ)
  • save() [req. here in flow]
  • map (repeater):
    • input (select fields from specified template to affect)
    • intake (corresponding DOM selectors / CSV col. letters/headers / xpath per field) (req.)
  • UID (unique ID; field reference to compare against, from selected input fields) (req.)
  • Lazy Cron interval

Scripts can be run via the import-update template; keep logs; show preview (iframe/ajax) for manual runs. ...

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By mtwebit
      I was looking for a module that allows the execution of long-running tasks (working with tens of thousands of pages) and could not find a suitable solution.
      It started with the import problem. I have lots of data in XML form (20k+ complex entries) that I want to import into ProcessWire.
      XMLReader() works fine but it takes a very long time to import all data so a simple "upload + process data on page save" would not work.
      So I've created a new module for this. Meet my first (well, third) PW module: Tasker.
      It's a simple module that executes long tasks in the background (using Cron or LazyCron) and reports their status to the user using a JQuery progressbar.
      Any suggestions are welcome. How do you solve similar problems?
      E.g. which is the best way to delete large number of pages? (max_exec_time will expire so I check it before the delete() call.)
      $children = $page->children('template='.$this->template.',include=all'); // creates lot of page objects, may not have time to delete all + ...->delete() OR $childIDs = $this->pages->findIDs('parent='.$page->id.',template='.$this->template.',include=all'); // will create page objects later + $pages->get(..id..)->delete() Be nice (I know you're always), I'm a PW-newbie, just started working with ProcessWire. I was developing sites with Drupal for a very long time but their non-existent module upgrade path finally has driven me away from it.
    • By Mike Rockett
         
      Docs & Download: rockettpw/markup-sitemap
      Modules Directory: MarkupSitemap
      MarkupSitemap is essentially an upgrade to MarkupSitemapXML by Pete. It adds multi-language support using the built-in LanguageSupportPageNames. Where multi-language pages are available, they are added to the sitemap by means of an alternate link in that page's <url>. Support for listing images in the sitemap on a page-by-page basis and using a sitemap stylesheet are also added.
      Example when using the built-in multi-language profile:
      <url> <loc>http://domain.local/about/</loc> <lastmod>2017-08-27T16:16:32+02:00</lastmod> <xhtml:link rel="alternate" hreflang="en" href="http://domain.local/en/about/"/> <xhtml:link rel="alternate" hreflang="de" href="http://domain.local/de/uber/"/> <xhtml:link rel="alternate" hreflang="fi" href="http://domain.local/fi/tietoja/"/> </url> It also uses a locally maintained fork of a sitemap package by Matthew Davies that assists in automating the process.
      The doesn't use the same sitemap_ignore field available in MarkupSitemapXML. Rather, it renders sitemap options fields in a Page's Settings tab. One of the fields is for excluding a Page from the sitemap, and another is for excluding its children. You can assign which templates get these config fields in the module's configuration (much like you would with MarkupSEO).
      Note that the two exclusion options are mutually exclusive at this point as there may be cases where you don't want to show a parent page, but only its children. Whilst unorthodox, I'm leaving the flexibility there. (The home page cannot be excluded from the sitemap, so the applicable exclusion fields won't be available there.)
      Sitemap also allows you to include images for each page at the template level, and you can disable image output at the page level.
      The module allows you to set the priority on a per-page basis (it's optional and will not be included if not set).
      Lastly, a stylesheet option has also been added. You can use the default one (enabled by default), or set your own.
      Note that if the module is uninstalled, any saved data on a per-page basis is removed. The same thing happens for a specific page when it is deleted after having been trashed.
          
    • By hellomoto
      Pressing problem at initial posting:  I am working on a module that runs/previews (for pub/unpub) imports on importer page view. The template has to be in site/templates, so far as I know, so one is created there on install which includes a file to run from within the module, but then when it's run and tries to read the importer csv file it cannot, since it's in the admin, I guess... Is it possible to set the template page to a file in the module directory? Will that work?
      https://bitbucket.org/chimmel/pw-import-everything