kongondo

Pages Export (export PW 2.x - import into PW 3.x)

Recommended Posts

Posted (edited)

Pages Export

This module is for specifically exporting ProcessWire 2.x sites for later importing into ProcessWire 3.x.

Github: Project Page

Modules Directory: https://modules.processwire.com/modules/process-pages-export/

Credits: Ryan Cramer

Background

As I make my modules ProcessWire 3.x-compatible only, I've had the need to re-create/mirror their ProcessWire 2.x test sites in respective ProcessWire 3.x sites. These ProcessWire 3.x sites (one for each module) were already in place and I didn't feel like re-doing them by exporting/importing site profiles. I also like working with JSON rather than other export formats. So, I decided to write a custom pages export/import script for moving the ProcessWire 2.x sites to their respective ProcessWire 3.x counterpart sites. I'd just finished the export side of things when I came across a post in the forums that reminded me that ProcessWire 3.x already boasts a pages export/import feature, although some of it is still in development. Great! I like its API (PagesExportImport.php) and GUI (ProcessPagesExportImport.module) so no need to re-invent the wheel. 

I still had the small problem of making sure my JSON export would be compatible with the JSON input that the ProcessWire 3.x import expects. No need to re-invent the wheel, again! I ditched my custom script and instead ported the export functionalities of ProcessWire 3.x Pages Export/Import for use in ProcessWire 2.2 - 2.7, specifically to help migrate older sites to ProcessWire 3.x.

Compatibility

The module and class have been tested and work in ProcessWire 2.2, 2.3, 2.4, 2,5, 2.6 and 2.7.  The module is currently tagged as 'in development' until Pages Import feature of ProcessWire 3.x is released as stable. Nonetheless, I have not encountered any issues so far in either the export or the ProcessWire 3.x import. I think Ryan is waiting until he can support more complex field types before tagging the ProcessWire 3.x Pages Export/Import as production-ready.

This is not a ProcessWire 3.x module and will never be. It has no such need :). Just in case you forget and try to install it in a ProcessWire 3.x site, the module will throw a WireException(). I will also not be porting the ProcessWire 3.x import functionality for use in ProcessWire 2.x. That will defeat the purpose here; to move sites to ProcessWire 3.x and not the other way round.

Supported Fields

  • All non-complex fields such as integer, text, textarea, etc
  • Page fields
  • Repeaters
  • File and Image fields

I think these cover most needs. Note: not yet tested with Multilingual fields.

Technical

To ensure exports will be compatible with ProcessWire 3.x Pages Import, where necessary, the module borrows (and amends as needed) methods from ProcessWire 3.x for use in ProcessWire 2.x. For instance, ProcessWire 3.x Pages Export/Import uses the new(-ish) $file functions found in WireFileTools. Rather than copy and include such files, the module only borrowed and amended needed methods. These are listed below.

PagesExport.php

  • From /wire/core/Functions.php: wireInstanceOf(), wireClassName() and wireClassParents()
  • From /wire/core/Fieldtype.php: getImportValueOptions() and getDatabaseSchema()
  • From /wire/core/WireFileTools.php: zip(), chmod() and mkdir()
  • From /wire/core/WireHttp.php: sendFile
  • From /wire/modules/Fieldtype/FieldtypeFile.module: exportValue() and exportDescription()
  • From /wire/modules/Fieldtype/FieldtypeImage.module: exportValue()
  • From /wire/modules/Fieldtype/FieldtypePage.module: exportValue() and exportValuePage()
  • From /wire/modules/Fieldtype/FieldtypeRepeater.module: exportValue()
  • From /wire/core/Fieldtype/WireTempDir.php: create(), createName() and getTempDir()
  • All the export methods from the /wire/core/PagesExportImport.php class

ProcessPagesExport.module

  • All the export methods from /wire/modules/process/ProcessPagesExportImport/ProcessPagesExportImport.module

Newer methods such as $this->wire() will gracefully degrade to the older wire() function, ensuring smooth and uniform operation in ProcessWire 2.2  - 2.7.

Use

This module and class is for supersusers only and has only 1 aim; to export ProcessWire 2.x sites ready for importing into ProcessWire 3.x sites. You can either install (like any other module) and use the process module (ProcessPagesExport.module) or skip the install and just include and use the class (PagesExport.php) to export your sites.

Both the module (Export GUI) and API require that you are logged in as a supersuser before you can use them. The PagesExport class has a gateway method and option not found in the original class (PagesExportImport). The method export() allows access to the three export methods in the original class, i.e. pagesToArray(), exportJSON() and exportZip(). See example usage below.

GUI/Process Module

On install, the module will create a new admin page Export Pages. Please note that unlike the original code, this page is created directly under /admin/ and not /admin/pages/. Click on Export Pages to start.

Nothing much has changed from the original ProcessPagesExportImport.

In older ProcessWire versions where InputfieldSelector was not available, the module will instead present you with a text input to specify a valid (for that version of ProcessWire!) selector for finding pages. The other two methods for adding pages (add pages manually or add by parent) are still available.

Custom JS ensures older installs without showIf functionality still get the inputfield dependency treatment.

API

export($items, $options) 

PageArray $items: The PageArray to export.

Array $options: In addition to the options in the original class that you can pass to pagesToArray(), there are two more options here. mode to specify the export type (array or json or zip) and fieldNamesExclude, to specify fields to exclude from the export. Note that the original class option fieldNames still works. It's for specifying the fields to include in the export.

 

// API USAGE

// get and include PagesExport class
/* @note: you'll need to include the path differently if you are using the
 class directly without installing the Process module
*/
$path = $config->paths->ProcessPagesExport . 'PagesExport.php';
require_once($path);

// create new instance of the class
$siteExport = new PagesExport();

// find items to export
/* 
	a. export whole site! (minus admin and children)
	careful! in some cases, better to export in batches
*/
//$items = $pages->get('/')->find('has_parent!=2');

// export a batch of pages
$items = $pages->find('template=basic-page|computer');

/* you could also use these methods directly
#$data = $siteExport->pagesToArray($items);
#$data = $siteExport->exportJSON($items);
#$data = $siteExport->exportZIP($items);
*/
$options = array(
	// @kongondo addition: export to screen as 'array' or 'json' OR export to zip
	// zip will be saved in /site/assets/backups/PagesExport/ 
    'mode' => 'array',// array or json or ZIP
	// export only these field names, when specified
    'fieldNames' => array('images', 'files', 'multi_pages'),
	// @kongondo addition: exclude fields from export. Here we exclude 'body' field
	'fieldNamesExclude' => array('body'), 
);
// get the export
$data = $siteExport->export($items, $options);

if(is_array($data)) {
    echo '<pre> EXPORTED SITE USING pagesToArray ';
    print_r($data);
    echo '</pre>';	
}
// JSON export
else echo $data;

Screenshots

See also the links to Ryan's blog posts above.

ProcessWire 2.2

5ab398144be01_PagesExport-001-PW2.2.thumb.png.501eae0c5ca448cb005c96177b356f2b.png

 

ProcessWire 2.4

5ab3983dd1753_PagesExport-002-PW2.4.thumb.png.4f748e07e1c2dcd0814bd529a6e94668.png

 

ProcessWire 2.5

5ab3984db5a6e_PagesExport-003-PW2.5.thumb.png.58b2fb9ebd4cdeb4ac6a3cd007002c92.png

 

ProcessWire 2.7

5ab39858de23b_PagesExport-004-PW2.7.thumb.png.eecf9eecb468b1b7dce647007b3c0153.png

 

Video Demo

(Sorry, long and boring) 

Demo shows various exports from ProcessWire 2.x and their importing into ProcessWire 3.x. Remember the old Skyscrapers site profile? See how a whole Skyscrapers site gets exported from a ProcessWire 2.7.3 site and imported into a ProcessWire 3.x starting from here.

 

Edited by kongondo
Link to modules directory
  • Like 7
  • Thanks 1

Share this post


Link to post
Share on other sites

Did you find some problems with upgrading sites from 2.x to 3.x? Just wondering why you would export/import pages rather than duplicate the site then upgrade to PW3?

Share this post


Link to post
Share on other sites
3 hours ago, Robin S said:

Did you find some problems with upgrading sites from 2.x to 3.x? Just wondering why you would export/import pages rather than duplicate the site then upgrade to PW3?

I needed clean installs free from some modules and pages in the 2.x sites. I didn't want those carrying over into the new install and then uninstalling/deleting them. 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By bernhard
      @Sergio asked about the pdf creation process in the showcase thread about my 360° feedback/survey tool and so I went ahead and set my little pdf helper module to public:
       
      Download & Docs: https://gitlab.com/baumrock/RockPdf
       
      You can combine it easily with RockReplacer: 
       
    • By Soma
      LogMaintenance
      A simple ProcessWire module to give some maintenance control over log files. I found myself often having lots of log files for different things that can grow more or less quickly to a size where they can be difficult to maintain. The built in Logger of PW does a good job of giving you the possibility to delete or prune logs. But it has to be done manually and sometimes a log grows into millions of lines, which makes it often impossible to even prune it as it's too large.
      LogMaintenance uses LazyCron to run the maintenance task and there's several settings you can setup on a global or per log basis.
      Archive: will create zip files for each log file in logs/archive/ folder and add the log each time the maintenance is run to a subfolder containing the datetime.
      Lines: keeps logs to a certain number of lines 
      Days: keeps the log to a certain number of days
      Bytes: keeps the log to a certain amount of bytes
      Each setting is checked from top down, the first setting to contain something is used. So if you check the "Archive" option, all other settings are ignored and logs are archived everytime the LazyCron is executed. If you want to keep your logs to a certain amount of bytes just leave all other settings to 0 or blank.
      Per Log Settings
      There's a textarea that you can use to setup a config for a specific log file one per line. All the logs you define here ignore the global settings above. The syntax for the settings is:
      logname:[archive]:[lines]:[days]:[bytes] 
      errors:1:0:0:0 // would archive the errors log messages:0:10000:0:0 // will prune the errors log to 10000 lines  
      The module can be found on github for you to check out. It's still fresh and I'm currently testing.
      https://github.com/somatonic/LogMaintenance
    • By teppo
      Fieldtype Page IDs is a third party Fieldtype that, simply put, stores Page references as integers (Page IDs).
      This fieldtype was built as a quick and dirty workaround for Page Reference fields' inability handle self-references due to circular reference issues. A project I've been working on for a while now includes a combination of RepeaterMatrix content blocks and tagging/categorization system that would've resulted in a lot of duplicate pages (and plenty of unnecessary manual work for content editors) had I used built-in Page Reference fields, and thus a new Fieldtype felt like the most sensible approach.
      Fieldtype Page IDs was designed to be loosely compatible with Page References in order to make conversions between the two feasible, but it is quite limited feature wise:
      largely due to the fact that stored values are actually just integers with no connection to Pages whatsoever some advanced selectors and related features are not supported, and page values can't be directly accessed configuration settings are limited to the bare essentials (selector string and Inputfield class) only a handful of Inputfields (AsmSelect, Checkboxes, Text) are (currently) supported Anyway, in case you need to store Page IDs (and Page IDs only) and are happy with the limitations mentioned above, feel free to give this Fieldtype a try. It has been working fine for me in one particular project, but hasn't been tested that much, so please tread carefully – and let me know if you run into any issues.
      GitHub repository: https://github.com/teppokoivula/FieldtypePageIDs
      Modules directory: https://modules.processwire.com/modules/fieldtype-page-ids/
    • By daniels
      This is a lightweight alternative to other newsletter & newsletter-subscription modules.
      You can find the Module in the Modules directory and on Github
      It can subscribe, update, unsubscribe & delete a user in a list in Mailchimp with MailChimp API 3.0. It does not provide any forms or validation, so you can feel free to use your own. To protect your users, it does not save any user data in logs or sends them to an admin.
      This module fits your needs if you...
      ...use Mailchimp as your newsletter / email-automation tool ...want to let users subscribe to your newsletter on your website ...want to use your own form, validation and messages (with or without the wire forms) ...don't want any personal user data saved in any way in your ProcessWire environment (cf. EU data regulation terms) ...like to subscribe, update, unsubscribe or delete users to/from different lists ...like the Mailchimp UI for creating / sending / reviewing email campaigns *I have only tested it with PHP 7.x so far, so use on owners risk
      EDIT:
      I've updated the module to 0.0.4. I removed the instructions from this forum, so I don't have to maintain it on multiple places. Just checkout the readme on github 🙂
      If you have questions or like to contribute, just post a reply or create an issue or pr on github. 
    • By bernhard
      WHY?
      This module was built to fill the gap between simple $pages->find() operations and complex SQL queries.
      The problem with $pages->find() is that it loads all pages into memory and that can be a problem when querying multiple thousands of pages. Even $pages->findMany() loads all pages into memory and therefore is a lot slower than regular SQL.
      The problem with SQL on the other hand is, that the queries are quite complex to build. All fields are separate tables, some repeatable fields use multiple rows for their content that belong to only one single page, you always need to check for the page status (which is not necessary on regular find() operations and therefore nobody is used to that).
      In short: It is far too much work to efficiently and easily get an array of data based on PW pages and fields and I need that a lot for my RockGrid module to build all kinds of tabular data.

      Basic Usage

       
      Docs & Download
      https://modules.processwire.com/modules/rock-finder/
      https://gitlab.com/baumrock/RockFinder/tree/master
       
      Changelog
      180528, v1.0.4 add custom select statement option 180516 change sql query method, bump version to 1.0.0 180515 multilang bugfix 180513 beta release <180513 preview/discussion took place here: https://processwire.com/talk/topic/18983-rocksqlfinder-highly-efficient-and-flexible-sql-finder-module/