kongondo

Pages Export (export PW 2.x - import into PW 3.x)

Recommended Posts

Posted (edited)

Pages Export

This module is for specifically exporting ProcessWire 2.x sites for later importing into ProcessWire 3.x.

Github: Project Page

Modules Directory: https://modules.processwire.com/modules/process-pages-export/

Credits: Ryan Cramer

Background

As I make my modules ProcessWire 3.x-compatible only, I've had the need to re-create/mirror their ProcessWire 2.x test sites in respective ProcessWire 3.x sites. These ProcessWire 3.x sites (one for each module) were already in place and I didn't feel like re-doing them by exporting/importing site profiles. I also like working with JSON rather than other export formats. So, I decided to write a custom pages export/import script for moving the ProcessWire 2.x sites to their respective ProcessWire 3.x counterpart sites. I'd just finished the export side of things when I came across a post in the forums that reminded me that ProcessWire 3.x already boasts a pages export/import feature, although some of it is still in development. Great! I like its API (PagesExportImport.php) and GUI (ProcessPagesExportImport.module) so no need to re-invent the wheel. 

I still had the small problem of making sure my JSON export would be compatible with the JSON input that the ProcessWire 3.x import expects. No need to re-invent the wheel, again! I ditched my custom script and instead ported the export functionalities of ProcessWire 3.x Pages Export/Import for use in ProcessWire 2.2 - 2.7, specifically to help migrate older sites to ProcessWire 3.x.

Compatibility

The module and class have been tested and work in ProcessWire 2.2, 2.3, 2.4, 2,5, 2.6 and 2.7.  The module is currently tagged as 'in development' until Pages Import feature of ProcessWire 3.x is released as stable. Nonetheless, I have not encountered any issues so far in either the export or the ProcessWire 3.x import. I think Ryan is waiting until he can support more complex field types before tagging the ProcessWire 3.x Pages Export/Import as production-ready.

This is not a ProcessWire 3.x module and will never be. It has no such need :). Just in case you forget and try to install it in a ProcessWire 3.x site, the module will throw a WireException(). I will also not be porting the ProcessWire 3.x import functionality for use in ProcessWire 2.x. That will defeat the purpose here; to move sites to ProcessWire 3.x and not the other way round.

Supported Fields

  • All non-complex fields such as integer, text, textarea, etc
  • Page fields
  • Repeaters
  • File and Image fields

I think these cover most needs. Note: not yet tested with Multilingual fields.

Technical

To ensure exports will be compatible with ProcessWire 3.x Pages Import, where necessary, the module borrows (and amends as needed) methods from ProcessWire 3.x for use in ProcessWire 2.x. For instance, ProcessWire 3.x Pages Export/Import uses the new(-ish) $file functions found in WireFileTools. Rather than copy and include such files, the module only borrowed and amended needed methods. These are listed below.

PagesExport.php

  • From /wire/core/Functions.php: wireInstanceOf(), wireClassName() and wireClassParents()
  • From /wire/core/Fieldtype.php: getImportValueOptions() and getDatabaseSchema()
  • From /wire/core/WireFileTools.php: zip(), chmod() and mkdir()
  • From /wire/core/WireHttp.php: sendFile
  • From /wire/modules/Fieldtype/FieldtypeFile.module: exportValue() and exportDescription()
  • From /wire/modules/Fieldtype/FieldtypeImage.module: exportValue()
  • From /wire/modules/Fieldtype/FieldtypePage.module: exportValue() and exportValuePage()
  • From /wire/modules/Fieldtype/FieldtypeRepeater.module: exportValue()
  • From /wire/core/Fieldtype/WireTempDir.php: create(), createName() and getTempDir()
  • All the export methods from the /wire/core/PagesExportImport.php class

ProcessPagesExport.module

  • All the export methods from /wire/modules/process/ProcessPagesExportImport/ProcessPagesExportImport.module

Newer methods such as $this->wire() will gracefully degrade to the older wire() function, ensuring smooth and uniform operation in ProcessWire 2.2  - 2.7.

Use

This module and class is for supersusers only and has only 1 aim; to export ProcessWire 2.x sites ready for importing into ProcessWire 3.x sites. You can either install (like any other module) and use the process module (ProcessPagesExport.module) or skip the install and just include and use the class (PagesExport.php) to export your sites.

Both the module (Export GUI) and API require that you are logged in as a supersuser before you can use them. The PagesExport class has a gateway method and option not found in the original class (PagesExportImport). The method export() allows access to the three export methods in the original class, i.e. pagesToArray(), exportJSON() and exportZip(). See example usage below.

GUI/Process Module

On install, the module will create a new admin page Export Pages. Please note that unlike the original code, this page is created directly under /admin/ and not /admin/pages/. Click on Export Pages to start.

Nothing much has changed from the original ProcessPagesExportImport.

In older ProcessWire versions where InputfieldSelector was not available, the module will instead present you with a text input to specify a valid (for that version of ProcessWire!) selector for finding pages. The other two methods for adding pages (add pages manually or add by parent) are still available.

Custom JS ensures older installs without showIf functionality still get the inputfield dependency treatment.

API

export($items, $options) 

PageArray $items: The PageArray to export.

Array $options: In addition to the options in the original class that you can pass to pagesToArray(), there are two more options here. mode to specify the export type (array or json or zip) and fieldNamesExclude, to specify fields to exclude from the export. Note that the original class option fieldNames still works. It's for specifying the fields to include in the export.

 

// API USAGE

// get and include PagesExport class
/* @note: you'll need to include the path differently if you are using the
 class directly without installing the Process module
*/
$path = $config->paths->ProcessPagesExport . 'PagesExport.php';
require_once($path);

// create new instance of the class
$siteExport = new PagesExport();

// find items to export
/* 
	a. export whole site! (minus admin and children)
	careful! in some cases, better to export in batches
*/
//$items = $pages->get('/')->find('has_parent!=2');

// export a batch of pages
$items = $pages->find('template=basic-page|computer');

/* you could also use these methods directly
#$data = $siteExport->pagesToArray($items);
#$data = $siteExport->exportJSON($items);
#$data = $siteExport->exportZIP($items);
*/
$options = array(
	// @kongondo addition: export to screen as 'array' or 'json' OR export to zip
	// zip will be saved in /site/assets/backups/PagesExport/ 
    'mode' => 'array',// array or json or ZIP
	// export only these field names, when specified
    'fieldNames' => array('images', 'files', 'multi_pages'),
	// @kongondo addition: exclude fields from export. Here we exclude 'body' field
	'fieldNamesExclude' => array('body'), 
);
// get the export
$data = $siteExport->export($items, $options);

if(is_array($data)) {
    echo '<pre> EXPORTED SITE USING pagesToArray ';
    print_r($data);
    echo '</pre>';	
}
// JSON export
else echo $data;

Screenshots

See also the links to Ryan's blog posts above.

ProcessWire 2.2

5ab398144be01_PagesExport-001-PW2.2.thumb.png.501eae0c5ca448cb005c96177b356f2b.png

 

ProcessWire 2.4

5ab3983dd1753_PagesExport-002-PW2.4.thumb.png.4f748e07e1c2dcd0814bd529a6e94668.png

 

ProcessWire 2.5

5ab3984db5a6e_PagesExport-003-PW2.5.thumb.png.58b2fb9ebd4cdeb4ac6a3cd007002c92.png

 

ProcessWire 2.7

5ab39858de23b_PagesExport-004-PW2.7.thumb.png.eecf9eecb468b1b7dce647007b3c0153.png

 

Video Demo

(Sorry, long and boring) 

Demo shows various exports from ProcessWire 2.x and their importing into ProcessWire 3.x. Remember the old Skyscrapers site profile? See how a whole Skyscrapers site gets exported from a ProcessWire 2.7.3 site and imported into a ProcessWire 3.x starting from here.

 

Edited by kongondo
Link to modules directory
  • Like 7
  • Thanks 1

Share this post


Link to post
Share on other sites

Did you find some problems with upgrading sites from 2.x to 3.x? Just wondering why you would export/import pages rather than duplicate the site then upgrade to PW3?

Share this post


Link to post
Share on other sites
3 hours ago, Robin S said:

Did you find some problems with upgrading sites from 2.x to 3.x? Just wondering why you would export/import pages rather than duplicate the site then upgrade to PW3?

I needed clean installs free from some modules and pages in the 2.x sites. I didn't want those carrying over into the new install and then uninstalling/deleting them. 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By blad
      Hi guys!
      I just uploaded a module to explore files based on elFinder. By default it will show the "Files" folder.
      Screenshots:

      Video:
       
      To do:
       More options To fix:
       The function of rotating or scaling an image fails  Image editors Github:
      https://github.com/LuisSantiago/ProcessElFinder/
      I hope you like it.
    • By BitPoet
      I'm really in love with FormBuilder, but the one thing missing to match all my end users' expectations were repeatable field groups. Think repeaters, in ProcessWire terms. Our primary application of PW is our corporate intranet, so "lines" of fields are quite common in the forms I build. We have all kinds of request forms where the information for a varying number of colleagues needs to be entered (from meal order to flight booking request) and where it is simply impractical to send a form for each, and I don't want to clutter my forms with multiple instances of fields that may only get used ten percent of the time.
      That's why I started to build FormBuilderMultiplier (link to GitHub).
      What it does:
      Adds an option to make a regular Fieldgroup repeatable Lets you limit the number of instances of a Fieldgroup on the form Adds an "Add row" button the form that adds another instance of the Fieldgroup's fields Adds a counter suffix at the end of every affected field's label Stores the entered values just like regular fields Makes the entered values available in preview and email notifications Supports most text based fields, textareas and selects (really, I haven't had enough time to test all the available choices yet) What it doesn't do (yet):
      Support saving to ProcessWire pages (i.e. real Repeaters) I haven't tested all the validation stuff, Date/Time inputs etc. yet, but since I'm utterly swamped with other stuff at work, I didn't want to wait until I have it polished. Any feedback is welcome. There might also be some issues with different output frameworks that I haven't encountered yet. The forms I work with mostly use UIKit.
      Status:
      Still alpha, so test well before using it in the field.
      Known issues:
      When rows are added, the form's iframe needs to be resized, which isn't completely clean yet.
      How it works:
      The Fieldgroup settings are added through regular hooks, as is the logic that adds the necessary field copies for processing the form and displaying previews.
      "Multiplied" field instances are suffixed with _NUM, where NUM is an incremental integer starting from 1. So if you have add two fields named "surname" and "givenname" to a fieldgroup and check the "multiply" checkbox, the form will initially have "surname_1" and "givenname_1" field (I'm still considering changing that to make the risk to shoot oneself into the foot by having a regular "surname_1" field somewhere else in the form less likely).
      When a "row" is added, the first row is cloned through JS and the counter in the fields' IDs, names and "for" attributes as well as the counter in the label are incremented before appending the copies to the Fieldset container in the form.
      To keep backend and frontend in sync, a hidden field named [name of the fieldset]__multiplier_rows is added to the form. Both the backend and the frontend script use this to store and retrieve the number of "rows".
      ToDo:
      Naturally, add the option to store the data in real repeaters when saving to pages. Do a lot of testing (and likely fixing). Make a few things (like the "Add row" button label etc.) configurable in field(set) context. Add a smooth API to retrieve the multiplied values as WireArrays. The mandatory moving screenshot:

    • By MoritzLost
      Hello there,
      I'm working on a tiny textformatter module that searches the text for titles of other pages on your site and creates hyperlinks to them. I'm not sure if something like this exists already, but I haven't found anything in the module directory, so I wrote my own solution 🙂
      It's not properly tested yet and is still missing some functionality I would like to implement, so at the moment it should be considered in BETA. Features include limiting the pages that will get searched by template, and adding a custom CSS class to the generated hyperlinks. As I'm writing this I noticed that it will probably include unpublished and hidden pages at the moment, so yeah ... it's still in development alright 😅
      You can download the module from Github:
      https://github.com/MoritzLost/TextformatterPageTitleLinks
      There's some more information in the readme as well.
      Anyway, let me know what you think! I'm happy about any feedback, possible improvements or ideas on how to improve the module. Cheers.
    • By adrian
      This module provides a way to rapidly generate Page fields and the required templates and pages for use as a drop down select (or any other Page field type).
      This module will let you create a full page field setup in literally a few seconds 
      To use, run Page Field Select Creator from the Setup Menu
      Enter a Field Title, eg: Room Types Select Options - These will become the child pages that will populate the page field select options. There are two different options.
       
      Option 1. TITLE FIELD ONLY - enter one option per line, eg:
       
      Single
      Double
      Suite
       
       
      Option 2. MULTIPLE FIELDS - the first line is used for the field names and the first field must be 'Title'. Subsequent lines are the values for the fields, eg:
       
      Title, Number of Beds, Number of People, Kitchen Facilities
      Single, 1, 1, Fridge Only
      Double, 2, 2, Fridge Only
      Suite, 3, 6, Full Kitchen
        Choose the parent where the page tree of options will be created, eg a hidden "Options" parent page Select a "Deference in API as" option depending on your needs Choose the input field type Check whether "Allow new pages to be created from field?" should be enabled. As an example, if you entered "Room Types" as the field title, you would end up with all of the following automatically created:
      a fully configured page field called: room_types MULTIPLE FIELDS OPTION - 3 additional fields - number_of_beds, number_of_people, kitchen a parent template called: room_types a child template called: room_types_items (with either just a title field, or with the 3 additional fields as well) a parent page called: Room Types a series of child pages named and titled based on the per line entries in the Select Options textarea The templates are configured such that the "room_types_items" child template can only have the main "room_types" template as a parent, and vice versa.

      Then all you have to do is add the newly created page field to any template you want and you're ready to go!
       
      You can grab it from:
       
      Modules directory: http://modules.processwire.com/modules/process-page-field-select-creator/
      Github: https://github.com/adrianbj/ProcessPageFieldSelectCreator
       

    • By bernhard
      WHY?
      This module was built to fill the gap between simple $pages->find() operations and complex SQL queries.
      The problem with $pages->find() is that it loads all pages into memory and that can be a problem when querying multiple thousands of pages. Even $pages->findMany() loads all pages into memory and therefore is a lot slower than regular SQL.
      The problem with SQL on the other hand is, that the queries are quite complex to build. All fields are separate tables, some repeatable fields use multiple rows for their content that belong to only one single page, you always need to check for the page status (which is not necessary on regular find() operations and therefore nobody is used to that).
      In short: It is far too much work to efficiently and easily get an array of data based on PW pages and fields and I need that a lot for my RockGrid module to build all kinds of tabular data.

      Basic Usage

       
      Docs & Download
      https://modules.processwire.com/modules/rock-finder/
      https://github.com/BernhardBaumrock/RockFinder
       
      Changelog
      180817, v1.0.6, support for joining multiple finders 180810, v1.0.5, basic support for options fields 180528, v1.0.4, add custom select statement option 180516, change sql query method, bump version to 1.0.0 180515, multilang bugfix 180513, beta release <180513, preview/discussion took place here: https://processwire.com/talk/topic/18983-rocksqlfinder-highly-efficient-and-flexible-sql-finder-module/