joey102030

Export pages to csv

Recommended Posts

kongondo    5,016

Not that I know of, but you can export your database tables (e.g. using phpMyAdmin) as CSV....Not sure if this will help you since I don't know your ultimate aim with the export :-)

Edited for clarity...

Edited by kongondo

Share this post


Link to post
Share on other sites
Soma    6,128

Too simple to be a module, consider a script like this:

$array = $pages->find("template=basic-page")->explode(function($item){
    return array(
        'id'=> $item->id,
        'title' => $item->title
        );
});

$fp = fopen('file.csv', 'w');
foreach ($array as $fields) fputcsv($fp, $fields);
fclose($fp);
 

Note, $pagearray->explode() used here is only available in 2.4 (2.3 dev)

http://cheatsheet.processwire.com/pagearray-wirearray/getting-items/a-explode/

And the anonymous functions requires php >= 5.3 http://php.net/manual/de/functions.anonymous.php

  • Like 13

Share this post


Link to post
Share on other sites
Soma    6,128

No just a example. Of course a module to create csv would be possible and cool, but some work required to make it flexible. ;)

Share this post


Link to post
Share on other sites
joey102030    18

Thanks for the replies

I think this could work as a module.

You would need to configure it at a template level with the fields you want to output, then in the list of page actions it could say something like 'export children as csv' (if there are any children)

Share this post


Link to post
Share on other sites
Macrura    2,142

thanks Soma! i was working on an export function sort of like this yesterday - this will help a lot!

Share this post


Link to post
Share on other sites
joey102030    18

OK I've had a play around with this.

I created a new module which extends ProcessPageSearch, but instead of outputting the results to screen they are outputted in .csv format to the browser as a downloadable file.

All parameters are accepted in the query string, so the download could be a full URL linked from anywhere in the admin area (ie a custom page action)

I'm new to module creation, so I'm not sure if extending a core module like ProcessPageSearch is a good idea as it could change in future releases.

Maybe someone more experience could advise?

Share this post


Link to post
Share on other sites
ryan    13,403

It's true that ProcessPageSearch may change down the road, though no specific plans on that at present. But if you are worried about that, rather than extending it (in the PHP class sense) you could just copy ProcessPageSearch to another module, and modify it to make your own. The PW API behavior doesn't change, so when you make something where that is the dependency (as opposed to the implementation of class that's already an endpoint) then it's a safer bet. 

  • Like 1

Share this post


Link to post
Share on other sites
joey102030    18

Thanks Ryan

That sounds sensible.

I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

I will get the csv output module I've created into a usable state ASAP. 

  • Like 1

Share this post


Link to post
Share on other sites
ryan    13,403
I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

That was the hope with the JSON API we put into it. Two modules that use it are the InputfieldPageListAutocomplete and ServicePages modules. Though of those two, only InputfieldPageAutocomplete uses it in the way originally intended, as ServicePages essentially translates ProcessPageSearch for front-end use. 

Share this post


Link to post
Share on other sites
SwimToWin    5

Soma has an excellent script above that uses the new WireArray::explode() function to bulk-export pages to CSV. But the script requires that you hard-code any custom fields for your pages. Is it possible to get the field names and field values dynamically (meaning, it is not necessary to hard-code each field key and value)?

Example: (to illustrate my point -- this code snippet does not work at present)

$array = $pages->find("template=foo")->explode(function($items){
    foreach ($items as $key => $item) {
        $arr[$key] = $item->$key;
    }
    return $arr;
    # Or, perhaps just cast using (array):
    # return (array) $items;
});

$fp = fopen('export.csv', 'w');
$i = 0;
foreach ($array as $fields) {
   // Add headings to the first line.
    if($i==0) fputcsv($fp, array_keys($fields), "\t");
    fputcsv($fp, $fields, "\t");
    $i++;
}
fclose($fp);

Note: Page fields are exported with their ID ("1001"), in a future version the script could look up these Page IDs.

See also:

  • Like 1

Share this post


Link to post
Share on other sites
horst    3,961

@SwimToWin: to get all fields from a page (template) you can use a snippet like this:

foreach ($page->template->fieldgroup as $field) {

But you have to check what types of field you can export to csv, I think. I'm not sure but I think so. Or maybe I'm wrong.

If you want to check for fieldtypes you do it like

if($field->type instanceof FieldtypeFile)  // or FieldtypeText, or ...
  • Like 1

Share this post


Link to post
Share on other sites
Soma    6,128

$page->fields is better. Does $page->template->fieldgroup work? Thought more like $page->template->fields or fieldgroup->fields.

Share this post


Link to post
Share on other sites
horst    3,961

Ok, I have read in the cheatSheet:

$template->fieldgroup = Get or set a template's Fieldgroup. Can also be used to iterate a template's fields.

$template->fields = Syntactical alias for $template->fieldgroup. Use whatever makes more sense for your code readability.

$page->fields = All the Fields assigned to this page (via it's template, same as $page->template->fields). Returns a FieldsArray.

So, this is all the same! And therefor I will use $page->fields in future. (It's more close for me. Ok, for Ryan it is more close to use $template->fieldgroup because in his thinking it is obvious that that is the part what gets invoked at the end) :)

Edited by horst
  • Like 1

Share this post


Link to post
Share on other sites
adrian    7,667

Some cross-linking and gratuitous self promotion :)

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

  • Like 2

Share this post


Link to post
Share on other sites
elabx    284
On 5/26/2015 at 3:55 PM, adrian said:

Some cross-linking and gratuitous self promotion :)

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

Share this post


Link to post
Share on other sites
elabx    284
3 hours ago, szabesz said:

@elabx Hi, FYI, Adrian is still away for more than a month, see: 

 

Oh!!! Thanks a lot for the heads up @szabesz !!

  • Like 1

Share this post


Link to post
Share on other sites
FrancisChung    129

Does it have to be CSV? I've recently migrated to a XML based export functionality because of the limitations of the CSV format.

Share this post


Link to post
Share on other sites
adrian    7,667
On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

Share this post


Link to post
Share on other sites
elabx    284
8 hours ago, adrian said:

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

Thanks for answering! Please don't mind the delay!

I do have access to ListerPro but I was looking to export from within the Page edit screen, with an Export button I place on a PageTable Inputfield through a hook (to export the fields data in a CSV file). So I thought "I must use something already done just for the export part!". Actions on ListerPro never crossed my mind!! Ended up coding it, found great comments on other posts dealing the data-to-csv problem. So now I have an ExportPageTableToCSV! :D

 

  • Like 2

Share this post


Link to post
Share on other sites
adrian    7,667
On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

Just in case someone else arrives in this thread, this is now implemented in Batch Child Editor: https://processwire.com/talk/topic/6102-batch-child-editor/?do=findComment&comment=149975

 

  • Like 2

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By hellomoto
      Pressing problem at initial posting:  I am working on a module that runs/previews (for pub/unpub) imports on importer page view. The template has to be in site/templates, so far as I know, so one is created there on install which includes a file to run from within the module, but then when it's run and tries to read the importer csv file it cannot, since it's in the admin, I guess... Is it possible to set the template page to a file in the module directory? Will that work?
      https://bitbucket.org/chimmel/pw-import-everything
    • By holmescreek
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
    • By modifiedcontent
      I am trying to install Processwire + an exported custom profile and keep getting this error:
      Does anyone recognize this? What could cause this?
      I have installed Processwire + exported profiles many times before, never had problems. I have reuploaded fresh downloads for this, both regular master and dev, but keep getting the same thing.
      I am probably doing something dumb. Probably not a structural PW issue. I am out of ideas, so any feedback appreciated.
      Edit:
      Finally got something to install using the standard blank profile, instead of an exported profile. I am now manually reconstructing my custom site, using the new import/export functions - hit or miss so far. Did those break the ProcessProfileExport module?
    • By hellomoto
      Would anyone here be able and inspired to develop a ProcessWire equivalent to the WordPress plugin WP All Import Pro? or to help me do so?
      This is what I'm envisioning...
      Upon installation the module creates an admin page titled "Import & Update". On the module config page you can specify allowed templates to run this on, otherwise allowing any. Include the following PHP libraries: hQuery for web scraping, Csv for CSV handling, and Parser for XML. Create template "import-update". On the "Import & Update" page, a list of current import/updaters will be displayed (0 initially), each with corresponding links to "edit" or "run". When you "Add New", this be the "import-update" template (with all module-specific fields tagged "impupd"):
      title destination (req.): parent, template source (req.): type (web, csv, xml) location (url, file, text) if web: opt. index URL & link selector, + paginator selector if csv: opt. ignore 1st row if xml: req. individual item node xpath actions (check): import (if none matching UID) update (if matching UID & field values differ) save() [req. here in flow] map (repeater): input (select fields from specified template to affect) intake (corresponding DOM selectors / CSV col. letters/headers / xpath per field) (req.) UID (unique ID; field reference to compare against, from selected input fields) (req.) Lazy Cron interval Scripts can be run via the import-update template; keep logs; show preview (iframe/ajax) for manual runs. ...
    • By benbyf
      Hi!
      I'm trying to export a large site but the export profile  module keeps timing out due to the size of the installation its trying to export (around 1500 pages - half with body content). Is there any work around or is it worth simply duplicating the site and files without using teh export profile module?
      Thanks