Export pages to csv

[joey102030]

Is it possible to export pages to a csv file, in the same way you can import with this module?

I couldn't find anything in the modules section

[kongondo]

Not that I know of, but you can export your database tables (e.g. using phpMyAdmin) as CSV....Not sure if this will help you since I don't know your ultimate aim with the export

Edited for clarity...

Edited January 16, 2014 by kongondo

[Soma]

Too simple to be a module, consider a script like this:

$array = $pages->find("template=basic-page")->explode(function($item){
    return array(
        'id'=> $item->id,
        'title' => $item->title
        );
});

$fp = fopen('file.csv', 'w');
foreach ($array as $fields) fputcsv($fp, $fields);
fclose($fp);
 

Note, $pagearray->explode() used here is only available in 2.4 (2.3 dev)

http://cheatsheet.processwire.com/pagearray-wirearray/getting-items/a-explode/

And the anonymous functions requires php >= 5.3 http://php.net/manual/de/functions.anonymous.php

[kongondo]

.....and then there was Soma....Better solution!

[Soma]

No just a example. Of course a module to create csv would be possible and cool, but some work required to make it flexible.

[joey102030]

Thanks for the replies

I think this could work as a module.

You would need to configure it at a template level with the fields you want to output, then in the list of page actions it could say something like 'export children as csv' (if there are any children)

[Macrura]

thanks Soma! i was working on an export function sort of like this yesterday - this will help a lot!

[joey102030]

OK I've had a play around with this.

I created a new module which extends ProcessPageSearch, but instead of outputting the results to screen they are outputted in .csv format to the browser as a downloadable file.

All parameters are accepted in the query string, so the download could be a full URL linked from anywhere in the admin area (ie a custom page action)

I'm new to module creation, so I'm not sure if extending a core module like ProcessPageSearch is a good idea as it could change in future releases.

Maybe someone more experience could advise?

[ryan]

It's true that ProcessPageSearch may change down the road, though no specific plans on that at present. But if you are worried about that, rather than extending it (in the PHP class sense) you could just copy ProcessPageSearch to another module, and modify it to make your own. The PW API behavior doesn't change, so when you make something where that is the dependency (as opposed to the implementation of class that's already an endpoint) then it's a safer bet. 

[joey102030]

Thanks Ryan

That sounds sensible.

I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

I will get the csv output module I've created into a usable state ASAP. 

[ryan]

I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

That was the hope with the JSON API we put into it. Two modules that use it are the InputfieldPageListAutocomplete and ServicePages modules. Though of those two, only InputfieldPageAutocomplete uses it in the way originally intended, as ServicePages essentially translates ProcessPageSearch for front-end use. 

[SwimToWin]

Soma has an excellent script above that uses the new WireArray::explode() function to bulk-export pages to CSV. But the script requires that you hard-code any custom fields for your pages. Is it possible to get the field names and field values dynamically (meaning, it is not necessary to hard-code each field key and value)?

Example: (to illustrate my point -- this code snippet does not work at present)

$array = $pages->find("template=foo")->explode(function($items){
    foreach ($items as $key => $item) {
        $arr[$key] = $item->$key;
    }
    return $arr;
    # Or, perhaps just cast using (array):
    # return (array) $items;
});

$fp = fopen('export.csv', 'w');
$i = 0;
foreach ($array as $fields) {
   // Add headings to the first line.
    if($i==0) fputcsv($fp, array_keys($fields), "\t");
    fputcsv($fp, $fields, "\t");
    $i++;
}
fclose($fp);

Note: Page fields are exported with their ID ("1001"), in a future version the script could look up these Page IDs.

See also:

• PHP.net: get_object_vars
• StackOverflow.com: How do I convert an object to an array?
• StackOverflow.com: Convert PHP object to associative array

[horst]

@SwimToWin: to get all fields from a page (template) you can use a snippet like this:

foreach ($page->template->fieldgroup as $field) {

But you have to check what types of field you can export to csv, I think. I'm not sure but I think so. Or maybe I'm wrong.

If you want to check for fieldtypes you do it like

if($field->type instanceof FieldtypeFile)  // or FieldtypeText, or ...

[Soma]

$page->fields is better. Does $page->template->fieldgroup work? Thought more like $page->template->fields or fieldgroup->fields.

[horst]

@soma: I have found this here in a script from Ryan and therefor thought it is save to use.

[horst]

Ok, I have read in the cheatSheet:

$template->fieldgroup = Get or set a template's Fieldgroup. Can also be used to iterate a template's fields.

$template->fields = Syntactical alias for $template->fieldgroup. Use whatever makes more sense for your code readability.

$page->fields = All the Fields assigned to this page (via it's template, same as $page->template->fields). Returns a FieldsArray.

So, this is all the same! And therefor I will use $page->fields in future. (It's more close for me. Ok, for Ryan it is more close to use $template->fieldgroup because in his thinking it is obvious that that is the part what gets invoked at the end)

Edited March 17, 2014 by horst

[adrian]

Some cross-linking and gratuitous self promotion

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

[elabx]

On 5/26/2015 at 3:55 PM, adrian said:

Some cross-linking and gratuitous self promotion

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

[szabesz]

@elabx Hi, FYI, Adrian is still away for more than a month, see: 

 

[elabx]

3 hours ago, szabesz said:

@elabx Hi, FYI, Adrian is still away for more than a month, see: 

 

Oh!!! Thanks a lot for the heads up @szabesz !!

[FrancisChung]

Does it have to be CSV? I've recently migrated to a XML based export functionality because of the limitations of the CSV format.

[adrian]

On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

[elabx]

8 hours ago, adrian said:

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

Thanks for answering! Please don't mind the delay!

I do have access to ListerPro but I was looking to export from within the Page edit screen, with an Export button I place on a PageTable Inputfield through a hook (to export the fields data in a CSV file). So I thought "I must use something already done just for the export part!". Actions on ListerPro never crossed my mind!! Ended up coding it, found great comments on other posts dealing the data-to-csv problem. So now I have an ExportPageTableToCSV!

 

[adrian]

On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

Just in case someone else arrives in this thread, this is now implemented in Batch Child Editor: https://processwire.com/talk/topic/6102-batch-child-editor/?do=findComment&comment=149975

 

[Fuzen]

On 1/16/2014 at 5:22 AM, Soma said:

Too simple to be a module, consider a script like this:

$array = $pages->find("template=basic-page")->explode(function($item){
    return array(
        'id'=> $item->id,
        'title' => $item->title
        );
});

$fp = fopen('file.csv', 'w');
foreach ($array as $fields) fputcsv($fp, $fields);
fclose($fp);
 

Note, $pagearray->explode() used here is only available in 2.4 (2.3 dev)

http://cheatsheet.processwire.com/pagearray-wirearray/getting-items/a-explode/

And the anonymous functions requires php >= 5.3 http://php.net/manual/de/functions.anonymous.php

@Soma This code works perfect. In my case the file is being saved to server, I want to initiate a download and not save it to the server. How could I do that? I know it has to do with headers, correct? Any help would be appreciated.