Jump to content
joey102030

Export pages to csv

Recommended Posts

Is it possible to export pages to a csv file, in the same way you can import with this module?

I couldn't find anything in the modules section

Share this post


Link to post
Share on other sites

Not that I know of, but you can export your database tables (e.g. using phpMyAdmin) as CSV....Not sure if this will help you since I don't know your ultimate aim with the export :-)

Edited for clarity...

Edited by kongondo

Share this post


Link to post
Share on other sites

Too simple to be a module, consider a script like this:

$array = $pages->find("template=basic-page")->explode(function($item){
    return array(
        'id'=> $item->id,
        'title' => $item->title
        );
});

$fp = fopen('file.csv', 'w');
foreach ($array as $fields) fputcsv($fp, $fields);
fclose($fp);
 

Note, $pagearray->explode() used here is only available in 2.4 (2.3 dev)

http://cheatsheet.processwire.com/pagearray-wirearray/getting-items/a-explode/

And the anonymous functions requires php >= 5.3 http://php.net/manual/de/functions.anonymous.php

  • Like 14

Share this post


Link to post
Share on other sites

No just a example. Of course a module to create csv would be possible and cool, but some work required to make it flexible. ;)

Share this post


Link to post
Share on other sites

Thanks for the replies

I think this could work as a module.

You would need to configure it at a template level with the fields you want to output, then in the list of page actions it could say something like 'export children as csv' (if there are any children)

Share this post


Link to post
Share on other sites

thanks Soma! i was working on an export function sort of like this yesterday - this will help a lot!

Share this post


Link to post
Share on other sites

OK I've had a play around with this.

I created a new module which extends ProcessPageSearch, but instead of outputting the results to screen they are outputted in .csv format to the browser as a downloadable file.

All parameters are accepted in the query string, so the download could be a full URL linked from anywhere in the admin area (ie a custom page action)

I'm new to module creation, so I'm not sure if extending a core module like ProcessPageSearch is a good idea as it could change in future releases.

Maybe someone more experience could advise?

Share this post


Link to post
Share on other sites

It's true that ProcessPageSearch may change down the road, though no specific plans on that at present. But if you are worried about that, rather than extending it (in the PHP class sense) you could just copy ProcessPageSearch to another module, and modify it to make your own. The PW API behavior doesn't change, so when you make something where that is the dependency (as opposed to the implementation of class that's already an endpoint) then it's a safer bet. 

  • Like 1

Share this post


Link to post
Share on other sites

Thanks Ryan

That sounds sensible.

I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

I will get the csv output module I've created into a usable state ASAP. 

  • Like 1

Share this post


Link to post
Share on other sites
I've expressed my love for ProcessPageSearch in these forums before... but seriously there is tonnes of potential to utilise the output in various ways, including datatables, charts, reports etc.

That was the hope with the JSON API we put into it. Two modules that use it are the InputfieldPageListAutocomplete and ServicePages modules. Though of those two, only InputfieldPageAutocomplete uses it in the way originally intended, as ServicePages essentially translates ProcessPageSearch for front-end use. 

Share this post


Link to post
Share on other sites

Soma has an excellent script above that uses the new WireArray::explode() function to bulk-export pages to CSV. But the script requires that you hard-code any custom fields for your pages. Is it possible to get the field names and field values dynamically (meaning, it is not necessary to hard-code each field key and value)?

Example: (to illustrate my point -- this code snippet does not work at present)

$array = $pages->find("template=foo")->explode(function($items){
    foreach ($items as $key => $item) {
        $arr[$key] = $item->$key;
    }
    return $arr;
    # Or, perhaps just cast using (array):
    # return (array) $items;
});

$fp = fopen('export.csv', 'w');
$i = 0;
foreach ($array as $fields) {
   // Add headings to the first line.
    if($i==0) fputcsv($fp, array_keys($fields), "\t");
    fputcsv($fp, $fields, "\t");
    $i++;
}
fclose($fp);

Note: Page fields are exported with their ID ("1001"), in a future version the script could look up these Page IDs.

See also:

  • Like 1

Share this post


Link to post
Share on other sites

@SwimToWin: to get all fields from a page (template) you can use a snippet like this:

foreach ($page->template->fieldgroup as $field) {

But you have to check what types of field you can export to csv, I think. I'm not sure but I think so. Or maybe I'm wrong.

If you want to check for fieldtypes you do it like

if($field->type instanceof FieldtypeFile)  // or FieldtypeText, or ...
  • Like 1

Share this post


Link to post
Share on other sites

$page->fields is better. Does $page->template->fieldgroup work? Thought more like $page->template->fields or fieldgroup->fields.

Share this post


Link to post
Share on other sites

@soma: I have found this here in a script from Ryan and therefor thought it is save to use. :)

Share this post


Link to post
Share on other sites

Ok, I have read in the cheatSheet:

$template->fieldgroup = Get or set a template's Fieldgroup. Can also be used to iterate a template's fields.

$template->fields = Syntactical alias for $template->fieldgroup. Use whatever makes more sense for your code readability.

$page->fields = All the Fields assigned to this page (via it's template, same as $page->template->fields). Returns a FieldsArray.

So, this is all the same! And therefor I will use $page->fields in future. (It's more close for me. Ok, for Ryan it is more close to use $template->fieldgroup because in his thinking it is obvious that that is the part what gets invoked at the end) :)

Edited by horst
  • Like 1

Share this post


Link to post
Share on other sites

Some cross-linking and gratuitous self promotion :)

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

  • Like 2

Share this post


Link to post
Share on other sites
On 5/26/2015 at 3:55 PM, adrian said:

Some cross-linking and gratuitous self promotion :)

I have just implemented highly configurable CSV exporting (admin and API) into Batch Child Editor:

https://processwire.com/talk/topic/6102-batch-child-editor/page-2#entry95855

It does lots of work for you, including supporting ProFields Textareas and Multiplier, along with outputting Page field title/name, rather than ID.

Export settings can be configured by the developer and/or the site editors. 

It might be useful for others searching for CSV export and finding this thread.

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

Share this post


Link to post
Share on other sites

@elabx Hi, FYI, Adrian is still away for more than a month, see: 

 

  • Like 2

Share this post


Link to post
Share on other sites
3 hours ago, szabesz said:

@elabx Hi, FYI, Adrian is still away for more than a month, see: 

 

Oh!!! Thanks a lot for the heads up @szabesz !!

  • Like 1

Share this post


Link to post
Share on other sites

Does it have to be CSV? I've recently migrated to a XML based export functionality because of the limitations of the CSV format.

Share this post


Link to post
Share on other sites
On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

I was looking to use the exportCsv method to export pages stored under a PageTable.

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

Share this post


Link to post
Share on other sites
8 hours ago, adrian said:

Hi @elabx - sorry for the long delay.

Currently there isn't an option to provide a selector to filter the child pages being exported. It wouldn't be a terribly complex addition, but remember that if you have ListerPro, you can do this using the CSV export action. Do you have access to this?

 

Thanks for answering! Please don't mind the delay!

I do have access to ListerPro but I was looking to export from within the Page edit screen, with an Export button I place on a PageTable Inputfield through a hook (to export the fields data in a CSV file). So I thought "I must use something already done just for the export part!". Actions on ListerPro never crossed my mind!! Ended up coding it, found great comments on other posts dealing the data-to-csv problem. So now I have an ExportPageTableToCSV! :D

 

  • Like 2

Share this post


Link to post
Share on other sites
On 5/17/2017 at 6:45 PM, elabx said:

Something I couldn't find or notice in your code Adrian, is there a way to filter the child pages you want as CSV by a selector?

Just in case someone else arrives in this thread, this is now implemented in Batch Child Editor: https://processwire.com/talk/topic/6102-batch-child-editor/?do=findComment&comment=149975

 

  • Like 3

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
       
    • By dragan
      Is it by design that a site/ready.php is not included when creating a new site profile? Is it possible to include it with a hook? Or are there any security thoughts? (I don't want to redistribute it in public, it's just so I have my own boilerplate)
    • By NorbertH
      I have trouble exporting fields via the buildin export. 
      For example when i export a single field  i get:
      { "bestellung_status_name": { "id": 194, "type": "FieldtypeText", "flags": 0, "name": "bestellung_status_name", "label": "Status Intern", "textformatters": [ "TextformatterEntities" ], "collapsed": 0, "minlength": 0, "maxlength": 100, "showCount": 0, "size": 0, "pattern": "[a-z\\A-Z\\(\\)]+", "showIf": "", "themeInputSize": "", "themeInputWidth": "", "themeOffset": "", "themeBorder": "", "themeColor": "", "themeBlank": "", "columnWidth": 100, "required": "", "requiredAttr": "", "requiredIf": "", "stripTags": "", "placeholder": "" } } exporting a secon single field i get :
      { "bestellung_status_name_ext": { "id": 195, "type": "FieldtypeText", "flags": 0, "name": "bestellung_status_name_ext", "label": "Status Extern", "textformatters": [ "TextformatterEntities" ], "collapsed": 0, "minlength": 0, "maxlength": 100, "showCount": 0, "size": 0, "pattern": "[a-z\\A-Z\\(\\)]+", "showIf": "", "themeInputSize": "", "themeInputWidth": "", "themeOffset": "", "themeBorder": "", "themeColor": "", "themeBlank": "", "columnWidth": 100, "required": "", "requiredAttr": "", "requiredIf": "", "stripTags": "", "placeholder": "" } } So far everything works fine .
      When i try to export both fields together  i get only an error message :
      Call to a member function getModuleInfo() on null File: .../wire/modules/Fieldtype/FieldtypeText.module:171 I added " bd($textformatter);" on line 170 to see whats wrong. so have a look at the screenshot i apended to this post.
       
      Its perfectly possible that one textformater module got removed by accident while experimenting whith some textformaters but my question is how to fix this maybe somewhere in the DB and possibly what went wrong?
      ProcessWire 3.0.120 © 2019
      Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k mod_fcgid/2.3.9
      PHP 7.1.2
           

       
      Edit: After adding
      if ($textformatter ===NULL) continue; I can export my fields , as there arent any Textformater missing in the fields , i get a perfect result. but still there is one textformater whith a NULL value.  
       
       
    • By psy
      I'm combining two PW sites into one, Site A into Site B.
      At each step, I did it bit by bit as the 'all at once' approach failed.
       
      First, I exported all the fields from Site A and imported into Site B. Any field types not supported by import/export, eg FieldtypeOptions I manually recreated. All good.
      Next I exported all the templates from Site A and imported them into Site B and copied across their associated template files. All good.
      Finally I exported the pages I needed from Site A into Site B - again, bit by bit to ensure it all went smoothly.
      From the admin side, it all looked and worked perfectly.
      Front end was a totally different story. All existing pages in Site B worked as expected. NONE of the pages imported from Site A displayed. They all ended in a redirect loop with no errors in the PW logs or Tracy Debugger.
      After some trial-and-error, I finally got it working with:
      - create a new template in Site B admin with no associated template file and just a title field
      - import the fields from the imported Site A template into the newly created template (both on Site B)
      - copy the Site A php template file into a new file that matched the new PW Site B template name and save in Site B site/templates
      I can deal with the above workaround. Just curious to know if I did something wrong or if the template import/export feature is problematic?
       
      ### Solution:
      While the export/import was a slow process, turned out the front end redirecting issue was unrelated. For reasons unknown, all templates marked as HTTPS only were the ones redirecting, ie all templates from Site A. Finally solved it by changing the $config->https to true in site/config.php
      Now the pages display correctly as https whether the template forces the issue or not.
       
    • By louisstephens
      I was attempting to export some fields from my dev branch to move them over to a live site when I got the following error:

      Has anyone experienced this before? I was thinking I could just write a script using the api to create the fields, but there are about 44 fields (2 are repeater matrix) that are all slightly unique. If anyone has experienced this, what was your work around?
×
×
  • Create New...