Jump to content
AndZyk

Export assets grouped by pages

Recommended Posts

Hello,

we have many projects as pages containing images inside regular image fields and repeater matrix elements. Now we would like to organize our assets.

ProcessWire saves assets with id as folder name and makes separate folders for repeater matrix elements.

Is there a way to collect all assets of one page and export them in a folder with the page name/title?

I think I could achieve this by using the command line wget for this website or maybe by a hook. But maybe someone has done this before. 😉

Regards, Andreas

Share this post


Link to post
Share on other sites

What do you want to achieve? Do you want to get all images for a project out of the system to use them externally? Or do you want to restructure how/where ProcessWire saves images in general?

For the first case, you can write a little script to iterate recursively through all fields, including any repeater / repeater matrix sub-fields, to get an array of images. Then you can use that to copy all the images to one folder (using $files->copy(), for example), get a link of filenames (see Pagefile::filename()) or do whatever you want with them, like export their meta data as JSON or anything else.

Super quick and dirty, will probably not work right away, but you get the idea:

function getImages(Page $page): array
{
	$images = new PageImages;
    $fields = array_map(function($field){ return $field->name; }, iterator_to_array($page->fields));
	foreach ($fields as $field) {
		$value = $page->get($field);
        $type = $page->fields->{$field}->getFieldType();
		if ($type instanceof FieldtypeImage) {
			$images->import($value);
		} elseif ($type instanceof FieldtypeRepeater) {
			$images->import(getImages($value));
		}
	}
	return $images;
}

$images = getImages($page);

 

  • Like 1

Share this post


Link to post
Share on other sites

Sorry, I should have explained myself better in my first post.

I would just like to export the assets and group them in folders with the name/title of the page they are used in. It is just for organizing the archiving the assets, because many are only available in the CMS right now.

So instead of folder „1234“, „2345“ etc. with the assets for page „Foo“, I want one folder „foo“ with all the assets of this page.

Thank you, your script seems to go in the right direction. 👍

Could you or somebody else please help me with this?

Regards, Andreas

Share this post


Link to post
Share on other sites

@AndZyk Alright, I had some fun with it. Here's an improved script for the asset export, which can handle nested repeater and matrix repeater fields:

/**
 * Get a flat array of all images on the given page, even in nested repeater / repeater matrix fields.
 *
 * @var Page $page		The page to get the images from.
 * @return Pageimage[]	Array of Pageimage objects.
 */
function getImages(Page $page): WireArray
{
	$images = new WireArray();
	$fields = $page->fields->each('name');
	foreach ($fields as $field) {
		$value = $page->get($field);
        $type = $page->fields->{$field}->getFieldType();
		if ($value instanceof Pageimage) {
			$images->add($value);
		} elseif ($value instanceof Pageimages) {
		    $images->import($value->getArray());
		} elseif ($value instanceof RepeaterMatrixPageArray || $value instanceof RepeaterPageArray) {
		    foreach ($value as $repeaterPage) {
		        $images->import(getImages($repeaterPage));
		    }
		} elseif ($value instanceof RepeaterMatrixPage || $value instanceof RepeaterPage) {
		    $images->import(getImages($value));
		}
	}
	return $images;
}

$images = getImages($page);

// create target folder for the page assets
$targetDir = $config->paths->assets . 'export/' . $page->name . '/';
$files->mkdir($targetDir, true);

// move all images to the target folder
foreach ($images as $image) {
    $name = $image->basename();
    $target = $targetDir . $name;
    $src = $image->filename();
    $files->copy($src, $target);
}

This could be extended in any number of ways:

  • Handle file fields as well as images.
  • Handle any other fields and dump them in the target folder as JSON.
  • Handle native Page properties.

At some point you got a complete page export module. Might want to look into the pages export module if you don't want to write all that stuff yourself 🙂

  • Like 5
  • Thanks 1

Share this post


Link to post
Share on other sites

I think you have a typo where you check for instances (all of them, here just one for brevity):

if ($value instanceof Pageimage) // Shouldn't be $type insted of $value?

and $type could be reached also like this? (as seen here)
 

$type = $value->type; // where $value is the variable you previously created

 

Share this post


Link to post
Share on other sites
15 hours ago, 3fingers said:

I think you have a typo where you check for instances (all of them, here just one for brevity):

@3fingers No that's actually correct, I'm checking the class of the formatted values there, not the class of the fieldtype. For example, for an image field, the $value will be an instance of Pageimage or Pageimages depending on the formatting settings, but the $type will be an instance of InputfieldImage. In fact, the $type variable is not needed at all. In the first version I started by checking the $type, but turned out checking the $value directly was more efficient. Though you could do it both ways.

15 hours ago, 3fingers said:

and $type could be reached also like this? (as seen here)

That would only work if you're working with field objects, not formatted values. The formatted value of a text field will be a string, so accessing $value->type will result in an error.

The code you linked is slightly different since it iterates Field objects, not field names:

// iterate over Field objects
// @see https://processwire.com/api/ref/field/
foreach($this->wire('fields') as $field) {}

// iterate over field names (strings) of fields on a page
// @see https://processwire.com/api/ref/wire-array/each/
$fields = $page->fields->each('name');
foreach ($fields as $field) {}

 

  • Thanks 1

Share this post


Link to post
Share on other sites

Thanks @MoritzLost now I got it, you've clarified my doubts. So many things to learn 🙂
I'll bookmark this thread for reference in the future 🙂

  • Like 1

Share this post


Link to post
Share on other sites
20 hours ago, MoritzLost said:

Alright, I had some fun with it. Here's an improved script for the asset export, which can handle nested repeater and matrix repeater fields:

Thank you very much @MoritzLost. This scripts works perfect. 😀

I have put it in a shell script to run it for all pages and modified it a little bit:

  • Added a foreach-loop for all pages
  • Only create folder if there are images
  • Added timestamps to see what the shell script is doing
Spoiler

#!/usr/bin/php -d display_errors
<?php
namespace {
	// Bootstrap ProcessWire
	include("./../../index.php");
}

namespace ProcessWire {

	echo date("H:i:s"). ": Start export of images.\n";

    /**
     * Get a flat array of all images on the given page, even in nested repeater / repeater matrix fields.
     *
     * @var Page $page		The page to get the images from.
     * @return Pageimage[]	Array of Pageimage objects.
     */
    function getImages(Page $page): WireArray {
        $images = new WireArray();
        $fields = $page->fields->each('name');
        foreach ($fields as $field) {
            $value = $page->get($field);
            $type = $page->fields->{$field}->getFieldType();
            if ($value instanceof Pageimage) {
                $images->add($value);
            } elseif ($value instanceof Pageimages) {
                $images->import($value->getArray());
            } elseif ($value instanceof RepeaterMatrixPageArray || $value instanceof RepeaterPageArray) {
                foreach ($value as $repeaterPage) {
                    $images->import(getImages($repeaterPage));
                }
            } elseif ($value instanceof RepeaterMatrixPage || $value instanceof RepeaterPage) {
                $images->import(getImages($value));
            }
        }
        return $images;
    }

    $allPages = pages()->find("template!=admin");
    echo date("H:i:s"). ": Pages count: " . count($allPages) . ".\n";

    foreach ($allPages as $page) {
        $images = getImages($page);

        // create target folder for the page assets
        $targetDir = $config->paths->assets . "export/{$page->name}/";

        if (count($images)) {
            $files->mkdir($targetDir, true);

            // move all images to the target folder
            foreach ($images as $image) {
                $name = $image->basename();
                $target = $targetDir . $name;
                $src = $image->filename();
                $files->copy($src, $target);
            }
        }

	    echo date("H:i:s"). ": Images of page '{$page->title}' exported.\n";
    }

	echo date("H:i:s"). ": Stop export of images.\n";

}

 

 

20 hours ago, MoritzLost said:

At some point you got a complete page export module. Might want to look into the pages export module if you don't want to write all that stuff yourself 🙂

I looked first in this module, but it only exports the pages as JSON without assets. The assets only get downloaded when importing. So this use case wasn't covered yet, as far as I know. 😉

 

I love this community. 😍

Regards, Andreas

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By hellerdruck
      Hi all
      I need to export all the texts from a website to a translation company (as json or csv or txt...). How can this be done? Of course manually, but this website is huge and it would take me years...
      Also, as a second step, importing the translation ...
      Any ideas anyone? Tutorials? Plugins?
      Thanks for your help.
    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
       
    • By horst
      Hi, on a site I want to disable access to original images and only allow to access thumbnails and watermarked image variations.
      EDIT:
      A good solution for protecting original images can be found a bit down in this thread:
       
      Old content of this initial post:
       
    • By dragan
      Is it by design that a site/ready.php is not included when creating a new site profile? Is it possible to include it with a hook? Or are there any security thoughts? (I don't want to redistribute it in public, it's just so I have my own boilerplate)
    • By NorbertH
      I have trouble exporting fields via the buildin export. 
      For example when i export a single field  i get:
      { "bestellung_status_name": { "id": 194, "type": "FieldtypeText", "flags": 0, "name": "bestellung_status_name", "label": "Status Intern", "textformatters": [ "TextformatterEntities" ], "collapsed": 0, "minlength": 0, "maxlength": 100, "showCount": 0, "size": 0, "pattern": "[a-z\\A-Z\\(\\)]+", "showIf": "", "themeInputSize": "", "themeInputWidth": "", "themeOffset": "", "themeBorder": "", "themeColor": "", "themeBlank": "", "columnWidth": 100, "required": "", "requiredAttr": "", "requiredIf": "", "stripTags": "", "placeholder": "" } } exporting a secon single field i get :
      { "bestellung_status_name_ext": { "id": 195, "type": "FieldtypeText", "flags": 0, "name": "bestellung_status_name_ext", "label": "Status Extern", "textformatters": [ "TextformatterEntities" ], "collapsed": 0, "minlength": 0, "maxlength": 100, "showCount": 0, "size": 0, "pattern": "[a-z\\A-Z\\(\\)]+", "showIf": "", "themeInputSize": "", "themeInputWidth": "", "themeOffset": "", "themeBorder": "", "themeColor": "", "themeBlank": "", "columnWidth": 100, "required": "", "requiredAttr": "", "requiredIf": "", "stripTags": "", "placeholder": "" } } So far everything works fine .
      When i try to export both fields together  i get only an error message :
      Call to a member function getModuleInfo() on null File: .../wire/modules/Fieldtype/FieldtypeText.module:171 I added " bd($textformatter);" on line 170 to see whats wrong. so have a look at the screenshot i apended to this post.
       
      Its perfectly possible that one textformater module got removed by accident while experimenting whith some textformaters but my question is how to fix this maybe somewhere in the DB and possibly what went wrong?
      ProcessWire 3.0.120 © 2019
      Apache/2.4.25 (FreeBSD) OpenSSL/1.0.2k mod_fcgid/2.3.9
      PHP 7.1.2
           

       
      Edit: After adding
      if ($textformatter ===NULL) continue; I can export my fields , as there arent any Textformater missing in the fields , i get a perfect result. but still there is one textformater whith a NULL value.  
       
       
×
×
  • Create New...