Export assets grouped by pages

[AndZyk]

Hello,

we have many projects as pages containing images inside regular image fields and repeater matrix elements. Now we would like to organize our assets.

ProcessWire saves assets with id as folder name and makes separate folders for repeater matrix elements.

Is there a way to collect all assets of one page and export them in a folder with the page name/title?

I think I could achieve this by using the command line wget for this website or maybe by a hook. But maybe someone has done this before. ?

Regards, Andreas

[MoritzLost]

What do you want to achieve? Do you want to get all images for a project out of the system to use them externally? Or do you want to restructure how/where ProcessWire saves images in general?

For the first case, you can write a little script to iterate recursively through all fields, including any repeater / repeater matrix sub-fields, to get an array of images. Then you can use that to copy all the images to one folder (using $files->copy(), for example), get a link of filenames (see Pagefile::filename()) or do whatever you want with them, like export their meta data as JSON or anything else.

Super quick and dirty, will probably not work right away, but you get the idea:

function getImages(Page $page): array
{
	$images = new PageImages;
    $fields = array_map(function($field){ return $field->name; }, iterator_to_array($page->fields));
	foreach ($fields as $field) {
		$value = $page->get($field);
        $type = $page->fields->{$field}->getFieldType();
		if ($type instanceof FieldtypeImage) {
			$images->import($value);
		} elseif ($type instanceof FieldtypeRepeater) {
			$images->import(getImages($value));
		}
	}
	return $images;
}

$images = getImages($page);

 

[AndZyk]

Sorry, I should have explained myself better in my first post.

I would just like to export the assets and group them in folders with the name/title of the page they are used in. It is just for organizing the archiving the assets, because many are only available in the CMS right now.

So instead of folder „1234“, „2345“ etc. with the assets for page „Foo“, I want one folder „foo“ with all the assets of this page.

Thank you, your script seems to go in the right direction. ?

Could you or somebody else please help me with this?

Regards, Andreas

[MoritzLost]

@AndZyk Alright, I had some fun with it. Here's an improved script for the asset export, which can handle nested repeater and matrix repeater fields:

/**
 * Get a flat array of all images on the given page, even in nested repeater / repeater matrix fields.
 *
 * @var Page $page		The page to get the images from.
 * @return Pageimage[]	Array of Pageimage objects.
 */
function getImages(Page $page): WireArray
{
	$images = new WireArray();
	$fields = $page->fields->each('name');
	foreach ($fields as $field) {
		$value = $page->get($field);
        $type = $page->fields->{$field}->getFieldType();
		if ($value instanceof Pageimage) {
			$images->add($value);
		} elseif ($value instanceof Pageimages) {
		    $images->import($value->getArray());
		} elseif ($value instanceof RepeaterMatrixPageArray || $value instanceof RepeaterPageArray) {
		    foreach ($value as $repeaterPage) {
		        $images->import(getImages($repeaterPage));
		    }
		} elseif ($value instanceof RepeaterMatrixPage || $value instanceof RepeaterPage) {
		    $images->import(getImages($value));
		}
	}
	return $images;
}

$images = getImages($page);

// create target folder for the page assets
$targetDir = $config->paths->assets . 'export/' . $page->name . '/';
$files->mkdir($targetDir, true);

// move all images to the target folder
foreach ($images as $image) {
    $name = $image->basename();
    $target = $targetDir . $name;
    $src = $image->filename();
    $files->copy($src, $target);
}

This could be extended in any number of ways:

• Handle file fields as well as images.
• Handle any other fields and dump them in the target folder as JSON.
• Handle native Page properties.

At some point you got a complete page export module. Might want to look into the pages export module if you don't want to write all that stuff yourself ?

[3fingers]

I think you have a typo where you check for instances (all of them, here just one for brevity):

if ($value instanceof Pageimage) // Shouldn't be $type insted of $value?

and $type could be reached also like this? (as seen here)
 

$type = $value->type; // where $value is the variable you previously created

 

[MoritzLost]

15 hours ago, 3fingers said:

I think you have a typo where you check for instances (all of them, here just one for brevity):

@3fingers No that's actually correct, I'm checking the class of the formatted values there, not the class of the fieldtype. For example, for an image field, the $value will be an instance of Pageimage or Pageimages depending on the formatting settings, but the $type will be an instance of InputfieldImage. In fact, the $type variable is not needed at all. In the first version I started by checking the $type, but turned out checking the $value directly was more efficient. Though you could do it both ways.

15 hours ago, 3fingers said:

and $type could be reached also like this? (as seen here)

That would only work if you're working with field objects, not formatted values. The formatted value of a text field will be a string, so accessing $value->type will result in an error.

The code you linked is slightly different since it iterates Field objects, not field names:

// iterate over Field objects
// @see https://processwire.com/api/ref/field/
foreach($this->wire('fields') as $field) {}

// iterate over field names (strings) of fields on a page
// @see https://processwire.com/api/ref/wire-array/each/
$fields = $page->fields->each('name');
foreach ($fields as $field) {}

 

[3fingers]

Thanks @MoritzLost now I got it, you've clarified my doubts. So many things to learn ?
I'll bookmark this thread for reference in the future ?

[AndZyk]

20 hours ago, MoritzLost said:

Alright, I had some fun with it. Here's an improved script for the asset export, which can handle nested repeater and matrix repeater fields:

Thank you very much @MoritzLost. This scripts works perfect. ?

I have put it in a shell script to run it for all pages and modified it a little bit:

• Added a foreach-loop for all pages
• Only create folder if there are images
• Added timestamps to see what the shell script is doing

Spoiler

#!/usr/bin/php -d display_errors
<?php
namespace {
	// Bootstrap ProcessWire
	include("./../../index.php");
}

namespace ProcessWire {

	echo date("H:i:s"). ": Start export of images.\n";

    /**
     * Get a flat array of all images on the given page, even in nested repeater / repeater matrix fields.
     *
     * @var Page $page		The page to get the images from.
     * @return Pageimage[]	Array of Pageimage objects.
     */
    function getImages(Page $page): WireArray {
        $images = new WireArray();
        $fields = $page->fields->each('name');
        foreach ($fields as $field) {
            $value = $page->get($field);
            $type = $page->fields->{$field}->getFieldType();
            if ($value instanceof Pageimage) {
                $images->add($value);
            } elseif ($value instanceof Pageimages) {
                $images->import($value->getArray());
            } elseif ($value instanceof RepeaterMatrixPageArray || $value instanceof RepeaterPageArray) {
                foreach ($value as $repeaterPage) {
                    $images->import(getImages($repeaterPage));
                }
            } elseif ($value instanceof RepeaterMatrixPage || $value instanceof RepeaterPage) {
                $images->import(getImages($value));
            }
        }
        return $images;
    }

    $allPages = pages()->find("template!=admin");
    echo date("H:i:s"). ": Pages count: " . count($allPages) . ".\n";

    foreach ($allPages as $page) {
        $images = getImages($page);

        // create target folder for the page assets
        $targetDir = $config->paths->assets . "export/{$page->name}/";

        if (count($images)) {
            $files->mkdir($targetDir, true);

            // move all images to the target folder
            foreach ($images as $image) {
                $name = $image->basename();
                $target = $targetDir . $name;
                $src = $image->filename();
                $files->copy($src, $target);
            }
        }

	    echo date("H:i:s"). ": Images of page '{$page->title}' exported.\n";
    }

	echo date("H:i:s"). ": Stop export of images.\n";

}

 

 

20 hours ago, MoritzLost said:

At some point you got a complete page export module. Might want to look into the pages export module if you don't want to write all that stuff yourself ?

I looked first in this module, but it only exports the pages as JSON without assets. The assets only get downloaded when importing. So this use case wasn't covered yet, as far as I know. ?

 

I love this community. ?

Regards, Andreas