Upgrading from Thumbnails module to Croppable Image 3

[jacmaes]

I have a few sites on PW 2.7 using the legacy Thumbnail module. I'd like to upgrade these sites to PW 3x (and the just-released PW3-compatible Croppable Image 3 module) to take advantage of the new goodness, but I'm not sure how best to proceed. I have easily more than a thousand pages that use the Thumbnail module. 

I think the safest way would be to create a new image field using Croppable as a fieldtype, and leave the current Thumbnails image field untouched as I can't possibly go and recrop thousands of images, especially on live sites. Anyone has tried upgrading? Any advice would be greatly appreciated. 

[horst]

It all depends on name formatting of the old crop variations!

Please, to be sure, can you give one or two examples of the filenames (original image + varition name of the crop)?

Also, how do you call them in templates. (with the older CroppableImage and the CroppableImage3, we use $image->getCrop("suffix")).

I'm sure, we will find a practicable solution, but want to be thouroughly and careful.

 

EDIT:

How many cropsettings / thumbsettings are defined with a typical field in your sites? Can you post an example of the textarea content?

[jacmaes]

@horst thanks a lot for your kind help. Being able to upgrade directly would be fantastic. 

 

My field is named simply "image" and its formatted value is set to "Automatic (single item or null when...)". Maximum files allowed is set to 0 (no limit). This is the only field in my site that's using the Thumbnails module. Here's a screenshot of the field settings that show that I have set three crops named "pequeno", "grande" and "destacado":

 

And here's how I'm calling the "grande" crop variation in one of my templates:

<img alt="<?= $page->title; ?>" src="<?= $page->image->first->getThumb('grande'); ?>">

 

An example of the filenames, original and output:

• Original: "spanish_fever.jpg"
• Output: "grande_spanish_fever.jpg"

I hope this is all you need to give me some pointers. Thanks again.

 

[horst]

It would be very easy to change from CroppableImage to CroppableImage3, but the old Thumbnail / CropImage uses a none-conform variation naming. It uses prefix_basename.ext, but since PW 2.5.11 we have basename.-suffix.ext.

The way to go would be to use a bootstrap script that iterates over all pages, looking for Thumbnailfields, and if one is found, iterate over all images and (to be save!) in a first run copy the old variations to imagevariations with the new naming scheme.

After successfully copied all crop variations, you should make a DB backup!! Than you can install the CroppableImage3 module and change the fieldtype of your images field to point to it. (Do a copy of your croppsettings before changing the type) Now you need to update your template files to use

$page->image->first->getCrop('grande')

instead of

$page->image->first->getThumb('grande')

After checking if all works well, you may run a variation of the bootstrap script that delete the old prefix_basename.ext variations instead to copy them to new naming scheme. Then uninstall the old Thumbnail module.

As you said it is a lifesite, I suggest to run the bootstrap copy of the variations under PW 2.7 version. This would have no effect on your site. You also may upgrade the Thumbnail module to the CroppableImage (intermediate) module first, what can be done completly under PW 2.7. This way, you would not have to update two parts at once (PW no namespace to namespace, Imagefieldtype from old to new over different naming schemes). Upgrading from CroppableImage to CroppableImage3 is easier. But, if you can lock down your site for 20 to 30 minutes while switching to PW 3, changing images fieldtype and updating template files, it should be save to change from Thumbnail to CroppableImage3 directly.

<?php
// for PW 2.7 (UPDATED with @jacmaes bugfix for $new filename from below post!)

$debugIteration = true;   // true to iterate only over the first page with a desired field, false to iterate over all pages
$doFilecopy     = false;  // true to really copy variation files, false for debug purposes

$oldFieldtype = 'CropImage';  // CropImage
$newFieldtype = 'CroppableImage3';

$timelimit = 60;  // per single page

// bootstrap PW
include(dirname(__FILE__) . '/index.php');


// collect fields and cropsetting names
$collection = array();
echo "<ul>";
foreach($fields as $f) {
    if($f->type != 'Fieldtype' . $oldFieldtype) continue;
    $collection[$f->name] = array();
    echo "<li>{$f->type} : {$f->name}</li>";
    $thumbSettings = preg_match_all('#(.*?),.*?\n#msi' , trim($f->thumbSetting) . "\n", $matches, PREG_PATTERN_ORDER);
    if(!$thumbSettings) continue;
    $collection[$f->name] = $matches[1];
    echo "<ul>";
    foreach($collection[$f->name] as $suffix) {
        echo "<li>{$suffix}</li>";
    }
    echo "</ul>";
}
echo "</ul>";
echo "<hr />";


// now iterate over all pages and rename or copy the crop variations
echo "<ul>";
foreach($pages->find("include=all") as $p) {
    set_time_limit($timelimit);  // reset the timelimit for this page
    foreach($collection as $fName => $suffixes) {
        if(!$p->$fName instanceof Pageimages) continue;
        $images = $p->$fName;
        if(0 == $images->count())continue;
        echo "<li>{$p->title}<ol>";
        foreach($images as $image) {
            echo "{$image->name}<ul>";
            foreach($suffixes as $suffix) {
                $old = dirname($image->filename) . "/{$suffix}_" . $image->name;
              	$new = dirname($image->filename) . "/" . str_replace(".", ".-{$suffix}.", $image->name);
                echo "<li>$suffix<ul><li>$old</li><li>$new</li></ul>";
                if($doFilecopy) {
                    if(!file_exists($old)) {
                        echo "ERROR: original variation is missing!";
                    } else {
                        if(file_exists($new)) {
                            echo "file already exists";
                        } else {
                            $res = @copy($old, $new);
                            echo "filecopy: " . ($res ? "Success!" : "!ERROR: $res");
                        }
                    }
                }
                echo "</li>";
            }
            echo "</ul>";
        }
        echo "</ol></li>";
        if($debugIteration) break;
    }
    $pages->uncache($p); // just in case we need the memory
}
echo "</ul>";
exit();

 

I have tested this with one page and two images, where I have done a little damage too. This was the output:

Edited September 17, 2016 by horst
fixed issue with $new filename

[jacmaes]

Thanks a million for the detailed walk-through and the full script @horst! I wouldn't have been able to do this without your expertise. You even took the time to test your script, which makes me more confident. It never ceases to amaze me how helpful the PW community is. I'm sure this will help other people who, like me, would like to upgrade but were not sure how to.

I'll definitely take the site down before launching the upgrade, and do a full backup of the site beforehand in case something goes wrong.  

[horst]

Yes, it may of help for others too. That's why the script first collects all CropImage fields and their cropsettings.

I will link to this post here from CroppableImage3 support thread.

[jacmaes]

@horst, I've tried running your bootstrapped script to copy the variations (on PW 2.7.3), but it stops short:

It collects the field and displays the crop settings correctly, but then does not iterate over all pages. Looks like it stops running after

echo "<hr />";

 

Edit: tried a test run on another site, and it almost works. Problem is with str_replace that also replaces the domain name (see what I've marked in bold):

 

'Sorolla and America'

1. sorolla-and-america-optimized.jpg
   • pequeno
     • /var/www/domain.com/site/assets/files/1228/pequeno_sorolla-and-america-optimized.jpg
     • /var/www/domain.-pequeno.com/site/assets/files/1228/sorolla-and-america-optimized.-pequeno.jpg
   • grande
     • /var/www/domain.com/site/assets/files/1228/grande_sorolla-and-america-optimized.jpg
     • /var/www/domain.-grande.com/site/assets/files/1228/sorolla-and-america-optimized.-grande.jpg
   • destacado
     • /var/www/domain.com/site/assets/files/1228/destacado_sorolla-and-america-optimized.jpg
     • /var/www/domain.-destacado.com/site/assets/files/1228/sorolla-and-america-optimized.-destacado.jpg

Edit 2: I've managed to fix my problem with a few tweaks, and all files were successfully copied.

// now iterate over all pages and rename or copy the crop variations
echo "<ul>";
foreach($wire->pages->find("include=all") as $p) {
    set_time_limit($timelimit);  // reset the timelimit for this page
    foreach($collection as $fName => $suffixes) {
        if(!$p->$fName instanceof Pageimages) continue;
        $images = $p->$fName;
        if(0 == $images->count()) continue;
        echo "<li>{$p->title}<ol>";
        foreach($images as $image) {
            echo "{$image->name}<ul>";
            foreach($suffixes as $suffix) {
                $old = dirname($image->filename) . "/{$suffix}_" . $image->name;
                $new = str_replace($suffix . '_', '', $old);
  
  				// These two lines are my tweaks
                $new = str_replace('.', '.-' . $suffix . '.', $image->name);
                $newer = dirname($image->filename) . "/" . $new;

  				// $newer is the new $new below:				
  
                echo "<li>$suffix<ul><li>$old</li><li>$newer</li></ul>";
                if($doFilecopy) {
                    if(!file_exists($old)) {
                        echo "ERROR: original variation is missing!";
                    } else {
                        if(file_exists($newer)) {
                            echo "file already exists";
                        } else {
                            $res = @copy($old, $newer);
                            echo "filecopy: " . ($res ? "Success!" : "!ERROR: $res");
                        }
                    }
                }
                echo "</li>";
            }
            echo "</ul>";
        }
        echo "</ol></li>";
        if($debugIteration) break;
    }
}
echo "</ul>";
exit();

 

[horst]

@jacmaes: many thanks for your feedback! I have updated the script in my post with your suggestion to work only with the image->name and not image->filename within str_replace(). Also I added a line at the end of the pages loop to uncache each no longer needed page, to save some memory.

[netcarver]

Here's an updated version of the script from @horst and @jacmaes

This version;

• Is PW3 compatible (but not 2.7/2.8)
• Fixes a bug that prevented the $debugIteration flag from working as described
• Tweaks the output formatting somewhat
• Adds some stats on pages/images/variations visited (and variations copied successfully)

<?php
// for PW3+
// (UPDATES by @jacmaes   bugfix for $new filename from below post!)
// (UPDATES by @netcarver debugIteration bugfix, PW3 compatibility & formatting tweaks)

$debugIteration = true;    // true to iterate only over the first page with a desired field, false to iterate over all pages
$doFilecopy     = false;   // true to really copy variation files, false for debug purposes

$oldFieldtype = 'CropImage';  // CropImage
$newFieldtype = 'CroppableImage3';

$timelimit = 120;  // per single page

// bootstrap PW
include(dirname(__FILE__) . '/index.php');


// collect fields and cropsetting names
$collection = array();
echo "<ul>";
foreach($fields as $f) {
    if($f->type != 'Fieldtype' . $oldFieldtype) continue;
    $collection[$f->name] = array();
    echo "<li>{$f->type} : {$f->name}</li>";
    $thumbSettings = preg_match_all('#(.*?),.*?\n#msi' , trim($f->thumbSetting) . "\n", $matches, PREG_PATTERN_ORDER);
    if(!$thumbSettings) continue;
    $collection[$f->name] = $matches[1];
    echo "<ul>";
    foreach($collection[$f->name] as $suffix) {
        echo "<li>{$suffix}</li>";
    }
    echo "</ul>";
}
echo "</ul>";
echo "<hr />";

$pages_visited      = 0;
$images_visited     = 0;
$variations_visited = 0;
$variations_copied  = 0;

// now iterate over all pages and rename or copy the crop variations
echo "<ul>";
foreach($pages->find("include=all") as $p) {
    set_time_limit($timelimit);  // reset the timelimit for this page
    foreach($collection as $fName => $suffixes) {
        if(!$p->$fName instanceof \ProcessWire\Pageimages) {
            continue;
        }
        $images = $p->$fName;
        if(0 == $images->count()) {
            continue;
        }
        echo "<li>Page \"<strong>{$p->title}</strong>\" <em>directory \"site/assets/files/{$p->id}/\"</em><ol>";
        foreach($images as $image) {
            echo "Image \"<strong>{$image->name}</strong>\"<ul>";
            $images_visited++;
            foreach($suffixes as $suffix) {
                $variations_visited++;
                $errors  = array();
                $dispold = "{$suffix}_" . $image->name;
              	$dispnew = str_replace(".", ".-{$suffix}.", $image->name);

                $dir = dirname($image->filename).'/';
                $old = $dir . $dispold;
                $new = $dir . $dispnew;

                $old_present = file_exists($old);
                $new_present = file_exists($new);

                if (!$old_present) $dispold = "<strike style='background-color:#F33'>$dispold</strike>";
                if ( $new_present) $dispnew = "<u style='background-color:#3F3'>$dispnew</u>";

                echo "<li><em>Variation: $suffix</em> — $dispold ⇒ $dispnew";

                if($doFilecopy && $old_present && !$new_present) {
                    $res = @copy($old, $new);
                    if ($res) {
                        $variations_copied++;
                        $res = "<span style='background-color:#3F3'>Success</span>";
                    } else {
                        $res = "<span style='background-color:#F33'>FAILED</span>";
                    }
                    echo " <strong>Filecopy: $res</strong>";
                }
                echo "</li>";
            }
            echo "</ul>";
        }
        echo "</ol></li>";
        if($debugIteration) break;
    }
    $pages->uncache($p); // just in case we need the memory
    $pages_visited++;
    if($debugIteration) break;
}
echo "</ul><p>Visited $pages_visited pages.</p>";
echo "<p>Visited $images_visited images.</p>";
echo "<p>Visited $variations_visited variations.</p>";
echo "<p>Copied  $variations_copied variations.</p>";
exit();

Looks like this now...

Thanks Horst for the detailed instructions and script - and to jacmaes for the testing and update!

[netcarver]

I also have a copy of the script that optionally handles deletion of the source variations too. I'll post that when its had some more testing.

[Pretobrazza]

Hi,

Is  the script that optionally handles deletion of the source variations available yet?

But first and foremost (forgive my ignorance), where do I put the script above? Croppable image 3 works as a dream!

ps. I had to change the limit of the image file from 1 to 0 - otherwise it refused to work as it kept on giving me an error: Call to a member function getCrop() on a non-object

Kind regards
Bernard

[adrian]

47 minutes ago, Pretobrazza said:

But first and foremost (forgive my ignorance), where do I put the script above?

There are several options:

1) The easiest option would be in the Console Panel in TracyDebugger

2) Converted to an action for the Admin Actions module - this would be a great option because I expect there will be lots of users wanting to convert from Thumbnails to Croppable image over the next few months - hint hint @netcarver, @horst, or @jacmaes

3) In a test hanna code

4) In a template file

5) In a bootstrapped script in the root of your site

[Pretobrazza]

Hello Adrian,

Firstly a great thank you for giving me the opportunity to discover TracyDebugger which I'll definitely keep as an extremely useful tool!

I seem to have found a but though in the script right above here for PW3+:

I did the following:

• made a copy of PW2.7.3 to a subdirectory on the site
• Removed the old cropimage modules and installed TracyDebugger (and did some other stuff)
• Upgraded to PW 3.0.44 and installed Croppable Image 3 properly.
• Ran the script for PW3+ with :

$oldFieldtype = 'CroppableImage3';  // CroppableImage3 instead of CropImage otherwise it doesn't do a thing
$newFieldtype = 'CroppableImage3';

Instead of taking an exact copy of eg. thumbnail_yyz_20140101_164521.jpg it seems to make its own cropfile called yyz_20140101_164521.-thumbnail.jpg 

Why doesn't it take an exact copy of the picture in thumbnail_  ...   and just rename the file? Do you know a solution or am I going wrong ?

Kind regards,
Bernard

 

[netcarver]

For those wanting to delete source images, you can find my script in this Gist. Make sure you have successfully converted the images to the CI3 standard before you run it to delete the source images. 

[MadHatter]

@netcarver I'm finding that the script is timing out on a large site I'm working on (there are ~1800 pages). I've tweaked the code to only find pages in which the found fields are in use and are not empty:

...
// collect fields and cropsetting names
$collection = array();
echo "<ul>";
foreach ($fields as $f) {
    if ($f->type != 'Fieldtype' . $oldFieldtype) {
        continue;
    }
    $collection[$f->name] = array();
    echo "<li>{$f->type} : {$f->name}</li>";
    $thumbSettings = preg_match_all('#(.*?),.*?\n#msi', trim($f->thumbSetting) . "\n", $matches, PREG_PATTERN_ORDER);
    if (!$thumbSettings) {
        continue;
    }
    $collection[$f->name] = $matches[1];
    echo "<ul>";
    foreach ($collection[$f->name] as $suffix) {
        echo "<li>{$suffix}</li>";
    }
    echo "</ul>";

    $field_names[] = $f->name;
}
echo "</ul>";
echo "<hr />";

$pages_visited = 0;
$images_visited = 0;
$variations_visited = 0;
$variations_copied = 0;

// now iterate over all pages and rename or copy the crop variations
echo "<ul>";

$selector = implode("|", $field_names) . "!='',include=all";
$found_pages = $pages->find($selector);

foreach ($found_pages as $p) {
...