Jump to content

PW 3.0.137 – Core updates


ryan
 Share

Recommended Posts

Core version 3.0.137 on the dev branch adds the ability to hook multiple methods at once, in a single call. This post details how it works and provides a useful example of how you might put it to use in your development environment. This version also refactors and improves upon several core classes—

https://processwire.com/blog/posts/pw-3.0.137/

  • Like 18
  • Thanks 1
Link to comment
Share on other sites

Hi @ryan,

On 8/3/2019 at 11:14 AM, wbmnfktr said:

On-demand mirroring of remote web server files to your dev environment ???

This is indeed an ingenious little helper! Thx for sharing this idea with us!

One problem though: When I'm only in the backend, the url and filename methods get never called. Could you please make the setFilename method hookable so that this method also works when the page is just edited in the backend? It would then only need this simple hook and all files are downloaded on demand:

$wire->addHookAfter('Pagefile::setFilename', function($event) {
  $file = $event->object->filename();
  if(!file_exists($file)) {
    // download file from source if it doesn't exist here
    $src = 'https://example.com/site/assets/files/';
    $url = str_replace($this->config->paths->files, $src, $file);
    $http = new WireHttp();
    $http->download($url, $file);
  }
}); 

Or is there a better way of doing it?

Edit: Ryans version does of course work without issues - just make sure to have the proper PW version installed ^^

  • Like 7
Link to comment
Share on other sites

  • 3 weeks later...

@ryan any news on this? It works really great in my setup and I think this approach can open up a lot of great options for a better DEV/LIVE workflow with processwire. In my current project where I modified the core file it was really just doing a

git clone myrepo.git
git submodule update --init --recursive
# restore db dump

And I had a working copy of the site that downloaded all assets on demand while working with it! This is really awesome but it would break on every core update. I could implement some hack that does a str_replace and file_put_contents in my dev environment to prevent updates breaking this feature, but it's really just adding three underscores to one core file thanks to the ingenious hook system ? 

PS: I've created an issue for this: https://github.com/processwire/processwire-issues/issues/954

  • Like 2
Link to comment
Share on other sites

@bernhard Making it hookable should be fine. But I'd like to better understand what you mean about it not working in the admin, and the url and filename methods never getting called? Are you using the core files/images field? Those methods/properties are the ones used for anything asking for the filename or url of a file, whether on the front-end or in the admin. For me at least, it works the same in the admin/page editor as it does on the front-end, just tested again to confirm—editing a page downloads the images, and they appear in the images field in the page editor. So I'm not sure I understand what you mean? Also, hooking setFilename() would make it less on-demand, as it would download any images in the field rather than only those that were specifically requested. Can you think of anything different about your environment that might account for it not working?

  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...

Thx @ryan

That's strange, I've just tried on a new install and I can confirm that everything seems to work as expected. I'll try on that special installation once more and report back if the problem persists.

One little change that I've made is catching and logging exceptions instead of throwing them (eg in db-backups process module):

    try {
      $http->download($url, $file);
    } catch (WireException $e) {
      $this->log($e->getMessage());
    }

Nz98Ugh.png

Link to comment
Share on other sites

  • 5 months later...

@ryan many thanks for that useful piece of code. I used it a lot lately, also on sites that changed during my locally work. Therefor I need to adapt it a bit to first check if the remote file is available (and the optional use of basic auth). So, if it is of use for someone too, here it is:

// use assets from remote host on demand for local development copies
    $wire->addHookAfter('Pagefile::url', function($event) {
        fetchRemoteAssetsToLocalDev($event);
    });
    $wire->addHookAfter('Pagefile::filename', function($event) {
        fetchRemoteAssetsToLocalDev($event);
    });
            function fetchRemoteAssetsToLocalDev($event) {

                if(!$event->wire('user')->isSuperuser()) return;  // only my personal preference

                $config = $event->wire('config');
                if(!$config->Remote2Local) return;  // I set the domain name of a remote host in the site/config.php

                $remoteHost = 'https://' . $config->Remote2Local;    // I always use https these days
                $file = $event->return;
                if('url' == $event->method) {
                    $file = $config->paths->root . substr($file, strlen($config->urls->root));  // convert url to disk path
                }
                if(!file_exists($file)) {
                    // download file from source if it doesn't exist here, but exist on source
                    $src = $remoteHost . '/site/assets/files/';
                    $url = str_replace($config->paths->files, $src, $file);
                    $http = new WireHttp();
                    if($config->Remote2LocalBasicAuth) {   // optionally use basic auth with credentials from config.php
                        $http->setHeader('Authorization', 'Basic '.base64_encode($config->Remote2LocalBasicAuth));
                    }
					// check if file exist on remote host
                    $http->head($url, [], ['use'=>'curl']);
                    if('404' != substr($http->getHttpCode($url), 0, 3)) {
						// file is available
                        if($config->Remote2LocalBasicAuth) {
                            $http->setHeader('Authorization', 'Basic '.base64_encode($config->Remote2LocalBasicAuth));
                        }
                        $res = $http->download($url, $file, ['use'=>'curl']);

                    } else {
						// file is missing, log it
                        $event->wire('log')->save('remoteFetch404', $url);

                    }
                }
            }
// use assets from remote host on demand for local development copies

 

  • Like 7
Link to comment
Share on other sites

  • 7 months later...

Just a quick note that the above example will need an adjustment if the site uses a non-default asset folder or url.

To make this work when serving assets from, say, /data/* or assets.domain.com/v1/*, we can use the path and url settings for the files folder instead of root.

if ('url' === $event->method) {
	$file = $config->paths->files . substr($file, strlen($config->urls->files));
}

 

  • Like 1
Link to comment
Share on other sites

  • 1 year later...

I'm using the on-demand feature a lot. That's why it is part of RockMigrations using one single config setting:

$config->filesOnDemand = 'https://example.com';

See https://github.com/BernhardBaumrock/RockMigrations/blob/b09bf8236d376913a63b6fce63f91e05a0da1f2a/RockMigrations.module.php#L413-L453

That works great with one exception: When I delete an image the hooks still try to download the image from remote and this makes two problems:

  1. If the file exists on remote, the file will be downloaded to the local filesystem and will never be deleted because the image does not exist in the DB any more so PW will not know about it
  2. If the file does not exist on remote I will always get an error that it tried to download an image which does not exist:
    yPFV7ZO.png

Any ideas how I can fix this?

  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...