CloudflareAssets

[nbcommunication]

Extends Pagefile to use Cloudflare Images, Stream and R2 storage to serve files.

https://github.com/nbcommunication/CloudflareAssets

The main purpose of this module is to allow ProcessWire to be used in an auto-scaling multi-instance environment. By serving file assets from Cloudflare, it is not necessary to have all file assets copied to all instances and we also get the benefits of serving assets from a CDN.

How it works

When a Pagefile is added in the admin or via the API, it is uploaded to Cloudflare's R2 storage service. Additionally, if the file is an image, it is uploaded to Cloudflare Images, and if the file is a video it is uploaded to Cloudflare Stream. When a URL for the Pagefile is requested e.g. $pagefile->url(), the appropriate Cloudflare URL is returned.

As ProcessWire's admin still requires the file to be available locally, in a multi-instance setup if a file is not available it is downloaded from the 'master' copy in R2.

This module is not yet being used in production.

There may be changes to how Images work in the coming months as features are still being rolled out to this Cloudflare service.

Cheers,

Chris

[MrSnoozles]

Freaking awesome!! I've been thinking about doing the same for bunny.net these last couple weeks and the approach I had outlined in my mind would have been very similar to yours. Very well done, I'm super excited for this ?

[nbcommunication]

It didn't take long for the changes to the Images implementation - it turns out that image options are available for named variants, just not documented. The module now uses named variants instead of flexible variants. More notes etc to be added to the README in the next month or so.

[Kiwi Chris]

This is super cool. I'd been thinking about ways to use Cloudflare images for a while.

On 4/17/2023 at 8:00 PM, nbcommunication said:

As ProcessWire's admin still requires the file to be available locally

This might be a bit of an issue for how I'd considered using it. One of the things that had attracted me to Cloudflare Images is the low price of storage and traffic, but if ProcessWire still needs a local copy, that's maybe not quite as useful for the scenario I'd had in mind, although it would certainly help with traffic.

Not sure whether a way around it would be to just store a thumbnail locally once full res file has been stored in Cloudflare.

 

[nbcommunication]

Hi @Kiwi Chris,

The admin needs the local copy, from what I can see it is mainly for the file size displayed beside the filename in the admin. I did look at hooking InputfieldFile::render() to replace this but I eventually decided that it made sense to keep the original file in place when uploaded, and if it didn't exist to download from the R2 copy. I was thinking here of worst case scenario where Cloudflare Images has a major service outage. In a multi-server environment I could perhaps query each server to see which has the most files and switch off the rest of the servers, and switch off the CF integration on the chosen server. That way a site could continue to function. A very unlikely situation but I'd like that option.

What it doesn't do (or shouldn't do, haven't fully tested it yet) is create the size variation image locally. Calls to Pageimage::size() should just create a variant in Cloudflare Images if it doesn't already exist. Unfortunately there is a limit of 100 variants (which can be applied to every image upload, it isn't a limit of 100 actual images), which while sufficient for a single development, means that the service cannot be used for multiple sites, unless they are creating the same variants.

I did expect to keep this and the other modules I posted at the start of the week internal as they are somewhat specific to our needs, but after consideration I figured it would make sense for them to get other eyes on them and some differing perspectives - if you do go down the route of Cloudflare Images with your own integration please use this work as reference and please do share any solution you reach!

Cheers,

Chris

[Kiwi Chris]

14 hours ago, nbcommunication said:

The admin needs the local copy, from what I can see it is mainly for the file size displayed beside the filename in the admin. I did look at hooking InputfieldFile::render() to replace this but I eventually decided that it made sense to keep the original file in place when uploaded, and if it didn't exist to download from the R2 copy.

Thanks. I'll look into this and have a go at replacing it.

14 hours ago, nbcommunication said:

I was thinking here of worst case scenario where Cloudflare Images has a major service outage.

In my usage scenario this isn't really an issue. I have some very small non-profits who don't want to pay much but have images and documents they want to host, and storing them locally on my VPS isn't really cost effective if they have a lot. If they're paying a low price and don't have to worry about storage, I don't think they're going to object if there's a temporary outage.

14 hours ago, nbcommunication said:

I did expect to keep this and the other modules I posted at the start of the week internal as they are somewhat specific to our needs, but after consideration I figured it would make sense for them to get other eyes on them and some differing perspectives

I really appreciate you making this available. It's something I'd been thinking about for a while, and having a working module to modify will be so much easier than starting from scratch.

I'd imagine my usage scenario may be useful to others too, so I'll be sure to update any modifications I make.

[Arklogic]

Hello @nbcommunication, thank you for creating this module.

I'd like to use it in my projects, I already configured it and it works, but there is a case when it doesn't.

Basically when I edit an image in ProcessWire, then save, the modifications are only saved locally in ProcessWire site/assets/files and do not upload to Cloudflare.

Also an issue, that cloning a page won't clone the related images in CF to a new folder with newly created page ID.

I'm wondering why these happening, if they are just bugs?

Right now I don't need to use Images and Stream in CF, my goal is to just use a CDN, and also to keep files in sync between my local and dev environments.

Edited May 27 by Arklogic

[nbcommunication]

Hi @Arklogic,

Apologies, I'd decided many months ago now that developing this module in a way that would be robust enough for production (and listing on the modules directory) was just too complex. I thought I'd posted here to that effect. I see I did not.

The latest code is here: https://github.com/nbcommunication/CloudflareAssets I think I did implement the page clone action, but did not implement anything for image editing in the admin, and have no plans to.

While we are using this module on the website I'd developed it for, it is tailored for its use-case. Were I approaching the same development now, I'd be looking at using a CDN which would be significantly less work.

Cheers,

Chris