Weekly update – 28 August 2020

[ryan]

This week I’ve been working on a combination of the core, and a PW audio-streaming project for a friend, working as a volunteer. There’s some overlap in these two things, as a lot of the core updates this week are somewhat related. It provided the needed motivation to finish a new feature I started working on in 3.0.149, which will be finally finished in 3.0.166. Actually, it’s finished now, but I haven’t bumped the version number on the dev branch yet—I’ll do that next week. The feature finished this week adds support for a new secure files option similar to the $config->pagefileSecure option (you may or may not have heard of), except that it allows you to configure secure files at the template level, rather than just site-wide level. 

In ProcessWire, when you unpublish or trash a page, or when you access control a page so that it’s not visible to guest users, the page’s files in /site/assets/files/page-ID/ can still be accessed directly by URL (like if someone bookmarked a PDF or something). So we have the $config->pagefileSecure option that lets you specify that the files should also be access controlled. This ensures (for example) that if a page is unpublished, its files can’t be loaded by anyone except a user that can view unpublished pages. But I rarely use this option because it is site-wide, and access controlled files take some overhead (being delivered by ProcessWire rather than Apache) and thus are something I’ve always wanted finer control over. That’s because most of the time, I don’t care too much about direct access to files because they are mostly images and endless variations of them. But there are also those specific cases where I really do need it. So being able to make it part of the template-level access control really makes it a lot more flexible, and hopefully makes the option a lot more obvious:

There’s also that “Yes always” option (in the screenshot above), which is something we’ve not had before. This is useful when you want to have full control over a file download (or stream). Because the files would always pass through ProcessWire, you can hook any part of the process, keep a download counter, log downloads, etc. ProcessWire supports partial file delivery and delivery of specific byte ranges and seeking, which comes in especially handy with <audio> tags.

In the case of the project I was working on this week, the audio streams are available only to logged in users (via LoginRegisterPro) and we had to find a way to make the audio files protected from direct access, so that they would only work as a stream through an <audio> tag to an authenticated user, not be accessible outside of that, and not be downloadable. That was all a bit of a challenge, but a fun one. If anybody ever runs into a similar need, be sure to send me a message. 

I have a few more things I want to work into version 3.0.166 before bumping the version number, so that’ll likely come by this time next week. I also hope to fully wrap up the new version of either FormBuilder or ProCache next week, followed by the remaining one the following week. Thanks for reading. Have a great weekend!

[horst]

50 minutes ago, ryan said:

That was all a bit of a challenge, but a fun one. If anybody ever runs into a similar need, be sure to send me a message. 

@ryan there is still one of my very first PW projects, the MP3-DB site profile, which doesn't work with any PW 3 version (or newer PHP versions ?). I've been wanting to put the essence of it into a module for quite some time, so that one can add MP3-DB support to any existing site with ease. It seems that I'm getting a bit closer to the goal thanks to your new features, because it will be much less work if partial file delivery and delivery of specific byte ranges and seeking is now natively supported by PW! ?

When starting with it (maybe around christmas this year), I will contact you before.

[teppo]

Hey @ryan!

This is very loosely related to the weekly update, but I noticed that this week (as well as in some earlier updates) you've started converting $this->wire('api_var_name') to $this->wire()->api_var_name. Just wanted to ask if there's some benefit to this, or if it's mostly just a matter of taste? ?

Not a big deal by any means of course, but I've been writing stuff like wire('session') all over my modules, so wondering if there's a reason why I should rather prefer wire()->session etc.

[ryan]

@horst The partial file delivery was added last year. I'm not sure of the exact right terms for it, but it's basically giving proper responses to the $_SERVER['HTTP_RANGE'] "bytes" requests which specify starting and ending ranges of bytes from a file to respond with. More details on the implementation in the WireHttp::sendFileRange method. Browsers basically send a test request to the server to see if it supports it, and if they give the proper response, then it starts requesting byte ranges and we just just give it the expected ranges. They aren't always sequential either, sometimes it wants a piece of the beginning and a piece of the end, sometimes different things, and it seems to vary between browsers. You don't have to do anything special to use it, as any file delivered by ProcessWire already supports it when the browser requests it. 

[ryan]

@teppo The only reason I've started using wire()->pages->find() rather than wire('pages')->find() is just because the IDE (PhpStorm in my case) recognizes what wire()->files is, but doesn't recognize what wire('files') is... well it does, but it only knows that it could be any one of a bunch of API vars, rather than knowing it's the $pages API var. Substitute any API var for "pages" here, I'm just using pages as an example. Phpdoc data is able to traverse through variable and method names like wire()->pages or pages(), but not through strings like wire('pages'). It makes zero difference technically which you use, but at least as far as IDEs like PhpStorm goes, it's the difference between the IDE knowing what you are doing and not really knowing what you are doing. I like it knowing what I'm doing because it helps to find potential issues more easily, and it knows arguments, types and return values on methods I'm going to call, etc. You can also do the same by using phpdoc to tell it what type a particular variable is. For instance, the two bits of code below would have the same effect in terms of the IDE knowing what $pages is: 

// this: 
/** @var Pages $pages */
$pages = $this->wire('pages');

// or this:
$pages = $this->wire()->pages; 

 

[adrian]

@ryan - I just tried working with this for the first time and the issue I am having is that I don't want to prevent guests from viewing the page, but I want to prevent them from being able to view the PDF associated with the page. Of course, the first level of protection is to simply not show them the link to the file, but if they were given (or guessed) the direct url to the file, I don't think I can actually use pagefileSecure and this new approach to block access, or am I missing something?

Perhaps pagefileSecure needs to have an optional permission, eg 'files-view' or something like that - I think that would allow much more flexibility in controlling access to files without relying on access to the page they are stored on. Does that make sense?

[thetuningspoon]

Seems like it should be mentioned somewhere that the pageFileSecure settings only works with files added after it is enabled.

[cb2004]

On 3/15/2021 at 6:51 PM, adrian said:

@ryan - I just tried working with this for the first time and the issue I am having is that I don't want to prevent guests from viewing the page, but I want to prevent them from being able to view the PDF associated with the page. Of course, the first level of protection is to simply not show them the link to the file, but if they were given (or guessed) the direct url to the file, I don't think I can actually use pagefileSecure and this new approach to block access, or am I missing something?

Perhaps pagefileSecure needs to have an optional permission, eg 'files-view' or something like that - I think that would allow much more flexibility in controlling access to files without relying on access to the page they are stored on. Does that make sense?

Yes could still do with a little extra work here. For example on a project I use I wanted to use an image field for a summary page that links to a pdf which should be locked down but of course the image doesn't load on the summary page. I ended up using the module by @Wanze to store the files outside the public folder and then add some logic. Would be great if we could pick what fields should be protected. 

[kongondo]

EDIT: solved; see post below

Could someone please confirm that $page->secureFiles() still works? Testing in ProcessWire 3.0.187 and irrespective of the page published status, the files are still publicly accessible. I have also tried this:

On 6/22/2021 at 9:20 PM, thetuningspoon said:

Seems like it should be mentioned somewhere that the pageFileSecure settings only works with files added after it is enabled.

And this:

but none of the proposed solutions works for me. 

Dumping $template->pagefileSecure  returns 2. Dumping $page->secureFiles() returns false. It only returns true if do you want to manage view and edit access for pages using this template is Yes.

Thanks.

Edited November 22, 2021 by kongondo
solved

[kongondo]

Ignore me. I forgot this:

?‍♂️