teppo

Thanks @adrian! First one was a PHP8 issue, while the second (and bigger) problem was related to fulltext index queries. Both should now be fixed in version 0.29.1, though please let me know if it doesn't work for you. Also: I had accidentally grouping by template enabled by default, this is now disabled by default.

Took a lot longer than anticipated to get this out of the oven, but version 0.29.0 of SearchEngine was just released. Word of warning: this is the biggest new release for the module so far (in terms of features as well as changed lines of code) so please test carefully! Main features included in this release: New group_by option in find_args. For now only supported values are null (default) and "template", the latter of which (obviously) groups results by template. Here grouping means two things: first of all results are rendered in tabs on the front-end, and second of all instead of a single Query object the module instead returns a QuerySet object, which in turn contains multiple Query objects. New results_grouped_by option in render_args. This accepts any field name, page property, etc. and essentially divides found results into groups by said field value / property. For an instance if you specify "template.label", you'll get a list of results with subheadings for templates. While both of these options are related to grouping results, they work on an entirely different level: group_by performs separate SQL queries and splits results into their own Query objects, while results_grouped_by is something that only affects the render phase (front-end). It's also possible to combine the two, in case that makes sense ? There's also a new custom sort setting "_indexed_templates" (results are sorted by the order of templates in the "indexed templates" config setting) and new argument "pinned_templates" for the find() method that will show results with provided templates at the top of the results list.

I'd say that these are more like two sides of the same coin: You've solved the file issue by directing static file requests to CDN and pointing file related requests to a single (?) master instance. This is of course a valid approach, but in this type of setup you've got at least two types of instances (read and read-write), which can make things more complicated than just replicating a single instance type based on the load, and may not work so well for sites that often require direct access to assets. This could also potentially impact fault tolerance: in case you indeed only have one write instance, that going down for any reason would be a big blow. Most cloud setups I've worked with have handled this the other way around: files offloaded to external buckets and database shared among instances. One benefit of this is that those instances are identical, there's no "master" to rely on. In most cases S3 hasn't had major impact on performance (plus there's always CDN) and I've never run into a case where database would've been an absolute bottleneck. Of course DB requests should be limited: most web requests can be served from static cache, and if advanced caching is needed, in comes Redis. The long story short is that I believe core level ability to store files somewhere other than local disk would be a splendid improvement for cloud use. It's true that you can get past this and it's not strictly necessary in order to run ProcessWire in the cloud, but it would simplify things quite a bit. That being said, I definitely get what you mean by this being the more complex part ? Also, just to be clear, I think that the database abstraction is a great addition!

Also interested in this. My impression is that offloading assets to Amazon S3 / Google Cloud Storage etc. is a relatively common need, and there's currently no bulletproof solution. Some third party modules have attempted to, but I'm not sure if any of them really solve all the related needs (image management etc.) — and while remote filesystems such as EFS are easier to work with, they also come with certain downsides (such as a higher price point). I seem to recall that Ryan mentioned something about this a while ago too ?

Doesn't sound like you're missing anything, and yes — the module should automatically redirect the user to the first branch configured for them. So far I've been unable to reproduce this issue. Just in case I installed a new ProcessWire site with only default site profile + AdminRestrictBranch + AdminRestrictBranchSelect, and everything worked as expected. Following the suggestions Adrian made above should give us a better idea of what's going on, but one more thing I'm wondering is that what version of ProcessWire you're running? The module should work on 3.0.123 and above, but I've only tested it on 3.0.165, so it could have something to do with that. It kind of sounds like AdminRestrictBranch::init() might run before AdminRestrictBranchSelect::init(), or the latter might not run at all; either would at least explain the error you're seeing. A simple way to test if this is happening would be adding some debug code (bd(), or just die() with some sensible message) in both methods and seeing which gets executed first (should be AdminRestrictBranchSelect::init()).

Yes ?

Thanks — looks like the modules directory defaults to master branch, while GitHub defaults to main instead. Fixed now.

You may have a specific reason for going this route, but first things first: Instagram normally prohibits this sort of use. They used to name "scraping" and "caching" specifically, but current version of their ToS just disallows "collecting information in an automated way without our express permission". So I'd say that while you may be able to do this technically, you probably shouldn't, especially if any of the data pulled from IG is going to be displayed publicly. You can of course request permission for this, but I'd wager that there's a very low chance of that working out ?‍♂️ Legal aspect aside, whether PW is a sensible platform for something like this would depend a lot on the specific case. For storing, displaying and searching through structured data? Sure, why not. Though in such a case it might make more sense to write the scraper in PHP as well, so you could just bootstrap PW. Alternatively the scraper could store data locally or in a DB, and PW would then read it from there... or you could provide an API endpoint at PW side ?

Hey @adrian! Just a heads-up that I posted an updated and extended version of my code as a new add-on module: The functionality seemed pretty specific to one use case so I figured it was best not to bundle it in the main module, but feel free to use anything from my module if it makes sense to you. Turns out that the original implementation missed a few key points and was very specific to a particular site, so this is basically a full rewrite... ?

Admin Restrict Branch Select is an add-on / companion module for Admin Restrict Branch. With this module enabled, you can manually select more than one branch parent per user via the branch_parent field, and users with more than one option selected will be able to switch between those while editing site content. Switching between branches is done via a select field injected at the top of the page tree. Note that users are still limited to one branch at a time: this module will not make it possible to view multiple branches at the same time. When the module is installed, it will automatically update the branch_parent field if deemed necessary, i.e. if it isn't yet configured to allow selecting multiple pages. You can make changes to the field later, if you want to restrict selectable options by template, use asmSelect instead of PageListSelectMultiple, etc. This is an early beta release, so be sure to test carefully before enabling this module in production! https://processwire.com/modules/admin-restrict-branch-select/ https://github.com/teppokoivula/AdminRestrictBranchSelect

Recently needed to add something similar to a site I've been working on, and ended up with a slightly different approach. Can't share the code right now, but here's the general concept: Store all allowed branches in user profile, similar to the screenshot above. Hook into ProcessPageList::execute and add a select option for choosing which branch is "active". Hook into AdminRestrictBranch::getBranchRootParentId and check if a branch GET param was provided and matches one of those stored in user profile; if yes, store the value in session. Return either the first branch from user profile, or the value from session. Technically it's only restricting the user to one branch at a time, so there are some limitations, but in my specific case that's actually preferable: this lets users focus on one section of the site at a time ?

Thanks! This wasn't particularly urgent, so thought I'd post it here in case others might stumble upon the same question. Anyway, will PM you if I find any more issues, though I'm pretty sure you covered them all ??

Hey @ryan — what's the proper process for getting access to a module in the directory? It looks like https://processwire.com/modules/process-changelog/ isn't connected to my profile, and it's also no longer auto-updating for some reason (looks like the latest update was in 2019) ? Edit: actually when I look at the public list of modules with my profile, I can see 19 modules, but after I log in only 15 of those show up under my own modules. Is this something to be expected? I'm assuming that the public list is not be using the same logic for connecting authors and modules, and this may even be by design, but it still seems a bit quirky.

Good to hear that it worked, eventually. The update notice should show up once when any superuser logs in, and every time you load said module's config page ?

Just released a new version (1.11.0) of Changelog. This version includes a schema update that requires action from a superuser, so the module will keep displaying a message when a superuser logs in until the update has been successfully executed. Technically this schema update could be handled just like the one before, behind the scenes, but since this one involves adding new indexes... well, if the custom database table (process_changelog) has a lot of data, this update could take quite a while, and might even result in a timeout. As such, I think it's best to let a superuser handle it. Just in case: the README.md file contains a simple PHP script that can be used if the update refuses to go through in the browser ? --- I recently migrated some sites to a new server, and ran into a serious performance issue right off the bat. After the usual process of blaming everything from the host to the MySQL version and even most recent ProcessWire updates, I almost accidentally noticed that some of my own queries for the process_changelog table were taking a very long time. Turns out there were ~2.5 million rows stored for the page in question, and no indexes whatsoever to help with those queries. "Whoops." With some new indexes — and one loosely related query optimization — in place, the heaviest page requests now take a few hundred milliseconds on my low-budget server. Still not blazing fast, but quite an improvement over the ~10-15 seconds they used to take. Not to mention that having millions of changelog entries for one page is probably a bit of a border case anyway ?‍♂️

I'd recommend hooking into Renderer::renderResult or Renderer::renderResultDesc, depending on your needs. Both are related to rendering a single result and have access to the Page object in question, as well as the Data object containing predefined template strings etc. Let me know if you run into any trouble. As for whether this might be a good feature addition — perhaps, though I've only had need for this once so far, so I'm not certain yet. If it's easy to add via hook and a relatively rare need, then it's probably best to leave it at that ? I'll take a closer look at the apostrophe issues later!

I'm pretty sure that aforementioned issue is now fixed in all the branches — but yes, you're correct, 2.0 branch is deprecated and the ones that are maintained — sort of — are dev (2.x) and master (1.x) ?

Hey kaz! This is just a forum policy: the Modules/Plugins area is reserved for support threads of existing modules, one per module. If a question is about an existing module, feel free to post it in the applicable thread. You can find a link ti the thread from the modules directory entry for that particular module. If its not (like this thread here, which is a general question about "which module to use") it's best suited for the general support area. Hope this clarifids things a bit ?

Not sure if I got this right, but when you say that the images are "in a custom folder directly from ERP", do you mean that this custom folder is at the local disk, not some external server/service/whatever? If that's the case, you can resize them with ImageSizer, but it requires going through some extra hoops. You may find this thread useful — it's about generating resized versions of Pagefiles, but the concept here is similar, with the difference that the path to file is not in a Pagefile object but rather your text field. That being said, it would be much easier if you would just store those images in an image field on one of your pages. The process would be more straightforward, you wouldn't need to worry about cleaning up old variations, and so on. I guess the TL;DR here is that if you have a good reason for not handling those images as regular ProcessWire images, then yes, technically it's possible to do what you've asked here with ImageSizer, but a) this is rarely the best approach, and b) even then you may find working with Imagick or something similar directly a tad easier ?

@bernhard, I'm feeling a little lazy so thought I'd just ask: does RockFinder support (template or field level) permissions? I haven't found any mention saying that it would, but neither did it state anywhere that it wouldn't (I think) ?

Just to make sure: do you mean that the chosen operator depends on whether the search query is wrapped in quotes, or that a single query can contain (multiple of) both? ? First one would be relatively simple; this would likely mean extending module config with an optional "literal match selector" setting, which would then be used in case there are quotes around the term. If it's the latter, I'd be interested in hearing how this works.

Just released a new version of the module — here's the full changelog: ### Added - Support for Indexer actions, the first one of which adds support for rendering FormBuilder forms as part of the search index. This feature is currently considered experimental. - Option to specify custom path for front-end themes via the Advanced settings section in module config. - EditorConfig (.editorconfig) and ESLint (.eslintrc.json) config files for defining coding style for IDEs. - Support for collapsible content blocks in Debugger. ### Changed - Indexed templates option in module config is now AsmSelect, which makes it possible to organize indexed templates by preferred priority. - Query class converts lesser than and greater than to HTML entities to allow matching said entities, as well as encoded HTML markup. - All Debugger CSS classes refactored to follow `.pwse-` format. The first part of this is now built in, i.e. the indexed templates setting uses AsmSelect and the order of indexed templates gets stored. The module doesn't yet actually do anything with this information, so there's more work to be done. That'll be the focus of next release ? I'll give this more thought once I (hopefully) figure out the initial template order thing.

Interesting idea — added to the backlog ? I have a "mostly functional" proof of concept of this on one of my sites. This will likely be the next feature I work on after I get current dev branch merged ? ?‍♂️

"form_action" is indeed part of the "render_args" array. The $args array that SearchEngine::renderForm() (actually implemented by SearchEngine\Renderer::renderForm()) takes in is also the "render_args" array from the parent scope — so this depends (or at least it should depend) on which context you're defining this argument: If you're defining it via site config or by manually calling SearchEngine::setOptions(), you should put it within the render_options array item, which itself is an array ($config->searchEngine = ['render_options' => ['form_action' => '...']]). If you're calling SearchEngine::renderForm() with an args array, you should pass it as a top level item ($searchEngine->renderForm(['form_action' => '...'])). Please let me know if this description doesn't match actual behaviour though. That is how it should work. I must admit that I manage to confuse myself with these arguments every now and then. While I thought this would be the best approach while building the module, right now I'm not so sure anymore — the basic idea here was that when you're specifically calling render* methods, you shouldn't have to worry about anything other than the render args. Argument handling is one of the things I might still change a bit once I decide that the module is ready for a 1.0 release... ? Makes sense to me. Since theming support was initially added I've been using either the default theme as-is, or completely custom markup and styles, so the whole theme concept is somewhat "underdeveloped" at the moment. I've added this on my backlog for now.

That's a different context: it seems to me that Tracy hooks after PageFinder::getQuery() and uses the getQuery() method of returned DatabaseQuerySelect object. In my case I'm calling PageFinder::getQuery() instead. It's potentially a bit confusing that PageFinder::getQuery() returns an object that has getQuery() method, but these are actually two unrelated things... ? Technically I could do the same thing in SearchEngine, but it feels "less straightforward" ?