Jump to content

Recommended Posts

33 minutes ago, teppo said:

Sorry @snck and @xportde, looks like I completely missed this question. Just to be clear, you mean the ___savedPageIndex() method?

Running this when the entire index is being rebuilt should be doable, but it looks like I'll have to move a few bits and pieces to another class. I'll take a closer look at this — hopefully later today, or perhaps tomorrow.

No problem at all! Excuse me, yes, I meant the ___savedPageIndex() method. Thanks in advance!

  • Like 1
Link to post
Share on other sites
On 6/9/2020 at 10:32 AM, xportde said:

I have a question concerning the savePageIndex-Hook: It works perfectly on saving a single page, but not, when I use the "Index pages now"-function in module settings page. How could I achieve this?

This is fixed in the latest release (SearchEngine 0.25.2).

  • Like 2
Link to post
Share on other sites
  • 2 weeks later...

how about "exact matches"?

When I use the searchengine-module's demo-site, it seems to check for the exact match of each word of a search query in that very order.

e.g. search queries (ignore the quotes):
"scalability issues" returns 1 result
"issues scalability" returns 0 results

Some times you do not want exact matches and sometimes you do. When using Google, you soon figure out the use of quotes for an exact-match-query does the trick. Needless to say, that won't work here since the lack of quotes already returns exact matches.

How to specify that on the frontend? Would there be a way to specify that by using/not using quotes? I want to avoid including yet another set of radio-buttons. 

thanks!

Link to post
Share on other sites

Hey @fruid,

SearchEngine finds content via the API and the operator used for text searches is configurable (see module config screen), so there's that. Depending on the operator you use you'll get a different set of results (obviously). You can read more about available selectors from the docs.

As for Google-style search queries, there's nothing exactly like that built-in, but if you've got a newish version of ProcessWire (3.0.160+) and a recent version of SearchEngine installed, the module supports the advanced text search operator. The downside is that it's currently a little unclear how such queries should be sanitized so this feature is not fully functional. If you don't mind writing a bit of code yourself, there's also an example in the README about how you could provide your own search query and frontend (which, in turn, would allow you to sanitize the query string and then perform a find operation using the "#=" operator).

... and, as for the last point, it's actually also possible to use markup generated by SearchEngine yet still provide a custom results object. It involves a couple of extra steps, but something like this should work (requires SearchEngine 0.26.0 and ProcessWire 3.0.160+):

// load and init SearchEngine
$searchEngine = $modules->get('SearchEngine');
$searchEngine->initOnce(); // (for autoloader)

// construct a Query object ($q stands for the _sanitized_ query string)
$query = new \SearchEngine\Query($q);
$query->results = $pages->find('search_index#=' . $q . ', limit=25');

// render results
echo $searchEngine->renderResults([], $query);

That doesn't remove the need to sanitize the query, though. That's something you'd (currently) have to do yourself if you want to use the advanced text search operator to the full extent.

  • Like 1
Link to post
Share on other sites
  • 1 month later...

Hi all,

This is likely a really stupid question, but how do I go about enabling the result descriptions and text highlighting?

I have seen @teppo mention it and have seen theres methods in the codebase relating to it but for the life of me I can't figure out how to enable it.

My search results are only returning a page title and a url at present, which I figured was the out the box default but not 100% sure.

 

  • Like 1
Link to post
Share on other sites

Hey @cosmicsafari — not a bad question at all.

The description field comes from a module config setting. By default the module is set up to look for field called "summary", but you can change this to something else:

$config->SearchEngine = [
        'render_args' => [
            'result_summary_field' => 'summary',
        ],
];

My guess is that your pages don't have the summary field?

You can use some other field instead (if there's a suitable field), or you could let the module auto-generate the description by setting the summary field as "_auto_desc"... though please note that the support for auto-generated descriptions is experimental, and comes with one major gotcha: SearchEngine doesn't know which parts of your search index are "public knowledge", so it may end up displaying anything stored there. If you end up using this option, be sure to test it and make sure that you haven't indexed anything you don't want to show up in the search results 🙂

 

  • Like 1
Link to post
Share on other sites
On 5/10/2020 at 10:30 PM, teppo said:

$wire->addHookAfter('SearchEngine::savedPageIndex', function(HookEvent $event) { $page = $event->arguments[0]; if ($page->template == 'ContainerPage' && $page->children->count()) { $searchEngine = $event->modules->get('SearchEngine'); foreach ($page->children as $child) { $child_index = $searchEngine->indexPage($child, false, [ 'return' => 'index', ]); $page->search_index .= "\n" . $child_index[0]; } $page->save('search_index', [ 'quiet' => true, 'noHooks' => true, ]); } });

This really helped us @teppo thank you :) We're now displaying a link to the parent page when it finds the search term within the child block (sub page rendered on the parent page as a block).

 We set the auto description to _auto_desc to enable the search term to show highlighted within its surrounding text.

'result_summary_field' => '_auto_desc', 

This works perfectly when the search term is found on the parent page:
Result = Parent page title, parent page url and auto description found within the parent page

However, we're not seeing the auto description appear when it finds the search term within the child page block that is rendered on the parent page:
Expected Result  = Parent page title, parent page url and auto description found within the child page.

So while it is searching and finding the right content and displaying the correct page from this, it's just missing the contextual auto description from the content that it has searched, meaning that some results have a summary and others don't where it can't display the child page summary.

Apologies for the awful description here. Are you able to give us any pointers?

Thanks again

 

 

Link to post
Share on other sites

Hi @teppo is it possible to order the results by the number of times the search query is found so that more important pages are listed first?

For example, a page containing the search term 5 times would appear higher than a page with the search query found only once or twice?

Link to post
Share on other sites
  • 1 month later...

Good day, @teppo!

Thanks for this timesaver module of yours! Really cool!

The topic is getting long. Sorry if I missed the answer scanning through. Is it possible to search by page id? Probably that means adding it to the indexable field, right? Maybe there is a hook for that?

Link to post
Share on other sites

Hey @Ivan Gretsky — thanks 🙂

With latest version (just released, 0.27.0) you can make the module store page ID as an indexable field. This will allow the page to be found by its ID (since that is now included in the index) OR you can use syntax such as page.id:1234 to specifically search for a page with this ID (though this would likely also match page.id:12345 etc.) Of course id:1234 will also work, but may result even more false positives.

If you update to the latest version of the module, note that you also need to add id as an indexable field and rebuild your index before this will work.

  • Like 1
Link to post
Share on other sites
  • 4 weeks later...

@teppo

just some basic questions, no issues (yet) 😄

I have different pro-tables throughout the site and their content is distinct, thus should be indexed separately, is that at all possible? If not, we can stop right here.

I set it up like this:
Selected indexed fields: the table-field
Indexed templates: the table-field's parent-template
Select index field: search_index (already auto-added to the parent-template) 

sort and operator is unclear to me. I will need to look into adding custom operators using the aforementioned PW-doc on selectors.
However, how do I make the sort direction and the operator optional depending on the user input?

How can I select the subfields of the table (i.e. columns) to be indexed? Only when I check the "Index pages now?" option I can select those subfields. 

I indexed the table and/or the template, did a search and got a result. But since it is not pages that it's finding but say 6 matching rows on 1 page, it only shows 1 result instead of 6 and links to the parent page. Can I adjust the results to list the rows? I have the entire markup already (you may or may not have noticed, I've been trying to accomplish a search function without SearchEngine for some time now), any chance I can just use that markup somewhere?

That's all the questions I have so far.

Thanks for help, have a nice weekend.

Link to post
Share on other sites
17 hours ago, fruid said:

I have different pro-tables throughout the site and their content is distinct, thus should be indexed separately, is that at all possible? If not, we can stop right here.

I'm not sure I fully understand what you're trying to achieve here, but no — this is not something SearchEngine does. It creates a single index for the entire site.

I have considered adding support for multiple indexes, but a solid need for that never surfaced. Would be interesting to hear if there's one now, though admittedly this seems more like a need for a custom search engine, and thus likely has little to do with this module.

The TL;DR seems to be that "this is not the module you're looking for", but I'll go through your questions just to clarify things:

17 hours ago, fruid said:

sort and operator is unclear to me. I will need to look into adding custom operators using the aforementioned PW-doc on selectors.
However, how do I make the sort direction and the operator optional depending on the user input?

I'm not entirely sure what you mean by adding custom operators. If you're referring to selecting some operator that isn't available via SearchEngine settings, that's possible, but to be honest most of the selectors that make sense within site search queries are already there.

As always it's good to read the docs, though; understanding selectors and selector operators is a key factor when working with ProcessWire 🙂

Anyway, the settings you see in this screen are defaults. Most site search tools don't let visitors select operators or even alter the sort order in any way, so setting them once here is enough. If you're working on something more complex — perhaps something like library databases often provide — these settings have little to do with that. In this case you're probably not going to have much use for SearchEngine, although it's of course possible to create a custom search feature and use the search_index field as a regular field in your queries.

17 hours ago, fruid said:

How can I select the subfields of the table (i.e. columns) to be indexed? Only when I check the "Index pages now?" option I can select those subfields. 

At the moment you can't. These are not fields, they are indeed (literal) table columns. Search Engine doesn't work on that level. It'd be possible to add this feature, but at the moment this seems like an extremely rare need, and as such it's not particularly high priority.

The selector inputfield you see when you use "Index pages now" feature is only a tool for finding the pages you want to index, and it's using the core Inputfield Selector for selecting applicable pages.

17 hours ago, fruid said:

I indexed the table and/or the template, did a search and got a result. But since it is not pages that it's finding but say 6 matching rows on 1 page, it only shows 1 result instead of 6 and links to the parent page. Can I adjust the results to list the rows? I have the entire markup already (you may or may not have noticed, I've been trying to accomplish a search function without SearchEngine for some time now), any chance I can just use that markup somewhere?

The first part is not really in the scope of SearchEngine: it's a tool for providing a site search, and since sites consist of pages, that's what it will list. It does indeed sound like you're looking for something more customized.

... although you can in fact a) completely skip any front-end markup generated by SearchEngine and provide your own instead (basically just use the search_index as a regular field in queries), or b) modify markup generated by SearchEngine using hooks, or even c) hook into the indexing process to fill in other custom fields in addition to the default search_index field. All of these would require plenty of custom work, and I'm honestly not sure if you'd find SearchEngine useful for your needs; you may find it easier to build the entire thing from scratch.

  • Like 1
Link to post
Share on other sites
  • 1 month later...
17 minutes ago, adrian said:

Hi @teppo - just playing around with this for the first time and got this error when using the Debug Query feature. Not sure if it's an issue with the way you are calling the PageFinderClass?


PHP Notice: Undefined index: returnAllCols in .../public_html/wire/core/PageFinder.php:1460

Hey Adrian!

First things first: which version of ProcessWire is this on? Line number doesn't seem to match either current master or dev branch.

Error you're seeing looks easy to fix, but at least in current master or dev branch of the core PageFinder takes care of setting default value for this option. My guess would be that there's a version of the core that doesn't set the default value, but I'd like to make sure before applying a "fix", just in case 🙂

 

  • Like 1
Link to post
Share on other sites

Sorry about that - I was running 3.0.171. I just updated to 3.0.172 and still seeing the notice, but now it's on line 1493

  • Like 1
Link to post
Share on other sites

Thanks @adrian!

Came to the same realization here in the meantime: PageFinder::find() sets the default value, but PageFinder::getQuery() doesn't, which means that in this case I do indeed need to specifically define returnAllCols. I'll commit a fixed version soon.

I'm accessing this method directly for debug purposes (in order to display the generated SQL query in the Debugger class), and that's likely the only situation where one might run into this.

  • Like 1
Link to post
Share on other sites

Sorry if I'm being dumb, but when I went to set the 'form_action' argument, I assumed it needed to be an element within the 'render_args' array, but if I do that, it doesn't work. It does work if I set it at the top level of the args/options array that I am passing to the renderForm() method though.

If I dump $args without setting this, I see it is at the top level. Did you change something intentionally since writing the docs, or am I just confused ?

image.png.492b22ec9d16147f05bef267e488276a.png

Link to post
Share on other sites
20 minutes ago, adrian said:

Weird thing is that I don't think I have seen those errors in Tracy which uses getQuery() and getDebugQuery()

https://github.com/adrianbj/TracyDebugger/blob/116324264c3d566c1cece8c53cc3a1075ca82862/panels/DebugModePanel.php#L232-L239

That's a different context: it seems to me that Tracy hooks after PageFinder::getQuery() and uses the getQuery() method of returned DatabaseQuerySelect object. In my case I'm calling PageFinder::getQuery() instead. It's potentially a bit confusing that PageFinder::getQuery() returns an object that has getQuery() method, but these are actually two unrelated things... 🙂

Technically I could do the same thing in SearchEngine, but it feels "less straightforward" 🙂

  • Like 1
Link to post
Share on other sites

Sorry @teppo - another question / request for you - what do you think about being able to store theme folders in /site/templates/SearchEngine/themes to avoid any issues with overwriting during updates, and also so it's cleaner for version control?

  • Like 1
Link to post
Share on other sites
1 minute ago, teppo said:

It's potentially a bit confusing that PageFinder::getQuery() returns an object that has getQuery() method

Yeah, I agree - I was confused by that when I was setting that stuff up in Tracy.

  • Like 1
Link to post
Share on other sites
29 minutes ago, adrian said:

Sorry if I'm being dumb, but when I went to set the 'form_action' argument, I assumed it needed to be an element within the 'render_args' array, but if I do that, it doesn't work. It does work if I set it at the top level of the args/options array that I am passing to the renderForm() method though.

"form_action" is indeed part of the "render_args" array. The $args array that SearchEngine::renderForm() (actually implemented by SearchEngine\Renderer::renderForm()) takes in is also the "render_args" array from the parent scope — so this depends (or at least it should depend) on which context you're defining this argument:

  • If you're defining it via site config or by manually calling SearchEngine::setOptions(), you should put it within the render_options array item, which itself is an array ($config->searchEngine = ['render_options' => ['form_action' => '...']]).
  • If you're calling SearchEngine::renderForm() with an args array, you should pass it as a top level item ($searchEngine->renderForm(['form_action' => '...'])).

Please let me know if this description doesn't match actual behaviour though. That is how it should work.

I must admit that I manage to confuse myself with these arguments every now and then. While I thought this would be the best approach while building the module, right now I'm not so sure anymore — the basic idea here was that when you're specifically calling render* methods, you shouldn't have to worry about anything other than the render args.

Argument handling is one of the things I might still change a bit once I decide that the module is ready for a 1.0 release... 🙂

27 minutes ago, adrian said:

Sorry @teppo - another question / request for you - what do you think about being able to store theme folders in /site/templates/SearchEngine/themes to avoid any issues with overwriting during updates, and also so it's cleaner for version control?

Makes sense to me. Since theming support was initially added I've been using either the default theme as-is, or completely custom markup and styles, so the whole theme concept is somewhat "underdeveloped" at the moment.

I've added this on my backlog for now.

  • Like 1
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By FireWire
      Hello community!

      I want to share a new module I've been working on that I think could be a big boost for multi-language ProcessWire sites.

      Some background, I was looking for a way for our company website to be efficiently translated as working with human translators was pretty laborious and a lack of updating content created a divergence between languages. I, and several other devs here, have talked about translation integrations and have recognized the power that DeepL has. DeepL is an AI deep learning powered service that delivers translation quality beyond any automated service available. After access to the API was opened up to the US, I built Fluency, a DeepL translation integration for ProcessWire.
      Fluency brings automated translation to every multi-language field in the admin, and also provides a translation tool allowing the user to translate their text to any language without it being inside a template's field. With Fluency you can:
      Translate any plain textarea or text input Translate any CKEditor content (yes, with markup) Translate page names for fully localized URLs on every page Translate your in-template translation function wrapped strings Translate modules Fluency is free, and now so is DeepL
      Since this module was first built DeepL has introduced free Developer accounts that allow anyone to start using Fluency at zero cost and beginning with the version 0.3.0 release Fluency now supports free DeepL accounts. As of June 2021 DeepL supports translation to 26 languages and continues to offer more!
      Installation and usage is completely plug and play. Whether you're building a new multi-language site, need to update a site to multi-language, or simply want to stop manually translating a site and make any language a one-click deal, it could not be easier to do it. Fluency works by having you match the languages configured in ProcessWIre to DeepL's. You can have your site translating to any or all of the languages DeepL translates to in minutes (quite literally).
      Let's break out the screenshots...
      When the default language tab is shown, a message is displayed to let users know that translation is available. Clicking on each tab shows a link that says "Translate from English". Clicking it shows an animated overlay with the word "Translating..." cycling through each language and a light gradient shift. Have a CKEditor field? All good. Fluency will translated it and use DeepL's ability to translate text within HTML tags. CKEditor fields can be translated as easily and accurately as text/textarea fields.

      Repeaters and AJAX created fields also have translation enabled thanks to a JavaScript MutationObserver that searches for multi-language fields and adds translation as they're inserted into the DOM. If there's a multi-language field on the page, it will have translation added.

      Same goes for image description fields. Multi-language SEO friendly images are good to go.

      Creating a new page from one of your templates? Translate your title, and also translate your page name for native language URLs. (Not available for Russian, Chinese, or Japanese languages due to URL limitations). These can be changed in the "Settings" tab for any page as well so whether you're translating new pages or existing pages, you control the URLs everywhere.

      Language configuration pages are no different. Translate the names of your languages and search for both Site Translation Files (including all of your modules)

      Translate all of the static text in your templates as well. Notice that the placeholders are retained. DeepL is pretty good at recognizing and keeping non-translatable strings like that. If it is changed, it's easy to fix manually.

      Fluency adds a "Translate" item to the CMS header. When clicked this opens up a modal with a full translation tool that lets the user translate any language to any language. No need to leave the admin if you need to translate content from a secondary language back to the default ProcessWire language. There is also a button to get the current API usage statistics. DeepL account owners can set billing limitations via character count to control costs. This may help larger sites or sites being retrofitted keep an eye on their usage. Fluency can be used by users having roles given the fluency-translate permission.

      It couldn't be easier to add Fluency to your new or existing website. Simply add your API key and you're shown what languages are currently available for translation from/to as provided by DeepL. This list and all configuration options are taken live from the API so when DeepL releases new languages you can add them to your site without any work. No module updates, just an easy configuration. Just match the language you configured in ProcessWire to the DeepL language you want it to be associated with and you're done. Fluency also allows you to create a list of words/phrases that will not be translated which can prevent items such as brands and company names from being translated when they shouldn't

       
      Limitations:
      No "translate page" - Translating multiple fields can be done by clicking multiple translation links on multiple fields at once but engineering a "one click page translate" is not feasible from a user experience standpoint. The time it takes to translate one field can be a second or two, but cumulatively that may take much longer (CKEditor fields are slower than plain text fields). There may be a workaround in the future but it isn't currently on the roadmap. No "translate site" - Same thing goes for translating an entire website at once. It would be great, but it would be a very intense process and take a very (very) long time. There may be a workaround in the future but it isn't on the roadmap. No current support for Inline CKEditor fields - Handling for CKEditor on-demand hasn't been implemented yet, this is planned for a future release though and can be done. I just forgot about it because I've never really used that feature personally.. Alpha release - This module is in alpha. Releases should be stable and usable, but there may be edge case issues. Test the module thoroughly and please report any bugs via a Github issue on the repository or respond here. Please note that the browser plugin for Grammarly conflicts with Fluency (as it does with many web applications). To address this issue it is recommended that you disable Grammarly when using Fluency, or open the admin to edit pages in a private window where Grammarly may not be loaded. This is an issue that may not have a resolution as creating a workaround may not be possible. If you have insight as to how this may be solved please visit the Github page and file a bugfix ticket.
      Requirements:
      ProcessWire  3.0+ UIKit Admin Theme That's Fluency in a nutshell. A core effort in this module is to create it so that there is nothing DeepL related hard-coded in that would require updating it when DeepL offers new languages. I would like this to be a future-friendly module that doesn't require developer work to keep it up-to-date.
      The Module Is Free
      This is my first real module and I want to give it back to the community as thanks. This is the best CMS I've worked with (thank you Ryan & contributors) and a great community (thank you dear reader).
      DeepL Developer Accounts
      In addition to paid Pro Developer accounts, DeepL now offers no-cost free accounts. Now all ProcessWire developers and users can use Fluency at no cost.
      Learn more about free and paid accounts by visiting the DeepL website. Sign up for a Developer account, get an API key, and start using Fluency today.
      Download & Feedback
      Download the latest version here
      https://github.com/SkyLundy/Fluency-Translation/archive/main.zip
      Github repository:
      https://github.com/SkyLundy/Fluency-Translation
      File issues and feature requests here (your feedback and testing is greatly appreciated):
      https://github.com/SkyLundy/Fluency-Translation/issues
       
      Thank you! ¡Gracias! Ich danke Ihnen! Merci! Obrigado! Grazie! Dank u wel! Dziękuję! Спасибо! ありがとうございます! 谢谢你!

    • By monollonom
      (once again I was surprised to see a work of mine pop up in the newsletter, this time without even listing the module on PW modules website 😅. Thx @teppo !)
      Github: https://github.com/romaincazier/FieldtypeQRCode
      Modules directory: https://processwire.com/modules/fieldtype-qrcode/
      This is a simple module I made so a client could quickly grab a QR Code of the page's url in the admin.
      There's not much to it for now, but if need be you can output anything using a hook:
      $wire->addHookAfter("FieldtypeQRCode::getQRText", function($event) { $event->return = "Your custom text"; }) You can also output the QR code on your front-end by calling the field:
      echo $page->qr_code_field; The module uses the PHP library QR Code Generator by Kazuhiko Arase. When looking for a way to generate a QR Code in PW I came across @ryan's integration in his TFA module. I'm not very familiar with fieldtype/inputfield module development so I blindly followed @bernhard (great) tutorial and his BaseFieldtypeRuntime. At some point I'll take a deeper look to make a module on my own.
      Some ideas for improvements :
      add the ability to choose what to ouput : page's url / editUrl / file(s) / image(s) / ... allow to output multiple QR codes ?
    • By Chris Bennett
      https://github.com/chrisbennett-Bene/AdminThemeTweaker
      Inspired by @bernhard's excellent work on the new customisable LESS CSS getting rolled into the core soon, I thought I would offer up the module for beta testing, if it is of interest to anyone.

      It takes a different approach to admin styling, basically using the Cascade part of CSS to over-ride default UiKit values.
      Values are stored in ModuleConfig Module creates a separate AdminThemeTweaker Folder at root, so it can link to AdminThemeTweaker.php as CSS AdminThemeTweaker.php reads the module values, constructs the CSS variables then includes the CSS framework Can be switched on and off with a click. Uninstall removes everything, thanks to bernhard's wonderful remove dir & contents function.
      It won't touch your core. It won't care if stuff is upgraded. You won't need to compile anything and you don't need to touch CSS unless you want to.

      It won't do much at all apart from read some values from your module config, work out the right CSS variables to use (auto contrast based on selected backgrounds) and throw it on your screen.
      You can configure a lot of stuff, leave it as it comes (dark and curvy), change two main colors (background and content background) or delve deep to configure custom margins, height of mastheads, and all manner of silly stuff I never use.

      Have been developing it for somewhere around 2 years now. It has been (and will continue to be) constantly tweaked over that time, as I click on something and find something else to do.
      That said, it is pretty solid and has been in constant use as my sole Admin styling option for all of those 2 years.

      If nothing else, it would be great if it can provide any assistance to @bernhard or other contributor's who may be looking to solve some of the quirkier UiKit behavior.
      Has (in my opinion) more robust and predictable handling of hidden Inputfields, data-colwidths and showIf wrappers.
      I am very keen to help out with that stuff in any way I can, though LESS (and any css frameworks/tools basically) are not my go.
      I love CSS variables and banging-rocks-together, no-dependency CSS you can write with notepad.



       

    • By opalepatrick
      I see old posts saying that repeaters are not the way to go in Custom Process Modules. If that is the case, when using forms (as I am trying to do) how would one tackle things like repeat contact fields where there can be multiple requirements for contact details with different parameters? (Like point of contact, director, etc) or even telephone numbers that have different uses?
      Just for background I am creating a process module that allows me to create types of financial applications in the admin area (no need to publish any of this, pure admin) that require a lot of personal or company information.
      Maybe I am thinking about this incorrectly?
    • By HMCB
      I ran across a reference to IftRunner module. The post was 6 years ago. I cant find it in available modules. Has it been pulled?
×
×
  • Create New...