module SearchEngine

BrendonKoz · December 5, 2022

Hi Clarity,

Can you clarify what you mean by, "modify the query during search," perhaps with an example of what you are looking for?

SearchEngine does provide the ability for you to not use its default ProcessWire searches, and instead simply prepare the index and form/results for you, so that you can form the query manually on your own depending on your own needs, as shown in the module page's documentation. Is that what you mean? If not, an example would be great. 🙂 Some of adrian's examples within this topic may be of use too.

Clarity · December 12, 2022

Hello, @BrendonKoz!

I need to change the erroneous keyboard layout after user performs the search. For example, I wrote 'ghbdtn' and that's supposed to be 'привет' (hello) on Russian. The code shown by the link is not similar to what I have. I'm using the following code ($searchEngine is a variable which contains the module):

<head>
  ...
	<?= $searchEngine->renderStyles() ?>
	<?= $searchEngine->renderScripts() ?>
  ...
</head>
<body>
  ...
	<?= $searchEngine->renderForm() ?>
  ...
  	<?= $searchEngine->renderResults() ?>
  ...
</body>

BrendonKoz · December 12, 2022

It seems like you could take advantage of the part of the module's documentation that I linked to above, something like the following:

<?php namespace ProcessWire;
$searchEngine = $modules->get('SearchEngine');
...
<head>
    <?= $searchEngine->renderStyles() ?>
    <?= $searchEngine->renderScripts() ?>
</head>
<body>
<?php
  	echo $searchEngine->renderForm();
	// This assumes your search form uses GET, and a field named 'q'
	if ($q = $sanitizer->selectorValue($input->get->q)) {
      	// Modify your value of the search query ($q) here...
      	// ...

        // This finds pages matching the query string and returns them as a PageArray:
        $results = $pages->find('search_index%=' . $q . ', limit=25');

        // Render results and pager with PageArray::render() and PageArray::renderPager():
        echo $results->render(); // PageArray::render()
        echo $results->renderPager(); // PageArray::renderPager()

        // ... or you iterate over the results and render them manually:
        // echo "<ul>";
        // foreach ($results as $result) {
        //     echo "<li><a href='{$result->url}'>{$result->title}</a></li>";
        // }
        // echo "</ul>";
    }
?>

The biggest benefit to the SearchEngine module is its creation of the (default) `search_index` field. You can query against that field just like any other field within ProcessWire. Rendering the form and search results are just additional benefits.

Clarity · December 14, 2022

On 12/12/2022 at 7:52 PM, BrendonKoz said:

It seems like you could take advantage of the part of the module's documentation that I linked to above, something like the following:

<?php namespace ProcessWire;
$searchEngine = $modules->get('SearchEngine');
...
<head>
    <?= $searchEngine->renderStyles() ?>
    <?= $searchEngine->renderScripts() ?>
</head>
<body>
<?php
  	echo $searchEngine->renderForm();
	// This assumes your search forum uses GET, and a field named 'q'
	if ($q = $sanitizer->selectorValue($input->get->q)) {
      	// Modify your value of the search query ($q) here...
      	// ...

        // This finds pages matching the query string and returns them as a PageArray:
        $results = $pages->find('search_index%=' . $q . ', limit=25');

        // Render results and pager with PageArray::render() and PageArray::renderPager():
        echo $results->render(); // PageArray::render()
        echo $results->renderPager(); // PageArray::renderPager()

        // ... or you iterate over the results and render them manually:
        // echo "<ul>";
        // foreach ($results as $result) {
        //     echo "<li><a href='{$result->url}'>{$result->title}</a></li>";
        // }
        // echo "</ul>";
    }
?>

The biggest benefit to the SearchEngine module is its creation of the (default) `search_index` field. You can query against that field just like any other field within ProcessWire. Rendering the form and search results are just additional benefits.

Thank you! I already did it. My code was the following (in search view):

<?php if($q = $sanitizer->selectorValue($input->get('q'))): ?>
    <?php if($pages->find('search_index%=' . $q)->count): ?>
   		<?= $searchEngine->renderResults() ?>
    <?php else: ?>
    	<?= $searchEngine->renderResults([], $searchEngine->find(Utils::handleSearch($q))) ?>
    <?php endif; ?>
<?php endif; ?>

I check first if there are search results in normal keyboard layout and then invert keyboard layout using Utils::handleSearch() method if there aren't any results.

gebeer · March 13

Hi @teppo and thank you for this module. I am evaluating Whether we can use this for a bigger project. The project makes heavy use of Repeater Matrix fields. And those seem not to be supported atm. Could you make the Indexer::indexPage method hookable? That way we could implement our own code for non supported fieldtypes. That would be awesome.

BrendonKoz · March 13

Hi @gebeer! I'm currently using RepeaterMatrix with SearchEngine successfully, though I'm not using any non-native PW FieldTypes. I did run into an issue where RepeaterMatrix-based fields, if customized by a template to not include one of it's available fields would cause an error and not complete the index. I have a pull request pending for the repository which, in my testing, seems to have fixed the problem.

Although I suspect this might not be your issue, I wanted to share on the small chance that it does help.

gebeer · March 13

40 minutes ago, BrendonKoz said:

Hi @gebeer! I'm currently using RepeaterMatrix with SearchEngine successfully, though I'm not using any non-native PW FieldTypes. I did run into an issue where RepeaterMatrix-based fields, if customized by a template to not include one of it's available fields would cause an error and not complete the index. I have a pull request pending for the repository which, in my testing, seems to have fixed the problem.

Although I suspect this might not be your issue, I wanted to share on the small chance that it does help.

Oh great, thank you. So this means that Repeater Matrix fields are supported out of the box or do you need to tell the module how to process them? If there is custom code/hooks involved would you mind sharing a snippet? 🙂

EDIT: I just saw that Indexer::getFieldIndex has logic for repeater fieldtypes. So no need for a snippet.

gebeer · March 13

3 hours ago, gebeer said:

Hi @teppo and thank you for this module. I am evaluating Whether we can use this for a bigger project. The project makes heavy use of Repeater Matrix fields. And those seem not to be supported atm. Could you make the Indexer::indexPage method hookable? That way we could implement our own code for non supported fieldtypes. That would be awesome.

Hi @teppo after checking your module code more carefully I found the already hookable methods ___getPageIndex and ___getFieldIndex. So please ignore my post above 🙂

snck · Monday at 03:28 PM

On 10/5/2020 at 2:29 PM, sambadave said:

Hi @teppo is it possible to order the results by the number of times the search query is found so that more important pages are listed first?

For example, a page containing the search term 5 times would appear higher than a page with the search query found only once or twice?

Hi @teppo, as I could not find an answer to this old question: Is there a way to sort the results by number of matches / occurences? That would be awesome!

BrendonKoz · Monday at 03:57 PM

Hi @snck! Although teppo might have a different answer, I suspect it'll be similar to this.

The SearchEngine module simply makes it dead simple to add standard search functionality into ProcessWire without handling it all manually yourself (i.e.: properly parsing/escaping fields, extrapolating searchable text from files [with the SearchEngine FileIndexer add-on module], and figuring out how to generate a search result list). Beyond that, it still uses ProcessWire's own search functionality; it doesn't expand upon it. ProcessWire can do some pretty significant things in search, but overall it still relies on MySQL's fulltext search to handle everything. MySQL can offer some level of relevancy (depending on the PW selector search you choose), but it can't, as far as I know, order by number of matches found.

Relevancy is not (necessasrily/typically) the same as number of matches (per matched database record). For anything outside of MySQL's default capabilities, something external would likely need to be integrated, such as Apache Lucene or ElasticSearch.

snck · Monday at 06:08 PM

2 hours ago, BrendonKoz said:

Relevancy is not (necessasrily/typically) the same as number of matches (per matched database record). For anything outside of MySQL's default capabilities, something external would likely need to be integrated, such as Apache Lucene or ElasticSearch.

Thanks for your reply! You're right that the number of matches is not necessarily an indicator for relevancy. As I stumbled over the quoted question I was just curious whether there was a selector or option that would allow me to quickly try it out because I have a project that could benefit from it. SearchEngine is a great module nonetheless. 🙂

teppo · Friday at 04:02 AM

On 5/22/2023 at 6:57 PM, BrendonKoz said:

Beyond that, it still uses ProcessWire's own search functionality; it doesn't expand upon it. ProcessWire can do some pretty significant things in search, but overall it still relies on MySQL's fulltext search to handle everything. MySQL can offer some level of relevancy (depending on the PW selector search you choose), but it can't, as far as I know, order by number of matches found.

This is true, with a couple of small twists:

SearchEngine supports "pinning" specific template(s) to the top of the list, or alternatively grouping results by template. These require making slight modifications (adding extra rules) to the query (DatabaseQuerySelect object) generated by ProcessWire.
In the dev branch of the module there is a work in progress "sort by relevance" feature, which also modifies the query. This is based on MySQL natural language full-text search, so it's still up to the database to decide how relevant each result really is.

Sorting results by number of matches, giving some fields more "weight" than others, etc. are not currently something that this module does, though I have occasionally considered if they should be. The main issue here is that it would require different storage and search mechanisms, so it's a lot of work, and additionally it would raise a few rather complicated issues (e.g. handling permissions, which is something that we currently get "for free" by relying on selectors.)

Not sure how sensible that would be, all things considered. It might make more sense to use SE to feed data to a separate search tool, or ditch SE altogether for that sort of use case 🙂

Sign In

module SearchEngine

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Similar Content