Jump to content

PW 3.0.160 – New search abilities


ryan
 Share

Recommended Posts

In ProcessWire 3.0.160 we’ve got some major upgrades and additions to our text-matching selectors and operators. This brings a whole new level of power to $pages->find() and similar API calls, especially when it comes to search engine type queries. This blog post also includes a demo search engine that lets you test everything out live—

https://processwire.com/blog/posts/pw-3.0.160/

  • Like 18
  • Thanks 2
Link to comment
Share on other sites

Wow, that's a motherload of new search operators. Awesome!

I usually want to include as many results as possible to so I tend use the LIKE operator for text searches. To bring in even more results it would be cool to use query expansion too, but I noticed that there isn't an operator that combines those two. Is that combination not possible for technical reasons?

  • Like 1
Link to comment
Share on other sites

@Robin S That's correct that the combination is not possible for technical reasons. The query expansion is a feature of MySQL fulltext indexes in a standard natural language match/against query, so it's not something that can be used with LIKE (which does not use fulltext index), and also can't be used with match/against boolean mode (which is how partial word matches can use fulltext). So there are only certain operators we can use query expansion with.

My experience so far is that query expansion is pretty inconsistent as to when it's useful. But since it's something that MySQL supports I thought PW should provide the option. If a search returns no results, query expansion isn't going to help, because it needs there to be matches in order to expand upon them. I'm currently thinking I might use it on one or two word searches that might otherwise only return a small number of results. If your goal is to bring in as many results as possible I would start experimenting with the "Contains match and expand" **+= operator, which uses the standard MySQL match/against logic, and doesn't require all search words to be present. Though depending on the search, once you go past one or two words with query expansion, it tends to add noise to the results. So I've been thinking a few of these new operators might be made configurable to automatically substitute another operator under certain conditions, like to a non-expand version depending on word count, or automatically substitute another operator when one produces no results. 

  • Like 3
Link to comment
Share on other sites

7 hours ago, ryan said:

So I've been thinking a few of these new operators might be made configurable to automatically substitute another operator under certain conditions, like to a non-expand version depending on word count, or automatically substitute another operator when one produces no results. 

That sounds like a powerful feature for sure. Thanks four these nice additions anyway.

Link to comment
Share on other sites

  • 10 months later...

@ryan - just playing around with some of the new options here and they are really awesome, thank you!

I was initially confused about how to use this example:

  • +(foo bar) or +"foo bar" indicates the phrase "foo bar" MUST appear

My experiments showed this. Unless I am missing something, to use the double quote option, you need to have two sets, with the inner set escaped. This is OK, but I feel like perhaps it could be better documented.

image.thumb.png.afbbec2a85a69e448923931fe404293b.png

 

The other thing I have noted is a feature request to add support for excluding stopwords when doing partial matches. For example, "University of Queensland" is great when you're searching for the phrase, but if you are using multiple operators at a time, eg: *=~=~|= you really don't want that last operator (~|=) to match the "of" if the other words don't exist. Is there a way to do this already, or is it something that you could support fairly easily? Or, would you recommend that we do this at the server level using MySQL's "ft_stopword_file" setting?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...