Jump to content
bernhard

RockFinder3 - Combine the power of ProcessWire selectors and SQL

Recommended Posts

img

Combine the power of ProcessWire selectors and SQL

img

Differences to previous RockFinder modules

  • RockFinder3 comes with extensive docs 😉 
  • RF3 supports chaining: $RockFinder3->find("template=foo")->addColumns(['foo']).
  • RF3 fully supports multi-language.
  • RF3 makes it super-easy to add custom columnTypes.
  • RF3 makes it easier to use custom SQL statements.
  • No bloat! The module does just do one thing: Finding data.

 

This module is built to stay, so I'm quite sure there will be no need for RockFinder4. The concepts are proven and used in many of my modules. RockFinder3 has seen some major refactoring and reduces LOC compared to RF2 by more than 50% from 1220 to 560 😎 I love the new API and if there where needs for changes the API should stay untouched meaning no need for a breaking version update in the future.

DOCS & DOWNLOAD: https://github.com/baumrock/rockfinder3

I'll do some more testing in the near future and add it to the modules directory once it proves as stable as I expect it to be.

Thank you

...for using RockFinder3. If you find RockFinder3 helpful consider giving it a star on github or saying thank you. I'm also always happy to get feedback in the PW forum!

Happy finding 🙂

  • Like 15

Share this post


Link to post
Share on other sites

Hey @bernhard - looking awesome - I think it's time I took this for a proper test spin 🙂

Just tested my first query and got this error:

image.png.b4c10064683a76ccfaf1d89e2273c6e3.png

  • Like 1

Share this post


Link to post
Share on other sites

Thx @adrian does the error disappear when you remove the "created" column?

Share this post


Link to post
Share on other sites
1 minute ago, bernhard said:

Thx @adrian does the error disappear when you remove the "created" column?

Well, it's not just the "created" field - the same error shows with the "name" field as well.

Loving some of the options this has BTW - having those docs has really shown me what this can do!

Just a thought about the Process module and the Tracy Console - I am not sure it's clear that the Process module is only needed to load Tabulator - otherwise loading the Console panel from any other page works for dumping the objects in the normal Tracy way.

I do think it would be great to figure out a way to load the Tablulator script on demand in Tracy if a RF dump() is called.

  • Thanks 1

Share this post


Link to post
Share on other sites

Please try the latest commit 🙂 

  • Like 1

Share this post


Link to post
Share on other sites
2 minutes ago, adrian said:

Just a thought about the Process module and the Tracy Console - I am not sure it's clear that the Process module is only needed to load Tabulator - otherwise loading the Console panel from any other page works for dumping the objects in the normal Tracy way.

True. Do I state somewhere that you need to use the processmodule for dumping?

Share this post


Link to post
Share on other sites
2 minutes ago, bernhard said:

Please try the latest commit 🙂 

That fixed it - thanks!

  • Like 1

Share this post


Link to post
Share on other sites

Thx @adrian works a treat 😎 I have slightly modified it so that it checks if Tabulator exists before loading it.

Seems you have some code prettifier active: https://github.com/baumrock/RockFinder3/commit/2863b76e9109673d7095e08d5d0d9b6ad5193f3c What are you using?

1 hour ago, adrian said:

@bernhard - you might be interested in this: https://github.com/processwire/processwire-issues/issues/1173#issuecomment-632868896 and Ryan's reply. Once you upgrade to the latest PW dev and use your dumpSQL() method you'll see what I mean 🙂

I've followed this issue and was a little afraid something would break anything in rockfinder, but I did the upgrade just before posting this topic and everything seemed to work. If you are referring to the screenshots containing "old" values instead of bind variables that's just because I made them before upgrading the core. Or is there anything I missed out or need to check again?

Share this post


Link to post
Share on other sites
2 minutes ago, bernhard said:

Seems you have some code prettifier active: https://github.com/baumrock/RockFinder3/commit/2863b76e9109673d7095e08d5d0d9b6ad5193f3c What are you using?

Just have VSCode set up to automatically remove trailing spaces at the end of lines. It's a pet peeve of mine and something I wish Ryan would do with the PW core.

3 minutes ago, bernhard said:

Or is there anything I missed out or need to check again?

Nothing new - it's just a shame that we can no longer see the values in query - makes it impossible to easily copy / paste a query into Adminer etc to test directly.

Are you going to completely remove the Process module now? Does it serve some other purpose?

Share this post


Link to post
Share on other sites
4 minutes ago, adrian said:

Are you going to completely remove the Process module now? Does it serve some other purpose?

That just happened with the latest commit 🙂 Thanks!

4 minutes ago, adrian said:

makes it impossible to easily copy / paste a query into Adminer etc to test directly.

Didn't think of that and that would have really made the development difficult! I'll post a comment in the issue 🙂 

5 minutes ago, adrian said:

Just have VSCode set up to automatically remove trailing spaces at the end of lines. It's a pet peeve of mine and something I wish Ryan would do with the PW core.

Thx for clarifying. I also don't like them, but it seems that Ryan likes them, because we have this setting in /wire/.editorconfig

trim_trailing_whitespace = false

Thx, just enabled the setting in VSCode as well 🙂 

  • Like 1

Share this post


Link to post
Share on other sites
13 minutes ago, bernhard said:

I also don't like them, but it seems that Ryan likes them, because we have this setting in /wire/.editorconfig

Assume you have read these:

https://github.com/ryancramerdesign/ProcessWire/issues/710
https://github.com/ryancramerdesign/ProcessWire/issues/1757

He says that "whitespace is preferred in many cases", but I just don't get it - not sure how trailing whitespace has any advantage, but maybe I am missing something 🙂

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites
4 minutes ago, bernhard said:

That just happened with the latest commit 🙂 Thanks!

Remember you have some wording in the docs that needs changing, eg "The ProcessModule of RockFinder3 does use Tracy for dumping the results, so TracyDebugger is required for the ProcessModule to run."

  • Thanks 1

Share this post


Link to post
Share on other sites

Thx, is removed 🙂 

Share this post


Link to post
Share on other sites
18 minutes ago, bernhard said:

Didn't think of that and that would have really made the development difficult! I'll post a comment in the issue

Yeah, please do - I think it's going to prove quite painful if we can't generate queries that work independently. I make use of the "Selector Queries" feature on the RequestInfo panel in Tracy a LOT and it's no longer as useful. Sounds like Ryan is keen to come up with a fix, but an extra voice might help motivate him 🙂

  • Like 1

Share this post


Link to post
Share on other sites

Done 🙂 Just realized that this update actually breaks a lot of the functionality provided by RockFinder3 - so it's critical to get an update for this! 😐 

  • Like 1

Share this post


Link to post
Share on other sites
Just now, bernhard said:

this update actually breaks a lot of the functionality provided by RockFinder3

Yeah, I just saw your post - that's too bad - hopefully there is a solution!

Share this post


Link to post
Share on other sites

Hey @bernhard - attached is a new version that loads Tabulator if someone wants to make a dump() call directly to the page within their template file.

Basically it just duplicates the JSloader functionality within this module.

I was also thinking about maybe making a barDump() version that would emulate bd() but with Tabulator output, but didn't get there yet.

I decided to take a look at adding a barDump() option. To test you'll need the new Rockfinder and TD.php files attached.

Now you can call the new barDump() method from anywhere (including template files, and the Console panel) and Tracy Dumps panel will receive the tabulated output, or the dumped object if you pass "true" for the title.

Please test thoroughly and feel free to tweak behavior as needed.

I should note that I changed the return for the dump method from $this to $out. I guess this will break chaining if someone wants to add another method after this. Maybe there is a better way to do this but was in a hurry.

 

RockFinder3.module.php TD.php

  • Like 1

Share this post


Link to post
Share on other sites

Hi @bernhard love your work.

Is RF2 upgradeable to RF3 or is RF3 a completely different module?

  • Thanks 1

Share this post


Link to post
Share on other sites

Thx @psy

It depends on how you have used it I guess. But you can definitely run both beside each other. That's why they have separate names so that I can use them together in my projects without the need of refactoring existing code. In the long run RockFinder3 should be the only module to stay. Meaning that if you build a new project you should definitely use RF3 only and if you find anything that does not work yet with RF3 just give me a ping and I'll implement it. Most of that should be easy and maybe just copy&pasting methods from RF2 to RF3 and doing some refactoring.

So I guess the answer is: It is a completely different module 🙂 

  • Like 2

Share this post


Link to post
Share on other sites
9 hours ago, adrian said:

Now you can call the new barDump() method from anywhere (including template files, and the Console panel) and Tracy Dumps panel will receive the tabulated output, or the dumped object if you pass "true" for the title.

Please test thoroughly and feel free to tweak behavior as needed.

I should note that I changed the return for the dump method from $this to $out. I guess this will break chaining if someone wants to add another method after this. Maybe there is a better way to do this but was in a hurry.

Great idea 🙂 I've implemented your changes on the bardump branch: https://github.com/baumrock/RockFinder3/commit/436257b8822fd89e76951db1521c8f2619ef880c

Once you have your code in Tracy please tell me and I can merge to master 🙂 

The new section of the readme:

 

Dumping data

For small finders Tracy's dump() feature is enough, but if you have more complex finders or you have thousands of pages this might get really inconvenient. That's why RockFinder3 ships with a custom dump() method that works in the tracy console and turns the result of the finder into a paginated table (using tabulator.info).

For all the dumping methods you can provide two parameters:

  1. The title of the dump*
  2. The settings for the rendered tabulator table

*Note that if you set the title to TRUE the method will not only dump the tabulator but also the current RockFinder3 object (see the next example).

dump() or d()

$RockFinder3
  ->find("template=cat")
  ->addColumns(['title', 'created'])
  ->dump(true);

img

barDump() or bd()

For situations where you are not working in the console but maybe in a template file or a module the barDump() method might be useful.

img

Dumping the SQL of the finder

To understand what is going on it is important to know the SQL that is executed. You can easily dump the SQL query via the dumpSQL() or barDumpSQL() methods. This even supports chaining:

$RockFinder3
  ->find("template=cat")
  ->addColumns(['title'])
  ->dumpSQL()
  ->addColumns(['owner'])
  ->dumpSQL()
  ->dump();

img

  • Like 2

Share this post


Link to post
Share on other sites

v1.0.3 adds two new features:

  • addPath()
  • each()

Callbacks

RockFinder3 supports row callbacks that are executed on each row of the result. Usage is simple:

each()

$RockFinder3
  ->find("template=cat")
  ->addColumns(['title', 'weight'])
  ->each(function($row) { $row->myTitle = "{$row->title} ({$row->weight} kg)"; })
  ->dump();

img

These callbacks can be a great option, but keep in mind that they can also be very resource intensive! That applies even more when you request page objects from within your callback (meaning there will be no benefit at all in using RockFinder compared to a regular $pages->find() call).

addPath()

A special implementation of the each() method is the addPath() method that will add a path column to your result showing the path of every page. This will not load all pages into memory though, because it uses the $pages->getPath() method internally.

$RockFinder3
  ->find("template=cat")
  ->addColumns(['title', 'weight'])
  ->addPath("de")
  ->dump();

img

If you need the path for linking/redirecting from your data to the pages it might be better to build a custom redirect page that works with the page id, so you don't need the overhead of getting all page paths:

<a href='/your/redirect/url/?id=123'>Open Page 123</a>

If you really need to access page objects you can get them via the $finder parameter of the callback:

$finder->each(function($row, $finder) {
  $row->foo = $finder->pages->get($row->id)->foo;
}
  • Like 5

Share this post


Link to post
Share on other sites
Posted (edited)

I have the same question than @psy but about RockFinder 1 😂

I think I have already my answer..

 

Anyway, thanks version 3 @bernhard 👍🏼

Edited by flydev 👊🏻
wrong ppl tag

Share this post


Link to post
Share on other sites
7 minutes ago, flydev 👊🏻 said:

but about RockFinder 1 😂

FockFinder1 had a totally different implementation (not based on DatabaseQuerySelect) and also a very different API, so there will be lots of differences. I think the concept of joins and relations should be very flexible though. And it's now also a lot easier to create custom column types that basically let you join whatever data you need. So I think RF3 should really be an improvement in all areas.

If you find any use cases that RF3 does not support, please let me know 🙂

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...