bernhard

OK finally I came back to this one and I found the problem. See these Tracy Console codes: Works: $p = $pages->get(1002); $p->of(false); $r = $p->repeater; $item = $r->getNew(); $item->title = 'created via api @ ' . date('YmdHis'); $item->save(); $p->repeater->add($item); $p->save(); d($p->repeater->count); Does NOT work: $p = $pages->get(1002); $p->of(false); $r = $p->repeater; $item = $r->getNew(); $item->title = 'created via api @ ' . date('YmdHis'); $item->save(); $r->add($item); // <<------------ here is the change $p->save(); d($p->repeater->count); Saying "does not work" I mean, that a d($pages->find('repeater.count>0')->each('path')); does NOT list a page with a repeater item added via the second code example, whereas it shows pages where I used the first (upper) example. Looking at the database it becomes obvious why: For the page with id 1002 there is no data and no count value present. @Robin S you should be able to reproduce it now. I've tested it on a fresh 3.0.99 dev version with the default profile and just added a repeater to the basic-page template. I'm not sure if this is a bug or I just have to use the api correctly... But at least I think it is an odd behaviour since it looks like everything is working correctly also with the bad code and you might not notice that you are in trouble... Items are available and even published: Here is the github issue: https://github.com/processwire/processwire-issues/issues/413

Same here, no time to test it right now, but what do you think of adding a field just like the exclude field but for manual INCLUDEs? Should be quite easy to implement, just pass the content of the field to a $files->find($dir) and scan that files. You are absolutely right that scanning the caches and compiled files is nonsense. But I think it's not a good solution to have a checkbox for the assets folder and then you have to exclude all unnecessary files... in my case I could just add /site/assets/RockGrid to my field and would have the todos show up in the tracy bar

Sorry, please try to rename your module's folder to "RockFinder" instead of "RockSqlFinder"

@adrian could you please add an option for the todo-panel to also scan /site/assets ? I always put my files for field configs there (runtime markup, rockgrid etc) so it would be great to have that files scanned as well. Thank you

@flydev please use dev3, it's a complete rewrite!

I get your point and it's not easy to formulate my answer properly. I'll come back to you.. I wouldn't see github as a place for grabbing software and removing brandings If they don't accept it, well, there are several other options: develop something on your own use formbuilder and support ryan find another client ( ) contribute something to this module (like you already did @gebeer ) sponsor new features for this module ... Please don't get me wrong - I just don't want to get too serious about that topic We also have those discussions for the pw Pro Modules from time to time... So you would prefer a commercial module? Don't get that point... Just contact me and suggest a proper price for removing that branding - and I'm not talking about thousands of euros here. Some contributions to the community would totally be fine BTW: I've started this experiment in my signature quite some time ago. Guess how many donations and likes I got until then? One donation, zero likes...

Of course it works. getForpage() returns the page where the repeater item lives. your code snippet was not clear to me, so you might use it differently, but using getForpage() is the key. You didn't provide any information of what your code does, which variables are what kind of data (pages, repeater items) etc.; Also you didn't provide information what you did exactly (where you placed my code) and what did not work. "doesn't work" is not a lot of information... Yes, that's correct and totally makes sense, since it is a repeater item and not a page with a parent page.

@Robin S I'm not an expert either and I will not pay a lawyer for consultancy on a module that I released for free... If anybody wants to make a fork with a removed branding: Go for it (bad karma for free ). As long as somebody uses MY version of the module (meaning he can also get all future updates), I don't think it is a problem to limit the MIT license by this detail. At least that is how it is meant... BTW: There was quite a similar discussion about that regarding pw itself long time ago:

Thanks for that one! Edit: Still seems mollie.com is better for my needs. But don't want to get offtopic...

~488 websites listed here using ProCache: https://nerdydata.com/search?query=%2Fsite%2Fassets%2Fpwpc%2Fpwpc

thx @szabesz, cheating like that would definitely be an option (and also not too hard in that case). just thought there might be some magical pw-way of doing that

$exhibitions->append($exhibition->getForpage());

Hey @adrian, glad you gave it a try 1) No, I guess this was just for development. PR welcome (really need to work on other projects atm, see rockgrid and rockfinder for example and some client work). 2) Everybody who wants to (or needs to) remove the branding can contact me via PM 3) I agree. I've used Nette Live Validation in an older project and this looked a lot better, but I've just not had the time to implement it. This might be more complex when thinking about supporting different css frameworks. Or it might be as easy as loading this script: https://github.com/contributte/live-form-validation. I don't know and I wanted to go with the default tools as I'm short on resources for this module. And the most important part was that it works and is quick to setup and save, of course If you come up with a better solution I'm happy to merge it, but at the moment, I'm sorry, I have other priorities.

Can you be more precise please? What kind of calculations are you talking about? I'll try an answer with a guess This is a quick test i did with 10.000 fake invoices. The result is great (but see later). Here the setup for the Tracy Console: $selector = 'parent=/invoices, limit=2'; // the limit is for demo of the sql $finder = new RockFinder($selector, ['value', 'date']); $finder->sort = false; $sql = $finder->getSQL(); d($sql, [6,999]); This finds all invoices and returns their "date" (a datetime field) and their "value" (random number between 1 and 100). The resulting SQL is this: SELECT `pages`.`id` AS `id`, `value`.`value` AS `value`, `date`.`date` AS `date` FROM `pages` /* --- join value --- */ LEFT JOIN (SELECT `pages_id` AS `pageid`, `value`.`data` AS `value` FROM `field_value` AS `value`) AS `value` ON `value`.`pageid` = `pages`.`id` /* --- end value --- */ /* --- join date --- */ LEFT JOIN (SELECT `pages_id` AS `pageid`, `date`.`data` AS `date` FROM `field_date` AS `date`) AS `date` ON `date`.`pageid` = `pages`.`id` /* --- end date --- */ WHERE `pages`.`id` IN (42063,42064) Now the funny part: You can do all kinds of custom SQL with that returned SQL statement, for example grouping the invoices by month and building a sum of their values: $results = $this->database->query("SELECT sum(value) as monthsum, DATE_FORMAT(date, '%Y-%m') as date FROM ($sql) AS rockfinder group by date"); And here the absolutely awesome result: So you get aggregated data for 12 months in no-time (166ms!!) with some VERY basic SQL. I'll think about how this can be made even easier in the future. Maybe by providing a method that does the query and returns the result. I'm open to suggestions. The maybe greatest thing about that is that you do not need to take care about processwire page statuses. That has always been a real pain when working with SQL because it makes things so complex. You always need to think about adding WHERE pages.status < ... and so on. That sucks. Now you just build your custom "view" with RockFinder and do some simple SQL on that result. You get the idea? Edit: I put $finder->getSQL() inside the timer, now it took 280ms - still great IMHO. And see how easy it is to add an invoice-count for that month for example

I've setup a gallery field on one site. I forgot to set a maxSize for those images, so the gallery images have >5MB per Image, which is of course not ideal. I changed the settings now to a max of 1024x1024 px - but the images are still in max resolution on the server. On the frontend gallery I changed the links to $img->maxSize(1024,1024) so the users will get the resized images, but I would like to save some space on the server and recreate (resize) the already uploaded images to fulfill this maxsize restriction. Any ideas how this can be done easily? Thanks

I guess Tracy is not loaded when your init fires. You can try to attach your hook in ready() instead of init(). Sidenote: You should also be able to use the d() shortcut method without the Tracy class

The "option" to sort on the server side is of course always there. But I set it to use sorting by default now. I think the worse performance will not matter in most of the cases. If it does, the user will realise it and can easily turn it off. It would be worse to think everything works as expected and then get some unexpected results due to sorting turned off while you think it should actually return a sorted resultset ( like from a selector containing the sort keyword ). Thanks for your feedback.

Sorry guys for all those posts... Found the performance-killer: It is the ORDER BY field(`pages`.`id`, 52066,52067,52068,52069,52070 ... ) part. Without retaining the sort order of the pages->findIDs it is a LOT faster (4s without sort compared to 60s with sort and 75 using findMany): 91300 items: rockfinder = 4385.5ms | findmany = 74213.9ms | 5.91% I'll add this as an additional option and switch sort order OFF by default since sorting will be done by RockGrid anyhow No problem at all. I need this stuff for my own work, so any help is welcome but of course not expected PS: again the tests without sort order 11 items: rockfinder = 4ms | findmany = 6.9ms | 57.97% 1000 items: rockfinder = 35.7ms | findmany = 744.2ms | 4.8% 5000 items: rockfinder = 165ms | findmany = 1675.4ms | 9.85% 10002 items: rockfinder = 327ms | findmany = 3359.5ms | 9.73% 35000 items: rockfinder = 1745.2ms | findmany = 28547.7ms | 6.11% 91300 items: rockfinder = 4385.5ms | findmany = 74213.9ms | 5.91% Now that looks a lot better, doesn't it?

Another performance test using findMany() So this looks like findMany() is a lot faster, but this is not true because creating the proper array of data takes longer than with RockFinder: $selector = 'parent=/data'; $finder = new RockFinder($selector, ['title', 'headline', 'summary']); t(); $result = $finder->getObjects(); d($rf = t()*1000, 'query time in ms (rockfinder)'); d(count($result), 'items'); d($result[0], 'first item'); t(); $result = $pages->findMany($selector); //$finder->getObjects(); d($fm = t()*1000, 'query time in ms (findmany)'); d($count = count($result), 'items'); d($result[0], 'first item'); t(); $arr = []; foreach($result as $p) { $arr[] = (object)[ 'id' => $p->id, 'title' => $p->title, 'headline' => $p->headline, 'summary' => $p->summary, ]; } d($fm2 = t()*1000, 'create array'); d($arr[0]); d("$count items: rockfinder = " . round($rf,2) . "ms | findmany = " . round($fm+$fm2,2) . "ms | " . round($rf/($fm+$fm2)*100, 2) . "%"); Result: Some other tests: $selector = 'parent=/persons'; // 11 items: rockfinder = 3.8ms | findmany = 7ms | 54.29% $selector = 'parent=/dogs'; // 1000 items: rockfinder = 41ms | findmany = 722.1ms | 5.68% $selector = 'parent=/cats'; // 5000 items: rockfinder = 221.4ms | findmany = 1660.8ms | 13.33% $selector = 'parent=/invoices'; // 10002 items: rockfinder = 526.6ms | findmany = 3385.3ms | 15.56% $selector = 'parent=/data'; // 35000 items: rockfinder = 7161.4ms | findmany = 27722.9ms | 25.83% $selector = 'parent=/data2'; // 91300 items: rockfinder = 59523.6ms | findmany = 76495.8ms | 77.81% What is very interesting (and not good), is that the time needed for RockFinder increases disproportionately when querying > 10.000 pages; 10.000 items = 500ms, but 3x10.000 pages = 7.000ms Maybe any sql experts have an idea?

Another update is online (branch DEV3): https://gitlab.com/baumrock/RockSqlFinder/tree/dev3 This is a major upgrade, a complete rewrite and finally a really good solution imho: I changed the syntax to object-notation in preference of nested associative arrays and it's a lot cleaner and easier to setup and read now. I rewrote the logic completely. It is NOT recursive any more but it takes away a LOT of complexity. You can add "fields" to your find-operation and those "fields" are fully customizable classes. The module ships with predefined classes for FieldtypeText (easy as it is just a column in the DB), FieldtypeFile (that shows concatenated filenames and descriptions) and FieldtypeRepeater (that shows concatenated data of the repeateritem's fields). The user can add custom FieldFinders in /site/assets/RockFinder (this will be the final name of the module). Possibility to add custom separators for concatenated fields Easy to create custom SQL by wrapping the created SQL in your own statement (SELECT ... FROM $finder->getSQL() GROUP BY ...) Examples: Query 10.000 invoices and return their date and value: Query all persons and show content of images (fieldtypeimage) and repeater (fieldtyperepeater): Same example but with image descriptions and all repeateritem's fields: Find 35.000 pages: This takes significantly longer than finds until 10.000 items. Maybe @adrian or @theo you have time to compare these results on your setups? Todos: Support for FieldtypePage (and other fieldtypes?) Support for Multilanguage (should not be hard, I've developed it with that in mind) Aggregation functions like sum(), avg(), min(), max() Support for closures? What do you guys think?

thx, this works looking forward to hearing their feedback!

I'm doing a complete rewrite (again ). What I have so far is this: <?php namespace ProcessWire; class RockFinderField extends Wire { private $name; // name of the final object property private $field; // name of the field to query private $closure = null; // closure to calc value /** * constructor */ public function __construct($name, $field) { // set the final column name for this field if(is_string($name)) $this->name = $name; else $this->name = $field; // check the field type if(is_callable($field) AND !is_string($field)) { $this->closure = $field; } else { $this->field = $field; } } } And in the module I do this: foreach($fields as $key => $value) $this->fields[] = new RockFinderField($key, $value); bd($this->fields);

Ok, thx. My class is indeed extending Wire... Maybe there is a possibility to check for the number of properties the debugInfo object returns and in case it is 0 it falls back to the original object dump?

Same here: I think they are really great! It makes the setup so much cleaner and more readable and also more flexible. For example I'm using a region "scriptsbottom" at the end of my <body> and whenever an included section needs some javascript I just write the markup and append the script to that region: // _main.php <region id="header"></region> [...] <script><region id="scriptsbottom"></region></script> // slider.php <section id="header">... my great slider markup ...</section> <region pw-append="scriptsbottom"> $('#slider').init(); </region> IMHO this is much more readable than using delayed output with php variables and then "messing" around with $config->scriptsbottom .= '$(\'#slider ... ' (doing some crazy escaping or ob_get_clean() or the like.

Hey @adrian, what do you think of adding a check if debugInfo is available for the requested object and if not, set it to "false" automatically? I'm working on my RockFinder module and wanted to debug my subclass. This was the result: Switching "debugInfo" to false I get this: Which is of course much more helpful. Luckily I remembered about that because we were talking about it here, but if someone missed that it could be quite hard to find out what is going on and why the information does not show up... Also it is quite annoying to always set ['debugInfo' => false] for every quick d($myobject)