Beluga

When thousands of queries (updating content) are bringing your system to its knees, InnoDB with transactions will bring a big quality of life improvement. Was just discovering this through this issue. Reading the issue comments, you can see Ryan is looking into making use of transactions more when they are available. You can already use them in your own templates and modules: https://processwire.com/blog/posts/using-innodb-with-processwire/

The plan is to have it as an interface to filter from two collections of proverbs, international with 35k items and Finnish with 8k. Filtering would target the proverb text. The db includes thematic categorisation, book references, cross-references for culture/language and various research-related tidbits. There will be other views on the material without the grid (like a digital card per proverb etc.). I like RockGrid very much and the new smart filter is a really nice addition. I will look into paying you after we launch the site.

I don't get the scrollbar with ag-Grid 19.0.1 (on latest Firefox). That is unfortunate ?

Can confirm everything works. I updated ag-grid soon after I started tinkering with RockGrid.

I see the Youshido lib is in need of a new maintainer.

Bah, this was my own blunder: I had failed to include leaflet.awesome-markers.css when I made my attempt to bump those versions. I had also overlooked that the readme has $map->getLeafletMapHeaderLines(); that should be echoed to our head element (the readme in Github is hard to read, so copying to a text editor helped). With this in mind I modified the getLeafletMapHeaderLines function MarkupLeafletMap.module like so to get all the latest hotness: $lines .= <<<LINES <!-- Styles supporting the use of Leaflet.js --> <link rel="stylesheet" type="text/css" href="https://unpkg.com/leaflet@1.3.4/dist/leaflet.css" /> <link rel="stylesheet" type="text/css" href="https://unpkg.com/leaflet.markercluster@1.4.1/dist/MarkerCluster.css" /> <link rel="stylesheet" type="text/css" href="https://unpkg.com/leaflet.markercluster@1.4.1/dist/MarkerCluster.Default.css" /> <!-- Scripts supporting the use of Leaflet.js --> <script type="text/javascript" src="https://unpkg.com/leaflet@1.3.4/dist/leaflet.js"></script> <script type="text/javascript" src="https://unpkg.com/leaflet.markercluster@1.4.1/dist/leaflet.markercluster.js"></script> <script type="text/javascript" src="https://unpkg.com/leaflet-providers@1.4.0/leaflet-providers.js"></script> <script type="text/javascript" src="{$url}MarkupLeafletMap.js"></script> <!-- Extend Leaflet with Awesome.Markers --> <link rel="stylesheet" type="text/css" href="https://unpkg.com/drmonty-leaflet-awesome-markers@2.0.2/css/leaflet.awesome-markers.css" /> <script type="text/javascript" src="https://unpkg.com/drmonty-leaflet-awesome-markers@2.0.2/js/leaflet.awesome-markers.js"></script>

Thanks. I had to delete the row from the modules db table and then I was able to reinstall it. Works now!

I upgraded to 0.9.4 without thinking. I don't think I had any newline-separated dependencies, only a config.js, but I am still getting an error. In the db, modules, AdminCustomFiles, the data field has {"files_folder":"AdminCustomFiles","process_filter":["ProcessPageAdd","ProcessPageEdit"],"js_config":1,"theme_files":"","dependencies":"AdminCustomFiles\/config.js"} When I am in the module config page, I get this in the console: "SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data" pointing to the line with data = input.value ? JSON.parse(input.value) : [], Being in debug mode, I get this on all admin pages: Warning: Invalid argument supplied for foreach() in /path/to/site/modules/AdminCustomFiles/AdminCustomFiles.module on line 249 What should I do?

What? You think Google makes things so easy for black hat SEO people? I'd like to see a source for your claim.

Yep, been discussing transactions in the issue

The link has rel nofollow, though.

I really hope the page creation/deletion slowness can be solved as it is quite disastrous (no blame on Ryan, just an unfortunate glitch). Who knows how many companies have evaluated PW and dropped it because of this without noticing InnoDB is at fault.

Issue created

InnoDB is your (really PW's) problem.

Now that I looked into it, it does not seem to do what I need. The example has a separate field that filters all the columns. I couldn't figure out how to make it work so it would act as a "normal" column filter. It seems simpler to just continue using the PersonFilter thing.

I think the reason is just lack of time on the part of the module devs, but it would also need to be adapted because of breaking changes. I have been thinking of giving another shot at debugging this thoroughly, but have lacked time and energy.

Cool, thanks for the hiding tip! Hmm, now that I tested it, changing null to empty in JS does not seem to affect perf in any way. I actually did a quick JS flame graph comparison in Firefox and the relevant call took the same amount of milliseconds either way. So without any PDO hackery, the rockgrid.js PersonFilter.prototype.doesFilterPass in my previous example can be modified like this: this.filterText.toLowerCase().split(" ").forEach(function (filterWord) { var value = valueGetter(params); if (value === null) value = ''; if (value.toString().toLowerCase().indexOf(filterWord) < 0) { passed = false; } }); Btw. has anyone tried this: https://www.ag-grid.com/javascript-grid-row-height/#auto-row-height I tested it, but the result was just huge CPU chewing for a while (for a large data set), but row height did not adapt for cells with overflowing content. So simply col.autoHeight = true;

Another topic: are you able to hide the id column? I tried with the methods in the readme, but I can't make it go away.

@dragan For me those blinking scrollbars and pagination occur reliably with a field height of 520 (Edit fields - Details).

I should have just used my brain. Luckily I am not prone to alcoholism, so this public embarrassment will not lead to pathological results. Anyway, let's pretend nothing happened last week and move on to the next topic. I noticed the ag-Grid demo has a cool filter, which allows us to use multiple strings. However, when used with RockGrid/PW, it breaks, if the field values include null ones. Long story short, I figured the most inexpensive change perf-wise would be telling PDO to treat nulls as strings. I did the change in RockFinder. Perhaps it would be unwise to include in upstream as some users would expect nulls to stay nulls. The change in RockFinder.module.php: public function getObjects($array = null) { $timer = $this->timer('getObjects'); try { $this->database->setAttribute(\PDO::ATTR_ORACLE_NULLS, \PDO::NULL_TO_STRING); $results = $this->database->query($this->getSql()); $objects = $results->fetchAll($array ? \PDO::FETCH_ASSOC : \PDO::FETCH_OBJ); } In RockGrid code, there is also a possibility to use sql query as datasource, but I don't see it in the interface. Anyway, in case one would need to use it somehow: InputfieldRockGrid.module.php: // sql query as datasource $this->database->setAttribute(\PDO::ATTR_ORACLE_NULLS, \PDO::NULL_TO_STRING); $results = $this->database->query($sql); return $results->fetchAll(\PDO::FETCH_OBJ); Edit: later I noticed that dealing with the nulls in JS does not seem to affect perf even with a big data set, so I modified the filter example below to just do if (value === null) value = ''; Using the filter from ag-Grid demo in your .js: document.addEventListener('RockGridItemBeforeInit', function(e) { if(e.target.id != 'RockGridItem_rockgrid') return; var grid = RockGrid.getGrid(e.target.id); var col = grid.getColDef('myfield'); col.headerName = grid.js.myfield; col.filter = PersonFilter; col.floatingFilterComponent = PersonFloatingFilterComponent; }); document.addEventListener('RockGridItemAfterInit', function(e) { if(e.target.id != 'RockGridItem_rockgrid') return; var col; var colDef; var grid = RockGrid.getGrid(e.target.id); }); function PersonFilter() { } PersonFilter.prototype.init = function (params) { this.valueGetter = params.valueGetter; this.filterText = null; this.params = params; this.setupGui(); }; // not called by ag-Grid, just for us to help setup PersonFilter.prototype.setupGui = function () { this.gui = document.createElement('div'); this.gui.innerHTML = '<div style="padding: 4px;">' + '<div><input style="margin: 4px 0px 4px 0px;" type="text" id="filterText" placeholder="Multi-string search..."/></div>' + '</div>'; var that = this; this.onFilterChanged = function () { that.extractFilterText(); that.params.filterChangedCallback(); }; this.eFilterText = this.gui.querySelector('#filterText'); this.eFilterText.addEventListener("input", this.onFilterChanged); }; PersonFilter.prototype.extractFilterText = function () { this.filterText = this.eFilterText.value; }; PersonFilter.prototype.getGui = function () { return this.gui; }; PersonFilter.prototype.doesFilterPass = function (params) { // make sure each word passes separately, ie search for firstname, lastname var passed = true; var valueGetter = this.valueGetter; this.filterText.toLowerCase().split(" ").forEach(function (filterWord) { var value = valueGetter(params); if (value === null) value = ''; if (value.toString().toLowerCase().indexOf(filterWord) < 0) { passed = false; } }); return passed; }; PersonFilter.prototype.isFilterActive = function () { var isActive = this.filterText !== null && this.filterText !== undefined && this.filterText !== ''; return isActive; }; PersonFilter.prototype.getModelAsString = function (model) { return model ? model : ''; }; PersonFilter.prototype.getModel = function () { return this.eFilterText.value; }; // lazy, the example doesn't use setModel() PersonFilter.prototype.setModel = function (model) { this.eFilterText.value = model; this.extractFilterText(); }; PersonFilter.prototype.destroy = function () { this.eFilterText.removeEventListener("input", this.onFilterChanged); }; function PersonFloatingFilterComponent() { } PersonFloatingFilterComponent.prototype.init = function (params) { this.params = params; this.eGui = document.createElement('input'); var eGui = this.eGui; this.changeEventListener = function () { params.onFloatingFilterChanged(eGui.value); }; this.eGui.addEventListener('input', this.changeEventListener); }; PersonFloatingFilterComponent.prototype.getGui = function () { return this.eGui; }; PersonFloatingFilterComponent.prototype.onParentModelChanged = function (model) { // add in child, one for each flat if (model) { this.eGui.value = model; } else { this.eGui.value = ''; } }; PersonFloatingFilterComponent.prototype.destroy = function () { this.eGui.removeEventListener('input', this.changeEventListener); };

Actually, this was it ? I can't believe I wasted 15 hours on this due to a missing echo.

Uh, and now Dragan provided a working solution that uses output buffering in the template as well! Would still love to know what is the big deal with this behaviour. Edit: well, it turned out it was all because I followed the RockGrid quickstart too literally and did not add an echo for some reason. What a great way to waste 15 hours.

In short: RockGrid will not render in the frontend in my dev environment. I narrowed it down to the use of output buffering in the inputfield module (it renders, if I comment the ob_ stuff out). Bernhard is unable to reproduce the problem. Output buffering works in template context just fine, I can ob_start() and get its status. The problem is limited to the inputfield rendering context. I suspected something in my specific Docker dev environment, so I tried a completely different setup, but was still able to reproduce the problem. My different setup used suzel/docker-processwire so Apache instead of Caddy. I did my further tests with the Default intermediate profile just like Bernhard. PW version used is 3.0.98 and PHP is 7.2.9 for my initial setup and 7.0.28 for the suzel env. What should I be looking into next?