Preview/Discussion: RockDataTables

[bernhard]

https://www.ag-grid.com/javascript-grid-features/

Everything with (e) is enterprise:

[kongondo]

Ah, I see, thanks. 

[bernhard]

Just added a check for empty find() result in RockSqlFinder: https://gitlab.com/baumrock/RockSqlFinder/commit/6d24cfd042d29a72a976db810e3d450c9166c531

If you are using it, this might be worth to update!

[szabesz]

8 hours ago, bernhard said:

I think it is superior to (and in most cases also easier than) datatables, that's why I'm switching.

Yeah, at firts sight ag-grid looks great but have you seen http://tabulator.info/ lately? Namely the new release: http://tabulator.info/news

I have also checked out a lot of these JS table libraries and Tabulator seems to be the only one which is completely open source (MIT)  and is being actively developed. I have not used it just yet but I am planning a test drive in the near future.

Edited April 4, 2018 by szabesz
typo

[bernhard]

thx @szabesz, haven't seen anything about tabulator so far. It looks quite similar to jquery datatables though and I have not seen anything that would not be possible using aggrid on the first sight. Or did I miss anything?

I'm quite far with my implementation of agGrid, so I'm quite sure I will not change the grid library (again) very soon  

[szabesz]

4 minutes ago, bernhard said:

Or did I miss anything?

Well, I am not suggesting you should switch to it, the only thing which is clearly better to have is its MIT license. I just thought you might want to reconsider it just once more but I know its not fun switching libraries "just because"  Feel free to ignore my notes.

Edited April 4, 2018 by szabesz
typo

[adrian]

5 minutes ago, bernhard said:

I have not seen anything that would not be possible using aggrid

Not saying it's a reason to switch, but it does support xlsx whereas that appears to require the paid version aggrid.

[bernhard]

Thanks for your feedback, I really enjoy the discussion.

@mit

Aggrid is also mit licensed: https://github.com/ag-grid/ag-grid/blob/master/LICENSE.txt

And I don't think it's bad to have paid pro features. Quite on the contrary. I think it can ensure a long life of the software and good quality. We know that principle from somewhere, don't we?! ?

@xlsx

I have looked at this closely already as it is a must for me to transfer data to excel. Aggrid has the ability to export data as CSV with custom separators. It's perfectly fine to use this for excel export IMHO. It even has custom renderers that only fire when data is accessed for export. Only thing missing at CSV compared to xlsx is fancy styling and formulas etc. Personally I don't need this in any of my projects.

Ok, simple links come to my mind. That might be nice to have. But we can still use the API to get data and then use any other JavaScript xlsx library to create our files. Or just pay a little fee to enable that feature.

 

Aggrid looks really well crafted. I'm actually quite happy that I forgot my version of datatables at home over Easter so I was forced to take a pause and take a closer look to aggrid ?

[kongondo]

49 minutes ago, szabesz said:

I have also checked out a lot of these JS table libraries and Tabulator

Thanks for this! A while back (actually, a long time ago), I thought I'd checked them all but it seems there's quite a number of new kids on the block (unless I missed them first time round ). I would have loved a Pivot Tables feature (aggrid has this in the paid version).

Sorry to hijack your thread Bernhard...

[bernhard]

I think I'm almost done

Plugins are now very easy to create and extremely useful as they are reuseable across multiple projects:

Sample setup of column statistics:

  // custom column stats
  grid.plugins.colStats.valueGetters = {
    id: function(column, selected) {
      return grid.pluck(0,{selected}).length + (selected ? ' selected' : ' rows');
    },
    percent: function(column, selected) {
      return grid.avg(column, {selected});
    },
  };

Everything works in the backend and in the frontend, just echo the field as you would do with any other field

echo $page->mygridfield

Data loads blazingly fast. The 10.001 rows here load in 500ms and are automatically compressed by the server to reduce transfer time (here it results in 90KB instead of 1MB of data!). Thx again @adrian for that idea.

Cell renderers can be combined as you like! That's really powerful and great! We can build libraries of different cellRenderers that we need often and then just wrap them in a function and return the one's we like:

In this column I combine the "actionItems" renderer and the "percentBar" renderer. The actionItems are only applied for regular rows. For rows of the "colStats" plugin we only append " (avg)" to the cell and don't show the icons because they are here to edit this lines related page (and of course the statistics row has no related page):

  colDef.cellRenderer = function(params) {
    var str = '';
    if(!params.data.colStatsRowType) {
      str = RockGrid.renderers.actionItems(params, [
        {
          icon: 'fa fa-bolt',
          href: '/admin/page/edit/?id=' + params.data.id,
          target: ' target="_blank"',
          str: 'Open external: ' + params.data.title,
        },{
          icon: 'fa fa-search',
          href: '/admin/page/edit/?field=percent&id=' + params.data.id,
          str: 'Open panel: ' + params.data.title,
          class: ' class="pw-panel hover"',
        },
      ]);
    }
    else {
      str = ' (avg)'
    }
    return RockGrid.renderers.percentBar(params, str, (!params.data.colStatsRowType ? 0 : 2));
  };

Updating the grid after closing the panel is on my todo-list

[bernhard]

Just tested the field on a custom admin page:

  public function ___execute() {
    $form = $this->modules->get('InputfieldForm');

    $f = $this->modules->get('InputfieldRockGrid');
    $f->label = 'Default Report';
    $f->themeBorder = 'none';
    $f->setData("id>0, limit=10", [
      'title',
      'templates_id',
      'created',
      'status',
    ]);
    $form->add($f);

    return $form->render();
  }

Thanks again @adrian for Tracy's awesome request info panel to get the proper "themeBorder" setting  

edit: ajax mode does not yet work in processmodules...

[bernhard]

Question to all of you: At the moment the module loads all assets separately. Each plugin and each renderer is placed in one file. I don't want to change this, because that way it stays clean and maintainable. But I thought it might not be ideal to load all those files separately for sometimes only one line of code inside this file. Do you have any ideas how i could combine those files to one js file and one css file? ProCache could do that on the frontend, but I don't want to rely on a pro module, and it would not work in the backend.

Thanks for any hints

[gebeer]

Check out https://github.com/mrclay/minify

But I don't know if it is worth the effort. For in the backend I wouldn't mind having multiple files loaded even if some of them have only one line.

[flydev]

2 hours ago, bernhard said:

Do you have any ideas how i could combine those files to one js file and one css file?

Maybe you could provide a Gulp tasks file along the module which could be called and generate a rockdatatables.min.js/css.

 

[adrian]

Hey @bernhard - totally OT, but any ideas why the full Tracy bar is showing twice in your screenshot?

[bernhard]

Just now, adrian said:

Hey @bernhard - totally OT, but any ideas why the full Tracy bar is showing twice in your screenshot?

I knew you would ask  Thats the screenshot tool

26 minutes ago, flydev said:

Maybe you could provide a Gulp tasks file along the module which could be called and generate a rockdatatables.min.js/css.

Hm, never worked with gulp and don't think it would be worth the effort. Maybe it does not matter at all? Does anybody know of any real issues when loading a lot of files? Does anybody know at which number this "a lot" could be?

I thought of a snippet that does a foreach and creates a file on the fly with file_put_contents. But the file would need to be different for all pages (thats how it works now). Hm... I could actually change that behaviour and create one single js and one single css file thats always loaded when any rockgrid item is available on the page. Since all the plugins load with eventlisteners this should work...

I'll leave this on the ideas-list...

[dotnetic]

@bernhard You could use the AIOM Module for this, it should also work in the backend. Please look at the sections "minify Javascript" and "Conditional loading". You can even parse LESS files with it. But who needs LESS, when you have SASS? Oooops, I didn't say anything  

[bernhard]

2 minutes ago, jmartsch said:

@bernhard You could use the AIOM Module for this, it should also work in the backend. Please look at the sections "minify Javascript" and "Conditional loading". You can even parse LESS files with it. But who needs LESS, when you have SASS? Oooops, I didn't say anything  

Thx. But I don't want to add any dependencies if not really necessary (like RockSqlFinder)

[dotnetic]

Then a simple gulp task or webpack would be the way to go. Are you interested in a simple minification script for gulp?

[dotnetic]

1 hour ago, bernhard said:

Hm, never worked with gulp and don't think it would be worth the effort. Maybe it does not matter at all? Does anybody know of any real issues when loading a lot of files? Does anybody know at which number this "a lot" could be?

 

Multiple files would be multiple HTTP Requests which would take a longer time, than loading one large file. There are some readings what is "a lot" and if you stay under 14KB you can do the whole file in one roundtrip. Don't know if this is correct, but this is something I remember.

Don't know if this still is relevant with HTTP 2.

[dotnetic]

Ok, here you go. It's simple.

Create two files in your root directory

gulpfile.js

var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var gulp = require('gulp');

var jsFiles = 'lib/**/*.js',
    jsDest = 'dist/';

gulp.task('scripts', function() {
   return gulp.src(jsFiles)
    .pipe(concat('scripts.js'))
    .pipe(uglify())
    .pipe(gulp.dest(jsDest))
})

gulp.task('default', ['scripts']);

Please change the paths to your javascript files according to your needs.

In my example every .js file under "lib" and its subdirectories are catched.

You can also change the jsDest. This is where the concatenated file "scripts.js" will be stored.

Create a package.json

{
  "name": "gulp-simple-concat",
  "version": "1.0.0",
  "description": "",
  "main": "gulpfile.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "gulp": "^3.9.1",
    "gulp-concat": "^2.6.1",
    "gulp-uglify": "^3.0.0"
  }
}

Then run

npm install

Node.js has to be installed for this to work. But it is very common now for web-devs to have this installed.

When npm is finished, just run `gulp`in your console/terminal.

That's it. Now you have a concatenated and minified javascript file.

 

[szabesz]

15 hours ago, bernhard said:

Does anybody know of any real issues when loading a lot of files? Does anybody know at which number this "a lot" could be?

If I were you, I would not put too much effort in it – as we are talking about the backend – but if you want to learn Gulp then why not?  Soon it is http2 time and as far as I can remember I read that when using http2 merging assets into one is often the wrong choice. But even with http1, since files are cached by the browser you loose a few millisecs first, that's all I guess.

I have seen WP sites with about a hundred js/css files loaded separately but the real issue with page load  speed never seemed to be loading those but mostly other bottlenecks are at play.

Of course, I biased, just read my current motto:
"It is unwise to relentlessly chase performance at the expense of everything else. Better is the enemy of good!"

 

[tpr]

I wouldn't bother with that neither. Perhaps you could set up a js loader (eg. Require.js) that would load all the js files but I think that's also unnecessary.

[bernhard]

Thanks @szabesz and @tpr, same feeling here. I just thought there might be a processwireish-3-lines-of-code-no-dependencies-at-all solution  

Also thanks @jmartsch, but the last thing i want is to include any dependencies for such a small priority  

Thanks for your feedback.

[flydev]

I don't want to pollute your official thread so I post here.

Perhaps you have an idea before I go further in my questions

I have a template which contain some fields. Two of them are timestamp, rest are integer.

What I want is to fetch pages that are in a date range, so I have this selector : 

Quote

has_parent=734756, template=transaction, range=(date>=1514761200, date<=1523483999), montant_especes=0, montant_carte_bancaire=0, montant_cheques=0

In this example, the system fetch 18119 pages. Results :

Quote

 

$this->pages->findIDs($selector); // 9387.67ms, 0.04 MB

$this->pages->findMany($selector); // 10494.93ms, 0.29 MB

$this->pages->findObjects($selector, ['title']); // 12259.55ms, 0.56 MB

 

It take around 10 seconds to fetch all pages, in the DBMS it take 0.3908 second(s) for 18119 pages.

I think I am missing something