Module Module: Matrix Fieldtype & Inputfield

[kongondo]

Link to the module in the modules directory:

Modules Directory: http://mods.pw/8s

[adrian]

Hey kongondo - I have just started using this on a site for the first time - it is going to be awesome!

The one thing I would like to be able to do though is have better control over the row/column selectors. What I need to do is be able to limit the matches to children of the current page because I will be using this field on several page trees on the site that have the same structure, so a template selector will grab all these pages, not just the one under the current parent. I think if the PW core supported 'page' as an argument that could be used for the parent selector I'd be OK.

parent=page, template=rows_template

Perhaps your module could have an extra config settings to limit to the current page? Or perhaps an option to use custom php code to find the pages - like you can when defining a page field. Is that something you'd be willing to support?

Any other thoughts on how to make this work?

Thanks again for your work on this!

PS Another feature suggestion - any chance of adding a browser warning alert for the reset button? I know that nothing is actually lost until the page is saved, but I bet some users will think they have lost everything already and start typing again, not realizing they can simply reload the page. Maybe the button could also be named "Clear Data" or something more descriptive than "Reset".

My thoughts continue I am not sure how to handle this, but I have a checkbox field on each of the pages that make up the columns - this is to determine whether the page should actually be included in the table. The thing I worry about is someone unintentionally unchecking this which will result in data loss the next time the page with the Matrix table field is edited and saved. Other than a warning in the description/notes of the checkbox field, I am not sure how to protect against this. Could you module perhaps warn when rows/column are about to be discarded - with an alert that actually says which ones are being deleted from the matrix table? Any other ideas?

Also, what about an API method to create data - it would be nice to make it easier to create an import routine from an existing database.

[adrianmak]

cool......

[kongondo]

@Adrian,

Thanks for the suggestions. I am trying to create some time this week to look at these and other pending issues RE my other modules....I know I have been saying this a lot lately but other things/distractions beyond my control keep cropping up.....

[ukyo]

Your module great !

I have a question for own needs.. is it possible to do this with your module ?

[adrian]

Update: version 0.0.3

Small but important update to correct oversight whereby records with empty values were being saved to the database, thx @netcarver.

Hey kongondo - I know you are super-busy at the moment, but I actually have a use case where it would be much easier if it did store empty values. Because I am foreaching through column names I actually ended up having to put "ND" (for No Data) in the cells to get the desired result. This isn't a huge problem because I control the data going into the Matrix tables, but I think it might be a nice option to have.

Thanks! 

[adrian]

Something else I am going to need shortly is the ability to check whether the CSV import appends rows, or overwrites - I actually haven't tested - does it currently append, or overwrite? Maybe if append, then it's ok because you can use the "Reset" to clear and then it is effectively an overwrite.  

Also, the ability to export to CSV would be very handy - it would be much easier to make major edits via Excel and then import when done.

Just some ideas for when you're bored, although I might need these sooner than later - if so I'll send you a PR.

[kongondo]

Hey kongondo - I know you are super-busy at the moment, but I actually have a use case where it would be much easier if it did store empty values. Because I am foreaching through column names I actually ended up having to put "ND" (for No Data) in the cells to get the desired result. This isn't a huge problem because I control the data going into the Matrix tables, but I think it might be a nice option to have.

Thanks! 

Do you mean to allow empty values in both CSV imports and manual entered values? Both should be quite easy to implement. I think I'll implement them as separate options though.

Something else I am going to need shortly is the ability to check whether the CSV import appends rows, or overwrites - I actually haven't tested - does it currently append, or overwrite? Maybe if append, then it's ok because you can use the "Reset" to clear and then it is effectively an overwrite.  

Also, the ability to export to CSV would be very handy - it would be much easier to make major edits via Excel and then import when done.

Just some ideas for when you're bored, although I might need these sooner than later - if so I'll send you a PR.

Overwrite vs Append:

Off the top of my head I think it overwrites. I'll have to confirm this one. 

Export to CSV:

Good idea. Added to my @todo

[kongondo]

@adrian,

Just to let you know that I've added all your requests the module's GitHub issues so that I don't forget. As for the append vs overwrite issue, append seems to be quite tricky. It would need to find the last populated row and append from there. But finding out what the last row can be tricky, especially where we also allowed 'empty values'. I don't know if this makes sense. Anyway, will have a think about this one. Btw, the suggestion about custom PHP code to find row/column matrix pages is a powerful one and will lead to many possibilities, thanks.

[kongondo]

Your module great !

I have a question for own needs.. is it possible to do this with your module ?

Sorry for the late response. I don't quite follow your question. Did you find a solution?

[adrian]

@adrian,

Just to let you know that I've added all your requests the module's GitHub issues so that I don't forget. As for the append vs overwrite issue, append seems to be quite tricky. It would need to find the last populated row and append from there. But finding out what the last row can be tricky, especially where we also allowed 'empty values'. I don't know if this makes sense. Anyway, will have a think about this one. Btw, the suggestion about custom PHP code to find row/column matrix pages is a powerful one and will lead to many possibilities, thanks.

Thanks kongondo - I saw all those posted issues!

I haven't thought about the issues involved in making append work, but I do think it is pretty critical functionality. Currently unintentional data loss is very possible and I would assume that append was actually the default behavior. 

Thanks very much for looking into the custom PHP code option!

[ralberts]

Great stuff this... great stuff indeed.

One question, would it be possible to pick a specific value from the matrix?

So on a certain page, I would like to spit out the value at row 2, column C, without having to loop the whole pricetable to find that specific value.

My current implementation is quite inefficient:

    foreach ($accommodation->pricetable AS $p) {
        if ($p->row == $weekPage->id && $p->column == $pricePage->id) {
            echo $p->value;
        }
    }

Thanks for a great module!

[ukyo]

Sorry for the late response. I don't quite follow your question. Did you find a solution?

I am busy with other projects and didn't check it again but i will check it. For the moment i don't have a solution.

[kongondo]

One question, would it be possible to pick a specific value from the matrix? So on a certain page, I would like to spit out the value at row 2, column C, without having to loop the whole pricetable to find that specific value.

Currently no (unless there is some selectors I don't know about). It is something I have been thinking about, creating a method() to accomplish this. I think it would make this module even more powerful. It wouldn't be too complicated to implement, I think; Just a simple raw SQL PDO query. Implementation would be similar to a get and a find (for many).

Edit: Silly me; half asleep probably. This already works. See next post

I also think for the row and columns, one should be able to search not only by ID but by $page and title. E.g. find the price (value) of a product that is 'red' (title [row]) in colour and 'small' (title [column]) in size. Could you please file your query as a request in GitHub so that I don't forget? Thanks. I can't promise when I'll be able to get to it though, unfortunately.

Edited October 17, 2015 by kongondo
Correct answer in my subsequent post

[kongondo]

Currently no (unless there is some selectors I don't know about). It is something I have been thinking about, creating a method() to accomplish this. I think it would make this module even more powerful. It wouldn't be too complicated to implement, I think; Just a simple raw SQL PDO query. Implementation would be similar to a get and a find (for many). I also think for the row and columns, one should be able to search not only by ID but by $page and title. E.g. find the price (value) of a product that is 'red' (title [row]) in colour and 'small' (title [column]) in size. Could you please file your query as a request in GitHub so that I don't forget? Thanks. I can't promise when I'll be able to get to it though, unfortunately.

I must have been half-asleep when I answered this one.  . Sorry for  wasting your time with the GitHub issue. This already works (in fact, there is an example in README). See examples below. The last example answers your question.

Here 'catalogue' is the name of the matrix field on the current page

//find all values for one row
$results = $page->catalogue->find("row=1024");

//find all values for one column
$results = $page->catalogue->find("column=1035");

//finding by object; no need for $p->id for 'column' or 'row'
$p = wire('pages')->get('template=basic-page, title=Sizes');
$results = $page->catalogue->find("column=$p, value>=67");//finding by object; no need for $p->id

foreach ($results as $r) echo $r->value . '<br>';

//alternatively user toString() formatted output
echo $results;

//get a specific matrix value (single intersection)
$result = $page->catalogue->get("column=1034, row=1020");
echo $result->value;

Will update the docs with these examples.

Edited October 17, 2015 by kongondo

[kongondo]

Toying with the idea of having 'non-page columns and rows mode'. In this mode, if you upload a csv file, the first row will be treated as column headers and first column as row headers/labels

[ralberts]

Ah, awesome stuff. Must've missed that. Thanks so much!

[kongondo]

Update version 1.0.1 (Dev only for now)

Note: Haven't had time to update README

Changes

1. As requested, added ability to find column/row pages using custom PHP code. Thanks @adrian. This is a really powerful feature. Note that the order of precedence for the 3 methods to return row/column pages is: 1. Selector 2. Custom PHP code 3. Page field (3 is greatest).
2. Browser warning alert for reset button + rename the button (@adrian). Done + enhancement by @mackski, the button styling now matches other PW buttons
3. Added two properties: rowLabel and columnLabel (in memory only; not saved to DB) to return user-friendly row/column header info. E.g. echo matrix->rowLabel could render 'Red' or 'Large' instead of echo matrix->row that would instead render the page's ID. These are searchable in-memory ONLY, e.g. $results = $page->results->find("columnLabel=Maths").
4. Added two methods getRow() and getColumn() for mainly syntactic convenience. These do in-memory searches. Can search by path, title, ID, page object (see examples below).

Examples

columnLabel

//find all results for one column - using 'user friendly selector';
$results = $page->results->find("columnLabel=Maths");//@note: in-memory selector
echo '<h3>The are the results of: ' . $results->first()->columnLabel . '</h3>';//maths results
foreach ($results as $r) echo $r->rowLabel . ': ' . $r->value . '<br>';//student name: score

rowLabel

$results = $page->results->find("rowLabel=Maurice|Joel|Mi Mong");//@note: in-memory selector
#$results = $page->results->find("");//will show all
echo '<h3>The are the MULTIPLE results for several students</h3>';
$row = array();
foreach ($results as $r) {

	if(!in_array($r->row, $row)) {
		echo '<h4>' . $r->rowLabel . '</h4>';//echo'es student name (e.g. Joel)
		$row[] = $r->row;
	}
	echo $r->columnLabel . ': ' . $r->value . '<br>';//echo'es each subject's results, e.g. Maths: 70
}

getRow()//accepts 3 arguments selector value, limit, sort

#$p = $pages->get(1088);
#$results2 = $page->results->getRow($p->id);//get by ID
#$results2 = $page->results->getRow($p->title);//get by Title
#$results2 = $page->results->getRow('Joel');//get by Title (case sensitive)
#$results2 = $page->results->getRow($p);//get by Page
#$results2 = $page->results->getRow($p->path);//get by Path
$results2 = $page->results->getRow(1087);//get by ID//get by ID
#$results2 = $page->results->getRow('/students/richard/');//get by Path
#$results2 = $page->results->getRow($p, 4, 'random');//get by Page, limit to 4 random values
#$results2 = $page->results->getRow($p, 4, 'asc');//get by Page, limit to 4 values, sort ascending
#$results2 = $page->results->getRow($p, 3, 'desc');//get by Page, limit to 3 values, sort descending
#$results2 = $page->results->getRow($p->title, 4);//get by Page title, limit to 4
#$results2 = $page->results->getRow($p, '', 'asc');//get by Page, no limit, sort ascending
#$results2 = $page->results->getRow($p->path, '', 'desc');//get by Path, no limit, sort descending

echo '<h3>The are the results for: ' . $results2->first()->rowLabel . '</h3>';//student name
foreach ($results2 as $r) echo $r->columnLabel . ': ' . $r->value . '<br>';//subject: score

getColumn()//accepts 3 arguments selector value, limit, sort

Similar to getRow() except for columns

$results3 = $page->results->getColumn('/subjects/physics/');
echo '<h3>The are the results for: ' . $results3->first()->columnLabel . '</h3>';//Subject
foreach ($results3 as $r) echo $r->rowLabel . ': ' . $r->value . '<br>';//student:score

Some feature requests still pending + thinking whether to implement findRows() and findColumns().

Please test and let me know (if you can, both fresh install and upgrades [test on non-critical data please])

Edited October 22, 2015 by kongondo
Added a few more examples

[adrian]

Thanks Kongondo - the new custom PHP code option is awesome - I just swapped out my old hacked version for this and it's working brilliantly!

[kongondo]

Update version 1.0.2 (Dev only for now)

Note: Haven't had time to update README

 

Change

1. Added method getValue() to get the value at the given coordinates(row, column). This is an extended syntactic alias for $page->matrix->get('row=1045, column=1098'). It is more versatile, taking two parameters (row, column), allowing you to get a value by specifying its row and column ID (integer), path, title or Page object.

Examples: getValue() (matrix field called 'results' with student names for rows and subjects for columns)

//get by path
#$result = $page->results->getValue('/students/ishak/', '/subjects/physics/');
//get by title		
#$result = $page->results->getValue('Ishak', 'Physics');//note title is case sensitive
//get by ID		
#$result = $page->results->getValue(1062, 1082);

//### get by Page
#$r = $pages->get(1064);//results row
#$c = $pages->get(1083);//results column

//pass Page object
#$result = $page->results->getValue($r, $c);
//pass Page title
#$result = $page->results->getValue($r->title, $c->title);
//pass Page ID
#$result = $page->results->getValue($r->id, $c->id);
//pass Page path
#$result = $page->results->getValue($r->path, $c->path);

###############

//get by mixed parameters
#$result = $page->results->getValue('Ishak', '/subjects/history/');
#$result = $page->results->getValue('/students/sarah/', 'French');//note title is case sensitive
#$result = $page->results->getValue($r, 'Music');
#$result = $page->results->getValue('Amani', $c->path);
$result = $page->results->getValue('James', $c->id);

if($result) echo '<strong>' . $result->rowLabel . ' </strong>scored ' . $result->value . ' in ' . $result->columnLabel;
else echo 'No result found';

Please test and let me know how it goes, thanks.

[kongondo]

Update version 1.0.3 (Dev only for now)
 
Note: Haven't had time to update README
 
Changes

1. Export to CSV feature: Can export whole matrix (default) or limit to specified row range (e.g. start at row#2 and end at row#10 ). These are set in the collapsed fieldset 'Import/Export CSV Data' below the matrix. Exported CSV files include timestamp in the file name.
2. Some CSS styling for buttons.
3. Option: Set alert message to show after clicking 'clear data' button. If message set, show alert if no message specified, do not show any alert. Note: now using a custom popup rather than the ugly + annoying browser popup .
4. Option: Save empty values. Default is no; do not save empty values. Current status shown at the top and bottom of the matrix table.
5. Option: Show row numbers. Default is no, do not show row numbers.
6. Fixed a bug where older saved values would still appear after one switched to using custom PHP code to return rows and columns.
7. Added checkboxes for row selections (see #8).
8. Clear Data: Rather than wholesale clearance, button will only clear values of selected rows (see #7). Will show a popup if you try to clear without first selecting at least one row.
9. Fixed some minor bugs in situations where there would be more than one matrix table on a page.
10. Some stylistic changes to rendered fields in line with above changes.
11. Code re-factoring.

Edited October 30, 2015 by kongondo

[adrian]

This looks fantastic kongondo - thank you!

I will try out soon and let you know if I have any problems/suggestions.

[kongondo]

Thanks kongondo - I saw all those posted issues!

I haven't thought about the issues involved in making append work, but I do think it is pretty critical functionality. Currently unintentional data loss is very possible and I would assume that append was actually the default behavior. 

Thanks very much for looking into the custom PHP code option!

@adrian: I think I misunderstood what you meant by append. I assumed you meant tucking new values at the end of the matrix starting from the last empty row. What I think you actually meant was for each value,  'if new value is empty, keep old value. If new value is not empty, overwrite old value'. Is that correct?

[kongondo]

Expounding on the above...in 'append mode' there's two possible ways to determine when a value should be appended or a db value left alone (i.e. it is not being updated).

1. This is as I describe above, If there is a BLANK/EMPTY value coming in, we assume we want to keep the db value, so we don't change it; OR
2. If a corresponding value DOES NOT EXIST in the new values coming in (sort of like null), then we keep the db value. In this approach, it means if an EMPTY corresponding value is coming in, we empty the existing db value. 

The problem with #1 is that it won't account for scenarios where you actually want an incoming blank value to wipe out a corresponding db value. Of course this can be accomplished by manually deleting said values in the table itself rather than the CSV import but it could get tedious. Maybe #2 is the more natural way to do it. However, it is the more difficult way (currently) since it would require new CSV values coming in differently. Consider a matrix like this:

              Small      Medium    Large

Red       12           11             13

Green    15           9               8

Blue       7             14             6

Currently, data import does not allow Row (Red, Green, Blue) and Column (Small, Medium, Large) labels/headers in the imported CSV data. As you know, data comes in like this (considering the above table):

12,11,13

15,9,8

7,14,6

This won't work for #2 since there is no way of telling what a null value is. If data came in like this

Red Small 12

Red Large 13

Green Small 15

Green Medium 9

Green Large 13

Blue Medium 14

...then, we would be able to tell that Red Medium, Blue Small and Blue Large were null because they are not part of the incoming dataset. In that case, we wouldn't touch the corresponding db values. However, I don't think this is the natural/normal way of structuring CSV data (verbosity). So, for now, unless I find another way or am nissing something obvious, I am sticking with #1 for append mode.

Edited November 1, 2015 by kongondo
Clarifications..

[adrian]

Kongondo - you raise some good points around the issue of appending when importing. I don't think I really thought it through myself when I mentioned it in the first place - of course this is not a normal field type where you can just append new rows. The row must already be part of the matrix. I was talking about appending new rows. In my current use case, each year there will be a new "year" row that will appear in the matrix. I would like users to potentially be able to import a CSV of the data for that row. Of course importing for one row may not be worth the effort, but in some cases I can have up to 20 columns, so it might just be worth it. In that case, I would want "import" to just append the new row to the blank year that is in the matrix table.

Here's my matrix so you can see what I am talking about:

 

However, I don't think this is the natural/normal way of structuring CSV data (verbosity).

 
I am not sure I agree with this - if I had a dataset in excel I would definitely have the years in the first column.
 
I agree that there are issues around how to structure the CSV to be imported - my initial thought is that it would be nice if it could (or must) contain the year in the first "column". I think it is a bit weird that you can't export from a matrix field and then import the same CSV file back into the matrix field, because of that first column issue. Maybe "ignore first column" and while you are at it, "ignore first row" options would be good - make sense?
 
I am concerned about the current behaviour though, because at the moment, any import completely wipes the existing data - I don't think this is a good situation.
 
Some other thoughts on your recent dev additions:

• my initial thought is that checking rows would mean that only those rows would be exported - I didn't realized that I had to enter row numbers - could this be changed to respect checked rows as well? It would be nice if Matrix supported Shift+Click for selecting a block of rows like the PW does.
• I think your new modal alert box needs some css tweaks - might be on your list, but thought I'd mention anyways.

This is getting better all the time - really super useful for my needs - thanks again!