Weekly update: Combo Q&A – 11 Dec 2020

[ryan]

Last week’s post was about the new ProFields Combo field and it seemed like there was a lot of interest. Thank you for that, it really got me motivated to finish it and release it sooner rather than later. I am very excited about the Combo field and worked on it most of this week, hoping that I may be able to have the Beta version ready for release in the ProFields board by this time next week. This is a little more break than I like to take from the core, but the focused development time means it gets done a lot sooner, and likewise in a stable release state a lot sooner. Since I don’t have core updates to walk through this week, I thought I’d focus on some Q&A about the Combo fields. Below are questions that were asked either here in the forums or in the blog post. Please feel free to reply with any other questions that come up too. 

Quote

…seems like potentially some huge performance gains depending on how you are querying and rendering the field data.

There are definitely cases where this type of arrangement Combo provides would provide some nice performance gains. And there are other cases where they might reduce it. Consider that fields on a page in ProcessWire are dynamically loaded when you access them. So when you access anything from a Combo field, all of the data in the Combo field loads at once (1 query). So if you’ve got 10 fields in a Combo field, and you are going to be using much of that data in the request, then you’ve just made some nice performance gains, at least relative to having those 10 fields split into dedicated/separate ProcessWire fields.

On the other hand, perhaps you are rendering a list of pages, and you only need one or two fields of data from the Combo field. In that case, it’s less efficient than regular ProcessWire fields because you’ve loaded a bunch of data in memory that isn’t going to get used. So I would say Combo fields are great for data that you use when rendering a full Page, but not as great for cases where you are rendering lists or summaries of pages (where you might not be using much of that data).

Quote

Could you say something about any potential downsides to this way of storing field data? E.g. situations where it wouldn't be optimal to use it. Or to put the question another way: why isn't all field data for every template stored this way by default in PW?

In the blog post I mentioned that Combo fields store data in individual DB columns, but also keep a separate index so that all of the combo data can be searched together. If that’s what you are referring to, I would say the primary downside of this storage method is that it takes up more space. It adds convenience and efficiency for searching, but at the cost of consuming more disk space [in the database]. This is something that may be beneficial to some and less so to others, and it may end up being an optional thing too. 

The index is similar to a JSON encoded version of all the fields. But it’s something that I think is worthwhile for another reason: backups. Like the Table field, the Combo field maintains a unique DB schema consistent with your selection of fields. Every time you save a Field, it checks the schema and determines whether to add columns, drop columns, or modify existing columns (and likewise for indexes). The JSON encoded version of the data is immune from data loss as a result of schema changes, so I see it as potentially useful as a way to backtrack, should it ever be needed.

As for your question about other fields in ProcessWire, Combo stores it exactly the same way as other fields, with the only difference being that it keeps a separate combined index of data; and is able to because the core Fieldtype architecture enables it to do so. Why not use the same duplication approach with other fields? I can't think of any core fields where this type of storage duplication would be beneficial in the way that it is with Combo. Outside of the core, the approach might be worthwhile in ProFields Table though.  

Quote

Will Combo field work within RepeaterMatrix field?

Yes! Combo fields work in Repeater and RepeaterMatrix fields. Since Combo fields are not a repeatable type, they play nicely with repeatable types. 

Quote

I don’t get what the difference to the page fieldset is?

They might look similar but they are also completely different. A page fieldset (FieldtypeFieldsetPage) is technically a single repeater item, but presented in a non-repeatable way. It’s a convenient way to make a group of fields that you could reuse across any number of templates. And in that respect it’s quite similar to Combo, at least in appearance. But that appearance is where the similarities end. Fields in a FieldsetPage are still individual ProcessWire fields defined on a template, and the Fieldset itself is an independent Page (an individual repeater item behind the scenes). So if you’ve got a FieldsetPage with 10 fields in it, then it’s still consuming the resources of 10 ProcessWire fields, plus 1-2 additional pages for structure. 

The goal of ProFields is to reduce the number of fields necessary to answer a particular need, and that is not the goal of FieldsetPage. If a Combo field can answer your need, it can do so a lot more efficiently than a FieldsetPage can. On the flip side, since FieldsetPage uses regular ProcessWire fields, it can support more types than Combo can. Nevertheless, once Combo is available, I would most often choose it over a FieldsetPage except in cases where I needed a type of field that only FieldsetPage could support. 

Quote

Will it be possible to use the individual fields in page selectors?

Yes, any field within a Combo field can be queried individually in selectors using the field.subfield syntax. For example: $pages->find(“combo.first_name=Ralph”); It’s just like querying a Page reference field on pages in ProcessWire. So far, my experience is that it can do all of the same things, and supports all of the same operators; whether querying text, numbers, Page references, selectable options, etc. There are technical differences behind the scenes though. All the data in a Combo field for a page is represented by a row in a database table, so matching some types of values (like multi-Page references) has to use indexes rather than lookup tables. This is similar to the approach taken in the Table field. Per last week's post, Combo fields can do one thing that other fields in ProcessWire can’t though. Because it keeps an index of combined data, you can perform text matching on all of the subfields in a combo field together, just by omitting the “subfield” from the selector. An example is that $pages->find(“combo~=Ralph”); would find the word “Ralph” in any of the fields in the Combo, rather than just the first_name field. So not only is it easy to do, but it can do that more quickly and efficiently than other fields in PW. 

Thanks for reading and have a great weekend!

[adrian]

6 minutes ago, ryan said:

On the other hand, perhaps you are rendering a list of pages, and you only need one or two fields of data from the Combo field. In that case, it’s less efficient than regular ProcessWire fields because you’ve loaded a bunch of data in memory that isn’t going to get used.

Thanks for the great post @ryan, but I am curious about this - why does the combo field have to load all subfields (db table columns). Why not have a way to select just the subfields we want? I know it's not a completely correct example, but think of SELECT field1, field2 FROM table vs SELECT * FROM table.

Perhaps I am missing something about how it all works though?

[ryan]

@adrian It's definitely possible, and maybe we'll add the option for that. For this first version though it loads all the data in the field in the same way that other PW fields do. But you are right that it would likely be relatively simple to have it load them dynamically. 

[adrian]

7 minutes ago, ryan said:

@adrian It's definitely possible, and maybe we'll add the option for that. For this first version though it loads all the data in the field in the same way that other PW fields do. But you are right that it would likely be relatively simple to have it load them dynamically. 

That's sounds great and if this is available, I wonder if this might change your answer to @Robin S's question about using this approach for all fields in PW. From my point of view I have always had a bit of a love / hate relationship with Tall-Thin vs Short-Wide approach to data storage in PW. There are obviously huge advantages in flexibility with Tall-Thin, and I know that in some cases they are more performant to query, but other times they are much less performant. While I don't expect moving the PW core in the direction of the Combo field actually makes sense, I do think that using it in certain templates for all fields (except page name / title) might make sense.

[ryan]

@adrian Having the option in something like Combo is definitely a nice choice. But like with the Table field, it places limitations on what kind of fields you can use in it (focused on simpler types), and what those fields can do. The approach isn't nearly flexible enough for a "supports any kind of field" modular system like PW. For instance, you could never have the likes of a Repeater field in a Combo field, among others. So it's kind of limited and primitive in that respect. But for a focused purpose module like Combo, that's where I think this approach is really useful and worthwhile. 

[adrian]

2 minutes ago, ryan said:

For instance, you could never have the likes of a Repeater field in a Combo field, among others.

Excellent point of course!

[awesomolocity]

Sometimes when I lay awake at night, I like to think about how cool the combo field is.

 

(But seriously, I'm very excited to see it)

[Chris-PW]

Hello, that reads well and I'm very curious.

Will it be possible to combine a CKEditor text field with an image field?
(I come from Typo3 and I've always found that very practical there. It was called text with image)

[Peter Falkenberg Brown]

Hi Ryan,

When I read about the combo field, it almost seems like a jury-rigged version of a standard MySQL data table with column definitions.

When I first came to ProcessWire, I had to get used to its splitting of the fields into separate tables.

I'm not an expert DBA kind of person on advanced MySQL queries and indexing and things like complex join SQL statements, but how would you compare the traditional SQL approach of complex queries to both the standard ProcessWire method and the new Combo fields?

My question might be too broad; not sure.

Peter

 

[HMCB]

@ryan one of the exemplary things about you is your ability to communicate. Writing clearly is important. You do a masterful job of it.