Query API

[mindplay.dk]

@ryan I'm hoping to hear from you on this subject.

I'm quoting from this article:

As time has gone on, I've come to question the value of this, as {$title} is equivalent to <?=$title?> in PHP syntax. In that case, the tagged version is slightly shorter, but at what cost? When you start to realize the redundancy and inefficiency of the tagged version, it starts to look a lot less attractive. Despite writing such template engines in the past, I always reverted back to PHP syntax for the direct simplicity, and efficiency of it. I didn't want to start down the road of creating a new syntax for people to learn. Especially when I knew I wouldn't use it myself. You don't need to know PHP in order to use it's syntax in a template, and much of the PHP syntax you learn is universal and non-proprietary. Tagged syntax seems to be based largely upon a myth.

I completely agree. I too wrote template engines in the past - and ended up not using them.

But my question is this: don't these arguments work against using custom syntax in the query engine, too?

As opposed to a native query builder API, the custom syntax is shorter, but at what cost? Some redundancy, some inefficiency, a learning curve - even if the syntax is great (it really is) and learning curve is minimal, for one, it won't receive IDE support, so you have to learn the syntax and symbols by heart.

Other than shorter syntax - the same argument for template engines that got overruled - what are the arguments in favor of custom query syntax?

I had to ask

[Pete]

It's short and simple would be my thought.

I'm not sure what the long-hand version would be but I suspect it would be a full-on database query, complete with some sub-queries in some cases, so I think the legibility and speed of coding gain outweighs the headache of the long version and the minute saving in query time.

Everything's easier when you don't even need to know what the DB structure is

[ryan]

This is a fun question to answer, so thanks for asking it. It really gets down to much of the motivation behind the system.

For me it really comes down to time. Time is the most important thing to me. I'm getting old and need to focus on the things that give me more time, and that's what it's all geared around. For others, it's money. But time is money. Either way, you have to ask: "what is saving me time?" That's all you need to know to answer the question. But here's the long version:

How much time does it save you to use a template engine vs. PHP API? I would suggest that using a template engine saves you zero time (maybe even costs time) vs. using a clean PHP API like in ProcessWire. Ultimately you are just typing different characters with a template engine, in about the same quantity, to do the same thing (example). Nor is a template engine performing any heavy logic, thinking or processing behind the scenes… it's mainly just substituting one thing for another. It's not saving you any time or mental/logical energy. It's this whole abstraction layer taking up space that isn't contributing anything to the bottom line. So the benefits of a template engine aren't very clear, relative to using something like ProcessWire's API. And I don't think it matters whether you know PHP or not, as the learning curve is identical between PHP and a template engine, for accomplishing the same things. The only difference is that PHP will go much farther, when/if you want it to.

Then look at the amount of time it would take you to construct raw DB queries (or through some abstraction layer like ActiveRecord). A simple selector that you write in a few seconds translates to a rather long and complicated SQL query that might have taken quite some time to create and optimize manually. Whether in ProcessWire or somewhere else, to create the SQL query, you would have had to maintain internal knowledge of the database, tables involved, indexes, left and right joins, orders, grouping, limits, security and much more…

So I would suggest that ProcessWire's selector syntax is saving HUGE amounts of time (and thus cost) by abstracting away a whole lot of technical details and optimization. At the same time, it's part of a larger system that has also abstracted away the complexities of DB design and optimization, and brought it all into the context and thinking of a website. You don't need to know anything about databases or indexes, or that they even exist.

I would also suggest that ProcessWire gives one the impression that they can easily pluck data from anywhere in their site as if it was already in memory… and in the context of a website, not a database. In terms of time and cost savings, is there any comparison between this and a template engine?

[mindplay.dk]

Wow, thank you for taking the time to answer this question so thoroughly!

I love how you accept a challenging question like this - a lot of people would have taken it as criticism.

I would submit though, that I do believe the real underlying strength of PW lies not in the query syntax, but in the data-model that supports it. You've sort of come up with a different kind of data-model, and I don't know what to call it, in some ways it's more like a "hierarchic document" database (not a real term) but it's not really relational - it just happens to be implemented using a relational database.

The document aspects of the data-model of course lends itself well to content management, which is really data-management or document-management, depending on how you look at it. While the hierarchic aspect lends itself extremely well to the hierarchic nature of URLs.

As Pete pointed out, it frees you from needing to know what the database structure is - again, I would attribute that quality to the data-model and not to the syntax.

As you said, no, there is no comparison between this and a template engine - that isn't what I was trying to suggest. What I'm suggesting is that the syntax itself is just syntax: it's a string of characters that can be interpreted.

The interesting thing about language, is not whether it uses parentheses or curly braces, single or double quotes - but rather the structure and instructions that those symbols express, and the functionality that they invoke.

A template engine has no real value, because all it really does, is invent different structure and symbols to express essentially the same things you can already express using existing native PHP syntax.

Okay, so PHP doesn't have built-in syntax for queries. It also doesn't have built-in syntax for MVC, event hooks, database access, document object models, or any number of other patterns and ideas - but (for the most part) people don't invent custom syntax for any of those, just APIs.

And maybe PHP syntax isn't as convenient for something as condensed as a query API.

But at the end of the day, isn't that the same selling point used by template engine peddlers?

Granted, query syntax in PHP is a more extreme case of the same thing - most query APIs (that I've seen) written in PHP are not pretty.

There are exceptions though - phinq for example manages to provide very familiar syntax by piggybacking on PHP itself for expressions, where other LINQ implementations for PHP (such as phplinq) usually resort to parsing strings. Of course, phinq has the luxury of just supporting native PHP data-structures like arrays - implementing efficient database-support would likely involve parsing and interpreting PHP, and then where are we going...

All things considered, I think custom query syntax is probably the best approach you could have chosen for the PW query engine.

The reason I brought it up in the first place, is that I strongly suspect that the day will come, when I want to do something that can't be done with the query engine.

When the day comes, I will return.

muhahahahhahaaaah!

[ryan]

But at the end of the day, isn't that the same selling point used by template engine peddlers?

I think that selling point may apply to CMSs that have their API as part of the template engine (Expression Engine?). Most CMSs that use template engines don't have a particularly nice or simple PHP API, and so the template engine is not just a template engine… it's the public API too. In those systems, it's not really even a question because you have to use the template engine if you want to use the system without being an expert on its code.

PHP itself makes a good template engine. In our case, trying to go beyond that doesn't present any real time-saving opportunities. I want to focus efforts on the things that will save designers/developers the most time. The template engine is an unnecessary layer that consumes overhead and doesn't deliver any real benefits for us. Whereas things like the find selectors in PW are delivering significant time savings relative to the alternative.

I think you might be able to realistically compare the difference of using a single selector string to using a query array (to the same function, like $pages->find). That would be a little closer comparison of template engine vs. native PHP (and maybe what you originally meant?). I probably wouldn't have originally bothered with selector strings if it weren't for the 'operator' part of them, which can't be succinctly represented in a PHP array (unless our only operator were "="). As the needs expanded to include things like OR fields/values, AND fields/values, negative expressions, and more, PHP (or any language) would have left us with a ugly, long and complex query (whether out of arrays or objects). Then we'd be asking "why not just use an SQL query?".

The reason I brought it up in the first place, is that I strongly suspect that the day will come, when I want to do something that can't be done with the query engine.

I'm sure you will too, most of us will at some point. The query engine is not a full-on replacement for database queries. But it is meant to cover the vast majority of common needs. But there are occasions where you may still want to go in and do a query yourself. Though for me, those occasions are pretty rare.