GraphQL for ProcessWire

[dadish]

NOTE: This thread originally started in the Pub section of the forum. Since we moved it into the Plugin/Modules section I edited this post to meet the guidelines but also left the original content so that the replies can make sense.  

ProcessGraphQL

ProcessGraphQL seamlessly integrates to your ProcessWire web app and allows you to serve the GraphQL api of your existing content. You don't need to apply changes to your content or it's structure. Just choose what you want to serve via GraphQL and your API is ready.

Warning: The module supports PHP version >= 5.5 and ProcessWire version >= 3.

Links:

• Zip Download
• Github Repo
• ScreenCast
• PW modules Page

Please refer to the Readme to learn more about how to use the module.

 

Original post starts here...

Hi Everyone! I became very interested in this GraphQL thing lately and decided to learn a bit about it. And what is the better way of learning a new thing than making a ProcessWire module out of it! 

For those who are wondering what GraphQL is, in short, it is an alternative to REST. I couldn't find the thread but I remember that Ryan was not very happy with the REST and did not see much value in it. He offered his own AJAX API instead, but it doesn't seem to be supported much by him, and was never published to official modules directory. While ProcessWire's API is already amazing and allows you to quickly serve your content in any format with less than ten lines of code, I think it might be convenient to install a module and have JSON access to all of your content instantly. Especially this could be useful for developers that use ProcessWire as a framework instead of CMS.

GraphQL is much more flexible than REST. In fact you can build queries in GraphQL with the same patterns you do with ProcessWire API.

Ok, Ok. Enough talk. Here is what the module does after just installing it into skyscrapers profile.

It supports filtering via ProcessWire selectors and complex fields like FieldtypeImage or FieldtypePage. See more demo here

The module is ready to be used, but there are lots of things could be added to it. Like supporting any type of fields via third party modules, authentication, permissions on field level, optimization and so on. I would love to continue to develop it further if I would only know that there is an interest in it. It would be great to hear some feedback from you. I did not open a thread in modules section of the forum because I wanted to be sure there is interest  in it first.

You can install and learn about it more from it's repository. It should work with PHP >=5.5 and ProcessWire 3.x.x. The support for 2.x.x version is not planned yet.

Please open an issue if you find bugs or you want some features added in issue tracker. Or you can share your experience with the module here in this thread.

[Robin S]

1 hour ago, Nurguly Ashyrov said:

I would love to continue to develop it further if I would only know that there is an interest in it.

I don't know much about GraphQL but I want to learn more, and your module looks like it will make it easy to get started with in PW. So definitely interested.

[teppo]

First of all, I think this looks great. I'm not entirely sure if I'll ever need something like this personally, but a module like this would no doubt be useful for some cases

To my best understanding the main point Ryan was making about REST was that since it's easy to add new views (or whatever you choose to call them) for a ProcessWire site, a separate module doesn't make as much sense as it does for a system that doesn't provide similar flexibility. I've got some, albeit very limited, experience with the REST API in WP, and in their case it definitely makes a lot of sense.

There's also the question of security: there may be cases where something is technically speaking public, but not accessible via your existing web site, and a "generic" REST API could result in some surprises there.. and, of course, if it's not read-only, that's a whole another thing to worry about.

All that being said, I'm looking forward to seeing where this module goes. An easy-to-use plug-n-play GraphQL API sounds like a great thing to have in one's toolbox 

[LostKobrakai]

I always though graphql would only be for qraph databases, but really this looks damn rad.

[dadish]

On 1/28/2017 at 6:10 PM, teppo said:

There's also the question of security: there may be cases where something is technically speaking public, but not accessible via your existing web site, and a "generic" REST API could result in some surprises there.. and, of course, if it's not read-only, that's a whole another thing to worry about.

By the way, in terms of security, this module follows the permission settings in ProcessWire. All it does is collects the templates that are viewable by the client via

$user->hasPermission('page-view', $template);

and for every request it makes sure to returns only those pages that have one of those templates. So, as long as user does not have permission to view the page, she won't be able to fetch it. I tried to make module reflect your existing settings as much as possible. It basically delegates everything possible to ProcessWire itself. 

 

On 1/28/2017 at 6:54 PM, LostKobrakai said:

I always though graphql would only be for qraph databases, but really this looks damn rad.

That is so true. I personally thought GraphQL was new SQL.

[LostKobrakai]

1 hour ago, Nurguly Ashyrov said:

All it does is collects the templates that are viewable by the client

There are also field access settings, just to be sure you're aware of them.

[dadish]

36 minutes ago, LostKobrakai said:

There are also field access settings, just to be sure you're aware of them.

Yes, I am aware of field permissions, thank you for reminding. I have not added support for them yet. Though it is definitely in my todo list for this module.

[apeisa]

Looks super cool!

Does it add some public endpoint to fetch data or how does it work outside the "console" example shown on the videos?

[dadish]

1 hour ago, apeisa said:

Looks super cool!

Does it add some public endpoint to fetch data or how does it work outside the "console" example shown on the videos?

Of course. In one of your templates (edit: In one of your template files) you simply do

<?php
echo $modules->get('ProcessGraphQL')->executeGraphQL();

and that's it. It will handle all GraphQL requests. There is more info in the repository.

[Ivan Gretsky]

Thank you very much for this module, @Nurguly Ashyrov!

I missed this graphql thing completely, though was messing around json api a bit. Had some great time reading about it. Seems like ProcessWire fits graphql like a glove. Your module should answer a lot of recent questions about integrating PW with vue.js and alike.

Dries Buytaert, the Drupal "godfather", has a nice article in his blog about the necessity for a contemporary CMS to have support for web-services built in. And his choice seems to be graphql and json api. I am sure that improving and promoting PW as a "headless CMS" kind of thing is something that could bring a lot of frontend developers to use PW. This module is the perfect start.

As I understand, mutations are a way to not only read, but write data? If so, that is certainly worth implementing, so a complete SPA could be possible with this graphql module alone.

 

[dadish]

16 hours ago, Ivan Gretsky said:

Seems like ProcessWire fits graphql like a glove.

You couldn't be more precise! GraphQL and ProcessWire fit each other very well. All this module does is just maps the ProcessWire's fieldtypes with GraphQL type system. It literally tells GraphQL that FieldtypeText is a StringType, FieldtypeDate is DateType and so on. And for getting the data, on average, it is less than a single line of code . Since you can access value of a page field like $pages->$fieldName all primitive fields inherit a method for accessing data from one place. I sure having lots of fun writing this module.

16 hours ago, Ivan Gretsky said:

Dries Buytaert, the Drupal "godfather", has a nice article in his blog about the necessity for a contemporary CMS to have support for web-services built in. And his choice seems to be graphql and json api. I am sure that improving and promoting PW as a "headless CMS" kind of thing is something that could bring a lot of frontend developers to use PW. This module is the perfect start.

I agree with Drupal "godfather" totally. The need for quick bootstrapping of an api service with flexible content structure is in very high demand. I had a hard time landing a job as a ProcessWire developer. So I target myself as a full-stack SPA developer in React.js/Node.js. I tried many of open source REST frameworks in Node.js that would help me get started with a project quickly. But non of them offered enough flexibility for my style of programming (I guess ProcessWire spoiled me ). At the time I figured out the best way to build REST api in Node.js I found out that REST is not flexible either. When an app starts evolving REST gets very messy. The Github built three versions of their REST api and still are not happy with it and now decided to release a GraphQL api which probably will not introduce breaking changes in the future, because GraphQL is designed that way.

I think if made correctly, this module could bring a great value to many ProcessWire users.

16 hours ago, Ivan Gretsky said:

As I understand, mutations are a way to not only read, but write data? If so, that is certainly worth implementing, so a complete SPA could be possible with this graphql module alone.

 That's right. That is the main goal of this module. I will eventually implement all the features that needed to build a complete SPA with this module. I just try to move carefully and a usage feedback from community would help a lot. Just installing it and making couple queries to confirm that it works as expected would be great.

[teppo]

On 28/01/2017 at 4:11 PM, Nurguly Ashyrov said:

By the way, in terms of security, this module mimics the permission settings in ProcessWire. All it does is collects the templates that are viewable by the client via [...] and for every request it makes sure to returns only those pages that have one of those templates. So, as long as user does not have permission to view the page, she won't be able to fetch it. I tried to make module reflect your existing settings as much as possible. It basically delegates everything possible to ProcessWire itself.

This sounds good, and that's basically everything that a "generic", publicly accessible API can do. I've got no complaints here

What I've found out while playing with the WP REST API a while ago is that even though permissions are sensible, it may end up showing more than the public site. Part of it is about things like (not just page but) field level permissions, and part is about pages that exist and are publicly viewable for technical reason, but are not *intended* for public consumption, if you get what I mean.

Surely it would be best to always use native permission rules to limit the visibility, but sometimes a site may have content that is viewable only if you know the direct URL, and a public API like this may make it "more public" than the developer intended. Another thing is that there may be a code-level permission check in place, and a module like this would have hard time figuring that out.

That being said, have you already implemented or are you considering implementing custom selector support for such limits? I.e. allow the developer to manually define a selector that returned pages must match, or alternatively should never match? I think that could make a lot of sense from a security point of view, particularly for public API endpoints, where it might actually work best as a per-endpoint setting 

[dadish]

41 minutes ago, teppo said:

...sometimes a site may have content that is viewable only if you know the direct URL, and a public API like this may make it "more public" than the developer intended.

Are you talking about pages with the status hidden? If thats the case, it should behave as expected. At this point this module accesses content only via $pages->find(). As long as $pages->find() does not return pages that are not intended for public this module should not make it accessible. I do not use $pages->get()as it bypasses some permission rules.

41 minutes ago, teppo said:

Another thing is that there may be a code-level permission check in place, and a module like this would have hard time figuring that out.

As a proper citizen of ProcessWire, one would implement code-level permission check by attaching a hook to User::hasPagePermission, User::hasTemplatePermission or any other equivalent, including field level permissions. For that cases this module wouldn't have to figure out anything, it will happen naturally. But for those cases where access to resources are checked outside of ProcessWire's permissions context, this module might not be a good fit for building service api.

[dadish]

1 hour ago, teppo said:

That being said, have you already implemented or are you considering implementing custom selector support for such limits? I.e. allow the developer to manually define a selector that returned pages must match, or alternatively should never match? I think that could make a lot of sense from a security point of view, particularly for public API endpoints, where it might actually work best as a per-endpoint setting 

I have not thought about this kind of security layer. Though it sounds reasonable. I will keep in mind this option. For now I plan to add an option to limit the templates that are meant to be accessible via public api by explicitly selecting them.

[teppo]

3 hours ago, Nurguly Ashyrov said:

Are you talking about pages with the status hidden? If thats the case, it should behave as expected. At this point this module accesses content only via $pages->find(). As long as $pages->find() does not return pages that are not intended for public this module should not make it accessible. I do not use $pages->get()as it bypasses some permission rules.

As a proper citizen of ProcessWire, one would implement code-level permission check by attaching a hook to User::hasPagePermission, User::hasTemplatePermission or any other equivalent, including field level permissions. For that cases this module wouldn't have to figure out anything, it will happen naturally. But for those cases where access to resources are checked outside of ProcessWire's permissions context, this module might not be a good fit for building service api.

Generally speaking you're, of course, right -- in most cases one should use built-in visibility settings and permission-related hooks, but it's not unheard of to check permissions in a template file either. Depends a bit on the use case. And yes, you're right that in such cases it may be preferable to avoid installing such a module at all.

Obviously this is mainly a problem with systems that include a enabled-by-default (or always enabled) built-in public API, and less so when enabling/installing the API itself is a conscious choice.

Either way, it's good to understand that exposing your content to the world via a publicly queryable API may uncover some surprises. This is one of the reasons why I find certain value in the idea of crafting the API per current needs and so that it only exposes the minimum viable amount of data 

Note: don't get me wrong, I'm definitely not against this module. What I've said here is mostly theoretical. I also think that your idea of being able to manually define queryable templates makes a lot of sense. While I'd still suggest enabling a selector instead, you obviously know the use cases (and the implementation) better.

 

[apeisa]

I hear what Teppo is saying, but choosing templates would pretty much nail that concern. Developer has chosen to show those templates through API, so leaves no place for confusion in my opinion. 

I will definitely give this one a good ride. 

[dadish]

On 1/30/2017 at 7:22 PM, teppo said:

Obviously this is mainly a problem with systems that include a enabled-by-default (or always enabled) built-in public API, and less so when enabling/installing the API itself is a conscious choice.

Either way, it's good to understand that exposing your content to the world via a publicly queryable API may uncover some surprises. This is one of the reasons why I find certain value in the idea of crafting the API per current needs and so that it only exposes the minimum viable amount of data 

You are completely right. I can't argue that "enabled-by-default" approach can lead to lots of security issues. That's why I am limiting the exposable pages only to selected templates. While the selector option is quite simple to implement I don't want to enable this kind of option because I believe it should not be this module's concern.

The way I see it, if this module stays consistent and retrieves data only through $pages->find() api (or it's equivalent like $page->children(), $page->siblings() etc) that should give the user any type of control with the security. For example what you suggest could be achieved with a single hook. Say this is your template file where you expose your GraphQL api (something like /site/templates/graphql.php).

<?php

echo $modules->get('ProcessGraphQL')->executeGraphQL();

What you suggest could be achieved like this.

<?php

wire()->addHookAfter('Pages::find', function($event) {
  $event->return = $event->return->filter($mySecuritySelector);
});

echo $modules->get('ProcessGraphQL')->executeGraphQL();

I would prefer users to approach security this way. This strategy to security gives full control for the user while allowing me to stick to a single rule when concerned about security and makes the code of the module much easier to reason about. I do realize that I could just insert the above code in the module and that's basically an implementation of what you suggest. But I don't want to encourage the user to solve security problems via module settings because no matter how hard I try, I won't be able to make this module dummy proof without limiting it's capabilities.

Another thing I wanted to mention is that I see this module as a GraphQL representation of ProcessWire api. Like @Ivan Gretsky mentioned, if done right, this could allow us to build lot's of useful developer tools on top of this module. Even a mobile app that gives you limited site administration capabilities. But only if module is consistent with how ProcessWire behaves. And that includes the security of course.

On 1/30/2017 at 7:22 PM, teppo said:

Note: don't get me wrong, I'm definitely not against this module. What I've said here is mostly theoretical. I also think that your idea of being able to manually define queryable templates makes a lot of sense. While I'd still suggest enabling a selector instead, you obviously know the use cases (and the implementation) better.

Oh no sir, not at all. I value your opinion very much. That's exactly what I wanted to hear from the community, opinions. I am thankful to you for mentioning this aspect of the module in it's early stage, before I started to implement other features that depend on it, like authentication or others that I might not think of right now.

[BitPoet]

Just a thought, since this is what I try to do with my own modules: could you add hookable methods in ProcessGraphQL that allow implementing custom restrictions? These hookables could be no-ops if not hooked and receive all information about the query at the time of calling, enabling users to filter or reject queries before or after they've run.

It's an intriguing module in any case. Thanks for sharing it with us!

[dadish]

1 hour ago, BitPoet said:

Just a thought, since this is what I try to do with my own modules: could you add hookable methods in ProcessGraphQL that allow implementing custom restrictions? These hookables could be no-ops if not hooked and receive all information about the query at the time of calling, enabling users to filter or reject queries before or after they've run.

That's a very good idea! Will do that. Thank you for the tip.

[Ivan Gretsky]

Here is another idea. What about the ability to make a number of API endpoints with different allowed templates and restrictions based on single instance of a module. By passing an argument to executeGraphQL() or something like that. With the ability do distinguish them in hooks.

A way to have a public and less public API on the same site. I guess than some authentication questions could be solved on a template level.

[dadish]

19 hours ago, Ivan Gretsky said:

Here is another idea. What about the ability to make a number of API endpoints with different allowed templates and restrictions based on single instance of a module. By passing an argument to executeGraphQL() or something like that. With the ability do distinguish them in hooks.

But all you said is already can be achieved.  No need to do anything on my side. Maybe add documentation on module's properties though. You can modify the module settings via api by overwriting them. So here how you can set different templates for different endpoints.

// /site/templates/graphql-endpoint1.php
<?php
$ProcessGraphQL = $modules->get('ProcessGraphQL');
$ProcessGraphQL->legalTemplates = array('skyscraper', 'city');
echo $ProcessGraphQL->executeGraphQL();
?>

// /site/templates/graphql-endpoint2.php
<?php
$ProcessGraphQL = $modules->get('ProcessGraphQL');
$ProcessGraphQL->legalTemplates = array('architect', 'basic-page');
echo $ProcessGraphQL->executeGraphQL();

// /site/templates/graphql-endpoint3.php
$ProcessGraphQL = $modules->get('ProcessGraphQL');
echo $ProcessGraphQL->executeGraphQL(); // here it will use default settings that you set via admin interface

 

For the ability to distinguish the versions of your GraphQL endpoints. That's also doable without much effort. We are talking about ProcessWire after all. Here how it might look like.

$config->GraphqlEndpointID = 123;
echo $modules->get('ProcessGraphQL')->executeGraphQL();
$config->GraphqlEndpointID = false;

And same thing in other template files with different endpoint id. Now anywhere you attach a hook, you can know which endpoint of your api is being executed, or if it is being executed at all. You just need to add a conditional block in your hook. Something like

if (wire('config')->GraphqlEndpointID === 123) {
  // some bussiness here
}

 

I am a bit confused though. One of selling points of GraphQL is that there is only one url that you need to deal with. It's just `example.com/graphql/` and nothing more. No more this

  GET example.com/graphql/skyscrapers/
  GET example.com/graphql/skyscrapers/{id}
 POST example.com/graphql/skyscrapers/
  PUT example.com/graphql/skyscrapers/{id}
PATCH example.com/graphql/skyscrapers/{id}
  GET example.com/graphql/architects/
 ...

It's only one endpoint for everything you need. That's actually is the way it is encouraged to build GraphQL api. Also it is only one HTTP verb you need to use, which is POST. You can stop thinking about dealing with PUT, PATCH, HEAD, OPTION and more. You only need this with GraphQL.

POST example.com/your-endpoint-url/

And that's it. One HTTP verb and one url to rule them all . 

[Ivan Gretsky]

I was thinking of a few endpoints to handle authentication with the template permissions. One for the public, another for the registered users with the ability to make changes via mutations when they'll be implemented. There are other ways to handle authentication of course. But then we will have to implement some restrictions via module itself. But maybe I am not seeing something obvious here.

[blynx]

2 hours ago, Nurguly Ashyrov said:

I am a bit confused though. One of selling points of GraphQL is that there is only one url that you need to deal with. It's just `example.com/graphql/` and nothing more. No more this [...]

Thats true - yet, when I was reading this thread and thinking about that restriction and security stuff and about the differences of graphQL and REST - 
I thought that it might actually be handy to simply combine the two.
Using REST for simple API calls - simple forms, logins... and so on - and using graphQL for sophisticated stuff for your interactive JS app.

You might want to do this to differ between the general public and logged in users.

/graphQL/public/ and /graphQL/private/

Since by defining legal templates in one endpoint those will be legal for anyone. But you want to restrain the public data for the guest user as much as possible - but at the same time allow more available data for logged in users, even granularly define legal data for different user roles.

You could do it just like that I think?
/public/ will only allow very few legal things in general.
/private/ will handle anything else automatically via ProcessWires granular permission system for logged in users.

Or maybe there is a way to handle that all in one endpoint? Idk ...  ... oh. of of course you can ask in your single endpoint for the user role ...... .... ...  as simple as that.

 

Maybe it would also be handy to have an option to define "legalFields" in the same way as templates? So you can restrain the amount of data which is instantly public a bit more granularly. So you can just hide anything which doesn't need to be public at all.

 

And by the way - good work!
I will definitely learn a couple of things here again ... thanks!

Edited February 1, 2017 by blynx
having a fancy fancy

[dadish]

3 hours ago, Ivan Gretsky said:

I was thinking of a few endpoints to handle authentication with the template permissions. One for the public, another for the registered users with the ability to make changes via mutations when they'll be implemented. There are other ways to handle authentication of course. But then we will have to implement some restrictions via module itself. But maybe I am not seeing something obvious here.

There is not need for different endpoint for users with different roles. The module does not have any authentication/authorization logic on it's own. The users that will be able to authenticate with this module are the same users in your ProcessWire installation. When I mentioned implementing authentication, I was talking about logging in via GraphQL api, like via AJAX. In reality it will be the same $session->login('username', 'password'), nothing more.

1 hour ago, blynx said:

Since by defining legal templates in one endpoint those will be legal for anyone.

No, no. Of course not. I am sorry for the confusion here. Legal templates mean legal for the api. It does not mean it will make it available to the public. Like I mentioned earlier the module checks if the requesting user has permissions to view, edit, create and etc. If say you select user template as legal. It does not mean it will be public. It means it is available via api to those who are authorized to view it, authorized via ProcessWire's access control system.

I personally don't think there is even a need for the legal templates option. But it is helpful if you have too many templates and selecting only few can reduce the schema size and make api faster.

I think there is a bit confusion about this. I want emphasize that this module does not make any data public, nor does it anything private. That is not the module's concern. The module's job is to make your data available in a JSON format, in addition providing the ability to consume that JSON data via GraphQL api. If the user does not have permissions to view a certain page according to ProcessWire's access control system then he won't be able to fetch it.

45 minutes ago, blynx said:

Maybe it would also be handy to have an option to define "legalFields" in the same way as templates? So you can restrain the amount of data which is instantly public a bit more granularly. So you can just hide anything which doesn't need to be public at all.

The same goes for fields. When implemented the user will be able to access only those fields that he is authorized via ProcessWire's access control. But I will add an option for legal fields also, because that also could help reduce the initial schema size.

[blynx]

50 minutes ago, Nurguly Ashyrov said:

I am sorry for the confusion here. Legal templates mean legal for the api. It does not mean it will make it available to the public. Like I mentioned earlier the module checks if the requesting user has permissions to view, edit, create and etc.

Ah sorry, I know - I expressed myself not accurately -

I also think legalTemplates and Fields are basically not necessary - but i think it is just a very convenient way to reduce the available data

 

50 minutes ago, Nurguly Ashyrov said:

I think there is a bit confusion about this. I want emphasize that this module does not make any data public, nor does it anything private. That is not the module's concern. The module's job is to make your data available in a JSON format, in addition providing the ability to consume that JSON data via GraphQL api. If the user does not have permissions to view a certain page according to ProcessWire's access control system then he won't be able to fetch it.

The confusion might be about this:

Normally in processwire templates you have to "make the fields public" by manually echoing data in a template (echo $page->title) - so actually for a guest user everything is hidden by default - though by permission actually authorized.

With this module - everything gets "unveiled" (to use another term here) automatically. This is what I meant by "public" and "private".

... am I right?

Edited February 1, 2017 by blynx
added second quote and blabla