Jump to content
thomasaull

Module: RestApi

Recommended Posts

Some time ago I created a site profile for creation of a REST API with ProcessWire. Since I kept struggeling with updating stuff between different projects which use this, I decided to convert it into a module. It is now ready for testing: https://github.com/thomasaull/RestApi

Additionally I added a few small features:

  • automatic creation of JWT Secret at module install
  • routes can be flagged as auth: false, which makes them publicly accessible even though JWT Auth is activated in module settings

To check things out, download and install the module and check the folder /site/api for examples.

If you find any bugs or can think of improvements, please let me know!

  • Like 15

Share this post


Link to post
Share on other sites

@nicolant had some problems to get the old site profile working with different domains for api and client:

It should work out of the box with the module, but apparently you need to add an OPTIONS route for every endpoint. I could automate this, but don't know if it's a good idea to do this for every route, since I'm not an expert on this  CORS / preflight. Opinions?

Share this post


Link to post
Share on other sites

Great, have to try this soon. Thanks for sharing!

Just flew over the readme and saw that you are always using http instead of https in your instructions. Is that a typo? Or does it also work with http? Maybe a hint that https would be more secure would make sense?

No idea how JWT Auth works and I also didn't have a look at your examples yet. Just wanted to say thank you, follow the thread and maybe my comment is even useful 🙂

Share this post


Link to post
Share on other sites

Thanks @bernhard! The API does not really care if it's served over http or https, it's just HTTP(s)-Requests after all. If your server is configured to redirect all http requests to https, it'll do so with these as well. However, it's always a bit of a hazzle to test locally, so I left  the examples as is and put a note that it's a good idea to use https 🙂

JWT Auth (in this case) works like the following:

  1. The client sends a login-request with username + password (this definitely should go over HTTPS)
  2. The server checks the login credentials and if correct, creates a unique token with an added encrypted signature
  3. The client uses this token to authentiate every following request

Since the client does not know the secret, he cannot modify the contents of the token without making it invalid

That's basically how I understood it 😉

  • Like 2

Share this post


Link to post
Share on other sites

I'd like to add that JWT is to be taken with a grain of salt. Once a token is issued there's not way to revoke it's validity unless one is storing something in the db to validate the jwt against. Having something stored on the server makes it essentially a more complex, more manual session handling, so there's at least in my opinion no longer a point to use JWTs. They're mostly useful for shortlived tokens or you can use them if you implement something like oauth or similar stuff. I personally wouldn't suggest using them like they're implemented currently in the module.

  • Like 2

Share this post


Link to post
Share on other sites

Thanks for your input @LostKobrakai , that's exactely why I put this up, since it's a security sensitive topic.

Again I'm not an expert on JWT, but I thought that's what the „exp" Parameter is for? In the module it's set to the "sessionExpireSeconds" of PWs config (which is 24h I think). I made a quick test and set it to 2 minutes and while it worked at first after a couple of minutes I got an Error: "Error: Exception: Expired token".

So I guess you're right, there is no way to revoke its validity but on the other hand it seems like it's not valid forever (at least if you don't set it to be)

Share this post


Link to post
Share on other sites

It's certainly not valid forever, but once a token is compromised there's no way to just invalidate that single token before it's going to expire on it's own. By using plain old sessions you have the ability to do so. And depending on the context 24h can be quite a long time. 

Share this post


Link to post
Share on other sites

Absolutely true. So what would be a feasable Alternative then if I don‘t want to use sessions? Say, because of multiple services (where the alternative would be to store alle the user data on every service)

Share this post


Link to post
Share on other sites

Hey Thomas,

Thanks for this module, looks really interesting, and I'm looking forward to giving it a go. Just a quick comment on this part of the README:

Quote

Currently the endpoint for the api is hardcoded to /api. That means a page with the name api is not going to work if you`ve installed this module. I might make the endpoint configurable via module settings in the future.

I think that a configurable endpoint name would definitely be a worthwhile option – or if that turns out to be difficult, perhaps you might want to consider something slightly more descriptive, such as /rest-api/ or something?

To be completely honest I'm being a bit selfish here: I've got the bad habit of using /api/ for any site-specific API implementations I might need, and that would pose an issue for this particular module 😅

Reminds me of the "two hard things in computer science" thing. This thread already covers both: cache (well, token...) invalidation and naming things 🤔

  • Like 3

Share this post


Link to post
Share on other sites

Thank you @teppo A configurable endpoint ist not really difficult, so I just did it – 0.0.2 has a field in the module settings for that 🙂

  • Like 3

Share this post


Link to post
Share on other sites

I'd really question why you don't want to use plain old sessions. I mean it doesn't have to be cookie-based even if that's imho still the easiest way to not get bitten by compromised sessions. To give my argumentation a bit more ground work you might want to look into the following blogposts. The latter has a quite simple flow-chart about why JWT just doesn't work well for session authentication.

http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/
http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-for-sessions-part-2-why-your-solution-doesnt-work/

There's also oauth, but that's mostly just another way to obtain a "session" with an service, where you allow a third party to access your content without giving away your credentials.

  • Like 1

Share this post


Link to post
Share on other sites

I actually stumbled upon these two articles when I did some resarch on saturday and I think I'm getting your point. However I don't agree with all of the statements made there:

On the flow chart on the far right it says “I'll just use refresh tokens" which he states couldn't be revoked – afaik usually you save the refresh token in the datebase of your Auth Server and everytime a user wants to refresh a token you check if it is still valid. So e.g. you could hand out short lived tokens (like 5-10 minutes) and everytime it expires the client has to obtain a new token via the refresh token if it's not revoked.

In an upcoming project we might have multiple endpoints for different task, where it just sounded good to have an Auth Server which holds all the user information and hands out tokens, which the client uses on the other server to access something. On "Footnote: microservice architectures" of part 2 the autor suggests to use single use tokens to get a session on the other service, which I think means, If I want to revoke a session I need to do it on multiple places right?

Aaaanyway, I did some tests with the API Module, sessions and a cross-origin client and it also works quite well, so with 0.0.3 you can choose your auth method in module settings between none / session / jwt

  • Like 3

Share this post


Link to post
Share on other sites
On 9/25/2018 at 7:00 PM, thomasaull said:

On the flow chart on the far right it says “I'll just use refresh tokens" which he states couldn't be revoked – afaik usually you save the refresh token in the datebase of your Auth Server and everytime a user wants to refresh a token you check if it is still valid. So e.g. you could hand out short lived tokens (like 5-10 minutes) and everytime it expires the client has to obtain a new token via the refresh token if it's not revoked.

As soon as you save anything on your server or in a db you loose the one feature JWTs really have over sessions, the one by which people generally choose JWTs: statelessness. So for a website adding an API for a JS frontend, why go through the length of trying to make JWT auth secure via some refresh token and some db table storing them, when php sessions already bring all you need to do basically the same without the fancy buzzwords and using cookies, which are more save in their handling on the browser side as well. Your argument about duration of authentication vs. refreshing the authentication often is a topic, which is actually totally unrelated to how any authentication prove is stored on the client. Both cookies as well as JWTs cannot be revoked once they're on the client. The difference is that php brings all the server side logic needed for sessions and their revokation, but you need to implement all of that for JWTs, which aren't really meant for stateful authentication in the first place.

On 9/25/2018 at 7:00 PM, thomasaull said:

In an upcoming project we might have multiple endpoints for different task, where it just sounded good to have an Auth Server which holds all the user information and hands out tokens, which the client uses on the other server to access something. On "Footnote: microservice architectures" of part 2 the autor suggests to use single use tokens to get a session on the other service, which I think means, If I want to revoke a session I need to do it on multiple places right?

That sounds like a use-case for oauth.

About your question: If you have multiple servers and you need to (be able to) actively revoke access for some user you need those server(s) to be aware of the revokation. Using client side stored tokens alone you just cannot revoke validity. Your server(s) could always ask your auth server about validity or your auth server could notify your task server(s) to drop sessions for users. You just need some way to make all your servers aware of the revoked access, so yeah multiple places need to be informed somehow. 

The usecase of getting a short-lived, single-use token from your auth server to authenticate against some task server for starting a session is one JWTs could fit in my opinion. 

Share this post


Link to post
Share on other sites

Hi @thomasaull

I have been using this module during the last week, to rebuild my API logic from the ground up. It's really nice work, thank you for releasing it! 😀

One thing I've noticed though, is that the exceptions handler (Router::handleException) is a bit overactive, and shuts everything down on non-critical exceptions. 

For instance, when running a PNG file through the PW ImageResizer ($img->size() etc), PW throws an exception, because the exif_read_data function is not available for PNG's. PW deals with this by using the error control operator (@method): the code still runs fine, while silently throwing an exception message. 

However, using the RestAPI module, this renders (and logs) an error message and stops any further output.   

It is of course easy to just comment out Router.php line 25, where you set the handler. But perhaps this could work in another way? Maybe making it a configurable option?

Share this post


Link to post
Share on other sites

@eelkenet Thanks for using this module and I'm glad it is useful to you 🙂

This is actually the first time, I've heard about the @Operator regarding errors. However, I found an interesting paragraph on the page you have linked to in your post:

Quote

If you have set a custom error handler function with set_error_handler() then it will still get called, but this custom error handler can (and should) call error_reporting() which will return 0 when the call that triggered the error was preceded by an @.

So I added a line, which checks for the error reporting before displaying an error: https://github.com/thomasaull/RestApi/commit/fe63cc48cfcc6d58489f019d5026764cb60d14e5

Could you please manually download the module from the develop branch on Github and give me quick feedback if this resolves your issue? https://github.com/thomasaull/RestApi/archive/develop.zip

  • Thanks 1

Share this post


Link to post
Share on other sites

I ran into an issue that is related to the way the RestAPI circumvents the pagetree structure (running the checkIfApiRequest hook before rendering any page).
This method made it impossible to use ProCache for API requests that could (and should) return a cached result, such as for static site content. I thought about creating a custom caching system on top of RestApi, but ProCache is just too well designed to ignore here.  

I wrote a post about this on the ProCache VIP-forum, but as this forum is not accessible to all people I'd like to share my (admittedly hacky) solution for this. Basically I add another (cacheable) endpoint in the pagetree, which pipes the request to the RestApi endpoint:

  1. Create a new template and corresponding page (I called both 'api').
  2. Set the content-type of this template to application/json, and disable any prepending/appending of files. 
  3. Add the following code to the template:
<?php //site/templates/api.php

$protocol = $config->https ? "https://" : "http://";
$endpoint = $modules->get("RestApi")->endpoint;
$hostname = $config->httpHost;
$segments = implode("/", $input->urlSegments);
$url =  $protocol.$hostname."/".$endpoint.$segments;

return file_get_contents($url);

I'm sure there would be a better, cleaner way of doing this. A current downside is that there now are 2 seemingly identical endpoints for my site.
One is cached, and the other is 'live'. 

Any ideas?

Share this post


Link to post
Share on other sites

Thomas,

I recently found your module, good job. 

Right now im thinkering with Electron and making Processwire to serve as a headless CMS. So your module is quite handy. 

I forked the module on GitHub and made it a little bit more connected to Processwire. To sum things up, I creates a "Endpoint Container" in the page tree where you can add your routes and methods. 

It still needs to add responding classes to provide content ;) I added a skeleton Class called "Blog" to get all contents under the "Home" Page or a specific Page via ID.

I created a Pull Request, maybe you like my approach.

https://github.com/Luis85/RestApi

 

  • Like 2

Share this post


Link to post
Share on other sites

@LuisM Hey Luis, yes I saw it on Github. I'm a little short on time from last week to in a few days, I'm going to get back at you at the PR as soon as possible!

  • Like 1

Share this post


Link to post
Share on other sites

@thomasaull Thanks for this module! It seems to be a great starting point for building an API. 

I have 2 questions:

1. How would you approach a multilanguage API? My idea would be to add a query param to the api call (e.g. /api/posts/?lang=fr ) and switch the user language before getting the field values. Is there another/better solution?

2. I don't need it right now, but how would I implement a session authentification when accessing the api?

 

Thanks!

Share this post


Link to post
Share on other sites

Hey,

Thanks for that  module.

 

in the module description it says:

 

Authorization: JWT

To use JWT-Auth you have to send a GET Request to http://yourhost/api/auth with two parameters, username and password. The API will create and return you the JWT-Token which you have to add as a header to every following request:

 

Actually I think it has to be a POST Request. In the "DefaultRoutes" the Route is defined like this.
 

['POST', '', Auth::class, 'login', ['auth' => false]],

 

Share this post


Link to post
Share on other sites
1 hour ago, pmichaelis said:

Hey,

Thanks for that  module.

 

in the module description it says:

 

Authorization: JWT

To use JWT-Auth you have to send a GET Request to http://yourhost/api/auth with two parameters, username and password. The API will create and return you the JWT-Token which you have to add as a header to every following request:

 

Actually I think it has to be a POST Request. In the "DefaultRoutes" the Route is defined like this.
 


['POST', '', Auth::class, 'login', ['auth' => false]],

 

You're totally right, good catch thank you! I updated the Readme accordingly.

  • Like 1

Share this post


Link to post
Share on other sites
On 12/18/2018 at 4:27 PM, Torsten Baldes said:

@thomasaull Thanks for this module! It seems to be a great starting point for building an API. 

I have 2 questions:

1. How would you approach a multilanguage API? My idea would be to add a query param to the api call (e.g. /api/posts/?lang=fr ) and switch the user language before getting the field values. Is there another/better solution?

2. I don't need it right now, but how would I implement a session authentification when accessing the api?

 

Thanks!

Thanks Thorsten!

How you handle incoming api requests is generally totally up to you – you have all the freedom 🙂 Your idea sounds like a good and easy solution though. However, currently there is no possibility to implement such thing globally on every request. For this maybe it would be a good idea to make the handle() method in Router.php hookable. Maybe you want to test it and provide a PR for this. It would be very welcome 🙂

For Session auth just activate the option in the module settings and make sure to provide the withCredentials option: https://github.com/thomasaull/RestApi/blob/master/README.md#authorization-session. In your frontend app just send a login request to the auth endpoint: https://github.com/thomasaull/RestApi/blob/master/README.md#authorization-jwt and it should (hopefully) work

  • Like 1

Share this post


Link to post
Share on other sites

@LuisM @thomasaull 
Hi, great module(s)!

 

But maybe it could be better 

  1. admin routes / endpoints from PW backend (map endpoint to a pw page / template or any other self defined php file)?
  2. maybe just use PW templates and (sub-)pages from PW page tree (/api/*)?
 
I searched for a simple module to deliver JSON output like Pages2JSON module, but would benefit from auth methods implemented with RestApi module...

Share this post


Link to post
Share on other sites

@pwFoo What exactely do you mean by 1)? Mapping an endpoint to a PW Page is as easy as

$page = wire('pages')->get(1042);

in your endpoint function.

Mapping an endpoint to a php file is the intendend behaviour of the module, check the example: https://github.com/thomasaull/RestApi/blob/master/apiTemplate/Example.php which get's mapped in the Routes.php: https://github.com/thomasaull/RestApi/blob/master/apiTemplate/Routes.php

2) That's basically the approach @LuisM used in his PR. I'm not sure if it's the best solution, check my comment on Github: https://github.com/thomasaull/RestApi/pull/1#issuecomment-450135767

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Process refunds and send customer notifications from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory (alpha version only available via GitHub) SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By d'Hinnisdaël
      Happy new year, everybody 🥬
      I've been sitting on this Dashboard module I made for a client and finally came around to cleaning it up and releasing it to the wider public. This is how it looks.
      ProcessWire Dashboard

      If anyone is interested in trying this out, please go ahead! I'd love to get some feedback on it. If this proves useful and survives some real-world testing, I'll add this to the module directory.
      Download
      You can find the latest release on Github.
      Documentation
      Check out the documentation to get started. This is where you'll find information about included panel types and configuration options.
      Custom Panels
      My goal was to make it really simple to create custom panels. The easiest way to do that is to use the panel type template and have it render a file in your templates folder. This might be enough for 80% of all use cases. For anything more complex (FormBuilder submissions? Comments? Live chat?), you can add new panel types by creating modules that extend the DashboardPanel base class. Check out the documentation on custom panels or take a look at the HelloWorld panel to get started. I'm happy to merge any user-created modules into the main repo if they might be useful to more than a few people.
       Disclaimer
      This is a pre-release version. Please treat it as such — don't install it on production sites. Just making sure 🍇
      Roadmap
      These are the things I'm looking to implement myself at some point. The wishlist is a lot longer, but those are the 80/20 items that I probably won't regret spending time on.
      Improve documentation & add examples ⚙️ Panel types Google Analytics ⚙️ Add new page  🔥 Drafts 🔥 At a glance / Page counter 404s  Layout options Render multiple tabs per panel panel groups with heading and spacing between ✅ panel wrappers as grid item (e.g. stacked notices) ✅ Admin themes support AdminThemeReno and AdminThemeDefault ✅ Shortcuts panel add a table layout with icon, title & summary ✅ Chart panel add default styles for common chart types ✅ load chart data from JS file (currently passed as PHP array) Collection panel support image columns ✅ add buttons: view all & add new ✅
    • By Robin S
      This module is inspired by and similar to the Template Stubs module. The author of that module has not been active in the PW community for several years now and parts of the code for that module didn't make sense to me, so I decided to create my own module. Auto Template Stubs has only been tested with PhpStorm because that is the IDE that I use.
      Auto Template Stubs
      Automatically creates stub files for templates when fields or fieldgroups are saved.
      Stub files are useful if you are using an IDE (e.g. PhpStorm) that provides code assistance - the stub files let the IDE know what fields exist in each template and what data type each field returns. Depending on your IDE's features you get benefits such as code completion for field names as you type, type inference, inspection, documentation, etc.
      Installation
      Install the Auto Template Stubs module.
      Configuration
      You can change the class name prefix setting in the module config if you like. It's good to use a class name prefix because it reduces the chance that the class name will clash with an existing class name.
      The directory path used to store the stub files is configurable.
      There is a checkbox to manually trigger the regeneration of all stub files if needed.
      Usage
      Add a line near the top of each of your template files to tell your IDE what stub class name to associate with the $page variable within the template file. For example, with the default class name prefix you would add the following line at the top of the home.php template file:
      /** @var tpl_home $page */ Now enjoy code completion, etc, in your IDE.

      Adding data types for non-core Fieldtype modules
      The module includes the data types returned by all the core Fieldtype modules. If you want to add data types returned by one or more non-core Fieldtype modules then you can hook the AutoTemplateStubs::getReturnTypes() method. For example, in /site/ready.php:
      // Add data types for some non-core Fieldtype modules $wire->addHookAfter('AutoTemplateStubs::getReturnTypes', function(HookEvent $event) { $extra_types = [ 'FieldtypeDecimal' => 'string', 'FieldtypeLeafletMapMarker' => 'LeafletMapMarker', 'FieldtypeRepeaterMatrix' => 'RepeaterMatrixPageArray', 'FieldtypeTable' => 'TableRows', ]; $event->return = $event->return + $extra_types; }); Credits
      Inspired by and much credit to the Template Stubs module by mindplay.dk.
       
      https://github.com/Toutouwai/AutoTemplateStubs
      https://modules.processwire.com/modules/auto-template-stubs/
    • By Mike Rockett
      Jumplinks for ProcessWire
      Release: 1.5.60
      Composer: rockett/jumplinks
      ⚠️ NOTICE: 1.5.60 is an important security patch-release for an XSS vulnerability discovered by @phlp. It's HIGHLY RECOMMENDED that all Jumplinks users update to the latest version as soon as possible.
      Jumplinks is an enhanced version of the original ProcessRedirects by Antti Peisa.
      The Process module manages your permanent and temporary redirects (we'll call these "jumplinks" from now on, unless in reference to redirects from another module), useful for when you're migrating over to ProcessWire from another system/platform. Each jumplink supports wildcards, shortening the time needed to create them.
      Unlike similar modules for other platforms, wildcards in Jumplinks are much easier to work with, as Regular Expressions are not fully exposed. Instead, parameters wrapped in curly braces are used - these are described in the documentation.
      Under Development: 2.0, to be powered by FastRoute
      As of version 1.5.0, Jumplinks requires at least ProcessWire 2.6.1 to run.
      View on GitLab
      Download via the Modules Directory
      Read the docs
      Features
      The most prominent features include:
      Basic jumplinks (from one fixed route to another) Parameter-based wildcards with "Smart" equivalents Mapping Collections (for converting ID-based routes to their named-equivalents without the need to create multiple jumplinks) Destination Selectors (for finding and redirecting to pages containing legacy location information) Timed Activation (activate and/or deactivate jumplinks at specific times) 404-Monitor (for creating jumplinks based on 404 hits) Additionally, the following features may come in handy:
      Stale jumplink management Legacy domain support for slow migrations An importer (from CSV or ProcessRedirects) Feedback & Feature Requests
      I’d love to know what you think of this module. Please provide some feedback on the module as a whole, or even regarding smaller things that make it whole. Also, please feel free to submit feature requests and their use-cases.
      Note: Features requested so far have been added to the to-do list, and will be added to 2.0, and not the current dev/master branches.
      Open Source

      Jumplinks is an open-source project, and is free to use. In fact, Jumplinks will always be open-source, and will always remain free to use. Forever. If you would like to support the development of Jumplinks, please consider making a small donation via PayPal.
      Enjoy! 🙂
    • By Robin S
      Add Image URLs
      Allows images/files to be added to Image/File fields by pasting URLs.

      Usage
      Install the Add Image URLs module.
      A "Paste URLs" button will be added to all image and file fields. Use the button to show a textarea where URLs may be pasted, one per line. Images/files are added when the page is saved.
       
      https://github.com/Toutouwai/AddImageUrls
      https://modules.processwire.com/modules/add-image-urls/
×
×
  • Create New...