New Module: AppApi

[Marcel_M]

Thanks @androbey, I found the folder api in site and the Example.php file. I'll try things out ?

EDIT:

I went through the Routs.php and tested example.com/api/test and in the browser I got "message: test successful"

But when I try /processwire/setup/appapi I get Unrecognized path.

In order to get it in the admin page, do I first have to edit the Routs.php file? Because in the manual I didn't see something like that or I have over seen it.

Best,

Marcel

[Sebi]

Hi everyone!

First, my two cents to @androbey: my first guess would be that maybe the spaces are not escaping correctly? Spaces in GET parameters need to be converted to %20. So "This is a test" becomes "q=this%20is%20a%20test" in the URL. Depending on the framework you use for your api request, this conversion may not happen automatically. In Javascript, for example, there is the encodeURIComponent() function that can do this conversion for you. In PHP there is the function urlencode() which does the same. Can you get further with it?

Now to @Marcel: The message "no endpoint defined" already shows that the module was installed correctly. This message comes from the default route definition:

['*', '', AppApiHelper::class, 'noEndPoint', ['auth' => false]]

And I can also tell you already that you are sending a valid api key along. Otherwise you would get an error message "Apikey not valid".

Next, you can call /api/auth once. This request will return the current user. Without auth token this should be the guest user:

{
  "id": 40,
  "name": "guest",
  "loggedIn": false
}

After that you can start implementing your own endpoints. Here in the wiki you can find everything you need to know: https://github.com/Sebiworld/AppApi/wiki/3.0:-Creating-Endpoints

EDIT @Marcel: Sorry, I did not see you last comment. Routes.php does only influence your endpoints. The admin-page should be created automatically during installation and should be accessible in the "setup" header menu. I appended a screenshot from a German-translated backend. "Verwaltung" means "setup". Maybe you have to delete and re-install AppApi if the page was not created automatically...

Edited October 23, 2021 by Sebi

[androbey]

Hi @Sebi,

thank you for your response. That was the first thing I checked. 
For the API request I am using vanilla JS (fetch) and I'm using the encodeURIComponent() function. Also in network tab in dev tools I can see the request is transmitted as it should be according to RFC 3986. I would digg deeper, but I don't know where the status code gets injected for a (in fact) successful request. 

[Sebi]

  On 10/23/2021 at 11:55 AM, androbey said:

For the API request I am using vanilla JS (fetch) and I'm using the encodeURIComponent() function. Also in network tab in dev tools I can see the request is transmitted as it should be according to RFC 3986. I would digg deeper, but I don't know where the status code gets injected for a (in fact) successful request. 

Expand  

Hey @androbey, I tried a few things and definitely found inconsistencies. It looks like spaces, but also other characters (e.g. "+") in the url path cause weird behavior.

In any case, any GET parameters that might be appended then seem to be ignored. At /api/search/my%20test?test=1 I cannot find the GET parameter test in $_GET of PHP. I can't tell exactly where this behavior is coming from (FastRoute? PHP? Apache?), but it probably won't work to specify the search string as you intended.

If you specify it as a GET parameter (/api/search?q=my%20test) instead, it works without problems. The corresponding route definition would look like this:

'search' => [
        ['GET', '', Search::class, 'getSearchResults', ["auth" => true]],
],

 

[androbey]

Hey @Sebi,

thanks for looking into the issue. You are absolutley right, when specifying it as a GET parameter it works without issues. This is anyhow the better way. 
Thank you again ? 

[Sebi]

Great news! Version 1.2.0 is here!
The changes in the code are actually not huge, but all the bigger are the additional possibilities we have now: With the new version, modules can register their own endpoints. These endpoints are then merged with the traditional definition in Routes.php.

You may ask: But @Sebi, why are you so excited about this? Well. This means that endpoints can now be easily installed and updated. And we can now easily share our great and useful endpoints with the ProcessWire community!

I'll get started then: My first AppApi module is AppApiFile. This module provides the /file endpoint for retrieving files uploaded to ProcessWire pages. 

I have extended the wiki with a small tutorial and sample code for creating an AppApi module:
https://github.com/Sebiworld/AppApi/wiki/4.0:-AppApi-Modules

I can't wait to see what you people will do with it! ?

[Sebi]

There was still a small problem that prevented the module from being automatically updated from within ProcessWire.
I had renamed the root branch in Github from master to main, but then the download path in the ProcessWire module directory didn't fit anymore.

Long Story Short: Now everything works again ?

[Pixrael]

@Sebi How to set the "API Endpoint" module configuration correctly? I need to create a website (subdomain) that is an API server only, no web pages will be served from there.. I test the parameter like "/" in order to works like: https://api.domain.com/users/ but this show a "404 Not Found".. I need the endpoint entry of the API in the "home" of the website

[Sebi]

Hi @Pixrael! 

At the moment, this is not possible. In the AppApi module, a route must be specified that is not yet occupied by a classic ProcessWire route. AppApi uses a hook on 404 (Page not found) error and then replaces the return with the AppApi content. 

I am facing a similar problem right now. I would like to use ProcessWire exclusively as a headless CMS for an app. If you have access to the domain and subdomain settings, it might work to run ProcessWire under a subdomain and link the main domain to /api.

However, I've been thinking for a while about how to add api responses to existing ProcessWire pages (e.g. the root page under "/").

[kongondo]

  On 11/16/2021 at 1:59 PM, Sebi said:

At the moment, this is not possible. In the AppApi module, a route must be specified that is not yet occupied by a classic ProcessWire route. AppApi uses a hook on 404 (Page not found) error and then replaces the return with the AppApi content. 

Expand  

Just wondering whether URL Path Hooks would be of help in this respect?

https://processwire.com/blog/posts/pw-3.0.173/#introducing-url-path-hooks

[Pixrael]

  On 11/16/2021 at 4:51 PM, kongondo said:

Just wondering whether URL Path Hooks would be of help in this respect?

https://processwire.com/blog/posts/pw-3.0.173/#introducing-url-path-hooks

Expand  

@kongondo This does not work, these new hook functions only work when the requested URLs are not mapped directly to a page and the home page is "always there".

@Sebi Could it be possible to use the homepage template file to "wireIncludeFile" the routes file, or to directly pass the request to your module, ex. initialize a module static function with the input object? AppApi::Execute($input);

[Sebi]

@Pixrael What I do on ProcessPageView::pageNotFound is calling this function:

	public function ___handleApiRequest(HookEvent $event) {
		if ($this->checkIfApiRequest()) { 			// Check if the current REQUEST_URI starts with /api
			$this->apiCall = true;					
			Auth::getInstance()->initApikey();		// Init authentication
			$router = new Router();					// Init Router
			$router->go($this->registeredRoutes);	// Start resolving the route. This function will echo() and
													// exit()
			$event->replace = true;					// Only needed in hook-function-call
		}
	}

In theory (not tested yet), it should work to copy the three relevant lines (Init Auth, Init Router, Resolve Route) to your home-template file. I would be really interested to know if it really works out that way. Don't have time to test it out right now.

This is my plan for a future module-update:
- Extend the routes definition that it is not only applied to the api endpoint from the module configuration but to the whole pagetree
- Check also on existing pages, not only in the 404 hook, if a route matches. If a valid api key is present, output the AppApi response instead. 
- Otherwise everything remains the same

[Pixrael]

  On 11/16/2021 at 6:51 PM, Sebi said:

@Pixrael What I do on ProcessPageView::pageNotFound is calling this function:

	public function ___handleApiRequest(HookEvent $event) {
		if ($this->checkIfApiRequest()) { 			// Check if the current REQUEST_URI starts with /api
			$this->apiCall = true;					
			Auth::getInstance()->initApikey();		// Init authentication
			$router = new Router();					// Init Router
			$router->go($this->registeredRoutes);	// Start resolving the route. This function will echo() and
													// exit()
			$event->replace = true;					// Only needed in hook-function-call
		}
	}

In theory (not tested yet), it should work to copy the three relevant lines (Init Auth, Init Router, Resolve Route) to your home-template file. I would be really interested to know if it really works out that way. Don't have time to test it out right now.

This is my plan for a future module-update:
- Extend the routes definition that it is not only applied to the api endpoint from the module configuration but to the whole pagetree
- Check also on existing pages, not only in the 404 hook, if a route matches. If a valid api key is present, output the AppApi response instead. 
- Otherwise everything remains the same

Expand  

Yes, it works with this three lines in the home template file, but you must enable the urlSegments in the home template.. now the AppApi configuration still have the "domain.com/api/users" as endpoint but "domain.com/users/" works too.. if you try "domain.com/foo/" the api respond with a 404 json response

[adrian]

I must be having a bad day because I keep getting "Apikey not valid" when testing via Postman or https://reqbin.com/

If I force the isApikeyValid() method to return true, then I get "Call to a member function getAuthtype() on bool" which if I look at, means that $this->application is returning a boolean.

Has anyone else had to troubleshoot through issues like this?

Thanks for any tips on getting started!

 

[adrian]

I am starting to think this is a PHP8 issue: https://github.com/Sebiworld/AppApi/issues/28#issuecomment-989300577

Is anyone using this successfully with v8?

[adrian]

Ok, so I am now testing on a different server with PHP7 and I can get the X-Api-Key header to work, but I really do need this working on PHP8.

The next problem (with PHP 7) is that if I try the /test endpoint (as included in the default Routes.php file), I get: 

"error": "Method not allowed"

Same goes for /users

Does anyone know what the problem with these two endpoints might be?

Thanks!

Solved the method issue - it was because the default test endpoint was set to GET and I was working with POST.

Now it seems to be just the PHP 8 with the API Key that needs solving.

[Sebi]

I just released version 1.2.1.  

When testing with PHP version 8, some strange changes in the naming of header values occurred in my test environment, which meant that the values in the AUTHORIZATION header could no longer be read correctly. I have therefore made the handling of the header values a little more general in the new version, and implemented that certain prefixes are ignored. As a result, everything now runs normally again under PHP 8.

How does it look for you? Have all the errors been solved? I can't rule out that things behave differently in other environments and with other server configurations. But I would like to take all possible deviations into account, so feel free to contact me if you notice any problems!

[Sebi]

And now I got for you: Version 1.2.2 ?

With @adrian's help I managed to find out, that the `date()` function in PHP 8 seems to return the current date, if NULL is passed in as the second parameter. In PHP 7 this was 01.01.1970 which was handled correctly. Long story short: Now I will write NULL to the database without any date-formatting. This should make AppApi fully compatible to PHP 8.

[adrian]

Thanks @Sebi - things are working for me properly now on PHP8.

No rush on this, but I would like to raise @kongondo's suggestion of using PW's new URL path hooks (https://processwire.com/blog/posts/pw-3.0.173/#introducing-url-path-hooks) instead of the PageNotFound hook. I am not sure how much work it would be to change to this approach, but when I noticed the PHP8 issues in this module, I actually ended up creating my API via URL path hooks and it was really easy to implement with very little code. Obviously your module is going to be awesome for handling multiple APIs, multiple api keys, revoking access, etc, etc, so I will probably switch to it now, but it would be nice to not rely on that PageNotFound hook, because a lot of modules are using that and as well as feeling hacky, I feel like there are bound to be conflicts at some point with different modules and the order in which they are executed.

[Jukka]

This is amazing module - big thanks to you @Sebi! I'm using this mostly for communicating with Vue3 frontend and the workflow is very smooth.

With my latest project, I just noticed that JSON processing time is starting to be too slow (~3.5sec) because there are so many searches. So I would have asked if you have any plans to further develop compatibility with ProCache or any other caching solutions?

[Sebi]

Hi @Jukka,
I'm glad the module helps you build Apis for your frontend projects!

Honestly, so far I'm not working on supporting caching solutions like ProCache. Not because I don't find it interesting or useful, but just because I don't have much experience with caching so far. 

Can you help me out with that? Maybe we can figure out together what the problem is or how the module would need to be extended to allow caching solutions to be docked without problems? 
Anyone else who is reading along here is of course also welcome to feel addressed :-) I am grateful for every advise!

[Jukka]

Nice to hear that you would be willing to develop cache compatibility for AppApi! Unfortunately, I don’t have the knowledge or skills to help you with the implementation ? Is there anyone in this forum that could help @Sebi with the development?

[toni]

@sebi @Jukka

Sorry I cannot help with pro cache. But caching with memcache or even flat files might be a start?
This is not tested at all but just to outline the idea:

 

  public static function getUser($data) {

    $data = AppApiHelper::checkAndSanitizeRequiredParameters($data, ['id|int']);
    $response = new \StdClass();
    $user = wire('users')->get($data->id);

    if(!$user->id) throw new \Exception('User not found', 404);
    $response->id = $user->id;
    $response->name = $user->name;

    // cache in a file
    $cache_file_name = hash('sha256', $data->id);
    $cache_path = "cache_path/$cache_file_name";
    if(!is_file($cache_path)){
        // the cache file does not exist let's add it
        file_put_contents( $cache_path, json_encode($response) );
    } else {
        // the cache exists, let's read it and return the data
        $response = file_get_contents($cache_path);
        return $response;
    }

    // the cache does not exist let's return it form the std class
    return $response;
  }

Of course it would be MUCH better to use ProCache or memcache.

@sebi Have you tested multilanguage? I wonder what would be the best option.

A) To have namend routes like

/api/test/
/apit/en/test

that map to the same function, which checks for the /en/ part or
 

B) handle the language switch by a get ver

/api/test/
/apit/test?lang=en

that controlls the language?

[Jukka]

Thanks @toni!!! ? With these guidelines I was able to do a solution that create temporary file from the content of my biggest json. That process otherwise take too long time when it was executed in every page load.

[Sebi]

  On 3/20/2022 at 11:30 AM, toni said:

@sebi Have you tested multilanguage? I wonder what would be the best option.

A) To have namend routes like

/api/test/
/apit/en/test

that map to the same function, which checks for the /en/ part or
 

B) handle the language switch by a get ver

/api/test/
/apit/test?lang=en

that controlls the language?

Expand  

Hey @toni!

Sorry, I totally missed your mention ? 
I recently had to deal with multilanguage urls in an api and tried a lot of things. It's something that must be handled in your endpoints and it's a matter of taste. If it comes to frontend-urls, the first way should be preferred because language-specific urls (not with language as GET-param) seem to be preferred by Google, so its better in terms of SEO optimization.

Have a look at the code of my additional module AppApiPage which adds an api/page endpoint. I've included handling for ProcessWire's multilanguage-urls and I've added setting the language via GET-param as a fallback.

Hope that helps you out!