New Module: AppApi

[Sebi]

On 5/11/2021 at 4:00 PM, fliwire said:

hi need to add custom variable to every request .
Token has an_id value and need to fetch user data from another database. Tried bottom code but $tokenString is empty.

Simply need to add $token->an_id value to request $data object.

Hi @fliwire! Auth::getBearerToken() is a protected function. I do not exactly know how PHP handles that, but maybe that results into an empty string? You could copy the logic from Auth::getBearerToken() and Auth::getAuthorizationHeader() to try out if that is the issue. 

Additionally, hooking into Router::params could be a better place to add the logic since it is called later - after all auth-checks and just before Router::handle calls the targetted function.

[fliwire]

hi @Sebi found that that was server fault.  AUTHORIZATION headers not set.
cant find what apache module should enable but .htaccess solve my issue. 

SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1

 

[Bacos]

Thank you very much my friends, 

The problem was the "api" instead "testeapi/api". 

Now I'm going to keep studying. 

[David Lumm]

Hi all. I've got something that I'm not 100% sure is a problem with AppApi or elsewhere, so I'm hoping someone can just help me cross the AppApi module off my list of the things I'm looking into.

I'm currently trying get my API to send an email when a certain POST is made. Locally this works fine, but on our test server I'm getting an unexpected error:

{
    "error": "Internal Server Error",
    "devmessage": {
        "message": "set_time_limit() has been disabled for security reasons",
        "location": "/home/runcloud/webapps/D2U-API/site/assets/cache/FileCompiler/site/modules/WireMailSmtp/WireMailSmtp.module",
        "line": 750
    }
}

Now a quick look through this forum suggests that this should be generating a PHP warning, that I should just be able to ignore, but for some reason it's stopping execution and generating the API error.

I'm wondering whether AppApi is handling this as a hard error and stopping execution, but really it would carry on fine if left, or whether the situation I'm seeing is more serious than the examples in that other forum post?

@Sebi or @thomasaull do you have any comments about how a PHP runtime warning would be handled?

[David Lumm]

10 minutes ago, David Lumm said:

I'm wondering whether AppApi is handling this as a hard error and stopping execution, but really it would carry on fine if left, or whether the situation I'm seeing is more serious than the examples in that other forum post?

Yeah, that's exactly what's happening. In Router.php:

public static function handleError($errNo, $errStr, $errFile, $errLine) {
        if (error_reporting()) {
            $return = new \StdClass();
            $return->error = 'Internal Server Error';
            $return->devmessage = [
                'message' => $errStr,
                'location' => $errFile,
                'line' => $errLine
            ];
            self::displayOrLogError($return, 500);
        }
    }

So if any error logging is enabled this will be fired, regardless of whether we actually care about that type of error. I've made some modifications, so I'll do a PR and you can see what you think @Sebi

[Bacos]

Hi friends, I'm here again with another newbie question, but...
I'm getting "failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized" when I try to access API using file_get_contents from PHP page.

I'm passing ApiKey (PHP Session) and logged in as superuser.

If I remove ["auth"=>true] from routes.php it works.

In my mind, if I'm logged in as superuser it should work with authorization enabled or am I wrong? 

Here is my code...

Thank you all!

$apiKey = $config->apikey;

$getdata = http_build_query(
    [
        'nome' => 'sample name',
        'endereco' => 'sample address'
    ]
);

$context = stream_context_create(
    [
        'http' => [
            'method' => 'GET',
            'header' => ["X-Api-Key: $apiKey", "Content-Type: application/x-www-form-urlencoded"],
        ],
        'ssl' => [
            'verify_peer'      => false,
            'verify_peer_name' => false
        ]
    ]
);

$url_returns = file_get_contents("$url" . $getdata, false, $context);

 

[Bacos]

Additional informations:

I believe the AppApi has a bug with PHP Session.

I've changed the line 187 in router.php to show the user name logged in and it shows "guest".

if (isset($routeParams['auth']) && $routeParams['auth'] === true && !$this->wire('user')->isLoggedIn()) {
            throw new AppApiException('User does not have authorization ' .$this->wire('user')->name, 401);

But running the same command inside a page it shows the real name of logged user.

Inside Page user name = 2069008

From API :
{"error":"User does not have authorization guest","devmessage":{"class":"ProcessWire\\AppApiException","code":401,"message":"User does not have authorization guest","location":"C:\\xampp\\htdocs\\cinemed\\site\\modules\\AppApi\\classes\\Router.php","line":187}

Any help please?

Thank you all!

[Sebi]

On 5/24/2021 at 2:38 PM, David Lumm said:

Yeah, that's exactly what's happening. In Router.php:

public static function handleError($errNo, $errStr, $errFile, $errLine) {
        if (error_reporting()) {
            $return = new \StdClass();
            $return->error = 'Internal Server Error';
            $return->devmessage = [
                'message' => $errStr,
                'location' => $errFile,
                'line' => $errLine
            ];
            self::displayOrLogError($return, 500);
        }
    }

So if any error logging is enabled this will be fired, regardless of whether we actually care about that type of error. I've made some modifications, so I'll do a PR and you can see what you think @Sebi

Hey @David Lumm, thank you for your pull request! 

I'm still testing out a few things at the moment, but I wanted to report back briefly at least. I catch all exceptions and errors at the top of Router.php to handle them myself:

    set_error_handler("ProcessWire\Router::handleError");
    set_exception_handler('ProcessWire\Router::handleException');
    register_shutdown_function('ProcessWire\Router::handleFatalError');

My main goal was to prevent a plaintext or HTML error message from being displayed when an API function was requested. Instead, the message should be output as JSON. @David LummDo I understand your commit correctly, that you disable this behaviour for warnings and only log the warning additionally? My goal is actually only that no PHP echo is made with the warning. A PHP echo before a JSON response would render the whole response useless.

Do any of you know how I can prevent this echo, but the warning is still treatable in the non-module code?

[Sebi]

On 5/27/2021 at 8:59 AM, Bacos said:

Additional informations:

I believe the AppApi has a bug with PHP Session.

I've changed the line 187 in router.php to show the user name logged in and it shows "guest".

if (isset($routeParams['auth']) && $routeParams['auth'] === true && !$this->wire('user')->isLoggedIn()) {
            throw new AppApiException('User does not have authorization ' .$this->wire('user')->name, 401);

But running the same command inside a page it shows the real name of logged user.

Inside Page user name = 2069008

From API :
{"error":"User does not have authorization guest","devmessage":{"class":"ProcessWire\\AppApiException","code":401,"message":"User does not have authorization guest","location":"C:\\xampp\\htdocs\\cinemed\\site\\modules\\AppApi\\classes\\Router.php","line":187}

Any help please?

Thank you all!

 

Hi @Bacos,  I have just tested it once through. At least it doesn't seem to be a general problem with AppApi's auth. In my Processwire test installation (running local with MAMP) I can protect a route with ['auth' => true]. It is then only accessible if I have logged in beforehand. In fact, there still seems to be a discrepancy, as I was also able to authenticate via the session with an apikey of type "Double JWT". But I will fix that soon.

Unfortunately, I'm not really familiar with stream_context_create. My API requests are mostly made from a Javascript context. Is it possible that the session is not passed on correctly? Does the session or the cookies perhaps have to be specified as a parameter in stream_context_create?

[Bacos]

Hi Sebi,

I guess I'm a little confused about phpSession feature. 

Calling API with curl getting the same problem.

Do you have some sample calling API inside a processwire template with route authenticated ?

Calling from Postman with JWT works fine. 

Thank you again!

[Sebi]

Hey @Bacos,
unfortunately I really don't have much experience with api requests from PHP. 

In my beautiful, carefree javascript world, the browser handles the session cookies, so I can log in normally via the ProcessWire backend, and then get the logged in user back in the frontend via a simple call to the /api/auth/ interface:

fetch(
	'/api/auth/', {
	headers: {
		'X-API-KEY': 'SHBaob3siaud8A'
	}
})
	.then(response => response.json())
	.then(response => console.log("RESPONSE", response));

Postman also handles session cookies for you automatically, so you don't have to worry about it manually.

I did a bit of research. If you want to make your API request in PHP, it seems you have to take care of the session cookies yourself. The answer under this stackoverflow post seems to be a usable example, but with a CURL request instead of stream_context_create, which is what you're using:https://stackoverflow.com/a/10307956/5477836
In short, you have to log in via a PHP request, and the session cookies are written to the specified file path. The next time you make a request, you can then reuse those cookies to authenticate yourself. (If anyone here knows better about this, please correct me). 

If I were in your place, I would probably try the Auth-Type Single JWT instead of the Auth-Type PHP-Session. The advantage here would be that you get back a login token (string) on the /api/auth/ request, which you could then send along as a header on the next requests. This seems easier to me than having to mess with a cookie file. But that's up to you to decide...

I hope you get somewhere with this? Also feel free to let me know if you have something executable. Maybe this would be a good example for documentation!

[abmcr]

I have setup the module and all work fine: my api send data to a client axios, but i have a problem with a file 

My api send this data json

 

{
    "data": [
        {
            "url": "http://p1.test/site/assets/files/1017/gpx.gpx",
            "title": "prova"
        }
    ]
}

and in the quasar app i need to use the file gpx for show a leaflet map.

I get this error

Access to fetch at 'http://p1.test/site/assets/files/1017/gpx.gpx' from origin 'http://localhost:8081' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

in the .htaccess file i have set this lines at the bottom 

 

RewriteEngine on
Header add Access-Control-Allow-Origin "*"
Header add Access-Control-Allow-Headers "origin, x-requested-with, content-type"
Header add Access-Control-Allow-Methods "PUT, GET, POST, DELETE, OPTIONS"

but the file is not fetched....

What i broken? Thank you (excuse me for the poor english language)

[Sebi]

Hey@abmcr,

I think that it is not a good idea to release all pages for cross-origin requests. But I have an alternative for you. You could also request the file via the api interface. Then the module would automatically set the CORS headers and the request would be a bit more secured by the apikey.

You can take the following class: https://github.com/Sebiworld/musical-fabrik.de/blob/f172a9ef4674e09cabce1fdcf4e55ddeea150d1b/site/api/FileAccess.class.php
It is in productive use in one of my projects. Copy the class into the directory where your Routes.php file is located (default is site/api/).

Now you can add the following endpoints to your Routes-definition:

<?php

namespace ProcessWire;

require_once wire('config')->paths->AppApi . 'vendor/autoload.php';
require_once wire('config')->paths->AppApi . 'classes/AppApiHelper.php';

require_once __DIR__ . '/FileAccess.class.php';

$routes = [
	'file' => [
		['OPTIONS', '{id:\d+}', ['GET']],
		['OPTIONS', '{path:.+}', ['GET']],
		['OPTIONS', '', ['GET']],
		['GET', '{id:\d+}', FileAccess::class, 'pageIDFileRequest'],
		['GET', '{path:.+}', FileAccess::class, 'pagePathFileRequest'],
		['GET', '', FileAccess::class, 'dashboardFileRequest']
	]
];

Full Routes.php for reference: https://github.com/Sebiworld/musical-fabrik.de/blob/f172a9ef4674e09cabce1fdcf4e55ddeea150d1b/site/api/Routes.php

After these routes are integrated, you can call the new /api/file/ endpoint:

const instance = axios.create({
  baseURL: 'https://some-domain.com/api/',
  headers: {
    'X-API-KEY': 'ThisIsYourCustomApiKey',
    'Authorization': 'Bearer ...' // optional if authentication needed
  }
});

instance.get('/file/1017', {
  file: 'gpx.gpx'
}).then(function (response) {
  console.log(response); // This should be your file
}).catch(function (error) {
  console.log(error.toJSON());
});

(Code not tested, only taken from Axios docs and changed to the correct parameters) 

I hope that I could help you with this!
[And please don't worry about your English. Everything is understandable, and there are many non-native speakers here. You don't have to apologize for anything :-)]

[abmcr]

Thank you

[Sebi]

Version 1.1.6 is out! ?

Changelog:

• Adds Router->registerErrorHandlers() Hook, that should allow you to overwrite the general error- and warning handlers of the module. That should fix the problem that @David Lumm mentioned above without breaking things for other users.
• Allows Apikey & Auth-token to be set as GET-params. That can be useful when it comes to loading images via api.
• Fixes a bug that made it possible to authenticate with the PHP session (cookie) even though token-auth was enabled.
• Adds Router->setCorsHeaders() Hook
• Updated Composer & Firebase dependencies

[csaggo.com]

Hi @Sebi

your module is of great help, we are using it in quite a view projects allready.

i just build a mobile app with a processwire backend using Double JWT. While implenting it i noticed the following:

In the docs the /auth route is described to be used like this (in addition to basic auth):

 

// Alternatively you can send username/pass in the request-body:
this.httpClient.post(
  'https://my-website.dev/api/auth',
  JSON.stringify({
    username: username,
    password: pass,
  }),
  {
    'x-api-key': 'ytaaYCMkUmouZawYMvJN9',
  }
);

but this does not work for me. It actually works with basic out and also when sending username and password as form-data. But not as JSON. Its no big deal with two working alternatives, but it would be actually nice to have the JSON option also (specially since form-data can be challenging to set up with some http-clients.

I allways get the following error with the above call:

 

{
  "username": null,
  "pass": null,
  "post": [],
  "test": "{\n\t\"username\": \"REMOVED\",\n\t\"password\": \"REMOVED\"\n}",
  "errorcode": "general_auth_exception",
  "error": "Login not successful"
}

Best,
Michael

[Sebi]

Hey @csaggo.com,

thank you very much for your input! This should not be a big problem, I will have a look at it in the next few days.
I'll get back to you soon :-)

[Studio Lambelet]

Hi @Sebi!

Thanks a lot for your module, really great, having fun with it for a little side project!

After some testing, I have a two questions:

1. First thing (not really essential and I may have made a mistake in the way I set things up), but the access-log shows all urls as starting with "/404/". JS side requests are working fine (http 200). The rest of the url shown in access-log is fine.
2. A little more significant, the function that handles authentication does not seem to respect the Session Login Throttle settings (I'm using double JWT). I'm a bit paranoid perhaps, but this seems to present a risk of brute-force attack. Is this the case, or is my implementation wrong?

Thanks a lot!

[Sebi]

Hi everyone!

@csaggo.com Sorry for the delay. I had some sick and stressful weeks, but I hope that it will become better now.

I just released a new version 1.1.7 that fixes @csaggo.com's problem that he mentioned above. The module's /auth-endpoint does now accept the login-credentials in different formats. You can now send username and password in a JSON-formatted (or FORM-URL-Encoded or Multipart) POST-body. It is even possible to send the credentials as GET-params - although I would not recommend it.

A short note to what went wrong in the previous versions: I myself forgot to use my own (extremely useful) parameter functionality, which extracts parameters from a wide variety of formats:

public function ___doLogin($data) {
	// ...

    // The right way to get params: 
    $username = $this->wire('sanitizer')->pageName($data->username);
    $pass = '' . $data->password;

    // Old code:
    // $username = wire('input')->post->pageName('username');
    // $pass = wire('input')->post->string('password');

    // ...

Please use the $data-object that your api-functions will receive automatically - this process is more tolerant than Processwire's wire('input')->post when it comes to different body formattings.

[Sebi]

Hey @Clément Lambelet !

Thank you for using AppApi and for mentioning the questions! 

1. I do not know a way to prevent ProcessWire to log the accesses starting with /404/. From a technical point of view, this is absolutely correct: to accept requests, the AppApi module hooks into ProcessWire's 404 handling. For example, you call the url /api/test, which does not exist in the ProcessWire page tree. So ProcessWire handles this as a 404 error, but the module intervenes and sends its own output, namely what is defined in your routes. However, when logging, the URL is still recognized as a 404. In the newer ProcessWire versions there is the possibility to handle requests programmatically without the detour via 404 pages, but because of the backward compatibility I haven't dared to do it yet. If someone already knows better about it: I would be very happy about pull requests or comments ?
2. No, that is absolutely legitimate and not paranoid! My goal is to make the module as secure as possible, which means that login via api must not open a way to undermine any security mechanisms. Speaking of Session LoginThrottle: I tested it out - LoginThrottle seems to be called correctly on the AppApi login calls as well. For example, if I enter an incorrect password three times in quick succession, I get the following feedback, which comes from LoginThrottle: 

   {
     "error": "Login not attempted due to overflow. Bitte warten Sie vor dem n\u00e4chsten Login-Versuch mindestens 5 Sekunden."
   }

   Do you have anything special set that is not being taken into account?

[Studio Lambelet]

Hi @Sebi !

Thanks a lot for your long reply!

Regarding the 404s, I totally understand. I remember now that I had used the same kind of hook for a module a while ago. But would it make sense to remove the "404" part of the url in the log? It's really a cosmetic thing though, not fundamental.

Thank you very much for testing whether Session Login Throttle works. I am reassured! I think it's my implementation that is not quite right yet. I'll keep testing and ask a new question if I really can't find a solution.

Thanks again for your help!

[studioweb]

On 5/16/2021 at 10:50 PM, fliwire said:

hi @Sebi found that that was server fault.  AUTHORIZATION headers not set.
cant find what apache module should enable but .htaccess solve my issue. 

SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1

 

This was the case for me too. Perhaps this could be added to the documentation ?

[Marcel]

Hi I am new to rest apis and I installed this module. I want to fallow the instructions but I cannot follow this:

Quote

After installing the module you will find "AppApi" as a new item under the "Setup" popup-menu in the header bar. Click on "Manage applications" and choose "Add", to create a new application.

This does not appear. In the module configuration page. I let all default. In the browser, when I have mywebsite.com/api this is the on the page:

Quote

message

"no endpoint defined"

What am I missing?

Thanks in advance,

Marcel

[androbey]

Hi @Marcel,

you can find the "AppAPI" config page with following schema: {{YOUR_DOMAIN_COM}}/{{ADMIN_PATH}}/{{SETUP_PATH}}/appapi/ 
For a "default" ProcessWire installation on "example.com" this would be: https://example.com/processwire/setup/appapi/
There you need to manage your applications.

Then, in your Router.php file you need to define routes (=endpoints). 

[androbey]

I am using AppAPI module for a project and it's amazing. Thanks @Sebi!
However I encountered a (for me) very strange behaviour. 

I defined an endpoint for a search functionality (for use via ajax). 

//...
//defined route in Router.php
'search' => [
		['GET', '{q}', Search::class, 'getSearchResults', ["auth" => true]],
],

I call the endpoint via ajax (content type application/json). The "q" parameter is free text and uri encoded. 

Now, when "q" is a single word no problem occurs.

However if "q" has a space like "my query" the response has a HTTP status code 404, although returning the correct results. So the route itself is working fine.

I am not sure what causes this. Has anybody some clues?

used versions:
AppAPI 1.1.7, ProcessWire 3.0.184, Php 7.4