New Module: AppApi

[benbyf]

3 minutes ago, David Lumm said:

Hey all!

I was wondering, is there any concept of "middleware" or hooks that I could attach to so I could run something on every request?

At the moment I've got a couple of things I've added into every route function, but I can anticipate that list getting longer. Rather than copy and paste the same lines into each and potentially miss something, it would be great if I could run it before the route specific code runs. Is that possible @Sebi?

Hi HI David, in PW any function in a module that has ___ at the start, like ___handleApiRequest() in the AppAPI module you can use as a hook in your own code. See https://processwire.com/docs/modules/hooks/

[David Lumm]

10 minutes ago, benbyf said:

Hi HI David, in PW any function in a module that has ___ at the start, like ___handleApiRequest() in the AppAPI module you can use as a hook in your own code. See https://processwire.com/docs/modules/hooks/

Ah that's great, thanks! I'll take a look at that and see what I can work out!

[thibaultvdb]

21 hours ago, Sebi said:

Hi @thibaultvdb,

thank you for reporting this issue! 
I'll be honest: In my Apis I actually always use arrays as return values, so I didn't notice this bug. With version 1.1.2, which I just released, you can use a stdclass again instead of an array as return value. 

I hope everything is running smoothly again with your Api? I would be very happy about a short feedback!

hi @Sebi

Awesome, thanks for the quick feedback and fix. Normally I use a lot of array's myself but I had a specific case where a object was the ideal way to go ?.
I'll test it out in the following days!
Version 1.1.1 is working good for me, however I'm not doing really special stuff with it at the moment so I'm not sure if my case is the best feedback ?.
I'm just opening the api  for visitors so I can use it a little like a restApi (doing some fancy ajax stuff) and it's doing exactly what I would expect so great work!

If you want me to check some other parts, let me know!

[eelkenet]

Hi @Sebi, I ran into a bug:

Currently the Router::handleError() function incorrectly reports HTTP 500 errors that result from methods that should have remained silent, because they are called with the error control operator '@'.

As the PHP documentation on error control operators puts it: 

Quote

If you have set a custom error handler function with set_error_handler() then it will still get called, but this custom error handler can (and should) call error_reporting() which will return 0 when the call that triggered the error was preceded by an @.

I bumped into this while working on a project with some semi-broken images, where the APP API module kept throwing 500's from PW's ImageInspector class.
Exif issues can be rather difficult to handle sometimes..

The solution is simple: just add a check error_reporting() before the handleError method, for instance

  public static function handleError($errNo, $errStr, $errFile, $errLine)
    {
        if (error_reporting()) {
            $return = new \StdClass();
            $return->error = 'Internal Server Error';
            $return->devmessage = [
                'message' => $errStr,
                'location' => $errFile,
                'line' => $errLine
            ];
            self::displayOrLogError($return, 500);
        }
    }

 

 

[Sebi]

I just released version 1.1.3 which resolves three issues that were reported recently:

• Fixes an issue with the constructor signature of the modules AppApiException class (by @David Lumm, thanks for PR ?)
• Fixes an issue with the error-handler, which made it mistakenly catch errors that should have been ignored via @ operator (Thanks to @eelkenet)
• Switched from `wire('input')->url` to `$_SERVER['REQUEST_URI']` for reading the base-url, because ProcessWire's internal function transferred everything to lowercase. (Thanks to Github-user @pauldro)

Thank you all for your contributions!

[psy]

I've read the doco, installed the module & Twack and no matter what I try when retrieving a page, all I get back is a successful Promise or worse, an error message about invalid json. Using React/Nextjs and the page is published, viewable & using the basic-page template. The authorisation is single JWT and the client JS code is:

export async function PWPageByUrl(path)  {
  // url is correct and returns a result, just not the one I was expecing
    const res = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/${process.env.NEXT_PUBLIC_API_VERSION}/page/${path}`, {
            headers: {
                'x-api-key': process.env.NEXT_PUBLIC_API_KEY,
                'authorization': process.env.NEXT_PUBLIC_TOKEN
            }
        }
    )
    const data = await res.json()
    if (!data) {
        return {
            notFound: true,
        }
    }
    return {
        props: {
            data
        }, // will be passed to the page component as props
    }
}

On the PW server side as per doco with routes modified for v1, I have:

Spoiler

<?php
namespace ProcessWire;

require_once wire('config')->paths->AppApi . "vendor/autoload.php";
require_once wire('config')->paths->AppApi . "classes/AppApiHelper.php";

require_once __DIR__ . '/TwackAccess.class.php';

$routes = [

    'v1' => [
        'page' => [
            ['OPTIONS', '{id:\d+}', ['GET', 'POST', 'UPDATE', 'DELETE']],
            ['OPTIONS', '{path:.+}', ['GET', 'POST', 'UPDATE', 'DELETE']],
            ['OPTIONS', '', ['GET', 'POST', 'UPDATE', 'DELETE']],

            ['GET', '{id:\d+}', TwackAccess::class, 'pageIDRequest'],
            ['GET', '{path:.+}', TwackAccess::class, 'pagePathRequest'],
            ['GET', '', TwackAccess::class, 'dashboardRequest'],

            ['POST', '{id:\d+}', TwackAccess::class, 'pageIDRequest'],
            ['POST', '{path:.+}', TwackAccess::class, 'pagePathRequest'],
            ['POST', '', TwackAccess::class, 'dashboardRequest'],

            ['UPDATE', '{id:\d+}', TwackAccess::class, 'pageIDRequest'],
            ['UPDATE', '{path:.+}', TwackAccess::class, 'pagePathRequest'],
            ['UPDATE', '', TwackAccess::class, 'dashboardRequest'],

            ['DELETE', '{id:\d+}', TwackAccess::class, 'pageIDRequest'],
            ['DELETE', '{path:.+}', TwackAccess::class, 'pagePathRequest'],
            ['DELETE', '', TwackAccess::class, 'dashboardRequest'],
        ],
        'file' => [
            ['OPTIONS', '{id:\d+}', ['GET']],
            ['OPTIONS', '{path:.+}', ['GET']],
            ['OPTIONS', '', ['GET']],

            ['GET', '{id:\d+}', TwackAccess::class, 'pageIDFileRequest'],
            ['GET', '{path:.+}', TwackAccess::class, 'pagePathFileRequest'],
            ['GET', '', TwackAccess::class, 'dashboardFileRequest']
        ]
    ]
];

and the TwackAccess class copy/pasted from the documentation, eg for pagePathRequest:

Spoiler

  <?php
	/**
     * General function for page-outputs:
     */
    protected static function pageRequest(Page $page) {
        // Exit if Twack is not installed
        if (!wire('modules')->isInstalled('Twack')) {
            throw new InternalServererrorException('Twack module not found.');
        }

        // This commands Twack to output a data-array instead of HTML:
        wire('twack')->enableAjaxResponse();

        // If the page has no template, is not accessable or is blocked (i.e. via hook),
        // we throw a ForbiddenException
        if (!$page->viewable()) {
            throw new ForbiddenException();
        }

        $ajaxOutput   = $page->render();

        // $ajaxOutput will contain JSON-code, so we have to decode it to prevent it is encoded twice:
        $results      = json_decode($ajaxOutput, true);

        // Now, $results is a clean PHP-array with the information generated by Twack-components:
        return $results;
    }

 

 

I'm obviously missing something. Help to get it working as expected much appreciated.

Thanks

 

[Sebi]

4 hours ago, psy said:

I'm obviously missing something. Help to get it working as expected much appreciated.

Hi @psy,

At first glance, I can't find any obvious error in your code. Can you please show me the server response you get for the "Invalid Json" errors? (You can see each request/response in your browser's developer-console in the network-tab. Feel free to DM me if you need support for that.)

I would try to take out some complexity first and leave Twack out of the queries for now. It's best to set up a test route that only returns a simple response. Insert this to your Routes.php:

'v1' => [
    'test' => [
        ['OPTIONS', '', ['GET']],
            ['GET', '', AppApiTest::class, 'test']
        ]
    ],
]

And create the AppApiTest-class:

<?php

namespace ProcessWire;

class AppApiTest {
    public static function test($data) {
        return [
          'test' => true,
          'success' => 'YEAH!'
        ];
    }
}

No token-authentication needed. If you get this response back in Javascript, we can be sure that the basic api connection works.

[psy]

@Sebi Thanks for the quick response. I'd already gone past that point both in PW and Postman so the connection is good.

I'd even written my own method in Examples to get a page by ID. Maybe not as sophisticated as yours but it too worked:

<?php namespace ProcessWire;

Class Example {
  
  	public static function test () {
		return ['test successful'];
	}

  
      public static function getPageById($data) {
        $data = AppApiHelper::checkAndSanitizeRequiredParameters($data, ['id|int']);

        $p = wire('pages')->get("id=$data->id");
        if(!$p->id) throw new \Exception('Page not found', 404);

        $response = new \StdClass();

        // Hardcode any data you need that does not have an inputfield field in the admin
        $response->id = $p->id;
        $response->name = $p->name;
        $response->path = $p->path;

        // This retrieves all the admin input fields. Does not (yet) get page images or files, just single entry data
        $fields = $p->getFields();
        foreach ($fields as $fld) {
            $fldName = $fld->name;
            $response->$fldName = $p->$fld;
        }

        return $response;
    }
}

This enabled me to get the page data into the NextJS client and embed in the 'template/component' as variables from any existing PW page.

My next exercise was to delve into the NextJS router so that I could retrieve pages via their path and those paths would appear in the URL. Couldn't work it out in my own PW code (no surprises there!). It was then I installed your Twack module and used the Twack routes & TwackAccess.class.php at which point I ran into the issues mentioned above.

I'm sure it's my lack of knowledge of REACT/NextJS rather than your code... Doesn't help when their doco doesn't match up with their examples in gitHub.

Even so, it would great to use your TwackAccess classes rather than reinventing the wheel.

 

 

 

[Sebi]

@psy So, just to make sure, that everything works well on the PHP side: You can make a request via Postman to the same url that you are trying to access via Javascript? And you get back the expected JSON-data?

Unfortunately, I don't know much about NextJS and React. I am actually an Angular or Vanilla Javascript developer. In an Angular environment I would use Angular's httpClient to make an api-call:

const params = new HttpHeaders({
	'x-api-key': environment.api_key
});
this.httpClient.get('https://my-test-domain.com/api/page/1042').subscribe(
	response => console.log("Api-Response: ", response)
);

I assume that React has a similar helper class as well. This makes a call to the page-route in my router-definition, which you can see here: https://github.com/Sebiworld/musical-fabrik.de/blob/master/site/api/Routes.php . It will return the ajax-results for the ProcessWire page with id 1042 in this case.

I prefer to use the httpClient (if available) instead of the fetch function, which you are using in your code-example above. Mainly because I found the fetch function very cumbersome to use when dealing with more complex parameter data. But for a vanilla-js project I needed to use it, so I wrote a helper class that is way more usable: https://github.com/Sebiworld/musical-fabrik.de/blob/master/site/templates/src/js/classes/AjaxCall.js

Here is how you can make an api-call:

const ajaxCall = new AjaxCall({
	method: 'GET',
	path: '/api/page/1042',
	headers: {
		'X-API-KEY': 'oiasnduz3498gfsubasd'
	}
});
ajaxCall.fetch().then(function(response){
  console.log('Api-response: ', response);
}).catch(function(response){
  console.log('Api-error: ', response);
});

I hope that helps you out ?

[Sebi]

I released v1.1.4 of AppApi. The update fixes a critical bug that occurred when routes were called with GET parameters.  (reported by @David Lumm, thanks for PR ?)

Because I was already at it, I outsourced the reading of the current route (which is then further used by FastRoute) to its own hookable function `___getCurrentUrl()`. This allows you in special use cases to subsequently influence the URL with your own hook function.

[csaggo.com]

Hi @Sebi

kann you confirm that the latest PW (3.0.173) breaks AppApi (404 on the routes) or is it something on my part.

EDIT: confirmed it myself, Downgrading to 3.0.172 with no other changes solved the issue

Thank you!

[psy]

On 3/9/2021 at 4:41 PM, csaggo.com said:

confirmed it myself

Yep, everything I had working with AppApi and REACT/NextJS broke with the PW 3.0.173 upgrade. Downgrading is a short-term option but limits future PW upgrade benefits

[Sebi]

Hi everyone! 

I did not have the time to look deep into it, but looks like version 3.0.173 has made some changes into the handling of hooks. Especially hooks for custom urls, like we do in AppApi. Previously we had to use a little workaround to get it done - we hook into 404 (site not found) exceptions and generate our own response, if the request was made for /api/... The new update seems to add a functionality, where our module can use an url, without doing the 404-hack. But the new functionality seems also to break some of the old functionality. ?

So, please wait with the 3.0.173 upgrade! Thank you @csaggo.com and @psy for mentioning it. I will have time to look into it on the weekend - pull requests or hints are very welcome!

[Sebi]

It is done. I have found the error. Version 1.1.5, which I just released, fixes the bug and makes AppApi fully compatible with ProcessWire versions >= 1.0.173 again.

For those interested in the details:
It was just a tiny little thing that caused the module to no longer be able to find out if an api url was requested. This is what the code for it looked like:

    protected function checkIfApiRequest() {
        $url = $this->sanitizer->url($this->input->url);

        // support / in endpoint url:
        $endpoint = str_replace('/', "\/", $this->endpoint);

        $regex = '/^\/' . $endpoint . '\/?.*/m';
        preg_match($regex, $url, $matches);

        return !!$matches;
    }

However, in ProcessWire versions >= 1.0.173, $this->input->url (or wire('input')->url) now no longer contains the requested URL (e.g. "/api/page/"), but already the URL of the 404 error page "/http404/". Thus, the module could no longer determine whether it should handle the request or not.

But the solution to the problem was easier than I thought. $_SERVER['REQUEST_URI'] still contains the correct value. So we use that now for this check. And because this would have worked before, we don't need to worry about AppApi not working with older ProcessWire versions now. The fixed version simply looks like this:

    protected function checkIfApiRequest() {
        $url = $this->sanitizer->url($_SERVER['REQUEST_URI']);

        // support / in endpoint url:
        $endpoint = str_replace('/', "\/", $this->endpoint);

        $regex = '/^\/' . $endpoint . '\/?.*/m';
        preg_match($regex, $url, $matches);

        return !!$matches;
    }

Finally, thank you again for your reports. And I hope that you can now run your apis with the latest ProcessWire versions again. Thank you for using AppApi! ?

[thomasaull]

Thanks for the quick fix @Sebi! Are you planning on using the new url hooks in the future? I read about it yesterday and it feels like a much cleaner way to handle the requests to the API endpoint

[Sebi]

Absolutely! But I'm glad, that I could fix the old handler, so it will work regardless which ProcessWire version is used. I think it would make sense to additionally add the new hook functionality. It should grab the request before it triggers 404. But that is something that must be tested very carefully. 

[Rudy]

How do you folks set up your front-end? Same as your back-end or using completely separate web space?

I am rethinking of how to structure my project using AppApi.

Cheers
Rudy

[fliwire]

Want to change Auth::login process with session::login hook but not working. 
Can i extend auth::login process ?

// Routes.php

wire()->addHookBefore('Session::login', function (HookEvent $event) {
	// Get the object the event occurred on, if needed
	$session = $event->object;

	// Get values of arguments sent to hook (and optionally modify them)
	$name = $event->arguments(0);
	$pass = $event->arguments(1);
	$force = $event->arguments(2);

	//HERE another database check for login

	// changed for testing not working
	$name = "prouser";
	$pass = '';
	$force = true;

	// Populate back arguments (if you have modified them)
	$event->arguments(0, $name);
	$event->arguments(1, $pass);
	$event->arguments(2, $force);
});

// extend auth login

wire()->addHookBefore('Auth::doLogin', function (HookEvent $event) {
	$auth = $event->object;
	$event->replace = true;

	$event->return = [
		'jwt' => $auth->createSingleJWTToken(),
		'username' => "3254235"
	];
});

 

Edited March 31, 2021 by fliwire
fixed

[a_dent]

Thank you for a great module! I have started to explore the many possibilities that this module opens up.

A note myself that might help someone else too: I have a shared hosting that uses FastCGI. I couldn't get the Basic authentication to work using the Authorization header. Adding the following line to .htaccess does the job (available in Apache HTTP Server 2.4.13 and later).

CGIPassAuth On

Earlier version of Apache may use

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
</IfModule>

Source: https://stackoverflow.com/questions/3663520/php-auth-user-not-set

 

[rick]

Welcome to the forum. Great post!

[fliwire]

hi need to add custom variable to every request .
Token has an_id value and need to fetch user data from another database. Tried bottom code but $tokenString is empty.

Simply need to add $token->an_id value to request $data object.

wire()->addHookBefore('Router::handle', function (HookEvent $event) {
    $auth = Auth::getInstance();
    $tokenString = Auth::getInstance()->getBearerToken();
    // if ($tokenString === null || !is_string($tokenString) || empty($tokenString)) {
    //     return false;
    // }

    // throws exception if token is invalid:
    // try {
    //     $secret = $auth->application->getTokenSecret();
    //     if (!$singleJwt) {
    //         $secret = $auth->application->getAccesstokenSecret();
    //     }
    //     $token = JWT::decode($tokenString, $secret, ['HS256']);
    // } catch (\Firebase\JWT\ExpiredException $e) {
    //     throw new AccesstokenExpiredException();
    // } catch (\Firebase\JWT\BeforeValidException $e) {
    //     throw new AccesstokenNotBeforeException();
    // } catch (\Throwable $e) {
    //     throw new AccesstokenInvalidException();
    // }

    $routeInfo = $event->arguments(0);
    $routeInfo[2]["an_id"] = $token->an_id;

    $event->arguments(0, $routeInfo);
});

 

[Bacos]

Hi masters,

I'm going to make a newbie question, but... I just can't make it work.

I've installed a new fresh processwire in xampp, the AppApi module, created an application and keeping the "api" endpoint.

But when I run "https://localhost/testeapi/api/test" in Insomnia, I get the standard "404 Page Not Found" instead AppApi error or sucess message.

It seems that the AppApi is never being called.

The https is correctly configured in xampp.

Thank you for any help.

[Sebi]

12 hours ago, Bacos said:

It seems that the AppApi is never being called.

Hi @Bacos,

You're right: If the AppApi module had received the Api request correctly, even an incorrect request would be answered with an exception and a JSON response. You get a HTML-response, so the request is not received by the module.

Let's see... You send your request to https://localhost/testeapi/api/test - so your processwire root is https://localhost/testeapi/, am I right?

Can you please double-check if your module's config looks like this:

Another reason for a 404 error could be that you have already created a page in the ProcessWire page tree that is accessible under the /api route. Since the module uses a hook on ProcessPageView::pageNotFound to intercept requests, there must not be a page serving the api route.

I think that's all the approaches I can think of for now. Was there perhaps already something suitable ?? 

Edited May 16, 2021 by Sebi

[thomasaull]

Doesnt the API Endpoint needs to be `testeapi/api` in this case?

[Sebi]

5 minutes ago, thomasaull said:

Doesnt the API Endpoint needs to be `testeapi/api` in this case?

@thomasaullThank you! You are right - AppApiModule->checkIfApiRequest() compares with the full path of $_SERVER['REQUEST_URI']. Because of that, we must give the full path to the module-config, even if the ProcessWire root is in a subfolder. @Bacos: If you have only api instead of testeapi/api in your configuration, that would result in an 404 error!