How to create a virtual page

[ferdinand]

We are planning to display course data from a non-pw-database in processwire and I'm looking for a way to hook the pws page data loading. But I haven't found a hook for that in the cheat sheet. Ideally we would like to provide all field data in more a less raw form to a hook whenever page xyz or a pw menu is build and have pw templates the render the data like it does for a normal page.

Sorry if this is a silly approach. We have just started

Thanks,
Ferdinand

[qtguru]

If I am to get this straight you are looking to have Processwire render a page but the details are not coming the PW Database, it's trying to set data to PW instance during runtime, making it seem like it's serving from a page. 

Indeed this is quite odd. Am curious why ?

 

 

 

[adrian]

You can easily make use of PW's $database to call an external database to populate content on a page - nothing fancy and no hooks required. It might help if you can provide more details of the data in these tables and how you want it displayed.

[qtguru]

Adrian is right if calling external database you can use WireDatabasePDO, it's a wrapper around PDO and additional method, 

[pwired]

Quote

You can easily make use of PW's $database to call an external database to populate content on a page - nothing fancy and no hooks required.

This is very interesting. Does anyone know some examples of processwire api commands to fetch data from an external database, how to store it inside processwire and then output it to a front ?

[kixe]

You can also have a look in my module http://modules.processwire.com/modules/fieldtype-select-ext-option/
This is a Select Fieldtype (single or multiple) which provides all the data of the selected table rows of any PW or Non-PW database table.

[pwired]

Thanks for that kixe. Do you might to know where to find more about $database and WireDatabasePDO ?

[adrian]

2 minutes ago, pwired said:

This is very interesting. Does anyone know some examples of processwire api commands to fetch data from an external database, how to store it inside processwire and then output it to a front ?

"how to store it inside processwire" - I am not sure what you mean here - do you mean save the content to PW pages? You can query content from an external database and simply iterate over it and output in your template file. 

Here is the docs on $database: https://processwire.com/api/ref/database/

Here is a rough idea of what you can do. Obviously I am querying the PW "pages" table, but you can do this on any table in the database whether PW or not. You can see in the output that you can easily iterate through each row in the returned array and grab whatever array keys (db fields) you want.

[ferdinand]

Wow! So many responses so soon. This is an active community. Thanks first of all.
Now some of you asked me to elaborate on the purpose. So let me try to be more specific.

The data we use is drawn from the clients internal course managment database which we access as mysql-tables.
Every day we display a limited selection of all courses (usually a window from today-10 days to today+1 year) on the website.

Access to the courses on display is provided by one- or two level access path-Strings that can be defined for each course (i.e. languages->german)  More than one access path may be defined to allow different ways of finding a path. (i.e. job->conflict AND communication->work-related). Each access path becomes a menu and submenu in the userinterface.

In a previous implementation of this in Drupal 6 we did in fact import all course data into the Drupal data model and ran an update of this database whenever a new import was done.

But with a system as flexible as pw I'd like to implement the real thing and hook database access.

In short:

Whenever PW receives a request for course "Dancing with the devil" a hooked call does all the queries of the external database (teh url tells me which data to retrieve) and returns all the desired field data to the calling procedure. So that as a result my $page-Object in the template is filled just like it would be if I rendered a normal page.

Ideally the same would apply when generating then menues (I'd just return an array of menu items to render) or the page that lists all the courses that have a certain access path (languages->german).

 

 

[ferdinand]

Thanks Adrian, this is what I was looking for. Or sounds like it. Will do some reading and check it out.

4 minutes ago, adrian said:

"how to store it inside processwire" - I am not sure what you mean here - do you mean save the content to PW pages? You can query content from an external database and simply iterate over it and output in your template file. 

Here is the docs on $database: https://processwire.com/api/ref/database/

Here is a rough idea of what you can do. Obviously I am querying the PW "pages" table, but you can do this on any table in the database whether PW or not. You can see in the output that you can easily iterate through each row in the returned array and grab whatever array keys (db fields) you want.

 

[ferdinand]

I think I'm missing something. The example shows how to access a database. PW or other run a query and retrieve data from it. And that part I understand. And of course I could also write all that into the template an thus fill my page with data.

But in doing so I would also mix business process code (generating the data) with presentation layer code (using templates to make the data look nice) and I would like to avoid that.

And I was thinking that a system full of hooks would als have a load-field oder load-page-data hook that I could intercept in my module and hand over my custom data to pw for presentation. Am I thinking in the wrong direction here. Or is this more that pw aims to provide?

 

11 minutes ago, adrian said:

Here is the docs on $database: https://processwire.com/api/ref/database/

Here is a rough idea of what you can do. Obviously I am querying the PW "pages" table, but you can do this on any table in the database whether PW or not. You can see in the output that you can easily iterate through each row in the returned array and grab whatever array keys (db fields) you want.

 

 

[adrian]

You can add a hook like this in your /site/init.php file:

$this->wire()->addHookProperty('Page::courseData', null, function($event) {
    $sql = "SELECT * FROM ......";
    $results = $this->wire('database')->query($sql);
    $event->return = $results->fetchAll();
});

And then in your template file you can call: $page->courseData

You can also make of $event->object->name inside the hook to get the name of the page that the courseData property is called from so you can use that in your sql query. Of course you can get any other field from the page, like $event->object->myfieldname and use that in your query as well.

Does that help?

[qtguru]

 

13 minutes ago, adrian said:

You can add a hook like this in your /site/init.php file:

$this->wire()->addHookProperty('Page::courseData', null, function($event) {
    $sql = "SELECT * FROM ......";
    $results = $this->wire('database')->query($sql);
    $event->return = $results->fetchAll();
});

And then in your template file you can call: $page->courseData

You can also make of $event->object->name inside the hook to get the name of the page that the courseData property is called from so you can use that in your sql query. Of course you can get any other field from the page, like $event->object->myfieldname and use that in your query as well.

Does that help?

Yep this works. Though once a cache is in place for that DB call there, I can say this seems like the solution and satisfies what OP needs. 

[ferdinand]

23 minutes ago, adrian said:

You can add a hook like this in your /site/init.php file:

$this->wire()->addHookProperty('Page::courseData', null, function($event) {
    $sql = "SELECT * FROM ......";
    $results = $this->wire('database')->query($sql);
    $event->return = $results->fetchAll();
});

And then in your template file you can call: $page->courseData

You can also make of $event->object->name inside the hook to get the name of the page that the courseData property is called from so you can use that in your sql query. Of course you can get any other field from the page, like $event->object->myfieldname and use that in your query as well.

Does that help?

Yes I think  I'm beginning to undertstand. Correct to say that I add a general property courseData to all the page objects and whenever that is read I call my function to provide the data? And using the $event->object->name property I can retrieve the name of the calling page and return data specific to the url?

That would work. And yet it makes you wonder, why there is not page_load_data field_load_data hook in pw. Any ideas why that is so?

 

[pwired]

Thanks for the examples guys. From here I can start trying out things. To let a website or a local computer (sometimes even a laptop) communicate with foreign databases is a much asked item. For example I know many call centers who are situated with local computers in Spain but want to fetch data from servers and databases in Germany.  Until know I lack how to code this but am learning bit by bit. This kind of work is payed very well if one knows how to do this.

[qtguru]

6 minutes ago, ferdinand said:

Yes I think  I'm beginning to undertstand. Correct to say that I add a general property courseData to all the page objects and whenever that is read I call my function to provide the data? And using the $event->object->name property I can retrieve the name of the calling page and return data specific to the url?

That would work. And yet it makes you wonder, why there is not page_load_data field_load_data hook in pw. Any ideas why that is so?

 

Not really, what the addHookProperty does is to run that code and inject your data from the database into Page::courseData in TLDR it dynamically creates the courseData property in the Page class. 

[qtguru]

5 minutes ago, pwired said:

Thanks for the examples guys. From here I can start trying out things. To let a website or a local computer (sometimes even a laptop) communicate with foreign databases is a much asked item. For example I know many call centers who are situated with local computers in Spain but want to fetch data from servers and databases in Germany.  Until know I lack how to code this but am learning bit by bit. This kind of work is payed very well if one knows how to do this.

 if you to make things easier, you can create an API sitting infront of the DB so from PW you simply call that API and based on the Data needed the API will fetch it from the relevant Database. However this might mean all call-centers having their own API so each can easily communicate because I doubt it would be easy to connect to their DB remotely directly. 

[adrian]

You will likely want to add some sort of conditional inside that AddHookProperty to check the template of the page so you only run the query on appropriate pages. eg.:

if($event->object->template == "template_that_needs_course_data") {

 

[adrian]

4 minutes ago, adrian said:

You will likely want to add some sort of conditional inside that AddHookProperty to check the template of the page so you only run the query on appropriate pages. eg.:

if($event->object->template == "template_that_needs_course_data") {

 

Actually I take that back - the function (and therefore the db query) won't be called unless you call $page->courseData so no need for this.

[ferdinand]

Ok, I understand how that works and will try to implement this for the pages.
Thanks you so much for your help.

 

[ferdinand]

As a follow up to the question above. How does pw determine

1. relationsships between pages (i.e. which children does a page have)
2. which template to use.

And is there a way to intercept this process?

Obviously both questions are normally answered by data that is created when creating a page (selecting a template and a location for a new page), but would not exist in virtual pages.

And exteme way of solving this would be to use the same standard template for all pages but I#d rather use a solution that extends the system rather then replacing it.

Thanks for any ideas,
Ferdinand

[LostKobrakai]

Both infos are stored in the "pages" table in the db: "parent_id" and "template_id". There's also "pages_parents", which lists all parents for each page, which is faster to query then building that table at runtime for each query. 

The template thing is probably quite easy to replicate for virtual pages, but searching pages by their parent-child relationship (deeper than 1 level down) is mostly dependent on the db. 

[ferdinand]

1 minute ago, LostKobrakai said:

Both infos are stored in the "pages" table in the db: "parent_id" and "template_id". There's also "pages_parents", which lists all parents for each page, which is faster to query then building that table at runtime for each query. 

The template thing is probably quite easy to replicate for virtual pages, but searching pages by their parent-child relationship (deeper than 1 level down) is mostly dependent on the db. 

Thanks for that info. Good to know where that data is stored. Is there any way to intercept pw when it retrieves that information so that I can substute it for virtual pages?

[LostKobrakai]

Your best bet would be to look at the PageFinder class. It's doing all the work of transforming selectors to mysql requests.

[ferdinand]

23 hours ago, LostKobrakai said:

Your best bet would be to look at the PageFinder class. It's doing all the work of transforming selectors to mysql requests.

Thanks for that hint. I checked out that class but I can't connect it to the problem.
It seems to me that the Page-class itself would be far more suitable.

Is it possible to create a subclass coursePage and overwrite some of its methods (like getParent, children() etc. with my own code that works within the logic of the courses-database? And if so, how can I make pw use such a modified object.