Headless ProcessWire - How Do You Do It?

[wbmnfktr]

Intro: Just to let you all know where this comes from...

In the last few weeks I had a lot of tasks and projects in regards to SSG (static site generators), mainly some *.js frameworks. While those *.js frameworks weren't anything of my taste I started to love and enjoyed 11ty. I've already re-built some of my personal projects with it (11ty) just on static files (js/json data files), which worked really great. BUT... and this is my reason I post this here... this was hard work to move data into static files. 

Question: How Do You Do It?

How do you handle ProcessWire as a Headless CMS for SSG projects?

I tried the Pages2Json and GraphQL modules and even custom URL-Hooks for JSON output... but those solutions didn't fit any of my needs.
Not even close.
It could be me, but I thought about solutions that were more kind of ready-made for almost all purposes.
(Yes, I know PW is more of a toolkit :D  But most projects just don't have the budget to allow such custom exports of data.) 

As I wanted to build more complex sites (even as those are mostly only prototypes whatsoever) with way more data, as in posts, pages and such... I installed a basic Wordpress instance, used Custom Post Type UI for custom post types, I was able to get everything done within minutes. Yes, minutes. It took me 3 minutes to set everything up (5+ post types), while adding data took another 10 minutes for each post type.

The result was, that I had specific JSON feeds for everything I needed.
wp.domain/wp-json/wp/v2/posts ... /pages .../movies .../authors .../whatever - all in perfectly formatted JSON, even those fields that where references, like categories, topics, and other taxomonies.

I'd love to see/hear about your solutions.

I'm almost certain that PW should or might need an additional (instead of new) focus.

As PW is able to do way more than any other Headless CMS out there... it might and should try to match up (in terms of marketing and onboard solutions).

I tried Forestry CMS, DatoCMS, SanityCMS, NetlifyCMS and a lot more Headless CMS the last few weeks... all were nice but FAR away from what a basic PW instance can do. I was shocked how basic, yet praised they (other HeadlessCMS) were in the community.
Even without ProModules ProcessWire is way ahead of any Headless CMS I tried/used so far. (SURE! It could be me, but I doubt it.)

Maybe I'm missing something but I for sure don't wanna miss out on PW for an awesome solution as a Headless CMS.
If there was a native JSON/RestAPI feed like in WordPress or any other like-minded (as in easy) solution in it... it could probably the one solution for all.

So...

Headless ProcessWire - How Do You Do It?

 

Therefore, as this is way more complex than an average question, I tag some of you right here, as you may know much more about anything in terms of ProcessWire or ProcessWire Headless as me. 

@ryan @adrian @Robin S @horst @psy @Jonathan Lahijani @teppo @dadish@thomasaull

[monollonom]

I don't know much about the tools you mentioned and regarding your JSON generation issue I'm not sure I see why the modules you mentionned don't fit the bill, but have you also checked/tried:

?

Assuming you have templates named "post", "movie", "page",... you could have something like:

$wire->addHook("/api/(post|movie|page)/(.*)", function($event) {
	$template = $this->templates->get($event->arguments(1));
	if(!$template) return;
	header("Content-type: application/json");
	$fields = $template->fieldgroup->explode("name", ["key" => "name"]);
	if($event->arguments(2)) {
		$page = $this->pages->findOne("template=$template,name=".$event->arguments(2));
		if($page->id) {
			return $page->pageQueryJson($fields);
		}
	}
	return $this->pages->find("template=$template")->pageQueryJson($fields);
});

This way you could have pw.domain/api/post returning all posts and then you could query a specific one with pw.domain/api/post/page-name.

(one thing to note is the module does not handle repeaters (or other third parties fields) out of the box, though the docs seem to mention how to do so actually it does but I have an issue in my setup, I think)

In case this doesn't help or I didn't get your issue, would you mind to elaborate ?

[dragan]

As far as I understand, Headless CMS and SSG aren't the same thing really. So which one do you really want?

If you'd like to use an SPA framework like React, you'd probably want headless. If you'd like to automagically create static pages, it's a typical SSG setup.

For headless, I'd try the AppApi module.

For SSG, I'd use Pro Cache.

I'm not sure what to make of your example comparison with WP and taxonomies. In PW, you create your own taxonomy or data model.

It's easy to create JSON output from your PW instance. Use URL segments, and handle each /json request to your own script.

11 hours ago, wbmnfktr said:

The result was, that I had specific JSON feeds for everything I needed.

It's ridiculously easy to create JSON feeds (just a bunch of URLs that match your selector). Is that all you really need? Sorry if I misunderstand you, but there's way more to a headless CMS than just having a URL feed.

Again, I don't get what you really want in the end. If you need static sites, what good are "JSON feeds" for you?

[gebeer]

.I guess the reason why @wbmnfktr resorted to WordPress is that you have the JSON feeds readily available without any coding. And I often thought that it would be really awesome if we had something like this in PW. With all the modules mentioned AppAPI GraphQl etc you still need to write code to get the desired output at the desired endpoint. WordPress handles this out of the box. And this makes it attractive. PW would definitely benefit and get more attention if there was a module that automatically creates RESTful API endpoints following the page tree structure.

[wbmnfktr]

Thanks @monollonom - I didn't know that module but I will have a look at it. Maybe that's more of what I need.

Thanks @dragan - you are right, a Headless CMS is something completely different than a SSG. I already found a nice SSQ solution and right now I'm looking for a fitting CMS. I looked at a lot of those mentioned above, yet ProcessWire is way more advanced and more capable of almost anything. And sure... creating JSON feeds is easy until you have a more complex setup of fields, references, repeaters, Matrix fields and such.

That's exactly that point @gebeer made here. In WordPress it was super easy to get it right from the start - sure still not a super complex structure, no real repeaters, but way ahead. 

I'm fine with going into the code, write my output but I sometimes just need something that works right out of the box. Wordpress could be a good fit but maybe even better one of those other CMS out there.

4 hours ago, gebeer said:

PW would definitely benefit and get more attention if there was a module that automatically creates RESTful API endpoints following the page tree structure.

This in the core, with support for (matrix) repeater, combo and all the good stuff... this would open a totally new and large user base.

[wbmnfktr]

Some additional feedback so far...

The PageQueryBoss module, which @monollonom recommened looks pretty nice and works out really well so far. There are issues, but way less than before.
I will use and play around with it for a bit. Maybe that's what I was looking for.

In regards to those other modules I mentioned before... 

GraphQL
In my setup I had to change each and every field to make it available for public (legal) requests.
That's awesome for a setup that needs control about each and everything. But that's nothing I needed or need.
Therefore changing a lot of fields without knowing anything about possible issues in other pleases, wasn't an option.

Pages2Json
The simple JSON feeds were easy to create but... customizing, repeater fields, matrix fields and such... were way out of focus and not that easy at that time (we haven't had a lot of budget and time to get things done)... so maybe it's a nice solution after all. I used it before and it's still in some other projects but in this special use-case... it wasn't that perfect for me.
So... I can't say anything bad about the module, yet it wasn't a solution for that specific project.

Another thing... 
While those modules work for some use-cases and in probably a lot of existing projects, they are still 3rd party (which isn't a bad thing, but...) and looking at the "last update" dates... they are somewhere between 2018 and 2021. Which isn't that bad compared to some core and 3rd party modules, yet I have to rely on those modules for the next couple of years. Changing PW versions, PHP versions and whatsoever. Some support threads are way outdated and sometimes even look "bad" with unanswered questions. Didn't and still don't feel comfortable with this - not only in this very case. Therefore I use so many ProModules for a lot of things.

 

 

[gebeer]

14 hours ago, wbmnfktr said:

This in the core, with support for (matrix) repeater, combo and all the good stuff... this would open a totally new and large user base.

Yes, it would for sure. I recall that Ryan was talking about making the PW API available through JS some years back. That would be exactly what we need. A RESTful API like the WP REST API in ProcessWire. This should ideally be available as an optional core module and support all Pro fields.

[3fingers]

6 hours ago, gebeer said:

I recall that Ryan was talking about making the PW API available through JS some years back.

So I recall, and still waiting ? It was on top of his list but...

EDIT:

https://processwire.com/blog/posts/roadmap-2017/#whats-in-store-for-processwire-in-2017

[flydev]

If you are interested I can take the time to publish a draft of a profile which you could use as starting point, and IMO it's better and easily than other solution that I could have tested. Basically it use a modified version of InertiaAdapter made by @clsource and let you code your app inside ProcessWire's template and using pages for everything fetched dynamicaly and internally, as JSON.

 

You can see it "in action" in this example url: https://blog.sekretservices.com/

(Do not take care of the issue while refreshing the blog post due to hanna codes, it's an old version)

 

For example, from your template you build the page properties as a simple PHP array, let's say I want to fetch the title of the page:

$component = "Home/BlogPost";
$properties = [ // accessible from app components with $page.props
    'name' => $page->name,
    'title' => $page->title,
    'subtitle' => $page->subtitle,
    'content' => $page->body,
    'author' => ucFirst($page->createdUser->fullname[0])."."." ".ucFirst($page->createdUser->surname),
    'date_created' => $page->created,
    'date_relative' => $datetime->relativeTimeStr($page->created)
];

Then from a component BlogPost.`{svelte, vue, react}` (or whatever) you can simply retrieve the page's props like:

(Svelte example)

<script>
  import {inertia, page} from '@inertiajs/inertia-svelte';
  export const { title:title, body:body } = $page.props
</script>

<div class="blog-post">
  <h1 class="font-sans mt-8 text-center font-bold">{@html title}</h1>
  <div>{@html body}</div>
</div>

 

Look at this sample, at the page's tree and the routes prop from the console :

 

About the workflow, it's not really different from what you should be used to. You write your logic in template in PHP then write some HTML/JS in your component.

Also, hot module replacement (HMR) is working ?

A word about server side rendering (called SSR and which is needed to get best SEO results), it's working for React and VUE, still a WIP for Svelte ?

Get a note of ?on every lighthouse performance.

Bundled with ViteJS, powered by ProcessWire, I am glad to say that it's the best stack I have ever built and used ??

 

Edit:

A small note, but the better one. You don't have to build your app again on every change on the backend, I mean for example when you add a new page, it's work out-of-the-box, you create and page, and it's available to your JS components.

Edit2: 

To understand better, here we can achieve what you already used/saw with Laravel. Having a `src` folder with the JS things, another folder with the Laravel `app` and running artisan and some npm command.

Here, in dev mode, you run `yarn dev` and you are ready. Simple as that.

 

Edited February 21, 2022 by flydev ??
added example url

[lokomotivan]

On 2/19/2022 at 2:01 PM, dragan said:

As far as I understand, Headless CMS and SSG aren't the same thing really

They are not, but they can be used together. Static sites doesn't have to be so static, if you are using static site generators (11ty, astro, nuxt, next...), you can fetch content from the headless cms, then you can trigger re-build when content is changed, and automate the deployment.  

On 2/20/2022 at 12:21 PM, gebeer said:

PW would definitely benefit and get more attention if there was a module that automatically creates RESTful API endpoints following the page tree structure.

+1 for this.
As processwire is in fact headless, having the REST api out of the box would be great. Take strapi as example...

Imo, having REST api would be much more beneficial, then live previews for example. Saying this because there was a discussion about live previews, but also of js api... If i could vote, would always put the time into the js api...

[bernhard]

3 hours ago, 3fingers said:

8 hours ago, gebeer said:

I recall that Ryan was talking about making the PW API available through JS some years back.

So I recall, and still waiting ? It was on top of his list but...

EDIT:

https://processwire.com/blog/posts/roadmap-2017/#whats-in-store-for-processwire-in-2017

To quote ryan (4. Jan 2022): https://processwire.com/talk/topic/24876-pw-30170-– core-updates/?do=findComment&comment=209625

Quote

As far as non-external headless, ProcessWire was the first to do this, at least as far as I know. The admin was later built in as an application in ProcessWire. For external API headless, this has been something I’d planned to do years ago (per old roadmaps and JS APIs). But, when it came to actually starting to implement it in the way I thought it would be useful, I found it opened a can of worms in terms of security. And security is one thing that you can trust I won’t make compromises on. I’ve come to settle on being comfortable with an external API so long as it is protected by very strong authentication. But then it also becomes less of a common strategy to build around, and more of a specialist thing… maybe useful to a small percent of users. Maybe there’s a middle ground and it’s worth revisiting this year. 

 

[3fingers]

5 minutes ago, bernhard said:

Maybe there’s a middle ground and it’s worth revisiting this year.

Let's hope then this is the year! ?

[lokomotivan]

20 minutes ago, bernhard said:

But then it also becomes less of a common strategy to build around, and more of a specialist thing… maybe useful to a small percent of users. Maybe there’s a middle ground and it’s worth revisiting this year. 

I could not agree with a "small percent of users"... Front-end is moving and has moved from the "traditional" approach. 
But let's hope in 2022 some opinions could be changed :).

[AndZyk]

Maybe this question is too general and I don't want to hijack this topic, but:

What are your reasons/advantages for using a headless CMS?
Is it performance, targeting different channels than browsers or is it more fun to structure your content?

I am curious, because I am still building websites the traditional way. ?

[lokomotivan]

39 minutes ago, AndZyk said:

Maybe this question is too general and I don't want to hijack this topic, but:

What are your reasons/advantages for using a headless CMS?
Is it performance, targeting different channels than browsers or is it more fun to structure your content?

I am curious, because I am still building websites the traditional way. ?

I also still build websites traditional way, depending on the project, and needs. If I need a CMS and more advanced features, using pw, if need a more basic website, then trying to use SSG. Usually without cms, content in markdown… Just started with SSG sites, but planning to move to it as much as I can.

• Don’t have to keep your front-end files on the same server
• Easier to deploy websites (static sites, SPA). You can deploy your website to github, netlify, cloudflare pages… no server needed
• Easy to build multi websites with one backend
• Easier to collaborate via git
• Easier to share data between websites
• Tooling, preprocessor parsers, purge css, live reload etc…
• Easier to switch front-end entirely, eg: change front-end framework, change static site generator…

[wbmnfktr]

4 hours ago, AndZyk said:

What are your reasons/advantages for using a headless CMS?

It's not really about the Headless CMS - at least not in my case. I'd be totally fine with just editing .md (content) or .njk (templates) files from the command-line.
While this works perfectly fine for me, my clients aren't that keen to work this way. They love their interfaces to edit content, creating pages and such.

So there is still need for some kind of interface of some kind for my clients.
As I love ProcessWire and all the things I can do with it, using it as a Headless CMS would be the perfect match.

Not only that but ProcessWire is way ahead compared to other CMSs, especially combined with ProModules. Yet it's not that easy to get this up and running to use it with 11ty, Hugo, Gatsby and other SSGs (Static Site Generators). Or at least it wasn't for me and still isn't. That's why used WordPress for quick solution. There is a REST API/JSON feed with everything I needed right from the start, some additional changes and I was ready to go. But as soon as I know how to get these JSON feeds from ProcessWire, I will switch that project over for obvious reasons.

And the reason why this even became a topic for me: static sites are way more comfortable in terms of "release and forget". If a client doesn't change anything within a year or two, it doesn't matter at all. There are no risks, no security issues, no updates, nothing. Sure you have to maintain a ProcessWire instance even though you rarely use it but you can disconnect it from the deployment and nothing will happen to the real site. But maybe in some cases it's already enough to use Forestry CMS, which right away talks to Github - kinda awesome!

@lokomotivan listed some very good reasons for static sites ans SSGs and I will go this route as soon as I can for existing projects and upcoming ones. Let alone the performance differences are quite insane, even when using ProCache. I hope to get one of my larger side-projects up and running with this kind of setup in the next months. Maybe with ProcessWire, maybe with another option (just to get a feeling for other tools).

While I already get awesome numbers in performance tests, the amount of unwanted traffic through bots, spiders and a lot of script-kiddies is an issue for me.
Using Netlify or Cloudflare as hosting solution this wouldn't be an issue anymore.

 

So... while classic websites are still a thing, I'd like to add another option to my tool belt and I'd love to use ProcessWire with it.

[bernhard]

1 hour ago, wbmnfktr said:

Not only that but ProcessWire is way ahead compared to other CMSs, especially combined with ProModules. Yet it's not that easy to get this up and running to use it with 11ty, Hugo, Gatsby and other SSGs (Static Site Generators). Or at least it wasn't for me and still isn't. That's why used WordPress for quick solution. There is a REST API/JSON feed with everything I needed right from the start, some additional changes and I was ready to go.

Could you show the exact steps that are necessary to get what you want when using one of these systems? And list what you are missing with PW and how in your opinion it would make sense to be?

[wbmnfktr]

14 minutes ago, bernhard said:

Could you show the exact steps that are necessary to get what you want when using one of these systems?

In which scenario?
Rebuilding ProcessWire in those Headless CMS or what's needed to add and import from JSON/API into those SSGs?

What I can already say and show are some parts from ForestryCMS in terms of fields that are available and what could be done so far there - right now Forestry is my option for the next couple of weeks to play around with.

The most interesting parts might be these... details in the docs.

Including templates: https://forestry.io/docs/settings/fields/#include-template

Repeatable Fieldgroup: https://forestry.io/docs/settings/fields/#repeatable-field-group

Blocks: https://forestry.io/docs/settings/fields/#blocks

And in terms of importing data into 11ty (the SSG I use right now) is quite easy... that's the guide I used and it worked out pretty well.
https://davedavies.dev/post/how-to-use-11ty-with-headless-wordpress/

 

Let me know if I missed the point. 

[bernhard]

This is what I meant:

1. I install Forestry on my host
2. Then I create Template XY
3. Then I add fields X, Y, Z to the template
4. Then I add Module X to get JSON Feed
5. Then I create a Frontend and access this feed

Steps 4+5 are missing in ProcessWire

[AndZyk]

Thank you @lokomotivan and @wbmnfktr for your insights. ?

I can see that a headless CMS + SSG can bring advantages in performance and flexibility.

I am not sure if this way of developing websites is something for me, but maybe I will try it out.

Especially the performance improvement with SSGs is something that interests me. ?

[flydev]

18 hours ago, wbmnfktr said:

While I already get awesome numbers in performance tests, the amount of unwanted traffic through bots, spiders and a lot of script-kiddies is an issue for me.

@wbmnfktr  this module is for you :  https://processwire.com/modules/blackhole/

Then add the 3 `wp-` magics folders and you will get rid of most bots :

[pwired]

Quote

I also still build websites traditional way, depending on the project, and needs. If I need a CMS and more advanced features, using pw, if need a more basic website, then trying to use SSG. Usually without cms, content in markdown… Just started with SSG sites, but planning to move to it as much as I can.

• Don’t have to keep your front-end files on the same server
• Easier to deploy websites (static sites, SPA). You can deploy your website to github, netlify, cloudflare pages… no server needed
• Easy to build multi websites with one backend
• Easier to collaborate via git
• Easier to share data between websites
• Tooling, preprocessor parsers, purge css, live reload etc…
• Easier to switch front-end entirely, eg: change front-end framework, change static site generator…

Is SSG not mostly being used by coders, git-users, documentation, etc. ? Is SSG really something to use for a client website ?

[lokomotivan]

9 hours ago, pwired said:

Is SSG not mostly being used by coders, git-users, documentation, etc. ? Is SSG really something to use for a client website ?

Everything comes down to the content management needs. I personally like managing content with markdown files and front matter, but obviously not ideal solution or non-technical non-dev users. Also, theres clients that want you manage content for them, and content is rarely changed.
In this case you dont really need a cms, but when you need it, this is where headless cms comes into play. And can work very well.

• Use headless cms to add/manage content
• Fetch content with SSG and build front-end with favorite generator / framework.
• Deploy your static site to the preferred service (github, cloudflare, netlify, vercel....).
• Use webhooks, or github actions to trigger rebuild when content is changed.

As re-generating static site takes a bit of time, and compute, this is not ideal for all situations and needs. It depends on how frequent content is changing. If you have a directory for example, and visitors can publishing content, obviously this would not be a good solution. 

There's a lot of headless cms-s out there, but non of them from my experience is good as processwire. What pw is missing is is  "external headless" features, REST api. Yes, we can build it our self, and can make json feeds fairly easy, but having it out of the box would be great. Im not an api or security expert, dont know implications, but from what i see most/all of the cms (especially headless) out there have this...

[bernhard]

I'm still curious how that "out of the box" would look like and how other platforms are working in that regard... As a detailed step by step use case.

[flydev]

I admit that this functionality could/should be included in the kernel (but disabled by default) as I personally use it for every installation, but still I didn't feel the need for it (getting it in the core) as I find the flexibility provided by the module AppApi made by @Sebi more interesting. 

Just include this module in your setup and you are ready. Anyway the argument is really good about that it will excite new users who have not yet tried ProcessWire but want to give a try when they see that they can use/build an API out-of-the-box.