Recently Browsing 0 members
No registered users viewing this page.
Hey Ryan, hey friends,
we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based).
So this is not your typical ProcessWire installation in terms of size.
Currently we count:
140 Templates (Some have 1 page, some have >6000 pages)
~ 15k Users (For an intranet portal? That's heavy.)
~ 195 431 Pages (At least that's the current AUTOINCREMENT)
I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like:
We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure.
We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec.
In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc.
We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above.
ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily.
That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long.
Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences.
We also have about 120 people working on the processwire backend the whole day concurrently.
What do you guys think?
Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would:
- Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time.
- Do you think at this point it makes sense to use ProcessWire as a simple REST API?
- In your experience, what fieldtypes are expensive? Page? RepeaterMatrix?
- Ryan, what do you consider as the primary bottleneck of processwire?
- Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible?
- Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors?
Thanks and lots of wishes,
Pascal from Mobile Trooper
What i wanna achive is a simple counter like that count up on visit (this is no problem) AND save the specific date (year/month/day) of the count...
in the end i will be able to get visits per day/per month/per year in a nice and dirty graph.
Just to have a way better simple counter system.
Should i only go with a complex setup of pages like this:
--stats (home template for pageviews)
---------29 ->page_views (integers on every day template)
Or just simple use:
--stats (home template for pageviews)
---->count (template) that holds simple field page_views and a date field
or could a fieldtype like tables (one table field for every month/year or so) be also a solution?
Or a own SQL table special for this and use it in a module? I don't have any experience on this topic...
What i have in mind of performance sideeffects on such a thing?
Or is there a solution that works with PW?
I wanna go the hard way and implement something like this:
only directly within PW and use the API to get the data...maybe create a simple module from it later i don't know if i could set it up right from the start 😉
this is the reason for my questions on more experienced devs
Kind regards mr-fan
Any help appreciated.
So I have a fairly unusual project, and I am still trying to wrap my head around it (so excuse me if this doesnt make much sense). A user will create parent page (in the backend) with a modal (piece of cake), but then have a custom js file for each parent page. I wouldn't have an issue with creating a new file everytime, but this instance calls for it pretty much to be an automated process as the users are not tech savy.
I was thinking that I could create the template for the modal (which will be iframed elsewhere), and using an approach found here, automatically create a "js" child page that I could then somehow output to a custom js file.
I guess I have several questions regarding this since the modal is iframed in:
1. Since the init script for the modal has to be outside of the iframe (and placed elsewhere), what is the best way to render the custom script (which will be generated from options on the page in the backend.
2. Would it even be safe/secure to even attempt this since the js file would be referenced elsewhere (though still in a file on the pw install)
One of our sites is suffering from very slow boot times, and I'm not sure how to diagnose the problem.
Here's a grab of the debug panel in Tracy debugger after loading the homepage.
A have a couple of questions -
Are all of the times listed separate items, or are some of them a breakdown? I ask because the number shown in the tracy debug bar is the total of all of the items but the wording suggests boot.load.modules, boot.load.fields etc are a breakdown of the boot.load. How do I find out what these times consist of? Currently, when using the site, and when running page speed tools, the server load time is consistently upwards of 1s often above 1.5s. This is before the browser even starts downloading resources - a quick grab from my firefox dev tools was even worse:
I would appreciate any advice on finding the cause here.
A few details:
Server is a digital ocean droplet (2GB memory + 2CPUs) running nginx and php7.0 - neither memory or cpu seem particularly taxed Site has 8 locales Using template cache and wirecache for heavy pieces of markup We're on the latest dev branch - the speed issue has been present for the last couple of versions. The speed is similar on when running locally (similar but stripped back nginx config)