Recently Browsing 0 members
No registered users viewing this page.
Hey Ryan, hey friends,
we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based).
So this is not your typical ProcessWire installation in terms of size.
Currently we count:
140 Templates (Some have 1 page, some have >6000 pages)
~ 15k Users (For an intranet portal? That's heavy.)
~ 195 431 Pages (At least that's the current AUTOINCREMENT)
I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like:
We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure.
We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec.
In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc.
We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above.
ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily.
That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long.
Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences.
We also have about 120 people working on the processwire backend the whole day concurrently.
What do you guys think?
Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would:
- Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time.
- Do you think at this point it makes sense to use ProcessWire as a simple REST API?
- In your experience, what fieldtypes are expensive? Page? RepeaterMatrix?
- Ryan, what do you consider as the primary bottleneck of processwire?
- Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible?
- Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors?
Thanks and lots of wishes,
Pascal from Mobile Trooper
Any help appreciated.
I started with Joomla on this CMS parade. Then later I put my hands on Wordpress and eventually settled into Concrete 5. Recently I came to know about ProcessWire CMS, while I was looking for a better replacement for Concrete 5 in terms of robust, easiness, compact. I went through few Video tutorials of PW and read the documentation. The installation of PW was smoothy.
The concept of Page is awsome. I like the relation of Page, Fields & Templates. There are many things I can equal to Concrete 5 from ProcessWire, yet PW offers a tiny minimal, code level approach for CMS. We have full control over design & data. There are some pitfalls I felt such no sitewide settings, global logo setup etc but it is easily tackled with some API codes. PW api seems amazing, it is par with what Concrete 5 Offers.
I have done a simple website in ProcessWire now and going to do a lot further :
Hello fellow PW devs!
This is a short story from the server management trenches.
These past couple of days trying to solve an unexpected problem: after DigitalOcean patched the droplets in NYC3 region last week, my client's droplet became almost useless and went down a couple of times. The droplet has 2GB RAM and was running Ubuntu 16.04 that was updated to kernel 4.4.0-116 after the patch. The server was provisioned using Forge (forge.laravel.com).
After sshing into it, and running "top" I've noticed the cause: "php-fpm7.1" processes (3-5 instances) were spiking the CPU to 100%. This was very odd, as the CPU usually kept around 33% most of the time.
The site uses ProCache and markupCache and was getting around 800-1000 visits/day last week. I checked everything on PW's side and nothing seemed out of place, so I went restarting PHP and Nginx but the problem continued. I checked access logs and no suspicious activity shown up. I upgraded PHP to 7.2 to see if anything will changed but the problem continued.
My only guess after all that is that the droplet in question got screwed up somehow, because I didn't see any complaints on the web of other people getting the same problem on DO (But I confess that I did a quick Google search only).
So in the end I decided to create a new droplet, now with 2 CPU cores and kept the 2GB (1 extra core and $5 cheaper ). Reinstalled PW there and pointed the floating IP to this new server.
The installation went smooth but to one issue: error log started to show messages of MySQL showing "to many files" error when the users were searching. I've never encountered this message before, so after reading some StackOverflow posts, I changed mysql.services config file to remove its file limit (https://stackoverflow.com/a/36807137)
Everything is normal now, but I think I'll never discover what truly happened. Anyone else had this kind of problem with MySQL before?