Jump to content

Recommended Posts

Hi all,

I have been asked by a client whether we can setup load balancing for their existing Processwire site.

From my investigations on Google and within these forums, it definitely seems possible but as a newbie with a basic understanding of the subject im a bit lost.

Does anyone know of any existing tutorials for settings up load balancing with PW?

What items would need to be changed on their current stand alone install, is there a list of best practices worth consulting etc?

As I understand it we would need to have some sort of copying mechanism (rsync script most likely) in order to make sure any uploaded assets are shared between the main server and the fallback ones, other than that im not sure what else would need to be ammended.

Any thoughts/help would be greatly appreciated.

Share this post


Link to post
Share on other sites

I'd look into AWS ELB: https://aws.amazon.com/de/elasticloadbalancing/

@ryan has set up his Villas site with some sort of load balancing (or whatever you would call it exactly):

Quote

this site involves 1 PW installation for managing the inventory, and another PW installation for presenting it, and they use web services to chat back and forth all day.

Maybe he would be willing to share some more in-depth technical details...

Share this post


Link to post
Share on other sites

There are three things to consider for load balancing: You need a central database, central file management (if people do upload things) and central session management (as the default also uses the filesystem). Each can be solved in various ways and depends mostly on your project's constraints.

Database
Probably as simple as running the db on it's own external server.

File Management
- Cloud storage (S3 or equivalent; minio for self hosted S3)
- rsync (this is at best eventually consistent as you need to wait for the sync)
- mount network share (I've not tried this and read a lot of bad things, but it might work)

Session Management
- Use the central db for session (SessionHandlerDB; might be slow)
- Use a central redis server (can be done directly in php)

  • Like 2

Share this post


Link to post
Share on other sites
1 hour ago, LostKobrakai said:

There are three things to consider for load balancing: You need a central database, central file management (if people do upload things) and central session management (as the default also uses the filesystem). Each can be solved in various ways and depends mostly on your project's constraints.

Database
Probably as simple as running the db on it's own external server.

File Management
- Cloud storage (S3 or equivalent; minio for self hosted S3)
- rsync (this is at best eventually consistent as you need to wait for the sync)
- mount network share (I've not tried this and read a lot of bad things, but it might work)

Session Management
- Use the central db for session (SessionHandlerDB; might be slow)
- Use a central redis server (can be done directly in php)

Thanks for the replies folks.

Given the above response does the following hypothetical setup sound reasonable.

  • Primary DB with mirrors hosted on Azure
  • Rsync contents of Assets/Files folder between primary server and mirrors
  • Enable database driven sessions, that way we don't need to copy Assets/Sessions between the servers

Obviously thats very vague in the details but as a general proposal is there anything glaring wrong with this approach?

Share this post


Link to post
Share on other sites

I've tried some experiments with Processwire on Heroku (with multiple Dynos).

It worked - I used

  • For Sessions: Heroku Redis + https://modules.processwire.com/modules/session-handler-redis/ + https://elements.heroku.com/addons/heroku-redis
  • For Uploads: Amazon S3 + https://modules.processwire.com/modules/amazon-s3-cloudfront/
  • For the DB: https://devcenter.heroku.com/articles/jawsdb-maria
  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By gebeer
      Although the PW backend is really intuitive, ever so often my clients need some assistance. Be it they are not so tech savvy or they are not working in the backend often.
      For those cases it is nice to make some help videos available to editors. This is what this module does.
      ProcessHelpVideos Module
      A Process module to display help videos for the ProcessWire CMS. It can be used to make help videos (screencasts) available to content editors.
      This module adds a 'Help Videos" section to the ProcessWire backend. The help videos are accessible through an automatically created page in the Admin page tree. You can add your help videos as pages in the page tree. The module adds a hidden page to the page tree that acts as parent page for the help video pages. All necessary fields and templates will be installed automatically. If there are already a CKEditor field and/or a file field for mp4 files installed in the system, the module will use those. Otherwise it will create the necessary fields. Also the necessary templates for the parent help videos page and it's children are created on module install. The module installs a permission process-helpvideos. Every user role that should have access to the help video section, needs this permission. I use the help video approach on quite a few production sites. It is stable so far and well received by site owners/editors. Up until now I installed required fields, templates and pages manually and then added the module. Now I added all this logic to the install method of the module and it should be ready to share.
      The module and further description on how to use it is available on github: https://github.com/gebeer/ProcessHelpVideos
      If you like to give it a try, I am happy to receive your comments/suggestions here.
    • By NehaPillai
      Hello Everyone, I was trying to update SEO meta title, description and meta keywords for my website in Process Wire CMS but it saving in the backend but it is not reflecting on my website, Please help me regarding this error. Please find below attached screen shot for your ref. TIA.


    • By Mobiletrooper
      Hey Ryan, hey friends,
      we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based).
      So this is not your typical ProcessWire installation in terms of size.
      Currently we count:
      140 Templates (Some have 1 page, some have >6000 pages)
      313 Fields
      ~ 15k Users (For an intranet portal? That's heavy.)
      ~ 195 431 Pages (At least that's the current AUTOINCREMENT)
       
      I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like:
      We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure.
      We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec.
      In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc.
      We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above.
      ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily. 
      That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long.
      Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences.
      We also have about 120 people working on the processwire backend the whole day concurrently.
       
      What do you guys think?
      Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would:
       
      - Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time.
      - Do you think at this point it makes sense to use ProcessWire as a simple REST API?
      - In your experience, what fieldtypes are expensive? Page? RepeaterMatrix?
      - Ryan, what do you consider as the primary bottleneck of processwire?
      - Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible?
      - Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors?
       
      Thanks and lots of wishes,
      Pascal from Mobile Trooper
       
       
    • By Sten
      Hello,
      I'd like to participate to a documentation about the page flow in PW. I am still a beginner and still have difficulties to see the big picture but I could help to the writing.
      What is the flow when a page is displayed or created ? It would be fine to draw a scheme with every stage in which one can add hooks. Something to describe the flow in time.
      May be it exists.
      page
      |
      fields
      |
      template
      What do you think ?
    • By paulb
      Hello,
      I recently started making a blog using Processwire and since i'm completely new, I struggle with a few things.
      I'm trying to make a comment section in my blog post, so I followed the tutorial and pasted the two code lines in my template.
      18 echo $page->comments->render(); 19 echo $page->comments->renderForm();  
      but now I have this error on my page :
      Error: Call to a member function render() on null (line 18 of C:\wamp64\www\processwire\site\assets\cache\FileCompiler\site\templates\Post.php)  
      Am I missing something?
      Thanks for your help.
×
×
  • Create New...