Jump to content

Server overload help


benbyf
 Share

Recommended Posts

HELLO!

I've had two Timeouts recently on a site due to the server being overloaded.

The site is hosted on digitalocean on a single distro with 512 MB Memory, 20 GB disk, Ubuntu 14.04.4 x64. as shown below before each outage I can a climbing peak in user bandwidth then the CPU hangs:

Screenshot 2016-09-20 16.28.32.png

(the light blue above is user, and the dark blue sys)

Not really had anything like this before with processwire as am puzzled. the site gets between 200 - 1000 visits a day, and I'm using the PW built in caching.

Any advice or tips would help loads!

Link to comment
Share on other sites

found this in logs:

exceptions:

errors

DirectoryIterator::__construct(/srv/users/serverpilot/apps/betaville/public/site/assets/cache/Page/1/): failed to open dir: No such file or directory (in /wire/core/CacheFile.php line 208)

 

  Error: Exception: DirectoryIterator::__construct(/srv/users/serverpilot/apps/betaville/public/site/assets/cache/Page/1/): failed to open dir: No such file or directory (in /srv/users/serverpilot/apps/betaville/public/wire/core/CacheFile.php line 208)
Link to comment
Share on other sites

1 minute ago, benbyf said:

not sure how to do that without breaking my site (which i did abit just now)

You might be able to do a: bd($this->path); before that line (if you have Tracy installed). Depending on when the CacheFile.php is loaded, this may or may not work. Turning on Tracy's Dump Recorder and Preserve Dumps option may also help.

Alternatively you could do: wire('log')->save('testing', $this->path); and then check the value recorded in the 'testing' log file.

Link to comment
Share on other sites

44 minutes ago, teppo said:

Am I reading your graphs correctly, i.e. is there a notable increase in inbound network activity when this happens? If so, is that natural traffic (regular visitors browsing the site) or something else?

 

Good point. Check for spiders on the log.

Link to comment
Share on other sites

44 minutes ago, teppo said:

Am I reading your graphs correctly, i.e. is there a notable increase in inbound network activity when this happens? If so, is that natural traffic (regular visitors browsing the site) or something else?

should be normal vistors, but not sure how i would tell...

1 minute ago, Sérgio said:

Maybe a silly question, but is Swap enable and it's at least 2GB?

not sure what you mean here...

 

2 hours ago, DaveP said:

Guessing, but this sounds like a permissions issue. /cache/ & children writeable?

thought so too but its currently 775 on a user and im not going to go 777 for obivous reasons.... but why would it not have a problem caching the other pages, or not...? thats the strange thing, im only ever seeing those two pages id's in teh cache folder and in errors

Link to comment
Share on other sites

9 minutes ago, benbyf said:

not sure what you mean here...

Disk swap is needed when some portion of disk space must be reserved to be used when the system RAM is full, so the disk acts like RAM (although much slower).

The default is zero on a fresh Ubuntu install and for lower RAM servers like this one should be at least the double (it's recommended 2GB for 512MB).

Serverpilot (that you're using) enables 512MB only: https://serverpilot.io/community/articles/swap-is-enabled.html

So I recommend you to log as root and increase it. 

https://www.digitalocean.com/community/questions/how-to-change-swap-size-on-ubuntu-14-04

And look here too: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-16-04

  • Like 2
Link to comment
Share on other sites

12 hours ago, benbyf said:

should be normal vistors, but not sure how i would tell...

First thing to check is whether those requests look natural, even if just to rule out the off chance of a DDoS attack and stuff like that. Tons of requests to a specific page and usually something uncacheable (like search page with the same search term every time) is a telltale sign that someone is intentionally bombarding your site.

It's also good to check where those requests are pointed at: if there's a lot of traffic to a specific part of the site at the same time when issues start, the problem might be related to that part too. Or it could just mean that a specific piece of content is extremely popular :)

Most properly configured and resourced servers should be able to handle bot traffic with ease. There's the possibility of a misconfiguration and the settings of Apache, MySQL, PHP, etc. can always be tweaked in order to handle traffic spikes better, but if the root cause is that you're being harassed, there are other (and more efficient) ways to handle that.

  • Like 1
Link to comment
Share on other sites

14 hours ago, Sérgio said:

As you're using Nginx in front of Apache, maybe it's worth to try this: https://github.com/mariusv/nginx-badbot-blocker

sweet! added, so hopefully that'll help

14 hours ago, Sérgio said:

Disk swap is needed when some portion of disk space must be reserved to be used when the system RAM is full, so the disk acts like RAM (although much slower).

The default is zero on a fresh Ubuntu install and for lower RAM servers like this one should be at least the double (it's recommended 2GB for 512MB).

Serverpilot (that you're using) enables 512MB only: https://serverpilot.io/community/articles/swap-is-enabled.html

So I recommend you to log as root and increase it. 

https://www.digitalocean.com/community/questions/how-to-change-swap-size-on-ubuntu-14-04

And look here too: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-16-04

Also good to know! not encountered swapfile before and its also to know.

 

I feel like that might be something else going on though. I'm running PW 2.7.2 on this site and I opened up permissions to 777 briefly on assets/cache/Page and still got the errors in the logs, e.g.:

  Error: Exception: DirectoryIterator::__construct(/srv/users/serverpilot/apps/betaville/public/site/assets/cache/Page/1/): failed to open dir: No such file or directory (in /srv/users/serverpilot/apps/betaville/public/wire/core/CacheFile.php line 208)

 

Link to comment
Share on other sites

  • 2 weeks later...

Added ProCache today which has helped alot on my server stats, as the CPU has gone from around 50-60% right down to 20%. however my RAM usage is still up the 80% which isnt great. Analytics are still at around 300-1500 hits a day. Will check out that site @Sérgio for sure.

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...