Cloudflare and Plesk

[Peter Knight]

Just posting this in case anyone else is running a PW site with a Plesk and Cloudflare combo.

This one bit me for months and it's relatively difficult to catch. Our dashboards were green, consultants were telling us it was an SEO issue, advertising reps told us we simply needed to increase Ads budgets.

At the end of the day, the falling traffic and conversions was much more insidious. I've asked AI to summarise below.

Heads-up if you run a Plesk VPS behind Cloudflare: check that Fail2Ban isn't banning Cloudflare itself. 

The mechanism: with Cloudflare proxying, every request reaches your server from a Cloudflare edge IP, not the visitor's.

Plesk's default Fail2Ban jails (ModSecurity, Apache, recidive) count failures per source IP. So when some bot scans your site for exploits through Cloudflare, Fail2Ban sees a burst of bad requests from one "IP", which is really a Cloudflare edge serving thousands of innocent visitors, and bans it.

Every visitor routed through that edge then gets Cloudflare's "521 Web server is down" page, while the site works perfectly for everyone else.

On one server the recidive jail had escalated to 7-day bans, so roughly 1 in 20 requests failed for six weeks;

The server looked healthy from the inside the whole time: load near zero, almost no 5xx in its own logs, because the banned requests never reached nginx.

The fix is two Plesk settings, no code.
Note: triple check these before adding them at your own risk.

1. Tools & Settings → IP Address Banning (Fail2Ban) → Trusted IP Addresses: add all of Cloudflare's published IPv4 ranges (cloudflare.com/ips, 15 of them). Fail2Ban then ignores edge IPs entirely.
2. Per domain, Apache & nginx Settings → Additional nginx directives: a set_real_ip_from line for each of those ranges plus real_ip_header CF-Connecting-IP;. nginx then logs the real visitor IP, so if Fail2Ban does ban someone, it bans the actual attacker.

Quick test for whether it's happening to you:

fail2ban-client status recidive and look for anything in 104.x, 162.158.x, 172.64 to 172.71.x, 141.101.x, 108.162.x, 188.114.x or 198.41.x in the banned list. But short bans hide it, so also count historic bans: zgrep -h "Ban " /var/log/fail2ban.log* | grep -cE ' (104\.|162\.158|172\.(6[4-9]|7[01])|141\.101|108\.162|188\.114|198\.41)\.'.

Anything above zero means it's been happening.

You can also check within Cloudflare itself:

The fix is two Plesk settings, no code:

1. Tools & Settings → IP Address Banning (Fail2Ban) → Trusted IP Addresses: add all of Cloudflare's published IPv4 ranges (cloudflare.com/ips, 15 of them). Fail2Ban then ignores edge IPs entirely.
2. Per domain, Apache & nginx Settings → Additional nginx directives: a set_real_ip_from line for each of those ranges plus real_ip_header CF-Connecting-IP;. nginx then logs the real visitor IP, so if Fail2Ban does ban someone, it bans the actual attacker.

 

Cheers
Peter

 

[sleepycron]

Good catch. One thing worth adding: Cloudflare also publishes IPv6 ranges (cloudflare.com/ips-v6). If the server has an AAAA record, a good part of the edge traffic comes in over IPv6, so those ranges need to go into the trusted list and the set_real_ip_from lines too, otherwise the same thing can still happen. The lists do change now and then, so it's worth checking them once in a while.

[Peter Knight]

ThanksI'll keep an eye out for that. I've also set up an automated CloudFlare alert which I discovered. Ive a bot reading it nightly too and reporting in every morning which I hope will flag em further.