Unable To Save An Image Or File - PW Websites hosted on Dreamhost [Solved]

[cstevensjr]

Good Day,

I'm having an issue with a client's PW websites that are hosted on a shared hosting package on Dreamhost.  Everything works fine, with the exception that recently you cannot save an image or file on the websites.  Files (images or pdfs) will load and the spinner never stops spinning, however the file shows 100 percent loaded. I honestly believe this is strictly a hosting related issue.

Troubleshooting So Far

These client websites (3) have been upgraded to PHP 7.3, 7.4 and 8.0.2 and the problem persists.  PW versions tested are versions 165, 172, 173 and 174.  Previously never had any issues with PW version 165 running on PHP 7.4

I run my own websites on Dreamhost (on VPS) and don't have this issue.  My PHP versions on the multiple VPS are 7.3 and 7.4.  PW versions 165, 172, 173 and 174 work with no problems noted.  No current issues with saving an image or file on the VPS hosted websites.

I have additional clients that are on PHP 7.3 and running PW Version 165 and 174 with no image or file issues.  These client websites are on Dreamhost shared hosting.

Conclusion

It seems odd that I am having this issue with one client and not others on shared hosting. I also haven't seen any recent reports in the. PW Forum of issues anyone is having with saving images or files.

If anyone has any specific idea of what's going on, I would greatly appreciate your comments and suggestions before I bring this up to the web hosting service.  Thanks.

[rick]

15 minutes ago, cstevensjr said:

cannot save an image or file ... the spinner never stops spinning ... shows 100 percent loaded

Howdy @cstevensjr

This sounds familiar in that I seem to recall similar symptoms mentioned in the forum (but cannot locate it) which I believe turned out to be related to permissions. I think it was due to the host running a script that inadvertently changed the permissions. I'll post the topic if I can locate it. In the meantime, can you verify if set_time_limit has been disabled on that account?

[Macrura]

@cstevensjr,

this started on Wednesday of this week when Dreamhost made some bizarre unilateral ModSecurity setting that broke every ProcessWire site's ability to upload images or files.

You have to open a support ticket and tell them to change the ModSec setting for every domain on your account to allow the CMS to work. They should be able to check their logs and know what to change, but you also might have to go back and forth with them and test it. I exchanged probably 10 emails with them on Wednesday while they repeatedly tweaked the ruleset until i was able to upload again.

Last night i had to send them a list of ~20 sites that they had to adjust the ruleset for and they replied today that it is now done, but only time will tell if I start to receive complaints from site owners that they can't upload.

This is certainly disappointing behavior from Dreamhost, and they should make amends. Hopefully they are going to learn a hard lesson once they get an avalanche of support complaints about this.

By the way, you can easily tell what the problem is, if you upload while viewing the network panel. You'll see probably a 418 error and when you view the response you'll see Internal Server Error. 418 is the DH response for anything related to ModSec. In addition you can go into the server logs and open the log file (which may be pretty huge by now) and see the mod sec errors.

[cstevensjr]

2 hours ago, Macrura said:

@cstevensjr,

this started on Wednesday of this week when Dreamhost made some bizarre unilateral ModSecurity setting that broke every ProcessWire site's ability to upload images or files.

You have to open a support ticket and tell them to change the ModSec setting for every domain on your account to allow the CMS to work. They should be able to check their logs and know what to change, but you also might have to go back and forth with them and test it. I exchanged probably 10 emails with them on Wednesday while they repeatedly tweaked the ruleset until i was able to upload again.

Last night i had to send them a list of ~20 sites that they had to adjust the ruleset for and they replied today that it is now done, but only time will tell if I start to receive complaints from site owners that they can't upload.

This is certainly disappointing behavior from Dreamhost, and they should make amends. Hopefully they are going to learn a hard lesson once they get an avalanche of support complaints about this.

By the way, you can easily tell what the problem is, if you upload while viewing the network panel. You'll see probably a 418 error and when you view the response you'll see Internal Server Error. 418 is the DH response for anything related to ModSec. In addition you can go into the server logs and open the log file (which may be pretty huge by now) and see the mod sec errors.

Thanks!

[cstevensjr]

@Macrura,

Thanks again for your detailed and to-the-point report on what happened at Dreamhost.  I was able to get Dreamhost to adjust the ModSec ruleset and things are working again.

[Hani]

A bunch of my old clients started having this issue on Dreamhost as well.  Like @cstevensjr, I noticed that any domains on a VPS weren't affected. However, some domains on shared hosting plans were affected while other ones were not.  Probably a difference between servers, but not certain why shared servers would have different ModSec settings.

Anyway, I was lucky to chat with a tech that was patient and knew his stuff.  (Shout out to John B!)  We worked through one of the affected domains and he figured out what ModSec rules were being tripped by the uploads. So he had to add exceptions to those rules until the uploads worked - even with the Extra Web Security option still enabled for the domain.  After we got one domain working, he replicated the same exceptions to the other domains and we tested each one as we went.

He kindly shared the list of rules that were causing the problem after I asked for them (in case this issue popped up again or if I had to speak to another agent about another domain).

The rules being tripped were:

• application-multi
• language-multi
• platform-multi
• event-correlation
• attack-generic

If you have to talk to a Dreamhost tech to get this problem resolved, it may be helpful to point them to this post or simply pass them the list of rules being tripped that need exceptions added.  Like I mentioned earlier, and unfortunately, these rule exceptions need to be added on a domain-by-domain basis.

[Hani]

Just adding another note to myself and anyone else that runs into this problem again.  Had a new domain on Dreamhost with the same problem and the agent found two additonal rules to whitelist.  He gave me the details in case it happens again:

SecRuleRemoveById 921150
SecRuleRemoveById 920270

 

[BrendonKoz]

Updating this for further future reference for Dreamhost customers. Their support team mentioned that they had added a specific rule to their custom ModSecurity handling for the /processwire/ admin URL path. If you change the administrative path for security-by-obscurity, then their ModSecurity rules will kick in and all of the things that used to get blocked, get blocked again. You'd need to contact their support and have the settings applied per-site, per custom URI path.

Quote

Thanks for your response! I found out why you're running into all of
these issues where you may not have previously. Our ModSecurity rules
have a specific whitelist entry for "/processwire/page/edit", but
changing the URL invalidated that.

Unfortunately, I can't apply the same whitelist to your new URL given how
domain-based exceptions are structured, so we'll just have to add
exceptions for individual rules to the domain as we see them.

Their support team is great in working with their customers, at least. ?

[FireWire]

I posted about this separately (missed this one) and this has become a recurring issue with DH. The following are some things (some mentioned above) I've confirmed through experience and information from DH support-

• Any new PW website uploaded requires a ticket into support to disable specific ModSec rules
• These rules must be modified for every domain separately
• When changes are made to servers, those ModSec rule exceptions for all sites will be wiped, requiring another support ticket to DH
• If the changes DH makes to a server are not disruptive (not requiring a restart) that could affect these rules, you won't be notified
• The only option for managing this yourself is to disable ModSec entirely in the admin panel, per-site
• Granular ModSec configurations are blocked, i.e. .htaccess cannot be used to modify rules (which is odd given that the alternative is turning it off entirely)
• The ModSec rules in place could be configured better but aren't- I have more complex applications on the same server that do not trigger violations
• Violations can still occur without triggering a 418 response or any error log entries, which can (has) lead to a lot of misspent time
• In lieu of a 418, access.log may show a request with a 200 response, but dev tools in Chrome show a connection error, in Firefox the request just hangs. No response/HTTP status/error log entry

I spent time with support a couple of weeks ago and the tech disabled the ModSec rules that were showing up in error.log. It looked like it solved the issue, but a week later a client of mine that contacted me with this issue said it was still broken. The server was rejecting uploads over ~120kb and not logging any errors. This time support just disabled ModSec entirely- we didn't even know where to start without logged errors. You might want to check your upload file sizes during this process.

I've been with DH almost 10 years and this didn't used to happen (maybe started when pushing DreamPress? just a guess). Being unable to handle this myself even though I'm on a private VPS (albeit managed) is frustrating. I'm confident in PW security, and I have a significant amount of .htaccess rules to block suspicious activity, but shutting ModSec off entirely isn't cool- especially when their admin, documentation, and support team (correctly) recommend strongly that you shouldn't.

At this point I'm considering spinning up a VPS at DigitalOcean. I only used managed hosting because it was supposed to "just work", but I've had better experiences with well-configured unmanaged VPS', which have other perks to boot.

C'est la vie.

 

[Macrura]

I turn off that "extra web security" or whatever they call it, otherwise it's just a time suck.