User Activity module AJAX forbidden due to updated apache

[adrian]

Hi everyone,

I don't have access to the support board for the User Activity module anymore, so am asking here if anyone else has started getting 403 forbidden errors with the the AJAX requests to URLs like: https://mysite.com/admin/?activity=PageEdit%40page%2Fedit%2F%3Fid%3D1032&started=1721087147&title=Edit Page%3A News • mysite.com&idle=0&hidden=0&changed=0&changes=&pwpdt=pwua

If I load it directly in the browser I also get the 403, but if I unencode the URL to be something like this, then it loads the raw html of the page.

https://mysite.com/admin/?activity=PageEdit@page/edit/?id=1032&started=1721087147&title=Edit page&idle=0&hidden=0&changed=0&changes=&pwpdt=pwua

I am seeing it on multiple servers, on different versions of PW, in different browsers, different sites, etc. I feel like I must be missing something obvious, but wondering if anyone else has seen similar issues start up recently?

[wbmnfktr]

32 minutes ago, adrian said:

User Activity module

What Pro module is this or where do I would get it?

[adrian]

@wbmnfktr - it's this one: https://processwire.com/store/pro-dev-tools/user-activity/

[wbmnfktr]

Oh, no... sorry. Don't have that. ?

[adrian]

Ok, I have got things working again by removing the encodeURIComponent() from '?activity=' + encodeURIComponent(activity) +

from UserActivityAdmin.js and I've pinged @ryan about it in a DM but at this stage, I'm really not sure why this became a problem all of a sudden.

 

[Robin S]

I haven't experienced the issue (I get normal 200 responses for User Activity requests) but it sounds like some sort of pre-emptive security blocking. If you're running WireRequestBlocker see my topic here because the default rules are quite broad.

Quote

It's up to each user to decide, but for me the sequence "/%" (which will match a slash followed by any URL-encoded character) is way too broad to be blocking users for.

Or maybe your host has introduced some security software without notifying you? I had an irritating situation like this with Cloudways recently (thankfully resolved now).

[adrian]

Hey @Robin S - thanks for your thoughts, but it's happening on VPS servers (all under my control) and also on my local dev machine and it's happening on sites on Digital Ocean and AWS so it's not that.

I had a similar issue recently with 7g firewall which I was playing with on one site, but I had already removed that and it was never on the other sites/servers anyway.

Oh, and I am not running WireRequestBlocker either.

I did recently update apache on all my servers, but I even tested on one with an older version and it was an issue there too.

I am seeing these errors in my logs:

[rewrite:error] [pid 34046:tid 34126] [remote xxx.xxx.xxx.xxx] AH10508: Unsafe URL with %3f URL rewritten without UnsafeAllow3F, referer: /admin/page/edit/?id=23917

Not certain it's the issue, but my hunch is that it is because of the %3F in the URL I posted in the first post above. But what has changed to make it a 403 issue now?

[Robin S]

6 minutes ago, adrian said:

I did recently update apache on all my servers, but I even tested on one with an older version and it was an issue there too.

Weird that you saw it on the old version, because like you say it's probably related to the changes in Apache 2.4.60

https://httpd.apache.org/security/vulnerabilities_24.html
https://www.cve.org/CVERecord?id=CVE-2024-38474

[adrian]

I have updated all but one server to 2.4.61, but yes, I am still seeing the issue on 2.4.41 (including that UnsafeAllow error) so not sure what's up with that :)

Figured this out - see my post below:

[adrian]

Turns out I can reproduce a forbidden error by simply loading a URL like http://pwtest.test/about/?id=%3f

It seems like you need to view a page that isn't the root url (hence the about child page) and that you have %3f in there. On some servers http://pwtest.test/about/%3f this also returns a 403 but on others I need the %3f to be part of a URL parameter.

This Drupal ticket reports the same issue and references the same apache change that Robin noted: https://www.drupal.org/project/drupal/issues/3460799 

If I make a similar change to PW's htaccess rule (as was changed in Drupal), ie:

change: 
RewriteRule ^(.*)$ index.php?it=$1 [L,QSA]

to: 
RewriteRule ^(.*)$ index.php [L,QSA]

or:
RewriteRule ^ index.php [L,QSA]

then everything works again. I expected that would break PW's page routing and/or admin, but initially it looks to be OK which honestly seems weird.

So other than the issue with UserActivity, we'll all need to be careful if we ever url_encode anything with a question mark (%3F) in it unless @ryan thinks an htaccess change like (or similar) that would be ok. Perhaps it will need some reworking of some of the $GET['it'] related code in PagesRequest.php to handle all scenarios?

Also, I just discovered that the server with what I thought was an old version of apache, was actually not old - I almost exclusively run Debian servers, but the "old one" I mentioned is Ubuntu and someone else is managing it now and they did update to 2.4.41 which in Ubuntu world includes that %3f fix: https://launchpad.net/ubuntu/+source/apache2/2.4.41-4ubuntu3.19

[Robin S]

@adrian you probably saw already that there's a flag you can add to the rewrite rule in htaccess to allow the encoded question mark: https://httpd.apache.org/docs/current/rewrite/flags.html#flag_unsafe_allow_3f
But who wants to reopen a fixed security hole?

It seems like this Apache change will affect a lot of websites as the built-in URL encoding functions in JS and PHP don't make any special exception for the question mark character, and it's surely not that rare to pass an encoded URL as a GET parameter. So it seems like the options are to specifically replace the encoded question mark with ? after getting the return value from a URL encoding function, or to use a completely different encoding like Base64 if all or part of a URL needs to be used in a GET parameter.

Edit: I've created a topic in the ProDevTools support forum to draw Ryan's attention to this discussion. Oh, you already PMd him so that probably wasn't necessary.

[adrian]

Thanks, I did see that flag, but as you said it's not a good approach for a fix. I agree this could be a potential issue on lots of sites.

Thanks also for posting to get Ryan's attention - the more pings he has on this the better. I tend to think that the correct fix has to be related to that "?it=$1" rewrite rule because it has the potential to break many more things than just the UserActivity module. 

[Jan Romero]

Omg I just got these errors last night, unrelated to the module. Guess my hoster updated Apache. Thank you for this thread ? 

I don’t understand the security issue and can’t seem to find many details on it, but must be pretty bad for them to break such a widespread use case…

[Jan Romero]

17 hours ago, adrian said:

or:
RewriteRule ^ index.php [L,QSA]

then everything works again. I expected that would break PW's page routing and/or admin, but initially it looks to be OK which honestly seems weird.

This seems to work because PW contains this “abnormal request” fallback, where it looks at $_SERVER['REQUEST_URI'] directly if the ”it“ query parameter is missing: https://github.com/processwire/processwire/blob/e508cfa2a74971490d6045bd5eb6f2a198d37037/wire/core/PagesRequest.php#L537

It involves some additional processing, removing the query string and PW’s subdirectory.

[adrian]

Good catch @Jan Romero - I wonder if PW could rely solely on parsing $_SERVER['REQUEST_URI'] ?

It seems like there might be an issue with NGINX if that happens: https://www.drupal.org/project/drupal/issues/2490870 but could be worked around with: https://api.drupal.org/api/drupal/includes!bootstrap.inc/function/request_uri/7.x

"Substitution encoding issue in mod_rewrite in Apache HTTP Server 2.4.59 and earlier allows attacker to execute scripts in directories permitted by the configuration but not directly reachable by any URL or source disclosure of scripts meant to only to be executed as CGI."

[adrian]

Just a follow up here for those who haven't found the Github issue: https://github.com/processwire/processwire-issues/issues/1952

It looks like the next version of Apache (2.4.63) will include a fix (https://github.com/apache/httpd/commit/a1a93beb58b81f1de2b713ae5f96c41ed5952a74) that should hopefully prevent this from being an issue for PW.

[Jonathan Lahijani]

At the time of this writing:

• Ubuntu 22.04 is at Apache 2.4.52
• Ubuntu 24.04 is at Apache 2.4.58

Both are experiencing the issue where going to example.com/some-folder/?foo=%3Fbar causes a 403, unless I modify the rewrite rule to be this updated version discussed above:
RewriteRule ^(.*)$ index.php [L,QSA]

@adrian You said the issue would be resolved when 2.4.63 is available.  So is the best plan to wait for that version to be released in apt package manager, update to it, then revert the rewrite rule back to what it originally was?

[adrian]

@Jonathan Lahijani - that's my plan for now. I don't know for certain that 2.4.63 will fix this issue, but from what I understand it sounds like it will.

That said, it still hasn't even been released on their Github repo yet: https://github.com/apache/httpd/tags so not sure how long it will be before it makes its way to APT etc, especially for Debian if it's not considered a security update.