BillH

Sadly, I spoke (or wrote) too soon! The problem continues, but is highly intermittent: sometimes people can log in, and sometimes they can't. As before, when the systems freezes (or whatever it's doing), eventually a server error appears, and it's necessary to reset things in the browser to get any access to pages in the site at all. It seems after all that it's (probably) nothing to do with +FollowSymLinks, which is set in the vhosts file anyway. It may well have been chance that the problem went away (aparently for over a day) when htaccess was changed. Support at the server company looked through their logs, and it seems that the 500 errors are caused by timeouts on the server, but the cause of these remains a mystery. They sent me this: [Tue Oct 15 12:43:01.590548 2019] [proxy_fcgi:error] [pid 14565] [client 127.0.0.1:57438] AH01068: Got bogus version 34, referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 12:43:01.590608 2019] [proxy_fcgi:error] [pid 14565] (22)Invalid argument: [client 127.0.0.1:57438] AH01075: Error dispatching request to : (passing brigade to output filters), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 12:43:02.804616 2019] [proxy_fcgi:error] [pid 14651] [client 127.0.0.1:57461] AH01068: Got bogus version 97 [Tue Oct 15 12:43:02.804649 2019] [proxy_fcgi:error] [pid 14651] (22)Invalid argument: [client 127.0.0.1:57461] AH01075: Error dispatching request to : (passing brigade to output filters) [Tue Oct 15 15:15:09.753656 2019] [proxy_fcgi:error] [pid 18607] (70007)The timeout specified has expired: [client 127.0.0.1:58757] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:15:39.741697 2019] [proxy_fcgi:error] [pid 18611] (70007)The timeout specified has expired: [client 127.0.0.1:58763] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/access/users/edit/?id=1027&s=1&c=1 [Tue Oct 15 15:16:03.569705 2019] [proxy_fcgi:error] [pid 18635] (70007)The timeout specified has expired: [client 127.0.0.1:58771] AH01075: Error dispatching request to : (polling) [Tue Oct 15 15:17:26.509702 2019] [proxy_fcgi:error] [pid 18684] (70007)The timeout specified has expired: [client 127.0.0.1:58785] AH01075: Error dispatching request to : (polling) [Tue Oct 15 15:18:00.381772 2019] [proxy_fcgi:error] [pid 18704] (70007)The timeout specified has expired: [client 127.0.0.1:58807] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:22:03.217694 2019] [proxy_fcgi:error] [pid 18758] (70007)The timeout specified has expired: [client 127.0.0.1:58835] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:31:09.990353 2019] [proxy_fcgi:error] [pid 2405] (70007)The timeout specified has expired: [client 127.0.0.1:49876] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:33:32.793776 2019] [proxy_fcgi:error] [pid 2495] (70007)The timeout specified has expired: [client 127.0.0.1:49902] AH01075: Error dispatching request to : (polling) [Tue Oct 15 15:37:36.761771 2019] [proxy_fcgi:error] [pid 2593] (70007)The timeout specified has expired: [client 127.0.0.1:49939] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:44:45.069830 2019] [proxy_fcgi:error] [pid 2859] (70007)The timeout specified has expired: [client 127.0.0.1:49982] AH01075: Error dispatching request to : (polling) [Tue Oct 15 15:51:48.169792 2019] [proxy_fcgi:error] [pid 2969] (70007)The timeout specified has expired: [client 127.0.0.1:50024] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ [Tue Oct 15 15:52:43.337813 2019] [proxy_fcgi:error] [pid 3013] (70007)The timeout specified has expired: [client 127.0.0.1:50041] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ At the moment I can think of only three things changing on the server that might account for the intermittent nature of the problem, though it could be nothing to do with these: Activity by the current user (though the freeze happens even if the user goes straight to log in). Other users logging in, and their activity in the system. Events in two other PW applications on the same server (separate installations using different databases). It would be nice to avoid having to rebuild the installation on a new server (it's large and complex), and not knowing the cause of the problem, there'd be no guarantee that it'd work. Any ideas or suggestions gratefully received!

THE SOLUTION IN THIS POST TURNS OUT NOT TO BE CORRECT - SEE THE NEXT POST I finally found a solution. I realised that, after a very long wait, apparently frozen browser windows were, at least sometimes, returning internal server errors. So I looked at the suggestions for error 500 in the PW readme and in the .htaccess file. The one that fixed it is in item 1 of .htaccess: uncomment Options +SymLinksifOwnerMatch and comment Options +FollowSymLinks. There are various things I'm not clear about, such as why this issue seemed to caused problems with database connections on log in, why it hasn't affected two other PW sites on the same cloud server (presumably there is some difference, such as this site being the Apache default), and so on, but the solution might be useful to others.

Many thanks for the replies. @Rudy The credentials do very occasionally work, so I think they must be correct. @dragan That seems like a good analysis. I was beginning to think it might be something to do with the server set up rather than on the PW end of things. I'll get on to hosting support and will report back if this fixes it.

I have moved a site to a new server and can't access the admin. Accessing the front end of the site seems to be working as normal. And I can get to the log-in page for the admin. But then when I enter username and password, everything freezes while waiting for the next page to load. Then, after this has happened, I can no longer access any page on the site (including the front end). If I restart the browser, however, we go back to square one - that is, I can access the site until I try to log in, when everything freezes again. There seems to be some instability in this behaviour - occasionally I or others can log in, for example - but mostly it is as just described. The error logs seem to indicate it's something to do with accessing MySQL (see below), but the fact that re-starting the browser seems to re-set things makes me think that perhaps it's primarily an issue with sessions or something like that. From the PW errors log: 2019-10-09 15:59:13 ? http://phpstack-273028-1014063.cloudwaysapps.com/ Error: Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382) 2019-10-09 15:59:13 ? http://phpstack-273028-1014063.cloudwaysapps.com/admin/ Error: Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382) 2019-10-09 15:59:15 ? http://phpstack-273028-1014063.cloudwaysapps.com/admin/ Error: Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382) And the entries from the corresponding time in the PW exceptions log: 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/ SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/ SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1 DB connect error 2002 - Connection refused (in /wire/core/Database.php line 79) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1 DB connect error 2002 - Connection refused (in /wire/core/Database.php line 79) 2019-10-09 15:59:10 ? ? SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/about/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1 SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1 SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423) 2019-10-09 15:59:10 ? ? SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 ? ? SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/about/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 ? ? SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:10 admin http://phpstack-273028-1014063.cloudwaysapps.com/ SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:13 ? ? Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:13 ? ? SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382) 2019-10-09 15:59:13 ? ? Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:13 ? ? SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382) 2019-10-09 15:59:15 ? ? Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217) 2019-10-09 15:59:15 ? ? SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382) It might also be worth noting that I have another PW installation running on the same cloud server (using a different MySQL database) in AFAIK the exactly the same environment with no problems at all. I am entirely stumped on where to go next, and I haven't managed to find any post in the forums regarding exactly this issue. Does anyone have any ideas?

I've come up with some seriously fast ways of doing this – following the suggestion from @Zeka and taking inspiration from the code in RockFinder. I'm posting them here in case anyone finds them useful. The RockFinder approach in my previous post takes about 0.95 seconds (across about 5,000 pages with about 1,500 associated images, resulting in about 30 pages in the final array), which is fine. But the following, using direct SQL access, produces exactly the same result in around 0.0055 seconds (much of which is making the final page array, so if there were more pages it would be somewhat slower). startDate = strtotime("-2 months"); // Run query on database $startDateString = date('Y-m-d H:i:s', $startDate); $queryString = "SELECT pages_id FROM field_article_images WHERE created >= '{$startDateString}'"; $queryResult = $this->db->query($queryString); // Get array of unique page IDs $pageIdArray = $queryResult->fetch_all(MYSQLI_NUM); $pageIdArray = array_unique($pageIdArray, SORT_REGULAR); // Make PW page array $pagesWithRecentImages = new PageArray(); foreach($pageIdArray as $pageIdSubArray) { $pwPage = $pages->get($pageIdSubArray[0]); $pagesWithRecentImages->add($pwPage); } And if an array of image data from the database table would be sufficient, the following is even faster, at around 0.0014 seconds. $startDate = strtotime("-2 months"); // Run query on database $startDateString = date('Y-m-d H:i:s', $startDate); $queryString = "SELECT * FROM field_article_images WHERE created >= '{$startDateString}'"; $queryResult = $this->db->query($queryString); // Make array of image data $imageData = $queryResult->fetch_all(MYSQLI_ASSOC); (Adding "ORDER BY created DESC" to the SQL query above could be a useful addition.) It's worth noting that the SQL queries above are quite straightforward because they are on a single table, and the image field in question appears only on relevant pages. For more complex situations, using RockFinder would be a great deal easier, and I'd guess the speed difference would be less.

After some experimentation, here's what I'm now using to get the relevant pages: $startDate = strtotime("-2 months"); // Use RockFinder to get recently saved pages with images $recentPagesWithImagesSelector = "has_parent=/articles/,article_images.count>0,modified>{$startDate},include=unpublished"; $finderRPWI = new RockFinder($recentPagesWithImagesSelector,['title']); $recentPagesWithImages = $finderRPWI->getPages(); // Use standard PW find to reduce to pages with at least one recent image $pagesWithRecentImages = $recentPagesWithImages->find("article_images.created>{$startDate}"); No doubt this could be improved (e.g. by maintaining an index of recent images), but it reduces the page-loading time to something acceptable. Note that I tried a one-step process using RockFinder only with "article_images.created>{$startDate}" in the selector, but this produced a few unexpected results, and anyway was slower than the above. I didn't investigate what was happening in any depth. I have also used RockFinder in another place on the page where a search was running slowly, and the page is now loading more quickly than before I added the list of images!

Thanks @Zeka, that makes a big difference, reducing the time taken to less than half of what is was previously. And thanks also @Autofahrn, that sounds a really interesting approach to improving performance further. The system is already quite well established, so I probably can't switch to having each image on a page (or at least not easily), but some variation on your idea might work well. For example, hooking on image load and storing relevant data (such as the time of the most recent image load) in a field on the page or some similar approach might work. And @Zeka, the SQL and RockFinder ideas seem interesting too. I now need to close down for the day, but I'll look into all these further over the next day or so.

I'm using the following code to get an array of recently added images with various bits of information about them, sorted in reverse order of when they were added. $startDate = strtotime("-2 months"); $pagesWithRecentImages = $pages->find("article_images.created>{$startDate},include=unpublished"); $recentImages = array(); foreach($pagesWithRecentImages as $pwri) { foreach($pwri->article_images as $image) { if ($image->created >= $startDate) { $sortKey = $image->created; $recentImages[$sortKey] = array($image->page->title, $image->page->url, $image->basename, $image->created); } } } krsort($recentImages); It works fine, and I'm displaying the contents of the array on a dashboard page on the admin side. However, it runs rather slowly, delaying the loading of the dashboard page by a few seconds (around six or seven seconds when getting images from about 5,000 pages). The page already takes 3 or 4 seconds to load (it shows other stuff too), and a total of ten seconds or so is a bit frustrating for users. I've been trying to work out a faster way of getting an array of images (perhaps using different selector), but so far with no success. Can anyone think of a way to get it to run more quickly? Thanks!

The issue that I raised on May 8 has now gone away. It was very probably caused by an entirely separate problem I was having with permissions, and thus is nothing to do with this module! I posted about the permissions issue at https://processwire.com/talk/topic/19266-page-edit-permission-not-working/.

Strangely, the problem has gone away all by itself! Obviously this is a good thing for the project, but it means that I don't know the cause of the trouble and can't offer a solution. The only thing I can think of that I changed and that might have affected permissions in some way - though this is a total guess - is that I discovered an image field which, for no reason I'm aware of, had access control enabled (I may have accidentally clicked something or done something through the API). It's possible, though I can't think why, that disabling this access control triggered the resolution of the issue. If anyone else comes across this problem, it'd be interesting to know how they fix it.

Thanks, Robin S, these look like really useful suggestions. I've certainly tried suggestion 3 already (many times!), but I'll get on with 1, 2 and 4. I'll report back.

I'm still completely stuck on this and have tried everything I can think of - including moving the system to a new database (in the same way as I would if moving to a new server). Does anyone have any clues at all? The horrible prospect of many days rebuilding the system from scratch is looming, so any help gratefully received!

I feel as if I must be missing something really massively obvious, but the page-edit permission has stopped working, except for superusers. For non-superusers with page-edit permission, the Edit button is not appearing on page listings. And if I go directly to a page's URL (e.g. ...page/edit/?id=123) there's the error message "ProcessWire: You don't have access to edit". Note that Edit Pages access is given on the template pages, so that's not the cause of the problem. There was no problem until recently. Unfortunately, I'm not quite sure when this issue started, as I've been working on the site as a superuser, and noticed only when testing as a non-superuser. Thus it's difficult to work out what might have triggered the issue. I have tried turning permissions on and off, creating new roles and users, clearing caches, removing and adding Edit Pages access to templates, and turning off any modules that I suspect might interfere with permissions (though I'm not sure any actually would). Any ideas for what to try next?

@Robin S Thanks for the advice on debugging. I'd never noticed the $log stuff in PW before, and have been wondering whether the Tracey Debugger module might be a good idea. I've tried every bit of cache clearing I could think of, but perhaps there's something about CKEditor I missed I'll investigate further and report back.

The bug seems to have returned. It's the same as before: the module doesn't work (the menu shows "No files for this page") if the user is not logged in as superuser. I'm using the latest version of the the module, 0.1.5. (And I'm getting the same problem with my own variants of the module.) I thought the issue might be caused changing to PW 3.0.98, but reverting to 3.0.96 and 3.0.62 doesn't help. In the browser, the Console isn't returning any errors, and there's nothing I recognize as relevant under Network. Something must have changed, but can't think what. In fact, not much (that I can remember) has changed on the back end of the site at all in the last week or so. I'm a bit stumped on what to try next!