Can't access the admin because PW freezes

[BillH]

I have moved a site to a new server and can't access the admin.

Accessing the front end of the site seems to be working as normal. And I can get to the log-in page for the admin. But then when I enter username and password, everything freezes while waiting for the next page to load.

Then, after this has happened, I can no longer access any page on the site (including the front end).

If I restart the browser, however, we go back to square one - that is, I can access the site until I try to log in, when everything freezes again.

There seems to be some instability in this behaviour - occasionally I or others can log in, for example - but mostly it is as just described.

The error logs seem to indicate it's something to do with accessing MySQL (see below), but the fact that re-starting the browser seems to re-set things makes me think that perhaps it's primarily an issue with sessions or something like that.

From the PW errors log:

2019-10-09 15:59:13    ?    http://phpstack-273028-1014063.cloudwaysapps.com/    Error:     Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382)
2019-10-09 15:59:13    ?    http://phpstack-273028-1014063.cloudwaysapps.com/admin/    Error:     Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382)
2019-10-09 15:59:15    ?    http://phpstack-273028-1014063.cloudwaysapps.com/admin/    Error:     Exception: SQLSTATE[HY000] [2002] Connection refused (in /home/273028.cloudwaysapps.com/ebmtwcxxzm/public_html/wire/core/ProcessWire.php line 382)

And the entries from the corresponding time in the PW exceptions log:

2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/  SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/  SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1  DB connect error 2002 - Connection refused (in /wire/core/Database.php line 79)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1  DB connect error 2002 - Connection refused (in /wire/core/Database.php line 79)
2019-10-09 15:59:10    ?    ?    SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/about/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1  SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/admin/?login=1  SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /site/assets/cache/FileCompiler/site/modules/ProDrafts/ProDrafts.module line 423)
2019-10-09 15:59:10    ?    ?    SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    ?    ?    SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/about/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    ?    ?    SQLSTATE[HY000]: General error: 2006 MySQL server has gone away (in /wire/core/WireDatabasePDO.php line 480)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:10    admin   http://phpstack-273028-1014063.cloudwaysapps.com/  SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:13    ?    ?    Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:13    ?    ?    SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382)
2019-10-09 15:59:13    ?    ?    Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:13    ?    ?    SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382)
2019-10-09 15:59:15    ?    ?    Unable to load WireDatabasePDO - SQLSTATE[HY000] [2002] Connection refused (in /wire/core/WireDatabasePDO.php line 217)
2019-10-09 15:59:15    ?    ?    SQLSTATE[HY000] [2002] Connection refused (in /wire/core/ProcessWire.php line 382)

It might also be worth noting that I have another PW installation running on the same cloud server (using a different MySQL database) in AFAIK the exactly the same environment with no problems at all.

I am entirely stumped on where to go next, and I haven't managed to find any post in the forums regarding exactly this issue.

Does anyone have any ideas?

 

 

 

[dragan]

23 minutes ago, BillH said:

MySQL server has gone away

+

23 minutes ago, BillH said:

DB connect error 2002 - Connection refused

= shaky mySQL server / setup.

24 minutes ago, BillH said:

(using a different MySQL database)

There you have it. Maybe ask hosting support or switch to another DB if you can.

[Rudy]

@BillH

Your issue is with MySQL connection.

Exception: SQLSTATE[HY000] [2002] Connection refused

Check your database user/credentials to make sure that you can connect.

[BillH]

Many thanks for the replies.

@Rudy The credentials do very occasionally work, so I think they must be correct.

@dragan That seems like a good analysis. I was beginning to think it might be something to do with the server set up rather than on the PW end of things. I'll get on to hosting support and will report back if this fixes it.

[BillH]

THE SOLUTION IN THIS POST TURNS OUT NOT TO BE CORRECT - SEE THE NEXT POST

I finally found a solution.

I realised that, after a very long wait, apparently frozen browser windows were, at least sometimes, returning internal server errors.

So I looked at the suggestions for error 500 in the PW readme and in the .htaccess file.

The one that fixed it is in item 1 of .htaccess: uncomment Options +SymLinksifOwnerMatch and comment Options +FollowSymLinks.

There are various things I'm not clear about, such as why this issue seemed to caused problems with database connections on log in, why it hasn't affected two other PW sites on the same cloud server (presumably there is some difference, such as this site being the Apache default), and so on, but the solution might be useful to others.

[BillH]

Sadly, I spoke (or wrote) too soon!

The problem continues, but is highly intermittent: sometimes people can log in, and sometimes they can't.

As before, when the systems freezes (or whatever it's doing), eventually a server error appears, and it's necessary to reset things in the browser to get any access to pages in the site at all.

It seems after all that it's (probably) nothing to do with +FollowSymLinks, which is set in the vhosts file anyway. It may well have been chance that the problem went away (aparently for over a day) when htaccess was changed.

Support at the server company looked through their logs, and it seems that the 500 errors are caused by timeouts on the server, but the cause of these remains a mystery. They sent me this:

[Tue Oct 15 12:43:01.590548 2019] [proxy_fcgi:error] [pid 14565] [client 127.0.0.1:57438] AH01068: Got bogus version 34, referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 12:43:01.590608 2019] [proxy_fcgi:error] [pid 14565] (22)Invalid argument: [client 127.0.0.1:57438] AH01075: Error dispatching request to : (passing brigade to output filters), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 12:43:02.804616 2019] [proxy_fcgi:error] [pid 14651] [client 127.0.0.1:57461] AH01068: Got bogus version 97
[Tue Oct 15 12:43:02.804649 2019] [proxy_fcgi:error] [pid 14651] (22)Invalid argument: [client 127.0.0.1:57461] AH01075: Error dispatching request to : (passing brigade to output filters)
[Tue Oct 15 15:15:09.753656 2019] [proxy_fcgi:error] [pid 18607] (70007)The timeout specified has expired: [client 127.0.0.1:58757] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:15:39.741697 2019] [proxy_fcgi:error] [pid 18611] (70007)The timeout specified has expired: [client 127.0.0.1:58763] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/access/users/edit/?id=1027&s=1&c=1
[Tue Oct 15 15:16:03.569705 2019] [proxy_fcgi:error] [pid 18635] (70007)The timeout specified has expired: [client 127.0.0.1:58771] AH01075: Error dispatching request to : (polling)
[Tue Oct 15 15:17:26.509702 2019] [proxy_fcgi:error] [pid 18684] (70007)The timeout specified has expired: [client 127.0.0.1:58785] AH01075: Error dispatching request to : (polling)
[Tue Oct 15 15:18:00.381772 2019] [proxy_fcgi:error] [pid 18704] (70007)The timeout specified has expired: [client 127.0.0.1:58807] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:22:03.217694 2019] [proxy_fcgi:error] [pid 18758] (70007)The timeout specified has expired: [client 127.0.0.1:58835] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:31:09.990353 2019] [proxy_fcgi:error] [pid 2405] (70007)The timeout specified has expired: [client 127.0.0.1:49876] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:33:32.793776 2019] [proxy_fcgi:error] [pid 2495] (70007)The timeout specified has expired: [client 127.0.0.1:49902] AH01075: Error dispatching request to : (polling)
[Tue Oct 15 15:37:36.761771 2019] [proxy_fcgi:error] [pid 2593] (70007)The timeout specified has expired: [client 127.0.0.1:49939] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:44:45.069830 2019] [proxy_fcgi:error] [pid 2859] (70007)The timeout specified has expired: [client 127.0.0.1:49982] AH01075: Error dispatching request to : (polling)
[Tue Oct 15 15:51:48.169792 2019] [proxy_fcgi:error] [pid 2969] (70007)The timeout specified has expired: [client 127.0.0.1:50024] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/
[Tue Oct 15 15:52:43.337813 2019] [proxy_fcgi:error] [pid 3013] (70007)The timeout specified has expired: [client 127.0.0.1:50041] AH01075: Error dispatching request to : (polling), referer: http://phpstack-273028-1014063.cloudwaysapps.com/admin/ 

At the moment I can think of only three things changing on the server that might account for the intermittent nature of the problem, though it could be nothing to do with these:

1. Activity by the current user (though the freeze happens even if the user goes straight to log in).
2. Other users logging in, and their activity in the system.
3. Events in two other PW applications on the same server (separate installations using different databases).

It would be nice to avoid having to rebuild the installation on a new server (it's large and complex), and not knowing the cause of the problem, there'd be no guarantee that it'd work.

Any ideas or suggestions gratefully received!

 

[BillH]

Just to round this topic off, in the end I got everything working fine, and it has been running without problems for several days now.

I achieved this simply by moving the site to a new cloud server. I set up a clean PW install, and then copied across all files and database content.

Straightforward enough, but I've still no idea what caused the original problem!