Jump to content
suntrop

Crashed mysql tables

Recommended Posts

Hi all

One of my sites (running PW since 2014, currently  v. 2.5.3) had problems with crashed tables.

Last week the body field and and image field crashed – noteworthy the latter crashed in a separate DB in my working copy. A couple hours ago another field table crashed (production system again). Funnily, today's table fixed itself. The site has nearly 20,000 pages, about 100 users a day and is installed on a managed server.

Unless the logs in sites/assets/logs/errors.txt I have no idea how to research the problem. The hosting company says 'system is fine' and 'mostly it's cased by the CMS'. I have seen others here having that problem, but unless fixing it immediately with repair table there wasn't much info to see what is wrong.

Perhaps you guys have some tips how to debug that kind of problem. I even don't know if it is possible to fix this on my own … if it's a problem with the filesystem for example. Any ideas? My code or system failure?

Share this post


Link to post
Share on other sites

Hey, 

Sorry to hear. Most hosting companies will tell you that the system is running fine (developers: it's the system. operations: it's the application). 

  1. Have you looked into the server logs by apache or mysql?
  2. Ask the hosting provider how they know it is being caused by ProcessWire.
  3. Is there enough ram?
  • Like 2

Share this post


Link to post
Share on other sites
1 hour ago, arjen said:

Most hosting companies will tell you that the system is running fine

I've read this several times :-) 

There should be enough memory (server has 16 GB RAM) and the monitoring doesn't indicate any problems the times I the tables crashed. My provider didn't blame the CMS directly, but support said this is often caused by sleeping queries and not properly closed connections (if I remember correctly).

Unfortunately  I only have access to the latest 500 log entries of Apache and no access to the MySQL log. I'll ask the provider to send me both.

Share this post


Link to post
Share on other sites

Sorry, couldn't find the time for that. Had some trouble elsewhere. But I saw today there was an error again. I think it won't be long till I see the next one and can have a look the logs.

Will keep updating here!

Share this post


Link to post
Share on other sites

Just to complete this …

My provider told me "there is no error on our side" always. About 5 weeks ago the provider moved to a new location and all my data were transferred (don't know exactly how) and for the last couple weeks I had no problems :-)

  • Like 1

Share this post


Link to post
Share on other sites
Quote

there is no error on our side

As always :)

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By jds43
      Hello,
      Does anyone have experience with migrating content from Django to Processwire? Or are there any suggestions for achieving this?
    • By Brawlz
      Hi,
      I hope this is the correct section for my problem.
      All I need is a connection to an external Database and a query gettings some data. I do this in a processwire Page-Template. I am honestly not sure if it is a problem with processwire or my code:
      $host = ‚XXXXX’; $user = ‚XXXXX‘; $pass = ‚XXXXX‘; $db = ‚XXXXX‘; $port = ‚3306‘; $mydb = new Database($host, $user, $pass, $db , $port);  $result = $mydb->query("SELECT * FROM char“);  while($row = $result->fetch_assoc()) {  print_r($row);  }  
      Produces the following error:
      Error: Exception: DB connect error 2002 - Connection timed out (in /customers/9/4/e/XXXX.de/httpd.www/wire/core/Database.php line 79)
       
      I also tried connecting without the $port variable but got the same error.
    • By Mobiletrooper
      Hey Ryan, hey friends,
      we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based).
      So this is not your typical ProcessWire installation in terms of size.
      Currently we count:
      140 Templates (Some have 1 page, some have >6000 pages)
      313 Fields
      ~ 15k Users (For an intranet portal? That's heavy.)
      ~ 195 431 Pages (At least that's the current AUTOINCREMENT)
       
      I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like:
      We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure.
      We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec.
      In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc.
      We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above.
      ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily. 
      That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long.
      Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences.
      We also have about 120 people working on the processwire backend the whole day concurrently.
       
      What do you guys think?
      Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would:
       
      - Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time.
      - Do you think at this point it makes sense to use ProcessWire as a simple REST API?
      - In your experience, what fieldtypes are expensive? Page? RepeaterMatrix?
      - Ryan, what do you consider as the primary bottleneck of processwire?
      - Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible?
      - Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors?
       
      Thanks and lots of wishes,
      Pascal from Mobile Trooper
       
       
    • By Sergio
      All of a sudden, with nothing changed on the database or server, a website was getting error when doing a search:
      Error: Exception: SQLSTATE[HY000]: General error: 23 Out of resources when opening file './your-database-name/pages_parents.MYD' (Errcode: 24 - Too many open files) (in /home/forge/example.com/public/wire/core/PageFinder.php line 413) #0 /home/forge/example.com/public/wire/core/Wire.php(386): ProcessWire\PageFinder->___find(Object(ProcessWire\Selectors), Array) #1 /home/forge/example.com/public/wire/core/WireHooks.php(723): ProcessWire\Wire->_callMethod('___find', Array) #2 /home/forge/example.com/public/wire/core/Wire.php(442): ProcessWire\WireHooks->runHooks(Object(ProcessWire\PageFinder), 'find', Array) #3 /home/forge/example.com/public/wire/core/PagesLoader.php(248): ProcessWire\Wire->__call('find', Array) #4 /home/forge/example.com/public/wire/core/Pages.php(232): ProcessWire\PagesLoader->find('title~=EAP, lim...', Array) #5 /home/forge/example.com/public/wire/core/Wire.php(383): ProcessWire\Pages->___find('title~=EAP, lim...') #6 /home/forge/example.com/public/wire This error message was shown because: you are logged in as a Superuser. Error has been logged.  
      I tried several things, listed in this thread: https://serverfault.com/questions/791729/ubuntu-16-04-server-mysql-open-file-limit-wont-go-higher-than-65536
      But for some reason, MySQL was not getting its limit increased, but in the end, the one that did the trick was this:
      This worked for me on Ubuntu Xenial 16.04:
      Create the dir /etc/systemd/system/mysql.service.d
      Put in /etc/systemd/system/mysql.service.d/override.conf:
      [Service] LimitNOFILE=1024000 Now execute
      systemctl daemon-reload systemctl restart mysql.service Yes indeed, LimitNOFILE=infinity actually seems to set it to 65536.
      You can validate the above after starting MySQL by doing:
      cat /proc/$(pgrep mysql)/limits | grep files
    • By Jarden Black
      Hi everyone,
      [edit: do not loose your time reading this post, I solved it by disabling cache in the Pages2Pdf module... sorry 😓]
       
      I do not know if I should post on the Pages2Pdf thread or here. Mods, feel free to move the post.
       
      Since three days I am scratching my head  to understand a weird thing happening with $session and $config->debug used in conjunction with Pages2Pdf module. For information, I tested it on a fresh install of ProcessWire 3.0.96 with PHP-7.0.28 and Pages2Pdf-1.1.7 (domain: http://session.sites.sek/). I will try to explain what is going on.
       
      What I am trying to achieve :
      In a template, I need to set some sessions variables which are then echo'd in the PDF document.
      (on the test installation, the basic-page template (page /about/?pages2pdf) serve the PDF, the home and sitemap template set the session variable.)
       
      The problem :
      From the template sitemap, I set a variable: $session->setFor('pdf', 'myvar', 'Session set from Sitemap template');
      From the template home, I set a variable: $session->setFor('pdf', 'myvar', 'Session set from Home template');
      Then in the PDF default template, I echo the session variable: <p>Session: <?= $session->getFor('pdf', 'myvar'); ?></p>
       
      Now I turn ON debug mode ($config->debug = true) :
      Then I navigate to  "http://session.sites.sek/home/" and the session variable "myvar" is set to "Session set from Home template". Then I navigate to  "http://session.sites.sek/sitemap/" and the session variable "myvar" is set to "Session set from Sitemap template". Now I want my PDF document, so I navigate to "http://session.sites.sek/about/?pages2pdf=1" and I get my PDF document with the right session var : "Session set from Sitemap template" For the moment, nothing special happen. Everything work great. We are in debug mode.
       
      Now I turn OFF debug mode ($config->debug = false) :
      Then I navigate to  "http://session.sites.sek/home/" and the session variable "myvar" is set to "Session set from Home template". Then I navigate to  "http://session.sites.sek/sitemap/" and the session variable "myvar" is set to "Session set from Sitemap template". Then I navigate back to "http://session.sites.sek/home/" and the session variable "myvar" is set back to "Session set from Home template". Now I want my PDF document - as expected, the "myvar" should be set to "Session set from Home template" - so I navigate to "http://session.sites.sek/about/?pages2pdf=1" and here the problem happen. Instead of echoing  "Session set from Home template" in the PDF document, the phrase "Session set from sitemap" is echo'd (the last value recorded before switching from debug ON).  
       
      I made two small screencasts to show the issue :
      DEBUG ON (Everything is OK)
       
      DEBUG OFF
       
       
      I am missing something ? EDIT: YES, you are dumb!
       
       
×
×
  • Create New...