ProcessDiagnostics

[netcarver]

Those are a couple of good questions - with the new core upgrade module I've started setting the dirs on my development boxes as 777 just to allow easy updates as the dev branch gets commits but I won't be doing that on production machines.

I wrote the rule about making the site folder unwriteable by the webserver process in case someone managed to craft an attack against PW that tried to write to the file system; I'd prefer that such an injection/scripting attack - if found - would not be able to write to the disk. So that's definitely a me thing rather than an official PW rule. To try and anwser your question about how to set it - well, it depends on a few things...

• Going with 0550 means the owning user (the first 5) gets read and traversal rights, the next 5 means users in the owning group get read and traversal rights and the final 0 means other users get no rights. This is the most secure setting but requires that the user the webserver process runs as be either the owning user or a member of the owning group.
• Going with 0750 (owning user can write to the tree) would be better if the owning user were different from the webserver user and if the webserver user were a member of the owning group.
• Going with 0755 (owning user can write the tree and any user can read & traverse) would be the one to go for if the owning user were different from the webserver user and if the webserver user were not a member of the owning group. In this respect, this option is the most universally applicable but it allows anyone on the box the right to read and traverse into your file structure and, on a shared host this is a security risk as they can then possibly do things like read your config file - with your DB access credentials in it - if that has read permissions for other users too! If you are on a VPS with no shared accounts - then this setting is fine.

To find out the user and group ownership you'd do an "ls -l" from a linux command line and it should show you the owning user and group names.

The diagnostic module works by checking if the directory is writeable by the user the webserver process is running as; it doesn't try to analyse or recommend how to fix it (if I remember correctly.) 

[ceberlin]

Login_persist
InnoDB / utf8_general_ci Rows: 5
Data: 16.00 KB
Index: 0 bytes
Overhead: 295.00 MB Warning Optimize table to reclaim space and reorganize indexes.

Do you think that "Overhead 295 MB" makes sense here? Or is this a calculation error?

Clicking on the "optimize" link has no effect, by the way.

[netcarver]

I think the size is likely to be correct. Can you verify with a tool like phpMyAdmin?

Once I'm less busy I can take a look at why this isn't optimizing.

[ceberlin]

I investigated more. This time also with a production site which is live:

It DOES work RIGHT on the LIVE server, I just found out. Only on localhost (MAMP PRO) on my testing ground it does not optimize - which does not matter much.

So ... a false alarm, regarding the live site.

[ceberlin]

DiagnoseFilesystem 

Root directory Exists, is readable, is writable, 0750 Failure Folder should not be writable for webserver user

That reads like DRAMA.  Are you sure?

What I would like to check frequently is that the config.php is read only (400). Unfortunately this gets reset by GIT updates to 755 now and then and a warning would be great here. Maybe with a "fix it"-link? - That possible feature was one of the reasons for me to try Diagnostics in the first place. 

(Drupal7 is very cool about this - it resets the settings files to 400 automatically.)

htaccess file Exists, is readable, is writable, 0664 Warning .htaccess must exist, must be readable, should not be writable.

If I set htaccess to read only (400), I receive a 403 error when going to the procache settings at ADMIN.

[netcarver]

Some nice feedback there - thank you.

Am I right in thinking that ProCache is the only part of the official PW system that needs write access to .htaccess? If so, does it need that access on install or all the time? If it is only at install I'll change the wording on that warning to indicate that it's a general steady-state recommendation and that some components might need write-access when installing them.

[ceberlin]

Hi Netcaver,

I double checked with my MYSQL software NAVICAT. I can confirm now that the problem of false errors is there.

I found what those affected fields have in common:

The module reports totally wrong MysQL overhead warnings of abt. 300 MB (always the same value for any of the fields in question by the way) for any field that has InnoDB set as engine (instead of MyISAM). This is even for freshly installed fields.

In those cases also the "optimize" link has no effect.

And for those other fields: a "WARNING" is very dramatic info if shown even for fields hat have something like 32 bytes overhead.

Maybe there should be a threshold before a warning is fired. So WARNINGS are not used to excessively with the effect that that real important warnings are taken more seriously within the module. Do you agree?

[netcarver]

Thank you ceberlin. I've been in two minds about allowing optimisation of InnoDB tables for a while and this has pushed me off the fence. I'll remove the optimisation for that engine with the next release.

Regarding a configurable level for issuing optimisation "warnings" that should be entirely possible.

[SteveB]

Just FYI: Noticed this error when installing DiagnoseModules.module on 2.5.10dev today though it seems to be working just fine. 

DiagnoseModules::getModuleConfigInputfields() or DiagnoseModulesConfig->getInputfields() did not return the expected value (InputfieldWrapper)

MySQL Collation:

Also have a question about warnings regarding MySQL Collation. Does this account for the fact that you can set collation at the Table level as well as at the Database level? It looks to me as though it ignores the settings on the individual tables. Do a query like " show table status from my_database;" to see them. Often the database is set Latin-1 (mysql default) but the tables are UTF8 because the application (or person) intentionally managed that aspect.

[FrancisChung]

Hi there,

I'm trying to investigate a sluggish performance issue with my local Processwire Admin module.

I came across this forum post, https://processwire.com/talk/topic/8805-slow-admin-localhost/ .

One of the things they mentioned was to optimise DB Tables.

So I ran the System Diagnostics module, and the DB Diagnostics and it had the following message for a table.

module_sert_keywords
InnoDB / utf8_general_ci Rows: 0
Data: 16.00 KB
Index: 0 bytes
Overhead: 9.00 MB Warning Optimize table to reclaim space and reorganize indexes.

I tried clicking on the Optimize link but it doesn't seem to do anything, as running the diagnostics again seem to return the same message.

Is this a bug, intended or otherwise, or is this step supposed to be done manually?

Thanks in advance.

[benbyf]

Loving this suite of modules. I was wondering what exactly it does when you click the Optimize button in the Database module - for my own understanding and safety of the data integraty of the site.

[adrian]

1 minute ago, benbyf said:

Loving this suite of modules. I was wondering what exactly it does when you click the Optimize button in the Database module - for my own understanding and safety of the data integraty of the site.

I haven't checked, the code, but I would assume: http://dev.mysql.com/doc/refman/5.7/en/optimize-table.html

[Macrura]

I noticed that if a table is marked as 'crashed', the process module won't load, e.g. you get a "process returned no content", instead of the usual interface.

We have had some server issues with Site5 hosting that has caused tables to randomly crash and only way to fix is from phpMyAdmin, but if we could use ProcessDiagnostics to repair the table(s), that would save a step..

[Mike Rockett]

This module family appears to work fine on PW3 (sans Macura's issue above). @netcarver, please could you tag it in the modules directory to show that it does indeed work?

[Juergen]

I use it on the latest PW 3 DEV without any problems.

[netcarver]

5 hours ago, Mike Rockett said:

This module family appears to work fine on PW3 (sans Macura's issue above). @netcarver, please could you tag it in the modules directory to show that it does indeed work?

Done

[netcarver]

On 26/05/2017 at 0:37 PM, Macrura said:

I noticed that if a table is marked as 'crashed', the process module won't load, e.g. you get a "process returned no content", instead of the usual interface.

We have had some server issues with Site5 hosting that has caused tables to randomly crash and only way to fix is from phpMyAdmin, but if we could use ProcessDiagnostics to repair the table(s), that would save a step..

Thanks for the report @Macrura. If you get a situation with a crashed table again, please could you turn Tracy on and try to get a stack trace for me? (I'm not in the habit of purposely corrupting tables!) I wonder if there is a tool we can use to force/simulate a corrupted table?

[froot]

I'm having the same issue as mentioned above

Optimize table to reclaim space and reorganize indexes.

I don't know what is going on here actually. Is that a reason for concern? Does that have to do with the database configuration? Also, I almost exclusively see this warning next to custom fields that are created for a custom module of mine upon installation. So naturally I wonder if I misconfigured something inside that module? 

Thanks for help!

[froot]

I want to hook into the module and log the warnings and failures, is that doable? This is what I tried with no luck:
 

//ready.php

$this->addHookAfter('ProcessDiagnostics::___collectDiagnostics', function($event) {

// Get the instance of the ProcessDiagnostics module
$module = $event->object;

// Get the title, value, status, and notes from the hook event
$title = $event->argument('title');
$value = $event->argument('value');
$status = $event->argument('status');
$notes = $event->argument('notes');

// Log the information to the console
wire('log')->save('diagnostics', $title);
wire('log')->save('diagnostics', $value);
wire('log')->save('diagnostics', $status);
wire('log')->save('diagnostics', $notes);
});

 

 

 

[froot]

OK, trying to answer my own question, Below is my so far working solution, the code I put on ready.php  

things to note:
- it doesn't log and therefor doesn't  render error messages
- I tried to put that in a separate module, but the problem seems to be, that both hooks would call methods from two different classes, so for the first hook I would need to extend the Wire class and for the second the Process class. Two modules for this simple functionality seems a bit overkill and then it can only do this…
- Maybe a better idea would be to extend the Diagnostics-module directly? Which ever one makes sense. (ProcessDiagnostics, DiagnoseModules, …)
- in order to make this code work, you'd still need to make the method GetDiagnostics hookable, so add ___ to GetDiagnostics in DiagnoseModules (I know, it's a hack)

// ready.php

$this->addHookAfter('DiagnoseModules::GetDiagnostics', function ($event) {
    wire('log')->delete('diagnostics');
    $results = $event->return;

    // Loop through results and extract the entries that are warnings and failures and log them
    foreach ($results as $result) {
        if ($result['status'] != '') {
            wire('log')->save('diagnostics', $result['status'] . ': ' . $result['title'] . ' - ' . $result['value']);
        }
    }

    // Return the modified results
    $event->return = $results;
});


function formatDiagnosticMessage($line) {
    $lineParts = explode(':', $line);
    $severity = $lineParts[0];
    $message = $lineParts[1];
    $formattedMessage[0] = $severity;
    $formattedMessage[1] = $message;
    return $formattedMessage;
}

$this->addHookBefore('PageRender::renderPage', function(HookEvent $event) {
    // Get the object the event occurred on, if needed
    $PageRender = $event->object;

    // Get values of arguments sent to hook (and optionally modify them)
    $event = $event->arguments(0);
    $page = $event->page;

    $info = ' - check Diagnostics-module for further investigation.';

    if ($page->template != 'admin') return;
    $ProcessDiagnostics = wire('modules')->get('ProcessDiagnostics');

    if (!file_exists('site/assets/logs/diagnostics.txt')) {
        $this->message('No warnings or errors found' . $info);
        return;
    }

    $diagnosticsFile = fopen('site/assets/logs/diagnostics.txt', 'r');
    $diagnosticsLines = [];

    while (($line = fgets($diagnosticsFile)) !== false) {
        $diagnosticsLines[] = $line;
    }

    fclose($diagnosticsFile);

    foreach ($diagnosticsLines as $line) {
        $parts = explode(' ', $line);
        $formattedMessage = formatDiagnosticMessage($parts[3]);

        if ($formattedMessage[0] == 'Warning') {
            $this->warning($formattedMessage[0] . ': ' . $formattedMessage[1] . $info);
        }
        if ($formattedMessage[0] == 'Failure') {
            $this->error($formattedMessage[0] . ': ' . $formattedMessage[1] . $info);
        }
    }

    $event->arguments(0, $event);
});

 

[typ9]

I receive warnings using ProcessDiagnostics together with php8.2:

Can someone advice what to do?

[froot]

Hi everybody, hello @netcarver,

Is this module maintained? 
In one of my projects I'm seeing status "Pending" among the Module Version Diagnostics and so no diagnostic is returned.
Does that have to do with the particular web server settings or what else could be the reason for that?

Thanks for help!

Edited August 22 by froot

[netcarver]

@froot Hi there - thanks for asking but I've not looked at this for a number of years, and I doubt I'll get to it any time soon.