Different logging providers

[BitPoet]

Since I'm doing a lot of detailed logging in our internal PW-based systems, that has become a bit of a bottleneck under heavy load and I was missing a centralized view with the growing number of separate PW instances. So I dug into the core a bit, namely WireLog.php and FileLog.php as well as ProcessWire.php. I managed to whip up my own WireLogDatabase and DbLog classes mimicking the behaviour of the regular logging classes, but not without a little bit of tweaking to the ProcessWire class itself to replace the regular logger. Now I'm logging to a MySQL server instead of plain files and ProcessLogger works smoothly with it without tweaking.

I thought it would be shame to keep this all to myself, but a release-worthy version would need or could benefit from:

• a bit of polishing in regards to error handling and proper treatment of conflicting concurrent operations without too much lock overhead (drop table vs. insert especially)
• more source code documentation
• a little more abstraction so all csv operations are deprecated in favor of database columns where avaible
• last but not least, an approved way to configure the substitute logger and load it early on, which means touching the core ProcessWire class

Before I invest too much into that, I'd love to hear all thoughts on this, especially if you think such a module may fit your requirements, and I would be especially happy to hear from @ryan - could you see such a mechanism in the core?

[horst]

Sounds interesting. - To ping ryan about it, you better submit an issue or a PR on github.

[LostKobrakai]

The most flexible version would probably be a monolog (or compatible) implementation, just because how ubiquitous it's in the php community and all it's adapters to various logging backends.

[adrian]

2 hours ago, LostKobrakai said:

The most flexible version would probably be a monolog (or compatible) implementation, just because how ubiquitous it's in the php community and all it's adapters to various logging backends.

There is a monolog plugin for Tracy: https://componette.com/nextras/tracy-monolog-adapter/ so if you're already using Tracy for logging errors in production mode (no debug bar), this might be an option.

If people are interested in that approach, I'd be willing to add.

PS - @BitPoet - not meaning to take away from your approach - just throwing another option out there.

[Andreas Augustin]

Monolog would be nice. With this I could use SaaS's like papertrailapp or loggly

[salmiak]

@BitPoet I'm quite interested in this as we have lots of problems with performance for file logs. Did you make any progress since you posted this?

[BitPoet]

On 1/22/2021 at 12:26 PM, salmiak said:

Did you make any progress since you posted this?

Unfortunately not. There were changes in the core that broke my own changes, and I simply didn't have the time (a recurring topic, lately...) to fix that at that time, so I switched back to regular logging.

[StanLindsey]

If you're running on a VPS you can use a service like Grafana Cloud or LogDNA to watch the various log files and send them at a system level. 

[salmiak]

 

On 3/17/2017 at 1:26 PM, LostKobrakai said:

The most flexible version would probably be a monolog (or compatible) implementation, just because how ubiquitous it's in the php community and all it's adapters to various logging backends.

I'm working making a module integrating monolog into processwire now. When hooking into WireLog I can intercept most logs. Except for fatal errors. These dont go through WireLog but are sent from WireShutdown to FileLog directly. 

I don't really know if there is a way to intercept these without hacking the core.

Does anyone have an idea?

[adrian]

@salmiak - did you see my suggestion above for incorporating monolog into Tracy? It might be the simplest option.

If you go with a separate module, perhaps hooking into Wire::trackException might be of use?

[horst]

1 hour ago, salmiak said:

I don't really know if there is a way to intercept these without hacking the core.

Does anyone have an idea?

Sorry I'm short in time ATM, but I think in general this is handled by PHP https://www.php.net/manual/en/function.set-error-handler.php

You will find such function calls in pws wire directories, for example in WireHttp.php and other files. As far as I remember right, it is possible to define multiple handlers in PHP that you can use by "bubbling" (or cascading) and maybe conditionally exit that at any point.

You definetly dont't need to hack the core. Maybe the PHP manual is a good starting point. Next would be to find where PW handles it and how to set you above this, do your logging and after that pass the event through to PWs handler where needed.