digitalmars.D - Suggestion/proposal regarding std.logger candidate
- David Osborne (79/79) May 24 2014 Yesterday, in the livestream chat, there was some discussion about
- David Osborne (5/5) May 24 2014 Just to be clear, what I am suggesting/proposing is that the
- Gary Willoughby (3/8) May 24 2014 Please be aware of KISS, YAGNI and other beware of over
- Tobias Pankrath (2/10) May 24 2014 Why not just copy the design of boost::log?
- Robert BuRnEr Schadek via Digitalmars-d (2/12) May 24 2014 because it is a c++ design based around streams
- Tobias Pankrath (4/23) May 25 2014 That's just the surface. See:
- Robert BuRnEr Schadek via Digitalmars-d (2/6) May 24 2014 why? you have just shown how easily extensible the current proposal is.
- HaraldZealot (2/2) May 24 2014 The logger is very good and need thing. But has anybody an idea
- Jonathan M Davis via Digitalmars-d (12/14) May 24 2014 That's impossible if you're doing I/O. By definition, that has to access
- Steven Schveighoffer (4/24) May 24 2014 IIRC, logging with 'debug' should work.
- Jonathan M Davis via Digitalmars-d (17/18) May 25 2014 On Sat, 24 May 2014 23:56:39 -0700
- Jeremy Powers via Digitalmars-d (11/12) May 24 2014 Overall, this approach seems good.
- Robert BuRnEr Schadek via Digitalmars-d (5/76) May 24 2014 and on a condition you can pass
Yesterday, in the livestream chat, there was some discussion about
the current std.logger proposal. The consensus seemed to be that
it's probably fine for most use cases and it's the best proposal
we've had so far, it doesn't look suitable for heavy duty,
enterprise-grade logging. In an ideal world, the std.logger
module
in phobos would be flexible enough to build an enterprise-grade
logging library on top of, but it seems the current proposal isn't
there yet. I got to thinking about how to make the logging system
more flexible, and this is what I came up with.
I identified three key parts to a logger: the filter, the format,
and the sink.
log.trace(...) ----> Filter ----> Format ----> Sink
The filter determines which log messages to act on, and which to
ignore. Usually this is done based on the log level, but one use
case that was brought up was automatically sending an email to the
developers only when a particular exception occurred (i.e. filter
based on Exception type). Currently, std.logger implements the
filtering for you, based on the global log level, and the log
level
of that particular logger.
The format determines what the output of the log message looks
like.
Usually, this winds up being some sort of call to formattedWrite,
though there is no need to assume that the output must be a
string,
or even human-readable. Currently, formatting tends to be
implemented as part of Logger.writeLogMsg, and is specific to each
Logger implementation.
The sink is where the log message goes. Popular destinations
include stderr, /dev/null, some text file somewhere, or a central
logging facility provided by your OS. Currently, the sink is
specified when implementing Logger.writeLogMsg.
Based on these three parameters, I came up with the following
proof
of concept implementation, built on top of the current proposal:
public class Clogger(alias format, alias filter, Sink) : Logger
if( isLogFilter!filter && isLogFormat!format &&
isLogSink!(Sink, format) )
{
public this(Sink sink, string newName, LogLevel lv) {
super(newName, lv);
this.sink = sink;
}
override
public void writeLogMsg(ref LoggerPayload payload) {
import std.algorithm : copy;
if( this.filterFun(payload) ) {
sink = formatFun(payload).copy(sink);
}
}
private:
Sink sink;
alias formatFun = unaryFun!format;
alias filterFun = unaryFun!filter;
}
The full code is at https://github.com/krendil/clogger.
As you can see, there are three template parameters, that
correspond
to the three components identified above.
- filter is a predicate that takes a LoggerPayload and returns
true
if that log message should be logged.
- format takes a LoggerPayload and converts it to something
suitable for output, in the form of a Range.
- Sink is an OutputRange that has the same element type as the
the output of format.
I think there are a few benefits to this setup:
- Separation of format from sink
- Custom, arbitrary filtering
- Simpler to implement new Loggers
With this setup, multiplexing can either be done at the Logger
level, as it is currently, or at the Sink level. Log messages can
still be converted to any sort of output. Aliases or convenience
functions could be created for common Logger types (I currently
have
convenience functions for a multiplexer, a stdout logger and a
generic dchar range logger).
Destroy :)
May 24 2014
Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.
May 24 2014
On Saturday, 24 May 2014 at 10:04:33 UTC, David Osborne wrote:Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.Please be aware of KISS, YAGNI and other beware of over engineering principles
May 24 2014
On Saturday, 24 May 2014 at 11:58:35 UTC, Gary Willoughby wrote:On Saturday, 24 May 2014 at 10:04:33 UTC, David Osborne wrote:Why not just copy the design of boost::log?Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.Please be aware of KISS, YAGNI and other beware of over engineering principles
May 24 2014
On 05/24/2014 05:39 AM, Tobias Pankrath via Digitalmars-d wrote:On Saturday, 24 May 2014 at 11:58:35 UTC, Gary Willoughby wrote:because it is a c++ design based around streamsOn Saturday, 24 May 2014 at 10:04:33 UTC, David Osborne wrote:Why not just copy the design of boost::log?Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.Please be aware of KISS, YAGNI and other beware of over engineering principles
May 24 2014
"Tobias Pankrath" <tobias pankrath.net> On Sunday, 25 May 2014 at 06:07:04 UTC, Robert BuRnEr Schadek via Digitalmars-d wrote:On 05/24/2014 05:39 AM, Tobias Pankrath via Digitalmars-d wrote:That's just the surface. See: http://boost-log.sourceforge.net/libs/log/doc/html/log/design.htmlOn Saturday, 24 May 2014 at 11:58:35 UTC, Gary Willoughby wrote:because it is a c++ design based around streamsOn Saturday, 24 May 2014 at 10:04:33 UTC, David Osborne wrote:Why not just copy the design of boost::log?Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.Please be aware of KISS, YAGNI and other beware of over engineering principles
May 25 2014
On 05/24/2014 03:04 AM, David Osborne via Digitalmars-d wrote:Just to be clear, what I am suggesting/proposing is that the current Logger class in std.logger be altered so that it looks more like this, with filter, format and sink template arguments that determine its behaviour, instead of using an inheritance-based approach.why? you have just shown how easily extensible the current proposal is.
May 24 2014
The logger is very good and need thing. But has anybody an idea how to preserve purity of functions?
May 24 2014
On Sat, 24 May 2014 11:42:30 +0000 HaraldZealot via Digitalmars-d <digitalmars-d puremagic.com> wrote:The logger is very good and need thing. But has anybody an idea how to preserve purity of functions?That's impossible if you're doing I/O. By definition, that has to access mutable, global state. The only way to do it would be to go the Haskell route and use something like monads - you'd have to basically pass an object around which built up the logging messages without actually logging anything and then have an impure function higher up (which is calling these pure functions) make the function call which actually caused the queued up logging to be output. Logging is basically anti-purity. So, unless you do something like I just described (which I doubt much of anyone will), then logging anything in your code means giving up purity. - Jonathan M Davis
May 24 2014
On Sat, 24 May 2014 20:10:21 -0700, Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Sat, 24 May 2014 11:42:30 +0000 HaraldZealot via Digitalmars-d <digitalmars-d puremagic.com> wrote:IIRC, logging with 'debug' should work. -SteveThe logger is very good and need thing. But has anybody an idea how to preserve purity of functions?That's impossible if you're doing I/O. By definition, that has to access mutable, global state. The only way to do it would be to go the Haskell route and use something like monads - you'd have to basically pass an object around which built up the logging messages without actually logging anything and then have an impure function higher up (which is calling these pure functions) make the function call which actually caused the queued up logging to be output. Logging is basically anti-purity. So, unless you do something like I just described (which I doubt much of anyone will), then logging anything in your code means giving up purity.
May 24 2014
On Sat, 24 May 2014 23:56:39 -0700 Steven Schveighoffer via Digitalmars-d <digitalmars-d puremagic.com> wrote:IIRC, logging with 'debug' should work.Yes, logging in debug blocks would work, because debug blocks ignore purity constraints (which means that they _can_ cause a function to have weird behavior if you're not careful, because the compiler is still free to do things like optimizations based on purity). It's that way to help with debugging (since not being able to print anything when debugging can be a _real_ pain), but since logging is frequently used for purposes other than simply debugging a program, putting logging statements in debug blocks doesn't really solve the problem - and abusing debug blocks by using them specifically for logging purposes could cause some weird stuff to happen if you're not careful, since you'd be doing impure stuff in a pure function and thereby violating guarantees that the compiler assumes hold. Realistically, if you're looking to do much logging in your program, you're probably going to have to give up on using pure in a lot of places. - Jonathan M Davis
May 25 2014
On Sat, May 24, 2014 at 2:57 AM, David Osborne via Digitalmars-d < digitalmars-d puremagic.com> wrote:Destroy :)Overall, this approach seems good. Not sure I like forcing one format for all sinks though - it is often desired/necessary to have different sinks (aka appenders) format the log messages differently. Aside: I'd recommend anyone looking at logging examine the logback architecture, if they haven't already: http://logback.qos.ch/manual/architecture.html Not saying std.logger should look like this, but shows one fairly well thought out solution.
May 24 2014
On 05/24/2014 02:57 AM, David Osborne via Digitalmars-d wrote:Yesterday, in the livestream chat, there was some discussion about the current std.logger proposal. The consensus seemed to be that it's probably fine for most use cases and it's the best proposal we've had so far, it doesn't look suitable for heavy duty, enterprise-grade logging. In an ideal world, the std.logger module in phobos would be flexible enough to build an enterprise-grade logging library on top of, but it seems the current proposal isn't there yet. I got to thinking about how to make the logging system more flexible, and this is what I came up with.Why is that?I identified three key parts to a logger: the filter, the format, and the sink. log.trace(...) ----> Filter ----> Format ----> Sink The filter determines which log messages to act on, and which to ignore. Usually this is done based on the log level, but one use case that was brought up was automatically sending an email to the developers only when a particular exception occurred (i.e. filter based on Exception type). Currently, std.logger implements the filtering for you, based on the global log level, and the log level of that particular logger.and on a condition you can passThe format determines what the output of the log message looks like. Usually, this winds up being some sort of call to formattedWrite, though there is no need to assume that the output must be a string, or even human-readable. Currently, formatting tends to be implemented as part of Logger.writeLogMsg, and is specific to each Logger implementation. The sink is where the log message goes. Popular destinations include stderr, /dev/null, some text file somewhere, or a central logging facility provided by your OS. Currently, the sink is specified when implementing Logger.writeLogMsg.That is the ideaBased on these three parameters, I came up with the following proof of concept implementation, built on top of the current proposal: public class Clogger(alias format, alias filter, Sink) : Logger if( isLogFilter!filter && isLogFormat!format && isLogSink!(Sink, format) ) { public this(Sink sink, string newName, LogLevel lv) { super(newName, lv); this.sink = sink; } override public void writeLogMsg(ref LoggerPayload payload) { import std.algorithm : copy; if( this.filterFun(payload) ) { sink = formatFun(payload).copy(sink); } } private: Sink sink; alias formatFun = unaryFun!format; alias filterFun = unaryFun!filter; } The full code is at https://github.com/krendil/clogger. As you can see, there are three template parameters, that correspond to the three components identified above. - filter is a predicate that takes a LoggerPayload and returns true if that log message should be logged. - format takes a LoggerPayload and converts it to something suitable for output, in the form of a Range. - Sink is an OutputRange that has the same element type as the the output of format. I think there are a few benefits to this setup: - Separation of format from sink - Custom, arbitrary filtering - Simpler to implement new Loggers With this setup, multiplexing can either be done at the Logger level, as it is currently, or at the Sink level. Log messages can still be converted to any sort of output. Aliases or convenience functions could be created for common Logger types (I currently have convenience functions for a multiplexer, a stdout logger and a generic dchar range logger). Destroy :)Thank you, for showing how easy it is to use std.logger for your purposes
May 24 2014