www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Early review of std.logger

reply "Dicebot" <public dicebot.lv> writes:
As `std.logger` is still marked as "work in progress" this thread 
is less formal that typical pre-voting review. Goal is to provide 
as much input about desirable `std.logger` functionality and 
current state and let module author to use this information for 
preparing proposal for actual review / voting.

Lets unleash the forces of constructive destruction.

===== Meta =====

Code: 
https://github.com/D-Programming-Language/phobos/pull/1500/files
Docs: 
http://burner.github.io/phobos/phobos-prerelease/std_logger.html

First announcement / discussion thread : 
http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com
Oct 14 2013
next sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 14.10.2013 13:39, schrieb Dicebot:

 As `std.logger` is still marked as "work in progress" this thread is
 less formal that typical pre-voting review. Goal is to provide as much
 input about desirable `std.logger` functionality and current state and
 let module author to use this information for preparing proposal for
 actual review / voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread :
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


Sorry in advance for the long list of issues. I think the general 
approach is fine, but over the years I've grown some preferences for 
this stuff. Generally, I think we should make sure that such a module is 
flexible enough to fulfill most people's needs, or it will probably fail 
the widespread adoption that is desired to actually improve 
interoperability.

  - LogLevel: enum values should start lower case according to the
    Phobos conventions.
  - The static methods in LogManager should be made global and the class
    be removed. It's not for objects so it shouldn't be a class.
  - For me this logger is completely worthless without any debug log
    levels. The last std.log entry had at least anonymous verbosity
    levels, but I'd prefer something like I did in vibe.d [1], where
    each level has a defined role. This should especially improve the
    situation when multiple libraries are involved.
  - My experience tells me that logging formatted messages by default
    (or even drop the non-formatted version) is perfectly fine, but
    others may differ of course and an additional "f" is not that bad.
  - Similarly, I've never felt the need for conditional logging, but
    without it being lazy evaluated what's the use for it, actually?
  - I really think there should be shortcuts for the different log
    levels. Typing out "LogLevel.xxx" each time looks tedious for
    something that is used in so many places.
  - There should be some kind of MultiLogger so that multiple log
    destinations can be configured (e.g. console + file). Or, instead of
    a single default logger, there could be a list of loggers.
  - Logger.logMessage: Exchanging the bunch of parameters with a single
    struct that is passed by reference makes the API much more
    flexible/future-proof and is more efficient when the data needs to
    be passed on to other functions.
  - "shared" - probably best to leave this out until we have a verified
    design for that - but in theory the loggers should probably be
    shared.
  - On the same topic, if I'm right and the default logger is stored as
    __gshared, it should be documented that Loggers need to be
    thread-safe.
  - GC allocations for each log message _must_ be avoided at all costs.
    Using formattedWrite() instead of format() on either a temporary
    buffer that is reused, or, better, on some kind of output range
    interface of the Logger would solve this.
  - A note on DDOC comments: The first paragraph of a doc comment is
    treated as a short description according to the DDOC spec. It should
    be kept to around a single sentence, followed by a more detailed
    paragraph. While this doesn't matter much with the current HTML doc
    layout, the short description is used in the single-page
    documentation inside of overview tables [2].

[1]: http://vibed.org/api/vibe.core.log/LogLevel
[2]: http://vibed.org/temp/d-programming-language.org/phobos/std/ascii.html
Oct 14 2013
next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 02:39 PM, Sönke Ludwig wrote:

 Am 14.10.2013 13:39, schrieb Dicebot:

 As `std.logger` is still marked as "work in progress" this thread is
 less formal that typical pre-voting review. Goal is to provide as much
 input about desirable `std.logger` functionality and current state and
 let module author to use this information for preparing proposal for
 actual review / voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread :
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


 Sorry in advance for the long list of issues. I think the general
 approach is fine, but over the years I've grown some preferences for
 this stuff. Generally, I think we should make sure that such a module
 is flexible enough to fulfill most people's needs, or it will probably
 fail the widespread adoption that is desired to actually improve
 interoperability.

  - LogLevel: enum values should start lower case according to the
    Phobos conventions.


will be fixed

  - The static methods in LogManager should be made global and the class
    be removed. It's not for objects so it shouldn't be a class.


LogManager also stores the global log level. Sure I can make another
static global function storing this log level, but I would like to keep
them together as they belong together IMO.

  - For me this logger is completely worthless without any debug log
    levels. The last std.log entry had at least anonymous verbosity
    levels, but I'd prefer something like I did in vibe.d [1], where
    each level has a defined role. This should especially improve the
    situation when multiple libraries are involved.


Logger.log(LogLevel.(d|D)ebug, "Your message");

  - Similarly, I've never felt the need for conditional logging, but
    without it being lazy evaluated what's the use for it, actually?


The conditional logging part is totally transparent.

  - I really think there should be shortcuts for the different log
    levels. Typing out "LogLevel.xxx" each time looks tedious for
    something that is used in so many places.


One could argue that writting logger.logDebug("...") is more tedious
than writing,
logger.logLevel = LogLevel.xxx;
logger.log("...");
logger.log("...");
...

This has been argued in the last logger discussion to some extend and it
looked to me like this is the mostly preferred version.

  - There should be some kind of MultiLogger so that multiple log
    destinations can be configured (e.g. console + file). Or, instead of
    a single default logger, there could be a list of loggers.


there is one default logger. I will create a MultiLogger, good point.
I'm currently sure how to store the multi logger (LL or Array or ... )

  - Logger.logMessage: Exchanging the bunch of parameters with a single
    struct that is passed by reference makes the API much more
    flexible/future-proof and is more efficient when the data needs to
    be passed on to other functions.


good point

  - "shared" - probably best to leave this out until we have a verified
    design for that - but in theory the loggers should probably be
    shared.


my thoughts exactly

  - On the same topic, if I'm right and the default logger is stored as
    __gshared, it should be documented that Loggers need to be
    thread-safe.


It is not stored __gshared, but If, you're right.

  - GC allocations for each log message _must_ be avoided at all costs.
    Using formattedWrite() instead of format() on either a temporary
    buffer that is reused, or, better, on some kind of output range
    interface of the Logger would solve this.


This was proposed in the last thread. A fixed size buffer would scream
bufferoverflow, a dynamic buffer not but both would raise the question
of thread safety. 

  - A note on DDOC comments: The first paragraph of a doc comment is
    treated as a short description according to the DDOC spec. It should
    be kept to around a single sentence, followed by a more detailed
    paragraph. While this doesn't matter much with the current HTML doc
    layout, the short description is used in the single-page
    documentation inside of overview tables [2].


Will be fixed

Awesome comments, thanks
Oct 14 2013
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 14.10.2013 15:12, schrieb Robert Schadek:

 On 10/14/2013 02:39 PM, Sönke Ludwig wrote:

  - The static methods in LogManager should be made global and the class
    be removed. It's not for objects so it shouldn't be a class.


 LogManager also stores the global log level. Sure I can make another
 static global function storing this log level, but I would like to keep
 them together as they belong together IMO.


The same could be said about the global "log" functions, which are
tightly coupled to that state. I think this is already nicely grouped
together by the logger module itself, since there is not much else in it.

Basically, I just wouldn't consider this style to be particularly
idiomatic D code, but of course that's just personal
perception/preference (there is also some precedence using "struct"
instead of "class" in Druntime). However, if it ends up like this in the
final version, it should get a " disable this();" to prevent misuse.


  - For me this logger is completely worthless without any debug log
    levels. The last std.log entry had at least anonymous verbosity
    levels, but I'd prefer something like I did in vibe.d [1], where
    each level has a defined role. This should especially improve the
    situation when multiple libraries are involved.


 Logger.log(LogLevel.(d|D)ebug, "Your message");


That would be my idea. Having at least two (diagnostic output for the
user and debug output for the developer), but better all four debug
levels can be very useful, though.


  - Similarly, I've never felt the need for conditional logging, but
    without it being lazy evaluated what's the use for it, actually?


 The conditional logging part is totally transparent.


But is there a compelling use case? It's transparent, but still
increases the size/complexity of the API, not much, but there should be
a reason for that IMO.


  - I really think there should be shortcuts for the different log
    levels. Typing out "LogLevel.xxx" each time looks tedious for
    something that is used in so many places.


 One could argue that writting logger.logDebug("...") is more tedious
 than writing,
 logger.logLevel = LogLevel.xxx;
 logger.log("...");
 logger.log("...");
 ...
 
 This has been argued in the last logger discussion to some extend and it
 looked to me like this is the mostly preferred version.


The last discussion resulted (if I remember right) in something like
"log.info(...)". This would be fine, too, although I think just
"logInfo" is slightly preferable due to its length and the principle of
least surprise.


  - There should be some kind of MultiLogger so that multiple log
    destinations can be configured (e.g. console + file). Or, instead of
    a single default logger, there could be a list of loggers.


 there is one default logger. I will create a MultiLogger, good point.
 I'm currently sure how to store the multi logger (LL or Array or ... )


I'd say a dynamic array is fine because the list of loggers will rarely
change and this would be most efficient for iteration.


  - On the same topic, if I'm right and the default logger is stored as
    __gshared, it should be documented that Loggers need to be
    thread-safe.


 It is not stored __gshared, but If, you're right.


So the defaultLogger is per-thread? That may result in unexpected log
output if you just do simple code like "spawn({ log("Hello, World!");
});" and have a custom logger set up during application initialization.
Anyway, let's just say the threading behavior in general should be
clearly documented here as this is critical for both log users and
Logger implementors.


  - GC allocations for each log message _must_ be avoided at all costs.
    Using formattedWrite() instead of format() on either a temporary
    buffer that is reused, or, better, on some kind of output range
    interface of the Logger would solve this.


 This was proposed in the last thread. A fixed size buffer would scream
 bufferoverflow, a dynamic buffer not but both would raise the question
 of thread safety. 


Something like a thread local buffer that grows when needed, or small
fixed buffer + a scoped heap allocation for large messages should be
fine. Using an output range interface could of course avoid buffering
each message altogether. That would just open up the question of a nice
API design for such.

One last thing just occurred to me, in the default logger in vibe.d I've
made it so (thanks to Jordi Sayol for requesting this) that the "info"
level gets output to stdout and all other levels to stderr. Either that
or always outputting to stderr would be conforming to the idea of
stdout/stderr and would avoid that some library with logging calls
interferes with the output of an application (when piping process output
in a shell script).

Thanks for bringing this forward!
Oct 14 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 04:44 PM, Sönke Ludwig wrote:

 Am 14.10.2013 15:12, schrieb Robert Schadek:

 On 10/14/2013 02:39 PM, Sönke Ludwig wrote:

  - The static methods in LogManager should be made global and the class
    be removed. It's not for objects so it shouldn't be a class.


 LogManager also stores the global log level. Sure I can make another
 static global function storing this log level, but I would like to keep
 them together as they belong together IMO.


 The same could be said about the global "log" functions, which are
 tightly coupled to that state. I think this is already nicely grouped
 together by the logger module itself, since there is not much else in it.

 Basically, I just wouldn't consider this style to be particularly
 idiomatic D code, but of course that's just personal
 perception/preference (there is also some precedence using "struct"
 instead of "class" in Druntime). However, if it ends up like this in the
 final version, it should get a " disable this();" to prevent misuse.


It is for ment for phobos not druntime. Anyway structs would mean all
templates and people will scream template bloat. And this would break
the design, which I find to be a valid use of classes and
polymorphisms.  The StdIOLogger can have its default constructor called IMO.

  - For me this logger is completely worthless without any debug log
    levels. The last std.log entry had at least anonymous verbosity
    levels, but I'd prefer something like I did in vibe.d [1], where
    each level has a defined role. This should especially improve the
    situation when multiple libraries are involved.


 Logger.log(LogLevel.(d|D)ebug, "Your message");


 That would be my idea. Having at least two (diagnostic output for the
 user and debug output for the developer), but better all four debug
 levels can be very useful, though.


Maybe I miscommunicated what I want to show by that example. The (d|D)
part is the rename to enum lower case.
The debug log level is given through the LogLevel.Debug, which will be
renamed to LogLevel.debug. I would call the developer the user of the
logger. Maybe log messages can be communicated to the user of the
applicaiton and the developer of the application through a MultiLogger
class.

  - Similarly, I've never felt the need for conditional logging, but
    without it being lazy evaluated what's the use for it, actually?


 The conditional logging part is totally transparent.


 But is there a compelling use case? It's transparent, but still
 increases the size/complexity of the API, not much, but there should be
 a reason for that IMO.


I had the case that a custom class made problems in some special case.
And IMO it was just prettier to have the condition if it should be
printed passed as first argument rather than adding a if branch in my
top level code.

log(cls.canBePrinted(), "%s", cls.toString());
is IMO prettier than

if(cls.canBePrinted()) {
    log("%s", cls.toString());
}


  - I really think there should be shortcuts for the different log
    levels. Typing out "LogLevel.xxx" each time looks tedious for
    something that is used in so many places.


 One could argue that writting logger.logDebug("...") is more tedious
 than writing,
 logger.logLevel = LogLevel.xxx;
 logger.log("...");
 logger.log("...");
 ...

 This has been argued in the last logger discussion to some extend and it
 looked to me like this is the mostly preferred version.


 The last discussion resulted (if I remember right) in something like
 "log.info(...)". This would be fine, too, although I think just
 "logInfo" is slightly preferable due to its length and the principle of
 least surprise.


At least jmdavis had a very strong argument against it. Something like
function should be verbs ... Maybe something like logCritical, logInfo
would be a compromise, but I think everybody here has a different idea
of what is correct.

  - There should be some kind of MultiLogger so that multiple log
    destinations can be configured (e.g. console + file). Or, instead of
    a single default logger, there could be a list of loggers.


 there is one default logger. I will create a MultiLogger, good point.
 I'm currently sure how to store the multi logger (LL or Array or ... )


 I'd say a dynamic array is fine because the list of loggers will rarely
 change and this would be most efficient for iteration.


Properly yes, at least will that be my first try.

  - On the same topic, if I'm right and the default logger is stored as
    __gshared, it should be documented that Loggers need to be
    thread-safe.


 It is not stored __gshared, but If, you're right.


 So the defaultLogger is per-thread? That may result in unexpected log
 output if you just do simple code like "spawn({ log("Hello, World!");
 });" and have a custom logger set up during application initialization.
 Anyway, let's just say the threading behavior in general should be
 clearly documented here as this is critical for both log users and
 Logger implementors.


Yes it is per thread. I concur that this needs to be documented well.

  - GC allocations for each log message _must_ be avoided at all costs.
    Using formattedWrite() instead of format() on either a temporary
    buffer that is reused, or, better, on some kind of output range
    interface of the Logger would solve this.


 This was proposed in the last thread. A fixed size buffer would scream
 bufferoverflow, a dynamic buffer not but both would raise the question
 of thread safety. 


 Something like a thread local buffer that grows when needed, or small
 fixed buffer + a scoped heap allocation for large messages should be
 fine. Using an output range interface could of course avoid buffering
 each message altogether. That would just open up the question of a nice
 API design for such.


Maybe, but I bet somebody else will bring up Allocator and Ref Counting
in that context.

 One last thing just occurred to me, in the default logger in vibe.d I've
 made it so (thanks to Jordi Sayol for requesting this) that the "info"
 level gets output to stdout and all other levels to stderr. Either that
 or always outputting to stderr would be conforming to the idea of
 stdout/stderr and would avoid that some library with logging calls
 interferes with the output of an application (when piping process output
 in a shell script).


First, the default logger is assignable so maybe we should talk about
the StdIOLogger which prints to StdIO or maybe StdErr. IMHO { I hate
when that happens, I hate writing redirects to the cmd. } But this is
properly a task issue.

 Thanks for bringing this forward!


To whom is this directed?
Oct 14 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-10-14 20:24, Robert Schadek wrote:


 At least jmdavis had a very strong argument against it. Something like
 function should be verbs ... Maybe something like logCritical, logInfo
 would be a compromise, but I think everybody here has a different idea
 of what is correct.


If "log.info" is used it minimizes the module level functions.

-- 
/Jacob Carlborg
Oct 14 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 14 October 2013 at 19:47:19 UTC, Jacob Carlborg wrote:

 On 2013-10-14 20:24, Robert Schadek wrote:


 At least jmdavis had a very strong argument against it. 
 Something like
 function should be verbs ... Maybe something like logCritical, 
 logInfo
 would be a compromise, but I think everybody here has a 
 different idea
 of what is correct.


 If "log.info" is used it minimizes the module level functions.


If we need to care about that, D module system is a failure.
But I don't think it is a valid concern.
Oct 14 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


People already complain about conflict function names in Phobos.

-- 
/Jacob Carlborg
Oct 15 2013
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 09:08, schrieb Jacob Carlborg:

 On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


 People already complain about conflict function names in Phobos.


And I'd agree with them. At least inside of a library, care IMO should 
be taken to minimize overlap (of course functionally equivalent ones in 
different overload sets are fine, though). But in case of "logXXX" this 
seems to be very unlikely, much in contrast to "log" (std.math.log).
Oct 15 2013
next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 09:32 AM, Sönke Ludwig wrote:

 Am 15.10.2013 09:08, schrieb Jacob Carlborg:

 On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


 People already complain about conflict function names in Phobos.


 And I'd agree with them. At least inside of a library, care IMO should
 be taken to minimize overlap (of course functionally equivalent ones
 in different overload sets are fine, though). But in case of "logXXX"
 this seems to be very unlikely, much in contrast to "log" (std.math.log).


yes and no. Of course does logXXX create less conflict, but I like to
simply write log and don't care about the LogLevel. So again pros and cons
Oct 15 2013
parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Tuesday, 15 October 2013 at 08:47:00 UTC, Robert Schadek wrote:

 On 10/15/2013 09:32 AM, Sönke Ludwig wrote:

 Am 15.10.2013 09:08, schrieb Jacob Carlborg:

 On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


 People already complain about conflict function names in 
 Phobos.


 And I'd agree with them. At least inside of a library, care 
 IMO should
 be taken to minimize overlap (of course functionally 
 equivalent ones
 in different overload sets are fine, though). But in case of 
 "logXXX"
 this seems to be very unlikely, much in contrast to "log" 
 (std.math.log).


 yes and no. Of course does logXXX create less conflict, but I 
 like to
 simply write log and don't care about the LogLevel. So again 
 pros and cons


I for once have never seen any log API with
log.level = INFO;
Logger.log("Here be dragons");

And this I believe for a good reason: in 99% of production code 
I've seen, several log levels are mixed, i.e INFO, CRITICAL and 
DEBUG for instance, so the case where a single log level is used, 
even in the same method, just never happens. The proposed 
solution looks extremely inconvenient to me as it will almost 
always necessit two lines of code instead of one.

I am with Sönke on this one, as well as the need for multi logger 
output. That's the absolute minimum requirement. If this doesn't 
exist, what will happen is, someone will make something better.
Oct 20 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/21/2013 06:19 AM, SomeDude wrote:

 On Tuesday, 15 October 2013 at 08:47:00 UTC, Robert Schadek wrote:

 On 10/15/2013 09:32 AM, Sönke Ludwig wrote:

 Am 15.10.2013 09:08, schrieb Jacob Carlborg:

 On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


 People already complain about conflict function names in Phobos.


 And I'd agree with them. At least inside of a library, care IMO should
 be taken to minimize overlap (of course functionally equivalent ones
 in different overload sets are fine, though). But in case of "logXXX"
 this seems to be very unlikely, much in contrast to "log"
 (std.math.log).


 yes and no. Of course does logXXX create less conflict, but I like to
 simply write log and don't care about the LogLevel. So again pros and
 cons


 I for once have never seen any log API with
 log.level = INFO;
 Logger.log("Here be dragons");

 And this I believe for a good reason: in 99% of production code I've
 seen, several log levels are mixed, i.e INFO, CRITICAL and DEBUG for
 instance, so the case where a single log level is used, even in the
 same method, just never happens. The proposed solution looks extremely
 inconvenient to me as it will almost always necessit two lines of code
 instead of one.


How good than, that you can pass the LogLevel as first argument to the
log function.

 I am with Sönke on this one, as well as the need for multi logger
 output. That's the absolute minimum requirement. If this doesn't
 exist, what will happen is, someone will make something better.


I added a MultiLogger five days ago...
Oct 21 2013
parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 21 October 2013 at 08:37:43 UTC, Robert Schadek wrote:

 I for once have never seen any log API with
 log.level = INFO;
 Logger.log("Here be dragons");

 And this I believe for a good reason: in 99% of production 
 code I've
 seen, several log levels are mixed, i.e INFO, CRITICAL and 
 DEBUG for
 instance, so the case where a single log level is used, even 
 in the
 same method, just never happens. The proposed solution looks 
 extremely
 inconvenient to me as it will almost always necessit two lines 
 of code
 instead of one.


 How good than, that you can pass the LogLevel as first argument 
 to the
 log function.


We repeat many times: you can add
Logger.logInfo("Here be dragons");
syntax. It's much better than
Logger.log(LogLevel.Info, "Here be dragons");
Oct 21 2013
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 October 2013 at 07:33:15 UTC, Sönke Ludwig wrote:

 Am 15.10.2013 09:08, schrieb Jacob Carlborg:

 On 2013-10-14 23:22, Dicebot wrote:


 If we need to care about that, D module system is a failure.
 But I don't think it is a valid concern.


 People already complain about conflict function names in 
 Phobos.


 And I'd agree with them. At least inside of a library, care IMO 
 should be taken to minimize overlap (of course functionally 
 equivalent ones in different overload sets are fine, though). 
 But in case of "logXXX" this seems to be very unlikely, much in 
 contrast to "log" (std.math.log).


I disagree. People complain because they try to use imports in a 
straightforward way, similar to includes. That should be 
discouraged as a bad style in D. Imports should always be either 
with explicit mention of exported symbol or aliased static 
imports. And global module imports should be discouraged too.
Oct 15 2013
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 14.10.2013 20:24, schrieb Robert Schadek:

 On 10/14/2013 04:44 PM, Sönke Ludwig wrote:

 Am 14.10.2013 15:12, schrieb Robert Schadek:

 On 10/14/2013 02:39 PM, Sönke Ludwig wrote:

   - The static methods in LogManager should be made global and the class
     be removed. It's not for objects so it shouldn't be a class.


 LogManager also stores the global log level. Sure I can make another
 static global function storing this log level, but I would like to keep
 them together as they belong together IMO.


 The same could be said about the global "log" functions, which are
 tightly coupled to that state. I think this is already nicely grouped
 together by the logger module itself, since there is not much else in it.

 Basically, I just wouldn't consider this style to be particularly
 idiomatic D code, but of course that's just personal
 perception/preference (there is also some precedence using "struct"
 instead of "class" in Druntime). However, if it ends up like this in the
 final version, it should get a " disable this();" to prevent misuse.


 It is for ment for phobos not druntime. Anyway structs would mean all
 templates and people will scream template bloat. And this would break
 the design, which I find to be a valid use of classes and
 polymorphisms.  The StdIOLogger can have its default constructor called IMO.


No no, I was talking about the JobManager, not the Logger classes. No 
templates involved.


   - For me this logger is completely worthless without any debug log
     levels. The last std.log entry had at least anonymous verbosity
     levels, but I'd prefer something like I did in vibe.d [1], where
     each level has a defined role. This should especially improve the
     situation when multiple libraries are involved.


 Logger.log(LogLevel.(d|D)ebug, "Your message");


 That would be my idea. Having at least two (diagnostic output for the
 user and debug output for the developer), but better all four debug
 levels can be very useful, though.


 Maybe I miscommunicated what I want to show by that example. The (d|D)
 part is the rename to enum lower case.
 The debug log level is given through the LogLevel.Debug, which will be
 renamed to LogLevel.debug. I would call the developer the user of the
 logger. Maybe log messages can be communicated to the user of the
 applicaiton and the developer of the application through a MultiLogger
 class.


But the statement of mine that you quoted was about debug levels (the 
case issue is clear)... Also right now there is no "(D|d)ebug" level, so 
I'm actually not sure about the statement that you want to make. But my 
example of having different levels for the application user and the 
developer is mostly important when the application user enables verbose 
log output to see where things go wrong. In that case things like system 
error codes and the like would make sense, but a repeated printout of 
some kind of internal buffer state would hardly help the user - it 
could, however, help the developer.


 Thanks for bringing this forward!


 To whom is this directed?


To you for attempting to revive the logger topic.
Oct 15 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 09:40 AM, Sönke Ludwig wrote:

 Am 14.10.2013 20:24, schrieb Robert Schadek:

 On 10/14/2013 04:44 PM, Sönke Ludwig wrote:

 The same could be said about the global "log" functions, which are
 tightly coupled to that state. I think this is already nicely grouped
 together by the logger module itself, since there is not much else
 in it.

 Basically, I just wouldn't consider this style to be particularly
 idiomatic D code, but of course that's just personal
 perception/preference (there is also some precedence using "struct"
 instead of "class" in Druntime). However, if it ends up like this in
 the
 final version, it should get a " disable this();" to prevent misuse.


 It is for ment for phobos not druntime. Anyway structs would mean all
 templates and people will scream template bloat. And this would break
 the design, which I find to be a valid use of classes and
 polymorphisms.  The StdIOLogger can have its default constructor
 called IMO.


 No no, I was talking about the JobManager, not the Logger classes. No
 templates involved.


Than I'm not sure what you're referring to.


 Maybe I miscommunicated what I want to show by that example. The (d|D)

 part is the rename to enum lower case.
 The debug log level is given through the LogLevel.Debug, which will be
 renamed to LogLevel.debug. I would call the developer the user of the
 logger. Maybe log messages can be communicated to the user of the
 applicaiton and the developer of the application through a MultiLogger
 class.


 But the statement of mine that you quoted was about debug levels (the
 case issue is clear)... Also right now there is no "(D|d)ebug" level,
 so I'm actually not sure about the statement that you want to make.
 But my example of having different levels for the application user and
 the developer is mostly important when the application user enables
 verbose log output to see where things go wrong. In that case things
 like system error codes and the like would make sense, but a repeated
 printout of some kind of internal buffer state would hardly help the
 user - it could, however, help the developer.


maybe something like:
auto devLogger = new StdIOLogger(LogLevel.info);
auto appLogger = new FencySelfWrittenGuiLogger(LogLevel.Warning);
auto multiLogger = new MultiLogger(devLogger, appLogger);

multiLogger.log("...");

otherwise, I think I don't follow you
Oct 15 2013
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 10:54, schrieb Robert Schadek:

 On 10/15/2013 09:40 AM, Sönke Ludwig wrote:

 Am 14.10.2013 20:24, schrieb Robert Schadek:

 On 10/14/2013 04:44 PM, Sönke Ludwig wrote:

 The same could be said about the global "log" functions, which are
 tightly coupled to that state. I think this is already nicely grouped
 together by the logger module itself, since there is not much else
 in it.

 Basically, I just wouldn't consider this style to be particularly
 idiomatic D code, but of course that's just personal
 perception/preference (there is also some precedence using "struct"
 instead of "class" in Druntime). However, if it ends up like this in
 the
 final version, it should get a " disable this();" to prevent misuse.


 It is for ment for phobos not druntime. Anyway structs would mean all
 templates and people will scream template bloat. And this would break
 the design, which I find to be a valid use of classes and
 polymorphisms.  The StdIOLogger can have its default constructor
 called IMO.


 No no, I was talking about the JobManager, not the Logger classes. No
 templates involved.


 Than I'm not sure what you're referring to.


What I meant is just that in Druntime there is something like this:

struct LogManager {
   static void somefunc();
}

instead of

class LogManager {
   static void someFunc();
}

In any case, such a struct/class should also have a member " disable 
this();" so that it cannot be uselessly instantiated.


 Maybe I miscommunicated what I want to show by that example. The (d|D)

 part is the rename to enum lower case.
 The debug log level is given through the LogLevel.Debug, which will be
 renamed to LogLevel.debug. I would call the developer the user of the
 logger. Maybe log messages can be communicated to the user of the
 applicaiton and the developer of the application through a MultiLogger
 class.


 But the statement of mine that you quoted was about debug levels (the
 case issue is clear)... Also right now there is no "(D|d)ebug" level,
 so I'm actually not sure about the statement that you want to make.
 But my example of having different levels for the application user and
 the developer is mostly important when the application user enables
 verbose log output to see where things go wrong. In that case things
 like system error codes and the like would make sense, but a repeated
 printout of some kind of internal buffer state would hardly help the
 user - it could, however, help the developer.


 maybe something like:
 auto devLogger = new StdIOLogger(LogLevel.info);
 auto appLogger = new FencySelfWrittenGuiLogger(LogLevel.Warning);
 auto multiLogger = new MultiLogger(devLogger, appLogger);

 multiLogger.log("...");

 otherwise, I think I don't follow you


The log messages are supposed to go to the same destination, just 
filtered by log level. A totally artificial example:

---
void main(string[] args)
{
     logDiagnostic("Application called as %s", args[0]);
     ubyte[] contents;
     try {
         logTrace("Going to read file");
         contents = read("somefile.dat");
         logTrace("Done reading file");
     } catch (Exception e) {
         logError("Failed to read input file.");
         logDiagnostic("Reported error: %s", e.msg);
         logDebug("Full exception: %s", e.toString());
	return 1;
     }
     logInfo("Input file is %d bytes", contents.length);
     logTrace("Computing sum");
     auto sum = sum(contents);
     logTrace("Removing file");
     remove("somefile.dat");
}

ulong sum(ubyte[] arr)
{
	ulong ret = 0;
	foreach (b; arr) {
		logDebugV("Adding %d", b);
		ret += b;
	}
	logDebugV("Sum result: %d", b);
	return ret;
}
---

A typical mode in my projects now is to output any level starting with 
"info" to the console by default, but allow to log "diagnostic" messages 
using "-v" (useful to the end user) and lower levels using other 
switches (useful for me to diagnose bugs).

At the same time there may be a log file or a network based remote 
logger that captures all levels regardless of command line switches, but 
needs to be able to filter based on the actual log level later in a GUI. 
Having multiple loggers for the different diagnostic/debug/trace levels 
would not really help there (except with some extensive hacking).

BTW you stated that there is a "debug" level in your implementation, but 
neither the docs, nor the pull request have it.
Oct 15 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 02:54 PM, Sönke Ludwig wrote:

 What I meant is just that in Druntime there is something like this:

 struct LogManager {
   static void somefunc();
 }

 instead of

 class LogManager {
   static void someFunc();
 }

 In any case, such a struct/class should also have a member " disable
 this();" so that it cannot be uselessly instantiated.


I see what you mean, good point.

 The log messages are supposed to go to the same destination, just
 filtered by log level. A totally artificial example:

 ---
 void main(string[] args)
 {
     logDiagnostic("Application called as %s", args[0]);
     ubyte[] contents;
     try {
         logTrace("Going to read file");
         contents = read("somefile.dat");
         logTrace("Done reading file");
     } catch (Exception e) {
         logError("Failed to read input file.");
         logDiagnostic("Reported error: %s", e.msg);
         logDebug("Full exception: %s", e.toString());
     return 1;
     }
     logInfo("Input file is %d bytes", contents.length);
     logTrace("Computing sum");
     auto sum = sum(contents);
     logTrace("Removing file");
     remove("somefile.dat");
 }

 ulong sum(ubyte[] arr)
 {
     ulong ret = 0;
     foreach (b; arr) {
         logDebugV("Adding %d", b);
         ret += b;
     }
     logDebugV("Sum result: %d", b);
     return ret;
 }
 ---


Ok, you can achieve this by assigning a LogLevel to the defaultLogger.
Than all message with a LogLevel >= to the assigned LogLevel will be
logged. Or do you want to, for example, filter all message with critical
and debug LogLevel, but don't display the messages with LogLevels in
between?

 BTW you stated that there is a "debug" level in your implementation,
 but neither the docs, nor the pull request have it.


You're right I was reading Debug and thought Info, my bad!
Oct 15 2013
prev sibling parent reply "Guillaume Chatelet" <chatelet.guillaume gmail.com> writes:
On Monday, 14 October 2013 at 12:40:16 UTC, Sönke Ludwig wrote:

 Am 14.10.2013 13:39, schrieb Dicebot:
  - For me this logger is completely worthless without any debug 
 log levels. The last std.log entry had at least anonymous 
 verbosity
 levels, but I'd prefer something like I did in vibe.d [1], 
 where each level has a defined role. This should especially 
 improve the situation when multiple libraries are involved.


Orthogonal to log levels one idea could be to define a namespace 
for a logger instance.

* The namespace could be in the form of a prefix (useful for 
grepping and simple to implement)

logger.setPrefix("myApi__");
logger.warn("a warning"); // would output "myApi__a warning"

* The namespace could be deduced from the module name and 
filtered at runtime by a flag 'à la' gtest ( 
https://code.google.com/p/googletest/wiki/AdvancedGuide#Running_a
Subset_of_the_Tests 
). This would be helpful when displaying fine grained log for 
debugging purpose without being flooded by other libraries logs.


./foo Has no flag, and thus output all the logs.
./foo --log_namespace=myLyb.* Output everything in myLyb module.


This option requires initializing the test library in main and 
would be more costly because of the namespace matching ( maybe 
the initialization could trigger a different implementation at 
runtime or we enable this feature at compile time through a 
version )
Oct 14 2013
parent reply Jeremy Powers <jpowers wyrdtech.com> writes:
 Orthogonal to log levels one idea could be to define a namespace for a


logger instance.
Take a look at the way log4j does it, with logger hierarchy:
http://logging.apache.org/log4j/1.2/manual.html

This is incredibly useful.
Oct 14 2013
parent reply "Guillaume Chatelet" <chatelet.guillaume gmail.com> writes:
On Monday, 14 October 2013 at 20:06:52 UTC, Jeremy Powers wrote:

 Take a look at the way log4j does it, with logger hierarchy:
 http://logging.apache.org/log4j/1.2/manual.html

 This is incredibly useful.


It is useful but it's also a pain to configure. I used logback ( 
http://logback.qos.ch/ ) which is a bit nicer but still... We 
should definitely step away from XML's configuration mess.
Oct 14 2013
next sibling parent Jeremy Powers <jpowers wyrdtech.com> writes:
 It is useful but it's also a pain to configure. I used logback (
 http://logback.qos.ch/ ) which is a bit nicer but still... We should
 definitely step away from XML's configuration mess.


Yeah, there's a reason I linked to the old docs and not the newer
xml-centric stuff.  The important bit is to have a well-defined hierarchy
of loggers, and be able to configure each level as needed.  Indispensable
for any large software product.

Haven't actually used logback yet myself, but it provides a great set of
functionality to shoot for.
Oct 14 2013
prev sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 11:10 PM, Guillaume Chatelet wrote:

 It is useful but it's also a pain to configure. I used logback (
 http://logback.qos.ch/ ) which is a bit nicer but still... We should
 definitely step away from XML's configuration mess.


Did somebody say XML, I first cast a stone. ;-) But seriously, XML does
not come to my mind when I think a flexible KISS logging library.
Oct 14 2013
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.


Would be nice if either FileLogger, or a "proxy" logger available 
to users, would add timestamps. Timestamps are very useful for 
log files, as they allow to correlate logged events with other 
events on the system (e.g. other applications' log files or file 
modification times), so I think they should be there by default.

My personal preference of timestamp format is "[YYYY-MM-DD 
HH:MM:SS.FFF] " (though it needs to be either UTC or additionally 
specify the timezone, to disambiguate overlapping local time when 
DST ends).
Oct 14 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 02:42 PM, Vladimir Panteleev wrote:

 Would be nice if either FileLogger, or a "proxy" logger available to
 users, would add timestamps. Timestamps are very useful for log files,
 as they allow to correlate logged events with other events on the
 system (e.g. other applications' log files or file modification
 times), so I think they should be there by default.


Good point, a timestamp will be part of the future struct passed down to
logMessage.
Oct 14 2013
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
My own few comments from quick overview of the documentation 
(must admit I did not pay much attention to previous logger 
discussion threads).

=== critical ===

1)

I don't like forced verbosity of mentioning logging levels. In my 
opinion logging is one place where laconic syntax elegance really 
helps. Would have been nice to have `error(...)` as an alias to 
`log(LogLevel.Error, ...)` etc.

2)

I don't like that default `log` function outputs conditionally. 
It is rather unexpected and less frequent use case. I think 
current `logf` should become `log` and current `log` turn into 
`logIf` (with matching `errorIf`, `debugIf` etc.)

3)

Simple way to define multiple loggers as default ones is missing. 
As far as I understand, currently it can be done by creating own 
Logger which embeds multiple simple ones and registering it as 
default in LogManager. I think such common task should have 
built-in solution.

=== possible features? ===

One interesting addition would be to embed module name into 
logger call and provide facilities to filter output on per-module 
basis (similar to log level). D introspection tools should allow 
to do that with no observable API complication.
Oct 14 2013
parent "Dicebot" <public dicebot.lv> writes:
On Monday, 14 October 2013 at 12:45:06 UTC, Dicebot wrote:

 === possible features? ===

 One interesting addition would be to embed module name into 
 logger call and provide facilities to filter output on 
 per-module basis (similar to log level). D introspection tools 
 should allow to do that with no observable API complication.


As a random idea this can be possibly done by providing default 
ModuleLogger in a similar way to default global logger, so that 
one can easily use both from the module context (i.e. logging 
most errors globally but making rest suppressible by 
module-specific settings)
Oct 14 2013
prev sibling next sibling parent reply Martin Drasar <drasar ics.muni.cz> writes:
Having skimmed through the docs I noticed that there are three features
missing that I use and would like to see in standard logger. First is
the ability to write to several loggers at once, the second is optional
formatting of log output and the third is an option to tell logger to
log only one concrete log level.

All these can be accomplished by writing my own logger that would do
this, but it would be nice to have some batteries included, e.g.:

1) MultiLogger class that takes references to other loggers and just
forwards the call to the log function.

2) Optional string parameter that describes the desired log output (i.e.
format and position of timestamp, line, file, message, ...) for
different needs, e.g. machine vs. human read log.

2.1) Allow more than one string to be logged. For example I want to add
component name, task identifier, etc., but I want to let correct
formatting on the logger.

***
Example:

auto machineLogger = new FileLogger(format = "%ts;%f;%l;{1};{2};{3};");
auto humanLogger = new FileLogger(format = "[%ts] :: %f:%l\nComponent:
{1}\nTask:{2}\n{3});

machineLogger.log("Some component", "Some task", "Everything is well");

output =>
2013-10-14 14:28:05;file.d;54;Some component;Some task;Everything is well;

humanLogger.log("Some component", "Some task", "Everything is well");

output =>
[2013-10-14 14:28:05] :: file.d:54
Component: Some component
Task: Some task
Everything is well
***

Regards,
Martin
Oct 14 2013
parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 14 October 2013 at 12:48:14 UTC, Martin Drasar wrote:

 1) MultiLogger class that takes references to other loggers and 
 just
 forwards the call to the log function.


+1
Also, we should support a few loggers whith same type. For 
example, I can use 2 file loggers: the 1-st only for debug 
messages and the 2-nd for all other messages. It can help for 
message sorting.

Also, we should support a reserve loggers. For example, I can use 
a file logger as a default and a syslog as a reserve logger (it 
will be used if the file logger fails). It increases logger 
reliability.

Also, it will be perfect to have logger failed notifications. For 
example, the file logger failed can indicate that we haven't got 
free disc space, a file system problems or hard disk problems. 
So, we should inform admin about this problems. We can do it via 
stderr (for local computer only), via syslog network logger or 
via e-mail logger.
Oct 15 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 09:44 AM, ilya-stromberg wrote:

 On Monday, 14 October 2013 at 12:48:14 UTC, Martin Drasar wrote:

 1) MultiLogger class that takes references to other loggers and just
 forwards the call to the log function.


 +1
 Also, we should support a few loggers whith same type. For example, I
 can use 2 file loggers: the 1-st only for debug messages and the 2-nd
 for all other messages. It can help for message sorting.

 Also, we should support a reserve loggers. For example, I can use a
 file logger as a default and a syslog as a reserve logger (it will be
 used if the file logger fails). It increases logger reliability.

 Also, it will be perfect to have logger failed notifications. For
 example, the file logger failed can indicate that we haven't got free
 disc space, a file system problems or hard disk problems. So, we
 should inform admin about this problems. We can do it via stderr (for
 local computer only), via syslog network logger or via e-mail logger.


I think File will throw anyway. What if stderr is piped to file? Again,
I don't think you can please everybody's needs. So we should not try,
but rather provide the tools to help yourself.
Oct 15 2013
prev sibling next sibling parent reply "Andrea Fontana" <nospam example.com> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: 
 https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: 
 http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread : 
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


Just for comparison, on Android you can write something like:
FileLogger.w(...)  instead of FileLogger.log(LogLevel.Warning...)

(and there's a "wtf" loglevel for "temporary" debugging)
Oct 14 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 02:51 PM, Andrea Fontana wrote:

 Just for comparison, on Android you can write something like:
 FileLogger.w(...)  instead of FileLogger.log(LogLevel.Warning...)

 (and there's a "wtf" loglevel for "temporary" debugging)


hm, I don't really like it. To me it sounds like, "I'm don't care about
log level just print this". Maybe changing the LogLevel type to an int
like "debug(int) { " would be an idea and than providing some immutable
int Info = 1, Debug = 256, Error = 1024 ...  would be a idea.
Oct 14 2013
parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Monday, 14 October 2013 at 13:25:00 UTC, Robert Schadek wrote:

 On 10/14/2013 02:51 PM, Andrea Fontana wrote:

 Just for comparison, on Android you can write something like:
 FileLogger.w(...)  instead of 
 FileLogger.log(LogLevel.Warning...)

 (and there's a "wtf" loglevel for "temporary" debugging)


 hm, I don't really like it. To me it sounds like, "I'm don't 
 care about
 log level just print this". Maybe changing the LogLevel type to 
 an int
 like "debug(int) { " would be an idea and than providing some 
 immutable
 int Info = 1, Debug = 256, Error = 1024 ...  would be a idea.


IMHO, Andrea's suggestion is still a million times better.
And in any case, logger.info(...) is much better than 
logger.log(Logger.INFO, ...), which is uselessly redundant and 
quite ugly.
Oct 20 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 02:32 PM, Martin Drasar wrote:

 Having skimmed through the docs I noticed that there are three features
 missing that I use and would like to see in standard logger. First is
 the ability to write to several loggers at once, the second is optional
 formatting of log output and the third is an option to tell logger to
 log only one concrete log level.

 All these can be accomplished by writing my own logger that would do
 this, but it would be nice to have some batteries included, e.g.:

 1) MultiLogger class that takes references to other loggers and just
 forwards the call to the log function.


will be done, see the reply to Sönke's post.

 2) Optional string parameter that describes the desired log output (i.e.
 format and position of timestamp, line, file, message, ...) for
 different needs, e.g. machine vs. human read log.


I thought about that, but I'm not sure if that won't the logger to
complex.  IMO it is better to have some reasonable hard-coded default
than a log format parser. If the user needs something else, subclassing
a logger and changing the format is a 7 liner. And this is the design
Idea behind the logger.

 2.1) Allow more than one string to be logged. For example I want to add
 component name, task identifier, etc., but I want to let correct
 formatting on the logger.


logf("name %s, taskId %s", name, taskId); already works

Thanks, keep it coming
Oct 14 2013
prev sibling next sibling parent Martin Drasar <drasar ics.muni.cz> writes:
On 14.10.2013 15:18, Robert Schadek wrote:

 On 10/14/2013 02:32 PM, Martin Drasar wrote:

 1) MultiLogger class that takes references to other loggers and just
 forwards the call to the log function.


 will be done, see the reply to Sönke's post.


Cool


 2) Optional string parameter that describes the desired log output (i.e.
 format and position of timestamp, line, file, message, ...) for
 different needs, e.g. machine vs. human read log.


 I thought about that, but I'm not sure if that won't the logger to
 complex.  IMO it is better to have some reasonable hard-coded default
 than a log format parser. If the user needs something else, subclassing
 a logger and changing the format is a 7 liner. And this is the design
 Idea behind the logger.


True, it would definitely make the logger more complex. Default
implementation would just help developers that want this feature avoid
the pitfalls of string formatting (e.g. buffers vs. allocation as was
discussed in previous thread on std.logger).


 2.1) Allow more than one string to be logged. For example I want to add
 component name, task identifier, etc., but I want to let correct
 formatting on the logger.


 logf("name %s, taskId %s", name, taskId); already works


This was tightly coupled with the previous request, i.e. not using
string format and logging one string, but passing several strings and
let the target logger assemble them as it see fit.


 Thanks, keep it coming


How about being able to log only certain log level(s) and not only
greater or equal?

Martin
Oct 14 2013
prev sibling next sibling parent reply "Byron" <byron.heads gmail.com> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: 
 https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: 
 http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread : 
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


** System logging: syslog and windows event logging support.
* log rolling, resuming
* sink - source logging
- buffered(speed, network logging anyone?) and unbuffered(crash 
safe) logging
- log configuration via environment or arguments
- log once/every Nth
Oct 14 2013
parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 14 October 2013 at 13:39:10 UTC, Byron wrote:

 ** System logging: syslog and windows event logging support.


+1, add System logging, and use as default windows event logging 
for windows and syslog for POSIX.

And remote log support for syslog to allow send log messages via 
network.

Also, we should support syslog for windows, but in that case user 
must provide valid syslog daemon (install it for windows or 
spesify valid network connection).
Oct 14 2013
next sibling parent Jeremy Powers <jpowers wyrdtech.com> writes:
Some comments from the peanut gallery:

* Mentioned already: configurable log output.  Configure style/contents of
log output based on output location.  More options for included info
(thread id, etc).  Allow custom info to be inserted based on logger context.

* Also mentioned: Configurable log rotation for file logger.

* Pass an exception to logger directly, have it output in message depending
on configuration.  Pretty vs. short vs. all-one-line for automated log
parsing.

* Support multiple output locations.  Code calls log.whatever(), be able to
have that spit out to various places (stdout, syslog, file, etc) depending
on configuration. Be able to configure different logging levels for each
output.

* Should be able to change log levels via configuration, for debugging of
existing code (during dev or production).  That is, be able to change each
logger level without recompiling.

* The 'critical' level seem extraneous - when would you use this instead of
error or fatal?

* Easy way to create logger per module, inheriting settings from logger
hierarchy.  Associated way to manage loggers and their configuration.


Basically, things I've found useful after using log4j heavily.  Most of
them seem fancy and extraneous, until you are dealing with long-running
production software you have to fix at three in the morning...


On Mon, Oct 14, 2013 at 9:02 AM, ilya-stromberg <
ilya-stromberg-2009 yandex.ru> wrote:


 On Monday, 14 October 2013 at 13:39:10 UTC, Byron wrote:


 ** System logging: syslog and windows event logging support.


 +1, add System logging, and use as default windows event logging for
 windows and syslog for POSIX.

 And remote log support for syslog to allow send log messages via network.

 Also, we should support syslog for windows, but in that case user must
 provide valid syslog daemon (install it for windows or spesify valid
 network connection).
Oct 14 2013
prev sibling next sibling parent Jeremy Powers <jpowers wyrdtech.com> writes:
Also:

* Asynchronous logging.  Log something and continue, not blocking on actual
message getting written.  Can write a custom logger that works this way,
but would be nice if this was supported at the base level.


On Mon, Oct 14, 2013 at 10:01 AM, Jeremy Powers <jpowers wyrdtech.com>wrote:


 Some comments from the peanut gallery:

 * Mentioned already: configurable log output.  Configure style/contents of
 log output based on output location.  More options for included info
 (thread id, etc).  Allow custom info to be inserted based on logger context.

 * Also mentioned: Configurable log rotation for file logger.

 * Pass an exception to logger directly, have it output in message
 depending on configuration.  Pretty vs. short vs. all-one-line for
 automated log parsing.

 * Support multiple output locations.  Code calls log.whatever(), be able
 to have that spit out to various places (stdout, syslog, file, etc)
 depending on configuration. Be able to configure different logging levels
 for each output.

 * Should be able to change log levels via configuration, for debugging of
 existing code (during dev or production).  That is, be able to change each
 logger level without recompiling.

 * The 'critical' level seem extraneous - when would you use this instead
 of error or fatal?

 * Easy way to create logger per module, inheriting settings from logger
 hierarchy.  Associated way to manage loggers and their configuration.


 Basically, things I've found useful after using log4j heavily.  Most of
 them seem fancy and extraneous, until you are dealing with long-running
 production software you have to fix at three in the morning...


 On Mon, Oct 14, 2013 at 9:02 AM, ilya-stromberg <
 ilya-stromberg-2009 yandex.ru> wrote:


 On Monday, 14 October 2013 at 13:39:10 UTC, Byron wrote:


 ** System logging: syslog and windows event logging support.


 +1, add System logging, and use as default windows event logging for
 windows and syslog for POSIX.

 And remote log support for syslog to allow send log messages via network.

 Also, we should support syslog for windows, but in that case user must
 provide valid syslog daemon (install it for windows or spesify valid
 network connection).
Oct 14 2013
prev sibling next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 07:01 PM, Jeremy Powers wrote:

 Some comments from the peanut gallery:

 * Mentioned already: configurable log output.  Configure
 style/contents of log output based on output location.  More options
 for included info (thread id, etc).  Allow custom info to be inserted
 based on logger context.


I don't think that this is a good way to good, complexity and speed wise.

 * Also mentioned: Configurable log rotation for file logger.

 * Pass an exception to logger directly, have it output in message
 depending on configuration.  Pretty vs. short vs. all-one-line for
 automated log parsing.


I think format("%s", someException) already calls toString on
someException (I need to this.)

 * Support multiple output locations.  Code calls log.whatever(), be
 able to have that spit out to various places (stdout, syslog, file,
 etc) depending on configuration. Be able to configure different
 logging levels for each output.


MultiLogger will come.

 * Should be able to change log levels via configuration, for debugging
 of existing code (during dev or production).  That is, be able to
 change each logger level without recompiling.


Debug Level are runtime only, You can disable all logging via a version
switch

 * The 'critical' level seem extraneous - when would you use this
 instead of error or fatal?


The system has tripped bad. Maybe some config file is not there, but
some system env might to the trick.

 * Easy way to create logger per module, inheriting settings from
 logger hierarchy.  Associated way to manage loggers and their
 configuration.


module myModule;
Logger myModuleLogger;
static this() {
    myModuleLogger = new StdIOLogger();
}

 Basically, things I've found useful after using log4j heavily.  Most
 of them seem fancy and extraneous, until you are dealing with
 long-running production software you have to fix at three in the
 morning...
Oct 14 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 14 October 2013 at 21:22:51 UTC, Robert Schadek wrote:

 module myModule;
 Logger myModuleLogger;
 static this() {
     myModuleLogger = new StdIOLogger();
 }


I'd like to have module-specific logging _without_ creating local 
module instance. And without referring to specific logger at all. 
As I have mentioned, something like:
```
import log = std.logger;
log.local.info(...); // uses special default logger that inspects 
__MODULE__
log.global.error(...); // uses normal default logger

```
Oct 14 2013
parent Jeremy Powers <jpowers wyrdtech.com> writes:
On Mon, Oct 14, 2013 at 2:27 PM, Dicebot <public dicebot.lv> wrote:


 I'd like to have module-specific logging _without_ creating local module
 instance. And without referring to specific logger at all. As I have
 mentioned, something like:
 ```
 import log = std.logger;
 log.local.info(...); // uses special default logger that inspects
 __MODULE__
 log.global.error(...); // uses normal default logger

 ```


This idea sounds great (esp. if paired with hierarchical loggers I have
been talking about).  If I'm importing std.logger, chances are I am going
to create a logger for the module.
Would eliminate a bunch of boilerplate 'logger log =
logfactory.getMeALog(modulename)'
Oct 14 2013
prev sibling parent reply Jeremy Powers <jpowers wyrdtech.com> writes:
Comments on comments.  And to be clear, this is just me wanting everything
to be amazing - I'm incredibly glad to have any logging framework at all,
and will use what I can get.


 Allow custom info to be inserted based on logger context.


 don't think that this is a good way to good, complexity and speed wise.


Configurable/custom logging output is required for (most) complicated
systems.  Aside from having to fit into a pre-existing log format, think
logging a request id for every action that happens within the context of a
request.  You can manually plum in the info everywhere and pass it to the
logger, but it is much easier/cleaner to be able to set info on a logger
context.  For some cases this is the only reasonable option.
Like: http://logback.qos.ch/manual/mdc.html


 * Pass an exception to logger directly, have it output in message
 depending on configuration.  Pretty vs. short vs. all-one-line for
 automated log parsing.


 I think format("%s", someException) already calls toString on


someException (I need to this.)

I'm thinking of cases where the standard exception toString may not be
appropriate, like when you need every log statement to occur only on one
line for log parsing, perhaps with a shortened stack trace.  But still want
to be able to run the same code in development and be able to look at logs
without eyes bleeding.  It might be useful to include the API to take an
exception directly, even if it doesn't actually do anything special without
a custom logger implementation.
Like:
http://logback.qos.ch/apidocs/ch/qos/logback/classic/Logger.html#error(java.lang.String,
java.lang.Throwable)



 MultiLogger will come.


Excellent.
Obligatory logback link: http://logback.qos.ch/manual/appenders.html



 * The 'critical' level seem extraneous - when would you use this instead




of error or fatal?

 The system has tripped bad. Maybe some config file is not there, but


some system env might to the trick.

If you have encountered a problem, is an error.  If you can't recover, is a
fatal.  I can't think of a time when I would want to use critical where an
error or fatal wouldn't be appropriate... personally, I'd rather have an
explicit debug/trace level instead.


 * Easy way to create logger per module, inheriting settings from logger




hierarchy.

 Associated way to manage loggers and their configuration.


module myModule;
Logger myModuleLogger;
static this() {
    myModuleLogger = new StdIOLogger();
}


Yes, but.  Should be able to create a logger per module, and have it
automatically fit into log hierarchy for configuration.  Example:

module foo -> has logger foolog
module foo.bar -> has looger barlog, child of foolog

Want to be able to configure log levels on foolog and have barlog inherit.
 Useful for turning on/off log levels for my code but not changing log
output from code I pull from elsewhere.  I'd say this is one of the most
critical parts of an effective logging framework.  Making it easy/default
gains a lot.
Like: http://logback.qos.ch/manual/architecture.html


Also:
I like log.error(foo) better than log(error, foo).  And the shed should be
blue.
Oct 14 2013
parent "Brian Schott" <briancschott gmail.com> writes:
On Monday, 14 October 2013 at 22:04:44 UTC, Jeremy Powers wrote:

 And the shed should be blue.


I vote that we paint it octarine: a nice color that's not too 
purple and yet not too green and can only be created with magic. 
So far nobody has painted a bike shed to my satisfaction and I 
can't understand why.
Oct 14 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 03:31 PM, Martin Drasar wrote:

 On 14.10.2013 15:18, Robert Schadek wrote:

 On 10/14/2013 02:32 PM, Martin Drasar wrote:

 1) MultiLogger class that takes references to other loggers and just
 forwards the call to the log function.


 will be done, see the reply to Sönke's post.


 Cool


Cool

 2) Optional string parameter that describes the desired log output (i.e.
 format and position of timestamp, line, file, message, ...) for
 different needs, e.g. machine vs. human read log.


 I thought about that, but I'm not sure if that won't the logger to
 complex.  IMO it is better to have some reasonable hard-coded default
 than a log format parser. If the user needs something else, subclassing
 a logger and changing the format is a 7 liner. And this is the design
 Idea behind the logger.


 True, it would definitely make the logger more complex. Default
 implementation would just help developers that want this feature avoid
 the pitfalls of string formatting (e.g. buffers vs. allocation as was
 discussed in previous thread on std.logger).


Yes, but you have to lookup the formatting parameter, which adds some
complexity. It would also a time complexity for each logging call,
because you would have to parse the format. IMO KISS.

 2.1) Allow more than one string to be logged. For example I want to add
 component name, task identifier, etc., but I want to let correct
 formatting on the logger.


 logf("name %s, taskId %s", name, taskId); already works


 This was tightly coupled with the previous request, i.e. not using
 string format and logging one string, but passing several strings and
 let the target logger assemble them as it see fit.


This plays in my hands exactly, you have properly some idea of "see fit"
and I can't anticipate this and properly can't create such a flexible
configuration that makes it all see fit. So write you're own logger and
handle all strings our own. Just ignore the printf style formatted
string at the beginning or just call logf("", string1, string2, ...) and
mix string1 and friend as you see fit ;-)

 Thanks, keep it coming


 How about being able to log only certain log level(s) and not only
 greater or equal?

 Martin


Well, Bitmasked come to mind, but I really don't want to go C-Style and
I think that defeats the purpose of having levels.
Oct 14 2013
prev sibling next sibling parent Martin Drasar <drasar ics.muni.cz> writes:
On 14.10.2013 15:43, Robert Schadek wrote:

 On 10/14/2013 03:31 PM, Martin Drasar wrote:
 Yes, but you have to lookup the formatting parameter, which adds some
 complexity. It would also a time complexity for each logging call,
 because you would have to parse the format. IMO KISS.


Ok, let's have it simple.


 This was tightly coupled with the previous request, i.e. not using
 string format and logging one string, but passing several strings and
 let the target logger assemble them as it see fit.


 This plays in my hands exactly, you have properly some idea of "see fit"
 and I can't anticipate this and properly can't create such a flexible
 configuration that makes it all see fit. So write you're own logger and
 handle all strings our own. Just ignore the printf style formatted
 string at the beginning or just call logf("", string1, string2, ...) and
 mix string1 and friend as you see fit ;-)


Yup, you are right. I somehow overlooked that these logf functions are
best suited for what I want.


 How about being able to log only certain log level(s) and not only
 greater or equal?

 Martin


 Well, Bitmasked come to mind, but I really don't want to go C-Style and
 I think that defeats the purpose of having levels.


I am not sure how that defeats the purpose, but one way to go around
this is to have the logger log also the logLevel of each message and
then grep for what you need. What do you say about adding this
functionality?

Martin
Oct 14 2013
prev sibling next sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.


Add e-mail logger (useful for critical errors) for example via 
`std.net.curl.SMTP`.
Oct 14 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 05:27 PM, ilya-stromberg wrote:

 Add e-mail logger (useful for critical errors) for example via
 `std.net.curl.SMTP`.


And than at a email format config parser that fulfills everyones wishes,
would bet the idea of the design. And I though people where trying to
remove curl.
Oct 14 2013
parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 14 October 2013 at 18:00:12 UTC, Robert Schadek wrote:

 On 10/14/2013 05:27 PM, ilya-stromberg wrote:

 Add e-mail logger (useful for critical errors) for example via
 `std.net.curl.SMTP`.


 And than at a email format config parser that fulfills 
 everyones wishes,
 would bet the idea of the design. And I though people where 
 trying to
 remove curl.


I said: "for example via `std.net.curl.SMTP`". If you don't want 
to use curl, you shouldn't. But yes, I think that e-mail logger 
can be useful. For example Linux use it (and bash can send errors 
via e-mail).

About "email format config parser that fulfills everyones 
wishes": you shouldn't do it, a simple string with error message 
should be enough: we should inform admins about the problem, not 
provide a marketing e-mail.

If you disagree, please tell why.
Oct 14 2013
next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 08:29 PM, ilya-stromberg wrote:

 I said: "for example via `std.net.curl.SMTP`". If you don't want to
 use curl, you shouldn't. But yes, I think that e-mail logger can be
 useful. For example Linux use it (and bash can send errors via e-mail).

 About "email format config parser that fulfills everyones wishes": you
 shouldn't do it, a simple string with error message should be enough:
 we should inform admins about the problem, not provide a marketing
 e-mail.

 If you disagree, please tell why.


I disagree on having a simple email layer among the default logger,
because I feel that having this "special" logger in would water the
design of the logging module. Maybe you are happy with a simple string
message mail with fixed subject and sender, the next guy will not. And
he will be asking for it and (we || I ) have to tell him, why the simple
one is in and his version is not. I would rather have a minimal
std(io|err), file logger version which is pure and easy to mod.
Oct 14 2013
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Monday, 14 October 2013 at 20:49:19 UTC, Robert Schadek wrote:

 I disagree on having a simple email layer among the default  
 logger,
 because I feel that having this "special" logger in would water 
  the
 design of the logging module. Maybe you are happy with a simple 
  string
 message mail with fixed subject and sender, the next guy will  
 not. And
 he will be asking for it and (we || I ) have to tell him, why  
 the simple
 one is in and his version is not. I would rather have a minimal
 std(io|err), file logger version which is pure and easy to mod.


Yes.  The really important thing about getting tools like this 
into the standard library is so libraries can build on the same 
framework that a user application is likely to use.  So now 
instead of hooking differently formatted callbacks into every D 
library I use so I can get log output, I can rely on them logging 
directly to the standard log device.

Different projects will have different specialized logger 
requirements, so trying to find a common back-end seems like a 
largely pointless effort anyway.  Though specialized loggers 
might be a good Dub project...
Oct 14 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 11:01 PM, Sean Kelly wrote:

 On Monday, 14 October 2013 at 20:49:19 UTC, Robert Schadek wrote:

 I disagree on having a simple email layer among the default  logger,
 because I feel that having this "special" logger in would water  the
 design of the logging module. Maybe you are happy with a simple  string
 message mail with fixed subject and sender, the next guy will  not. And
 he will be asking for it and (we || I ) have to tell him, why  the
 simple
 one is in and his version is not. I would rather have a minimal
 std(io|err), file logger version which is pure and easy to mod.


 Yes.  The really important thing about getting tools like this into
 the standard library is so libraries can build on the same framework
 that a user application is likely to use.  So now instead of hooking
 differently formatted callbacks into every D library I use so I can
 get log output, I can rely on them logging directly to the standard
 log device.

 Different projects will have different specialized logger
 requirements, so trying to find a common back-end seems like a largely
 pointless effort anyway.  Though specialized loggers might be a good
 Dub project...


I think you got, at least, my point ;-)

The Dub part is a good at least from my perspective
Oct 14 2013
prev sibling parent "ponce" <contact gam3sfrommars.fr> writes:
On Monday, 14 October 2013 at 18:29:09 UTC, ilya-stromberg wrote:

 On Monday, 14 October 2013 at 18:00:12 UTC, Robert Schadek 
 wrote:

 If you disagree, please tell why.


If you want a logger with a particular feature, this module will 
allow to create a custom logger.
It would be a mistake to include something that specific.

eg: - we can create new RNG on top of std.random, which follow 
the same interface
     - we can use any such std.random-compatible RNG to feed 
random distributions algorithms
Oct 15 2013
prev sibling next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
It's weird that LogManager.defaultLogger exists as a read-only 
property, and if I want to change it I have to use the global 
static "log" variable.

I think this was mentioned elsewhere, but is the global logger 
thread-local, shared, or __gshared?  Or is this something we can 
control?  I can see maybe wanting to have a separate logger per 
thread in some cases, for example.

I kind of wanted channels per Boost::Log, but simply creating 
multiple loggers that I can reference by name seems reasonable.  
Channels add a ton of complexity anyway.

We really need timestamps, and the timestamp format should be 
configurable.  One thing I like about Boost::Log is that I can 
statically specify the format of log lines.  This would be 
particularly useful when trying to integrate a D app into a setup 
that already expects a particular log line format.

This is an implementation detail, but I imagine the buffer these 
lines are written into is a static buffer that grows as needed 
and is just reused for each log line written?  I understand why a 
custom logger would receive only the generated log string, but if 
this is allocating unnecessarily in the background it could be a 
problem.  Maybe we could get some way to specify the initial size 
of this internal buffer as well, or limit the total buffer size 
to some value?  In some cases, you don't want to log more than N 
bytes per entry even if it means truncating the message.

Formatted logging should be the default.  I almost never want to 
just long a string with no parameters.

I'm not entirely happy with the conditional just sitting in the 
parameter list.  Maybe if the command were "logIf" instead of 
just "log" or there were some way to mark up the conditional in 
code as that it determines whether the log line is written...  I 
think this goes with my general aversion to using magic numbers 
as function parameters.  I either like the function call to read 
line a sentence or if that isn't reasonable, for there to be some 
way to indicate what a particular parameter is for.

For what it's worth, I think the conditional is a reasonable 
substitute for not having the LogLevel be a bitmask.

For the rest, I think the current API is sufficient to make most 
use cases work.  I might sometimes create a proxy logger than 
fans out to multiple discrete loggers underneath, for example.  
But that seems straightforward.
Oct 14 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 08:55 PM, Sean Kelly wrote:

 It's weird that LogManager.defaultLogger exists as a read-only
 property, and if I want to change it I have to use the global static
 "log" variable.


this will be properly be fixed in the next incarnation (tonight, no
promises)

 I think this was mentioned elsewhere, but is the global logger
 thread-local, shared, or __gshared?  Or is this something we can
 control?  I can see maybe wanting to have a separate logger per thread
 in some cases, for example.


currently all are thread local. the default logger will properly be
gshared. the others, I wish I could predict the shared discussion outcome.

 I kind of wanted channels per Boost::Log, but simply creating multiple
 loggers that I can reference by name seems reasonable.  Channels add a
 ton of complexity anyway.

 We really need timestamps, and the timestamp format should be
 configurable.  One thing I like about Boost::Log is that I can
 statically specify the format of log lines.  This would be
 particularly useful when trying to integrate a D app into a setup that
 already expects a particular log line format.


timestamps will be in

 This is an implementation detail, but I imagine the buffer these lines
 are written into is a static buffer that grows as needed and is just
 reused for each log line written?  I understand why a custom logger
 would receive only the generated log string, but if this is allocating
 unnecessarily in the background it could be a problem.  Maybe we could
 get some way to specify the initial size of this internal buffer as
 well, or limit the total buffer size to some value?  In some cases,
 you don't want to log more than N bytes per entry even if it means
 truncating the message.

 Formatted logging should be the default.  I almost never want to just
 long a string with no parameters.

 I'm not entirely happy with the conditional just sitting in the
 parameter list.  Maybe if the command were "logIf" instead of just
 "log" or there were some way to mark up the conditional in code as
 that it determines whether the log line is written...  I think this
 goes with my general aversion to using magic numbers as function
 parameters.  I either like the function call to read line a sentence
 or if that isn't reasonable, for there to be some way to indicate what
 a particular parameter is for.


I'm not sure what is the best, all fitting way here.

 For what it's worth, I think the conditional is a reasonable
 substitute for not having the LogLevel be a bitmask.

 For the rest, I think the current API is sufficient to make most use
 cases work.  I might sometimes create a proxy logger than fans out to
 multiple discrete loggers underneath, for example.  But that seems
 straightforward.
Oct 14 2013
prev sibling next sibling parent reply "ponce" <contaact g3mesfrommars.fr> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.

 Lets unleash the forces of constructive destruction.


Thanks for taking care of that.
My logger class logs warning in yellow and errors in red ;),
but honestly I will take _anything_ since the lack of std.logger 
is a big problem right now for library writers.
I cannot write a nice wrapper for a C library without forcing my 
pet logger implementation [that no one wants].
Oct 14 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/14/2013 09:59 PM, ponce wrote:

 Thanks for taking care of that.
 My logger class logs warning in yellow and errors in red ;),


colors are awesome I now, but hard to do across all shells and even
worse when you want to pipe the messages in a file afterwards. So I
suggest you implement your own custom logger. Btw. thanks for good
example, that people have differend ideas of what a logger should do.

 but honestly I will take _anything_ since the lack of std.logger is a
 big problem right now for library writers.
 I cannot write a nice wrapper for a C library without forcing my pet
 logger implementation [that no one wants].


my motivation
Oct 14 2013
prev sibling next sibling parent reply "Kapps" <opantm2+spam gmail.com> writes:
A few concerns:

There doesn't seem to be a debug or trace log level. This is 
quite a useful thing to have once your program is deployed.

I don't like the returning by ref for log methods. For example, 
it seems like you can do: log(5 < 4, "This is a test log") = new 
StdIOLogger();
It could potentially be useful to return the Logger so you can 
chain calls and such, but it should not be possible to set the 
logger by assigning the result of a log call.

The simple act of logging a message is very verbose right now: 
log(LogLevel.trace, "Creating new pool") is a lot of boiler 
plate. I'd prefer something like log.trace("Creating new pool") 
and log("Creating new pool") where the latter would use opCall to 
forward to the default log level. If it's intentional that you 
can assign the result of log, this also helps that because log = 
new StdIOLogger would be possible (log being a property that 
returns a Logger, and so a setter could be made), but 
log("Creating new pool") = new StdIOLogger() would not be.


There's a lot of suggestions for things like network logging, but 
I think this is far beyond the scope of the module, and the sheer 
amount of different ways of doing it means people will likely 
write their own anyways. For example, synchronous vs 
asynchronous, what protocol to use, authentication, client data, 
encryption, etc.
Oct 14 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 02:44 AM, Kapps wrote:

 A few concerns:

 There doesn't seem to be a debug or trace log level. This is quite a
 useful thing to have once your program is deployed.


there is a LogLevel.debug and a LogLevel.info

 I don't like the returning by ref for log methods. For example, it
 seems like you can do: log(5 < 4, "This is a test log") = new
 StdIOLogger();
 It could potentially be useful to return the Logger so you can chain
 calls and such, but it should not be possible to set the logger by
 assigning the result of a log call.


I saw this, but this comes from the way you get the default logger. I
don't think that this is that bad, but I bet somebody will disagree.

 The simple act of logging a message is very verbose right now:
 log(LogLevel.trace, "Creating new pool") is a lot of boiler plate. I'd
 prefer something like log.trace("Creating new pool") and log("Creating
 new pool") where the latter would use opCall to forward to the default
 log level. If it's intentional that you can assign the result of log,
 this also helps that because log = new StdIOLogger would be possible
 (log being a property that returns a Logger, and so a setter could be
 made), but log("Creating new pool") = new StdIOLogger() would not be.


The LogLevel is optional. And always writing log.trace might become more
typing work and assigning a LogLevel and than calling log("..."). Both
have pros and cons

 There's a lot of suggestions for things like network logging, but I
 think this is far beyond the scope of the module, and the sheer amount
 of different ways of doing it means people will likely write their own
 anyways. For example, synchronous vs asynchronous, what protocol to
 use, authentication, client data, encryption, etc.


my point exactly
Oct 15 2013
parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 10:41, schrieb Robert Schadek:

 On 10/15/2013 02:44 AM, Kapps wrote:

 The simple act of logging a message is very verbose right now:
 log(LogLevel.trace, "Creating new pool") is a lot of boiler plate. I'd
 prefer something like log.trace("Creating new pool") and log("Creating
 new pool") where the latter would use opCall to forward to the default
 log level. If it's intentional that you can assign the result of log,
 this also helps that because log = new StdIOLogger would be possible
 (log being a property that returns a Logger, and so a setter could be
 made), but log("Creating new pool") = new StdIOLogger() would not be.


 The LogLevel is optional. And always writing log.trace might become more
 typing work and assigning a LogLevel and than calling log("..."). Both
 have pros and cons


What happens when a called function alters the default log level?

---
void func1() {
	log.logLevel = LogLevel.debug;
	log("This is a debug message");
	func2();
	log("This is supposed to be a debug message");
}

void func2() {
	log.logLevel = LogLevel.warning;
	log("This is a warning");
}
---


I don't think it's a good idea to use such kind of global state, 
especially for a logging framework that is supposed to be shared between 
libraries, so that it is difficult to predict what a particular function 
does. With a logger that is shared between threads, things get worse of 
course.

A related question: It seems like Logger.logLevel at the same time means 
the minimum log level that is output and the default log level, is that 
right?
Oct 15 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 03:21 PM, Sönke Ludwig wrote:

 Am 15.10.2013 10:41, schrieb Robert Schadek:

 On 10/15/2013 02:44 AM, Kapps wrote:

 The simple act of logging a message is very verbose right now:
 log(LogLevel.trace, "Creating new pool") is a lot of boiler plate. I'd
 prefer something like log.trace("Creating new pool") and log("Creating
 new pool") where the latter would use opCall to forward to the default
 log level. If it's intentional that you can assign the result of log,
 this also helps that because log = new StdIOLogger would be possible
 (log being a property that returns a Logger, and so a setter could be
 made), but log("Creating new pool") = new StdIOLogger() would not be.


 The LogLevel is optional. And always writing log.trace might become more
 typing work and assigning a LogLevel and than calling log("..."). Both
 have pros and cons


 What happens when a called function alters the default log level?


The default log level is altered.

 ---
 void func1() {
     log.logLevel = LogLevel.debug;
     log("This is a debug message");
     func2();
     log("This is supposed to be a debug message");
 }

 void func2() {
     log.logLevel = LogLevel.warning;
     log("This is a warning");
 }
 ---


If you don't specify a logger nor a LogLevel the currently set default
logger will log the message with its currently set LogLevel.

 I don't think it's a good idea to use such kind of global state,
 especially for a logging framework that is supposed to be shared
 between libraries, so that it is difficult to predict what a
 particular function does. With a logger that is shared between
 threads, things get worse of course.


I think this is good, as it gives you a way to quite libraries down. The
idea behind the free standing "log" function is to provide an ultra easy
way to log. It is not meant to be used for the 2<<31 line program. In
that case you will properly have very specific needs on how to log.
Hence implement the abstract Logger class to your needs.

 A related question: It seems like Logger.logLevel at the same time
 means the minimum log level that is output and the default log level,
 is that right?


Yes. The LogLevel of each logger is the LogLevel used if non is
specified and only messages are logged by this logger if their LogLevel
is greater equal to that Level. Additionally the LogLevel must be >= to
the global LogLevel.
Oct 15 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 October 2013 at 13:52:17 UTC, Robert Schadek wrote:

 I think this is good, as it gives you a way to quite libraries 
 down. The
 idea behind the free standing "log" function is to provide an 
 ultra easy
 way to log. It is not meant to be used for the 2<<31 line 
 program. In
 that case you will properly have very specific needs on how to 
 log.
 Hence implement the abstract Logger class to your needs.


I'll consider any logging library that forces me to use logger 
instance explicitly for typical tasks a failure. Once the system 
is configured upon program startup, using free functions and/or 
system-wide defaults should be enough.
Oct 15 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 03:57 PM, Dicebot wrote:

 On Tuesday, 15 October 2013 at 13:52:17 UTC, Robert Schadek wrote:

 I think this is good, as it gives you a way to quite libraries down. The
 idea behind the free standing "log" function is to provide an ultra easy
 way to log. It is not meant to be used for the 2<<31 line program. In
 that case you will properly have very specific needs on how to log.
 Hence implement the abstract Logger class to your needs.


 I'll consider any logging library that forces me to use logger
 instance explicitly for typical tasks a failure. Once the system is
 configured upon program startup, using free functions and/or
 system-wide defaults should be enough.


I don't think that there are enough compiler generated marked that you
can pass as default parameter to a free standing function in a way that
you can create a logger that fulfills this need. At some point you
properly have to write a string of and identifier to specify a logger.
Oct 15 2013
prev sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 15:52, schrieb Robert Schadek:

 On 10/15/2013 03:21 PM, Sönke Ludwig wrote:

 Am 15.10.2013 10:41, schrieb Robert Schadek:

 On 10/15/2013 02:44 AM, Kapps wrote:

 The simple act of logging a message is very verbose right now:
 log(LogLevel.trace, "Creating new pool") is a lot of boiler plate. I'd
 prefer something like log.trace("Creating new pool") and log("Creating
 new pool") where the latter would use opCall to forward to the default
 log level. If it's intentional that you can assign the result of log,
 this also helps that because log = new StdIOLogger would be possible
 (log being a property that returns a Logger, and so a setter could be
 made), but log("Creating new pool") = new StdIOLogger() would not be.


 The LogLevel is optional. And always writing log.trace might become more
 typing work and assigning a LogLevel and than calling log("..."). Both
 have pros and cons


 What happens when a called function alters the default log level?


 The default log level is altered.


Believe it or not, for some reason I suspected as much.


 ---
 void func1() {
      log.logLevel = LogLevel.debug;
      log("This is a debug message");
      func2();
      log("This is supposed to be a debug message");
 }

 void func2() {
      log.logLevel = LogLevel.warning;
      log("This is a warning");
 }
 ---


 If you don't specify a logger nor a LogLevel the currently set default
 logger will log the message with its currently set LogLevel.


Yes, but the point is that when looking only at func1, you might expect 
that all messages are logged as debug messages, but the last one will be 
logged as a warning instead. func2 may be hidden in library where the 
function body is not readily available.


 I don't think it's a good idea to use such kind of global state,
 especially for a logging framework that is supposed to be shared
 between libraries, so that it is difficult to predict what a
 particular function does. With a logger that is shared between
 threads, things get worse of course.


 I think this is good, as it gives you a way to quite libraries down. The
 idea behind the free standing "log" function is to provide an ultra easy
 way to log. It is not meant to be used for the 2<<31 line program. In
 that case you will properly have very specific needs on how to log.
 Hence implement the abstract Logger class to your needs.


But if it's available people _will_ use it in complex contexts. Also if 
the writer of a 2<<8 loc library uses it and the library is used by a 
large piece of software, that will also be affected. The point is that 
it is unhygienic and requires non-trivial extra work when using a logger 
in a multi-threaded environment. Some kind of scoped stack of default 
log levels would get around this issue, but that smells like over 
engineering.


 A related question: It seems like Logger.logLevel at the same time
 means the minimum log level that is output and the default log level,
 is that right?


 Yes. The LogLevel of each logger is the LogLevel used if non is
 specified and only messages are logged by this logger if their LogLevel
 is greater equal to that Level. Additionally the LogLevel must be >= to
 the global LogLevel.


But these are two different concepts and it's hard for me to imagine why 
they should be conflated. But I guess it's time to stop complaining 
about the whole log("message") complex.
Oct 15 2013
next sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 14:12:38 UTC, Sönke Ludwig wrote:

 Am 15.10.2013 15:52, schrieb Robert Schadek:

 On 10/15/2013 03:21 PM, Sönke Ludwig wrote:

 Am 15.10.2013 10:41, schrieb Robert Schadek:

 On 10/15/2013 02:44 AM, Kapps wrote:

 The simple act of logging a message is very verbose right 
 now:
 log(LogLevel.trace, "Creating new pool") is a lot of boiler 
 plate. I'd
 prefer something like log.trace("Creating new pool") and 
 log("Creating
 new pool") where the latter would use opCall to forward to 
 the default
 log level. If it's intentional that you can assign the 
 result of log,
 this also helps that because log = new StdIOLogger would be 
 possible
 (log being a property that returns a Logger, and so a 
 setter could be
 made), but log("Creating new pool") = new StdIOLogger() 
 would not be.


 The LogLevel is optional. And always writing log.trace might 
 become more
 typing work and assigning a LogLevel and than calling 
 log("..."). Both
 have pros and cons


 What happens when a called function alters the default log 
 level?


 The default log level is altered.


 Believe it or not, for some reason I suspected as much.


 ---
 void func1() {
     log.logLevel = LogLevel.debug;
     log("This is a debug message");
     func2();
     log("This is supposed to be a debug message");
 }

 void func2() {
     log.logLevel = LogLevel.warning;
     log("This is a warning");
 }
 ---


 If you don't specify a logger nor a LogLevel the currently set 
 default
 logger will log the message with its currently set LogLevel.


 Yes, but the point is that when looking only at func1, you 
 might expect that all messages are logged as debug messages, 
 but the last one will be logged as a warning instead. func2 may 
 be hidden in library where the function body is not readily 
 available.


 I don't think it's a good idea to use such kind of global 
 state,
 especially for a logging framework that is supposed to be 
 shared
 between libraries, so that it is difficult to predict what a
 particular function does. With a logger that is shared between
 threads, things get worse of course.


 I think this is good, as it gives you a way to quite libraries 
 down. The
 idea behind the free standing "log" function is to provide an 
 ultra easy
 way to log. It is not meant to be used for the 2<<31 line 
 program. In
 that case you will properly have very specific needs on how to 
 log.
 Hence implement the abstract Logger class to your needs.


 But if it's available people _will_ use it in complex contexts. 
 Also if the writer of a 2<<8 loc library uses it and the 
 library is used by a large piece of software, that will also be 
 affected. The point is that it is unhygienic and requires 
 non-trivial extra work when using a logger in a multi-threaded 
 environment. Some kind of scoped stack of default log levels 
 would get around this issue, but that smells like over 
 engineering.


+1
I dislike syntax:
log.logLevel = LogLevel.warning;
log("This is a warning");

Much better:
log.warning("This is a warning");
Oct 15 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 04:17 PM, ilya-stromberg wrote:

 On Tuesday, 15 October 2013 at 14:12:38 UTC, Sönke Ludwig wrote:

 But if it's available people _will_ use it in complex contexts. Also
 if the writer of a 2<<8 loc library uses it and the library is used
 by a large piece of software, that will also be affected. The point
 is that it is unhygienic and requires non-trivial extra work when
 using a logger in a multi-threaded environment. Some kind of scoped
 stack of default log levels would get around this issue, but that
 smells like over engineering.


 +1
 I dislike syntax:
 log.logLevel = LogLevel.warning;
 log("This is a warning");

 Much better:
 log.warning("This is a warning");


You can also write log(LogLevel.warning, "This is a warning"); And that
also allows you to treat the LogLevel as a variable.
log(myComputedLogLevel, "..."); Anyway, functions should be verbs.
Oct 15 2013
parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 14:25:55 UTC, Robert Schadek wrote:

 On 10/15/2013 04:17 PM, ilya-stromberg wrote:

 On Tuesday, 15 October 2013 at 14:12:38 UTC, Sönke Ludwig 
 wrote:

 But if it's available people _will_ use it in complex 
 contexts. Also
 if the writer of a 2<<8 loc library uses it and the library 
 is used
 by a large piece of software, that will also be affected. The 
 point
 is that it is unhygienic and requires non-trivial extra work 
 when
 using a logger in a multi-threaded environment. Some kind of 
 scoped
 stack of default log levels would get around this issue, but 
 that
 smells like over engineering.


 +1
 I dislike syntax:
 log.logLevel = LogLevel.warning;
 log("This is a warning");

 Much better:
 log.warning("This is a warning");


 You can also write log(LogLevel.warning, "This is a warning"); 
 And that
 also allows you to treat the LogLevel as a variable.
 log(myComputedLogLevel, "..."); Anyway, functions should be 
 verbs.


Yes, I know.
Idea: let all "as is" and just add `log.xxx("...");` functions. 
It looks like that solves all problems:
- we can log with configurable default log level (setup for all 
programm):
log("This is a warning");
- we can specify local log level (setup for class or function):
log(myComputedLogLevel, "...");
- and we can call log with specific log level:
log.warning("...");
or maybe
log.logWarning("..."); //yes, it is a verb
Oct 15 2013
prev sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 04:12 PM, Sönke Ludwig wrote:

 Believe it or not, for some reason I suspected as much.

 Yes, but the point is that when looking only at func1, you might
 expect that all messages are logged as debug messages, but the last
 one will be logged as a warning instead. func2 may be hidden in
 library where the function body is not readily available.


Logging is the most unpure functionality I can think of. It is side
effect heaven.

 But if it's available people _will_ use it in complex contexts. Also
 if the writer of a 2<<8 loc library uses it and the library is used by
 a large piece of software, that will also be affected. The point is
 that it is unhygienic and requires non-trivial extra work when using a
 logger in a multi-threaded environment. Some kind of scoped stack of
 default log levels would get around this issue, but that smells like
 over engineering.


 But these are two different concepts and it's hard for me to imagine
 why they should be conflated. But I guess it's time to stop
 complaining about the whole log("message") complex.


...
Oct 15 2013
parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 14:20:15 UTC, Robert Schadek wrote:

 On 10/15/2013 04:12 PM, Sönke Ludwig wrote:

 Believe it or not, for some reason I suspected as much.

 Yes, but the point is that when looking only at func1, you 
 might
 expect that all messages are logged as debug messages, but the 
 last
 one will be logged as a warning instead. func2 may be hidden in
 library where the function body is not readily available.


 Logging is the most unpure functionality I can think of. It is 
 side
 effect heaven.


Yes, but we should minimise possible side effects.
Oct 15 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 04:23 PM, ilya-stromberg wrote:

 On Tuesday, 15 October 2013 at 14:20:15 UTC, Robert Schadek wrote:

 Logging is the most unpure functionality I can think of. It is side
 effect heaven.


 Yes, but we should minimise possible side effects.


Of course, but having global state aka. a global default logger and no
side effect are opposing design goals.
Oct 15 2013
parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 16:33, schrieb Robert Schadek:

 On 10/15/2013 04:23 PM, ilya-stromberg wrote:

 On Tuesday, 15 October 2013 at 14:20:15 UTC, Robert Schadek wrote:

 Logging is the most unpure functionality I can think of. It is side
 effect heaven.


 Yes, but we should minimise possible side effects.


 Of course, but having global state aka. a global default logger and no
 side effect are opposing design goals.


"minimise" != "eliminate"

The global default logger will usually be a set-once thing, so it's far 
less problematic.
Oct 15 2013
prev sibling next sibling parent reply "Eric Anderton" <eric.t.anderton gmail.com> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 Lets unleash the forces of constructive destruction.


So, not to be too heavy-handed with criticism on this library, 
but I think this should come up to par with solutions like log4j, 
log4cpp, or log4cxx, with respect to features and capabilities.  
Libraries like these have enjoyed a lot of very serious use, and 
once you have something like that in your project, it's hard to 
not use most of what they have to offer.  There's really not a 
lot of fluff in those solutions.

Here's what I think is missing:
- System log support (as others have mentioned).  This would be 
syslog or WEL, depending on environment.

- Guarantees or options for working with log rotation 
(logrotate.d).  It's nice to either know that you must restart 
your daemon once logs are rotated, or can configure logging to 
re-open handles automatically or when it detects rotation has 
occurred.

- Guarantees about threading and thread safety, with concessions 
to help keep log event streams coherent from thread to thread.  
Log formatting in particular could enjoy the ability to emit a 
thread id, so logs can be analyzed without confusing which thread 
is responsible for which chain of events.

Here's what I think would make this an amazing library:
- Nested Diagnostic Context (NDC) support.  This isn't heavily 
used in all projects, but it does a fantastic job of cutting down 
on the tendency to put tons of redundant information into every 
call to log().  In practice, this helps tremendously for 
debugging, as engineers stop pulling punches as adding rich 
contextual data to log lines becomes painless.

- Log "category" support.  Just some way to add an axis for 
filtering, so you can configure logging to block all log messages 
from one library, or just errors from another, at the same time. 
Under log4j, this is simply the module where the log event 
originates from - other libs let you use an arbitrary string.

- Filtering log events on another axis.  Loggers can already be 
configured with a log level.  But it would be nice to be able to 
set a global log level to dial in how much information comes out 
of the system across all logger instances.

That said, I do appreciate the compactness of this library.  It 
provides some very straightforward logging support and covers all 
the basic and important use cases.  But after using more 
feature-rich solutions, I can't help but think of all the places 
that I would feel inclined to go with a stronger solution, or 
extend this to do more of the kinds of things I'm used to doing 
elsewhere.  I think D has a chance to make an implementation of 
something on   par with log4j or log4cxx, easy to do, without 
needing anywhere near as much code.

- Eric
Oct 14 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 04:06 AM, Eric Anderton wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 Lets unleash the forces of constructive destruction.


 So, not to be too heavy-handed with criticism on this library, but I
 think this should come up to par with solutions like log4j, log4cpp,
 or log4cxx, with respect to features and capabilities.  Libraries like
 these have enjoyed a lot of very serious use, and once you have
 something like that in your project, it's hard to not use most of what
 they have to offer.  There's really not a lot of fluff in those
 solutions.


IMO these libraries are to heavy. Especially with phobos inclusion in mind.

 Here's what I think is missing:
 - System log support (as others have mentioned).  This would be syslog
 or WEL, depending on environment.


This is sort of the idea of the design, I can't anticipate your needs
therefor I should not try. I should try to give you guidelines or a
framework to work against.

 - Guarantees or options for working with log rotation (logrotate.d). 
 It's nice to either know that you must restart your daemon once logs
 are rotated, or can configure logging to re-open handles automatically
 or when it detects rotation has occurred.


See previous point

 - Guarantees about threading and thread safety, with concessions to
 help keep log event streams coherent from thread to thread.  Log
 formatting in particular could enjoy the ability to emit a thread id,
 so logs can be analyzed without confusing which thread is responsible
 for which chain of events.


Passing a thread id with a log message, ok! shared!?

 Here's what I think would make this an amazing library:
 - Nested Diagnostic Context (NDC) support.  This isn't heavily used in
 all projects, but it does a fantastic job of cutting down on the
 tendency to put tons of redundant information into every call to
 log().  In practice, this helps tremendously for debugging, as
 engineers stop pulling punches as adding rich contextual data to log
 lines becomes painless.


See previous point

 - Log "category" support.  Just some way to add an axis for filtering,
 so you can configure logging to block all log messages from one
 library, or just errors from another, at the same time. Under log4j,
 this is simply the module where the log event originates from - other
 libs let you use an arbitrary string.


at one point the logger had names, but I thought on how to get the
correct names to them. This lead to some config file and I don't want that.

 - Filtering log events on another axis.  Loggers can already be
 configured with a log level.  But it would be nice to be able to set a
 global log level to dial in how much information comes out of the
 system across all logger instances.


there already is a global log level

 That said, I do appreciate the compactness of this library.  It
 provides some very straightforward logging support and covers all the
 basic and important use cases.  But after using more feature-rich
 solutions, I can't help but think of all the places that I would feel
 inclined to go with a stronger solution, or extend this to do more of
 the kinds of things I'm used to doing elsewhere.  I think D has a
 chance to make an implementation of something on   par with log4j or
 log4cxx, easy to do, without needing anywhere near as much code.

 - Eric
Oct 15 2013
next sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 07:52:28 UTC, Robert Schadek wrote:

 On 10/15/2013 04:06 AM, Eric Anderton wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:
 Here's what I think is missing:
 - System log support (as others have mentioned).  This would 
 be syslog
 or WEL, depending on environment.


 This is sort of the idea of the design, I can't anticipate your 
 needs
 therefor I should not try. I should try to give you guidelines 
 or a
 framework to work against.


Totally disagree. We need a powerful logger, not only file logger.
I can implement a file logger myself for a few hours, and it will 
cover 90% of my needs. For other 10% I would like to have a 
standart logger with advanced features like speed and reliability.

If you need help, please tell us. For example, jkm already 
implemented syslog for Vibe.d with support files (via file 
streams) and the network (via TCP or SSL streams):
https://github.com/rejectedsoftware/vibe.d/pull/294
It's under the MIT license, that similar to the Boost license.

Sönke Ludwig, can you allow to use syslog code for `std.logger` 
under the Boost license? Robert Schadek can use it as initial 
point.
Oct 15 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 October 2013 at 13:31:40 UTC, ilya-stromberg wrote:

 ...


I think such stuff should go as an extra module in same package 
with various useful out-of-the box logger implementations at the 
very best. Probably even dub package built on top of std.logger;

Phobos has very specific general design I like a lot - 
self-contained modules/packages with zero external dependencies 
(please kill std.net.curl!), which do work by simply importing 
certain modules. No extra configuration, no creation of weird 
useless classes - just reasonable defaults that work as-is in 
most cases.

In that sense what is 100% needed is enhancing current API so 
that it may allow more fine grained tweaking of loggers (addition 
of module info, providing built-in multiplexing logger). There 
should be no temptation to build own stuff with own API because 
you can't write own logger that fits standard one.

But actual batteries - no, this does belong to Phobos.
Oct 15 2013
next sibling parent "Dicebot" <public dicebot.lv> writes:
 But actual batteries - no, this does belong to Phobos.


* does not belong
Oct 15 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 03:54 PM, Dicebot wrote:

 On Tuesday, 15 October 2013 at 13:31:40 UTC, ilya-stromberg wrote:

 ...


 I think such stuff should go as an extra module in same package with
 various useful out-of-the box logger implementations at the very best.
 Probably even dub package built on top of std.logger;

 Phobos has very specific general design I like a lot - self-contained
 modules/packages with zero external dependencies (please kill
 std.net.curl!), which do work by simply importing certain modules. No
 extra configuration, no creation of weird useless classes - just
 reasonable defaults that work as-is in most cases.

 In that sense what is 100% needed is enhancing current API so that it
 may allow more fine grained tweaking of loggers (addition of module
 info, providing built-in multiplexing logger). There should be no
 temptation to build own stuff with own API because you can't write own
 logger that fits standard one.

 But actual batteries - no, this does belong to Phobos.


+1
Oct 15 2013
prev sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 13:54:12 UTC, Dicebot wrote:

 On Tuesday, 15 October 2013 at 13:31:40 UTC, ilya-stromberg 
 wrote:

 ...


 I think such stuff should go as an extra module in same package 
 with various useful out-of-the box logger implementations at 
 the very best. Probably even dub package built on top of 
 std.logger;

 Phobos has very specific general design I like a lot - 
 self-contained modules/packages with zero external dependencies 
 (please kill std.net.curl!), which do work by simply importing 
 certain modules. No extra configuration, no creation of weird 
 useless classes - just reasonable defaults that work as-is in 
 most cases.

 In that sense what is 100% needed is enhancing current API so 
 that it may allow more fine grained tweaking of loggers 
 (addition of module info, providing built-in multiplexing 
 logger). There should be no temptation to build own stuff with 
 own API because you can't write own logger that fits standard 
 one.

 But actual batteries - no, this does belong to Phobos.


I did not talk about additional external libraries. As I know, 
Vibe.d use OpenSSL to provide SSL streams. Since we haven't got 
encryption support in Phobos, we can provide only TCP streams.
Oct 15 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 October 2013 at 14:09:36 UTC, ilya-stromberg wrote:

 I did not talk about additional external libraries. As I know, 
 Vibe.d use OpenSSL to provide SSL streams. Since we haven't got 
 encryption support in Phobos, we can provide only TCP streams.


For example, sending mail is clearly relying on external stuff 
and should never be in Phobos (again, std.net.curl was a terrible 
mistake)
Oct 15 2013
next sibling parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 14:13:53 UTC, Dicebot wrote:

 On Tuesday, 15 October 2013 at 14:09:36 UTC, ilya-stromberg 
 wrote:

 I did not talk about additional external libraries. As I know, 
 Vibe.d use OpenSSL to provide SSL streams. Since we haven't 
 got encryption support in Phobos, we can provide only TCP 
 streams.


 For example, sending mail is clearly relying on external stuff 
 and should never be in Phobos (again, std.net.curl was a 
 terrible mistake)


I didn't know about it, sorry.
Oct 15 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-10-15 16:13, Dicebot wrote:


 For example, sending mail is clearly relying on external stuff and
 should never be in Phobos (again, std.net.curl was a terrible mistake)


I don't see why this couldn't be included in Phobos, if it doesn't have 
any external dependencies.

-- 
/Jacob Carlborg
Oct 16 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 03:31 PM, ilya-stromberg wrote:

 On Tuesday, 15 October 2013 at 07:52:28 UTC, Robert Schadek wrote:

 On 10/15/2013 04:06 AM, Eric Anderton wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:
 Here's what I think is missing:
 - System log support (as others have mentioned).  This would be syslog
 or WEL, depending on environment.


 This is sort of the idea of the design, I can't anticipate your needs
 therefor I should not try. I should try to give you guidelines or a
 framework to work against.


 Totally disagree. We need a powerful logger, not only file logger.
 I can implement a file logger myself for a few hours, and it will
 cover 90% of my needs. For other 10% I would like to have a standart
 logger with advanced features like speed and reliability.


I bet your 10% and mine 10% do not overlap. And than either you or I
will complain about it.

 If you need help, please tell us. For example, jkm already implemented
 syslog for Vibe.d with support files (via file streams) and the
 network (via TCP or SSL streams):
 https://github.com/rejectedsoftware/vibe.d/pull/294
 It's under the MIT license, that similar to the Boost license.

 Sönke Ludwig, can you allow to use syslog code for `std.logger` under
 the Boost license? Robert Schadek can use it as initial point.


Before any time is spent to implement x number of logger the design must
be done.
Oct 15 2013
prev sibling parent =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig outerproduct.org> writes:
Am 15.10.2013 15:31, schrieb ilya-stromberg:

 On Tuesday, 15 October 2013 at 07:52:28 UTC, Robert Schadek wrote:

 On 10/15/2013 04:06 AM, Eric Anderton wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:
 Here's what I think is missing:
 - System log support (as others have mentioned).  This would be syslog
 or WEL, depending on environment.


 This is sort of the idea of the design, I can't anticipate your needs
 therefor I should not try. I should try to give you guidelines or a
 framework to work against.


 Totally disagree. We need a powerful logger, not only file logger.
 I can implement a file logger myself for a few hours, and it will cover
 90% of my needs. For other 10% I would like to have a standart logger
 with advanced features like speed and reliability.


In this case I do agree with Robert that it will be better to keep the 
std.log module basic and as dependency free as possible. The main 
advantage of it is to enhance interoperability of different libraries 
that use logging, but any advanced features can be implemented by 
external libraries just as well (at least at the beginning). But of 
course the general design allow for all complex use cases. And then over 
time it may well show that it makes sense to include more standard 
loggers. /IMHO


 If you need help, please tell us. For example, jkm already implemented
 syslog for Vibe.d with support files (via file streams) and the network
 (via TCP or SSL streams):
 https://github.com/rejectedsoftware/vibe.d/pull/294
 It's under the MIT license, that similar to the Boost license.

 Sönke Ludwig, can you allow to use syslog code for `std.logger` under
 the Boost license? Robert Schadek can use it as initial point.


Should this be needed, it would be absolutely no problem. MIT -> Boost 
works automatically and I would also happily transfer the code ownership 
if needed.
Oct 15 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/15/13 12:52 AM, Robert Schadek wrote:

 On 10/15/2013 04:06 AM, Eric Anderton wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 Lets unleash the forces of constructive destruction.


 So, not to be too heavy-handed with criticism on this library, but I
 think this should come up to par with solutions like log4j, log4cpp,
 or log4cxx, with respect to features and capabilities.  Libraries like
 these have enjoyed a lot of very serious use, and once you have
 something like that in your project, it's hard to not use most of what
 they have to offer.  There's really not a lot of fluff in those
 solutions.


 IMO these libraries are to heavy. Especially with phobos inclusion in mind.


I agree. A bunch of stuff at Facebook is heavily relying on logging for 
statistics and debugging, yet we're fine with the relatively scarce API 
of Google log. That said, I'm clearly biased because I've never used 
log4xxx.

One note - log4j, log4cxx, and log4cpp are not part of the respective 
languages' standards. That doesn't mean much (in fact it may be a 
competitive advantage to integrating log4d in std) but it is one factor 
to consider.

Eric, could you please enumerate a short list of features of log4j that 
you think would be really missed if absent?


Andrei
Oct 15 2013
next sibling parent Jeremy Powers <jpowers wyrdtech.com> writes:
On Tue, Oct 15, 2013 at 8:17 AM, Andrei Alexandrescu <
SeeWebsiteForEmail erdani.org> wrote:


 One note - log4j, log4cxx, and log4cpp are not part of the respective
 languages' standards. That doesn't mean much (in fact it may be a
 competitive advantage to integrating log4d in std) but it is one factor to
 consider.


It also gave rise to slf4j, to tie the various (java) logging solutions
together.


From a core library standpoint, the slf4j model might be a good one to


emulate - provide a basic logging abstraction that can then be plumbed to
whichever logging implementation is needed.  Logback is essentially the
logging framework written by the slf4j guys, which is why I used it as an
example.

And though I am not Eric, I do have a short list.  These are things that
log4j/slf4j/etc provide that I'd consider required of any log framework
before I use it in a production* environment:

Multiple log destinations (sinks/appenders), configurable.
  - required for logging to file, syslog, etc as appropriate
  - different running instances of same code may need different log
names/locations/appenders

Hierarchical logs, with inheritance of levels, configure at runtime.  Turn
on/off log level for part of hierarchy.
  - for debugging own code without being overwhelmed with log statements
from elsewhere
  - turn off extraneous logging in dependencies, or turn it on for deep
diving

Configurable log ouput with custom fields (time, thread, etc).
  - required for making log output match predefined formats
  - include needed metadata in the log line

Allow 'lazy' evaluation/formatting of log output (parameterized logging
equivalent).
  - no performance excuse not to log

Log rotation
  - if this isn't there out of the box, guarantee will be first
customization


* where 'production' is biased towards high availability services
Oct 15 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/16/2013 01:34 AM, Jeremy Powers wrote:

 On Tue, Oct 15, 2013 at 8:17 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org <mailto:SeeWebsiteForEmail erdani.org>>
 wrote:

     One note - log4j, log4cxx, and log4cpp are not part of the
     respective languages' standards. That doesn't mean much (in fact
     it may be a competitive advantage to integrating log4d in std) but
     it is one factor to consider.


 It also gave rise to slf4j, to tie the various (java) logging
 solutions together.

 From a core library standpoint, the slf4j model might be a good one to
 emulate - provide a basic logging abstraction that can then be plumbed
 to whichever logging implementation is needed.  Logback is essentially
 the logging framework written by the slf4j guys, which is why I used
 it as an example.

 And though I am not Eric, I do have a short list.  These are things
 that log4j/slf4j/etc provide that I'd consider required of any log
 framework before I use it in a production* environment:

 Multiple log destinations (sinks/appenders), configurable.
   - required for logging to file, syslog, etc as appropriate
   - different running instances of same code may need different log
 names/locations/appenders


MultiLogger is planed.

 Hierarchical logs, with inheritance of levels, configure at runtime.
  Turn on/off log level for part of hierarchy.
   - for debugging own code without being overwhelmed with log
 statements from elsewhere
   - turn off extraneous logging in dependencies, or turn it on for
 deep diving


no hierarchical logs, KISS just create a logger with different
destination. new FileLogger("myDevLog"); $ tail -f myDevLog

 Configurable log ouput with custom fields (time, thread, etc).
   - required for making log output match predefined formats
   - include needed metadata in the log line


I think this has been discussed twice already, no configuration can
anticipate all possible needs and will fail fast and hard. So why try, I
rather write 7 lines, than wait for a patch to the configuration parser
to appear in my production env.

 Allow 'lazy' evaluation/formatting of log output (parameterized
 logging equivalent).
   - no performance excuse not to log

 Log rotation
   - if this isn't there out of the box, guarantee will be first
 customization


see above argument

 * where 'production' is biased towards high availability services
Oct 16 2013
prev sibling next sibling parent Jeremy Powers <jpowers wyrdtech.com> writes:
Short version of below:
I want a powerful logging system.  Maybe std.logging should provide the
interface with some basic functionality, allow other solutions to fill in
gaps.  Should be able to always code against std.logging, complications
added as needed w/o code calling log() caring or changing.


On Wed, Oct 16, 2013 at 12:30 AM, Robert Schadek <realburner gmx.de> wrote:


  no hierarchical logs, KISS just create a logger with different
 destination. new FileLogger("myDevLog"); $ tail -f myDevLog


Without this provided out of the box, I'd have to create my own framework
for such on top for any serious use.  This is arguably the most important
feature of a logging framework for a large product.  Once you get multiple
people/teams/companies/monkeys contributing to a system, you _need_ a way
to distinguish and filter logging from each part.  Spitting to different
log files is not a solution in most cases, the 'create my own' would have
each module logger spitting to the same place as the root, with the root
logger controlling what actually made it to the log.

Simple logging framework should be simple.  But it should also be powerful,
without requiring custom boilerplate for more complex usage...  Earlier was
mention of getting a module's log via import, this seems a good solution
interface wise - basic implementation would just return basic logger, but
would allow for a hierarchical solution to be plumbed in without the
logging code knowing/caring.


 Configurable log ouput with custom fields (time, thread, etc).

   - required for making log output match predefined formats
    - include needed metadata in the log line

 I think this has been discussed twice already, no configuration can
 anticipate all possible needs and will fail fast and hard. So why try, I
 rather write 7 lines, than wait for a patch to the configuration parser to
 appear in my production env.


There are two parts to this: making sure the log output conforms to some
format, and making sure required information is included.  You can never
anticipate what everyone needs for either, but you can provide the tools to
enable them.  Conceptually, this means separating the information being
logged from the actual output - the basic logging framework doesn't need to
even try to cover every case, so long as it provides hook points.

Once the output is separated from the information, custom output is as
simple as a format string.  Important bit is that this is decided by the
logger (or whoever configured it) not the code where it is logged.  Can
provide basic information by default, or not, so long as there is a
mechanism to include custom information.

MDC in slf4j/logback is one approach to the 'information included' problem:
http://logback.qos.ch/manual/mdc.html

Not suggesting std.logging needs the same thing, but illustrates one way to
solve.  Personal experience has shown this is useful.
Oct 16 2013
prev sibling next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/16/2013 08:18 PM, Jeremy Powers wrote:

 Short version of below:
 I want a powerful logging system.  Maybe std.logging should provide
 the interface with some basic functionality, allow other solutions to
 fill in gaps.  Should be able to always code against std.logging,
 complications added as needed w/o code calling log() caring or changing.


 On Wed, Oct 16, 2013 at 12:30 AM, Robert Schadek <realburner gmx.de
 <mailto:realburner gmx.de>> wrote:

     no hierarchical logs, KISS just create a logger with different
     destination. new FileLogger("myDevLog"); $ tail -f myDevLog


 Without this provided out of the box, I'd have to create my own
 framework for such on top for any serious use.  This is arguably the
 most important feature of a logging framework for a large product.
  Once you get multiple people/teams/companies/monkeys contributing to
 a system, you _need_ a way to distinguish and filter logging from each
 part.  Spitting to different log files is not a solution in most
 cases, the 'create my own' would have each module logger spitting to
 the same place as the root, with the root logger controlling what
 actually made it to the log.

 Simple logging framework should be simple.  But it should also be
 powerful, without requiring custom boilerplate for more complex
 usage...  Earlier was mention of getting a module's log via import,
 this seems a good solution interface wise - basic implementation would
 just return basic logger, but would allow for a hierarchical solution
 to be plumbed in without the logging code knowing/caring.


I still don't feel that hierarchy building should be part of std.logger,
as it is way to heavyweight. But thinking about MultiLogger made me
realize, that I need a way to identifier Logger to remove them. So
currently, I think MultiLogger will have an AA and Logger will have
names (string). You do the math ....

     Configurable log ouput with custom fields (time, thread, etc).
       - required for making log output match predefined formats
       - include needed metadata in the log line


     I think this has been discussed twice already, no configuration
     can anticipate all possible needs and will fail fast and hard. So
     why try, I rather write 7 lines, than wait for a patch to the
     configuration parser to appear in my production env.


 There are two parts to this: making sure the log output conforms to
 some format, and making sure required information is included.  You
 can never anticipate what everyone needs for either, but you can
 provide the tools to enable them.  Conceptually, this means separating
 the information being logged from the actual output - the basic
 logging framework doesn't need to even try to cover every case, so
 long as it provides hook points.


logMessage(LoggerPayload); is your all powerful hookpoint.
Oct 16 2013
prev sibling parent reply "Eric Anderton" <eric.t.anderton gmail.com> writes:
On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei Alexandrescu 
wrote:

 Eric, could you please enumerate a short list of features of 
 log4j that you think would be really missed if absent?


Certainly.  Here's my top 3, with some background to explain why 
I think they'd be missed.

- Hierarchical logging with a centralized (singleton) logging 
facility.  The strength of this is that it would allow us to 
freely integrate D libraries that use std.logger, yet filter 
their log output from *outside* the library through the 
std.logger API. This can be accomplished by tagging each log 
event with a category name.  Log events are then filtered by 
prefix matching of the category name, as well as by log level.  
Without this feature, library authors would have to provide 
explicit API calls to manipulate their library's logging, or 
require API users to pass logging contexts forward to the library.

- Configurable output strategies for each log category.  In 
log4cpp, these are known as "appenders."  Appenders need not be 
registered explicitly for each category, but can be registered by 
category name prefix match, just like the filtering for the 
hierarchical system (above). The idea is to allow for different 
formatting strategies and output targets, including the logrotate 
issue I mentioned earlier.  This provides a nice integration 
point to tackle basic capabilities today, like file logging and 
syslog support, and more advanced features by 3rd party authors.

- Nested Diagnostic Context support ("Mapped Diagnostic Context" 
in log4j).  The NDC facility in log4cpp/log4cxx is incredibly 
handy for cutting down on the amount of data one usually puts 
into a given log event.  The gist of an NDC is just a way to 
bracket a series of log calls with a prefix that is emitted with 
the rest of the log line.  These contexts are maintained on a 
thread-specific stack, such that each log event is prefixed with 
all the information in the entire stack, at the time of the 
event. Without this, one winds up re-inventing the concept 
(usually poorly) to forward the same information to each call to 
emit a log message.  It also eliminates the need for a stack 
trace in a log message in most cases, which is something that 
people who use SIEM software (e.g. Splunk) would appreciate.

There are other things that would be nice - configuration file 
support, lazy evaluation of log event arguments, custom output 
formats - but I think the above is really core of what's needed.

For what it's worth: my opinions mostly come from my experience 
in integrating with log4j and log4cpp.  Log4j is a very 
heavyweight library - I don't think we need anything anywhere 
close to that nuanced.  In contrast, log4cpp is very small 
library and worth a look as something that may be a good model 
for what std.logger could accomplish.

- Eric
Oct 16 2013
next sibling parent reply "qznc" <qznc web.de> writes:
On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:

 The strength of this is that it would allow us to freely 
 integrate D libraries that use std.logger, yet filter their log 
 output from *outside* the library through the std.logger API.


This is one of the most important aspects in my opinion.
Std.logger should be easy to use, so library writers are
encouraged to use it. Compare this with the "unittest" keyword,
which makes it easy to write some simple tests. Of course,
flexibility to use complex machinery for using the messages/tests
is necessary. Just like we can hook up more advanced unit testing
frameworks, we should be able to hook up more advanced logging
machinery. The advanced stuff is not for Phobos though. Advanced
stuff for unittests is for example, parallel execution and
graphical reports. Advanced stuff for logging is for example log
rotation and networking.
Oct 17 2013
next sibling parent Robert Schadek <realburner gmx.de> writes:
On 10/17/2013 09:34 AM, qznc wrote:

 On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:

 The strength of this is that it would allow us to freely integrate D
 libraries that use std.logger, yet filter their log output from
 *outside* the library through the std.logger API.


 This is one of the most important aspects in my opinion.
 Std.logger should be easy to use, so library writers are
 encouraged to use it. Compare this with the "unittest" keyword,
 which makes it easy to write some simple tests. Of course,
 flexibility to use complex machinery for using the messages/tests
 is necessary. Just like we can hook up more advanced unit testing
 frameworks, we should be able to hook up more advanced logging
 machinery. The advanced stuff is not for Phobos though. Advanced
 stuff for unittests is for example, parallel execution and
 graphical reports. Advanced stuff for logging is for example log
 rotation and networking.


+1
Oct 17 2013
prev sibling parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Thursday, 17 October 2013 at 07:34:28 UTC, qznc wrote:

 On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton 
 wrote:

 The strength of this is that it would allow us to freely 
 integrate D libraries that use std.logger, yet filter their 
 log output from *outside* the library through the std.logger 
 API.


 This is one of the most important aspects in my opinion.
 Std.logger should be easy to use, so library writers are
 encouraged to use it. Compare this with the "unittest" keyword,
 which makes it easy to write some simple tests. Of course,
 flexibility to use complex machinery for using the 
 messages/tests
 is necessary. Just like we can hook up more advanced unit 
 testing
 frameworks, we should be able to hook up more advanced logging
 machinery. The advanced stuff is not for Phobos though. Advanced
 stuff for unittests is for example, parallel execution and
 graphical reports. Advanced stuff for logging is for example log
 rotation and networking.


There is no contradiction. Complex log libraries become 
(relatively) complex when one wants to use their advanced 
features, but are as simple as the others when one wants to use 
them simply.
That's why in the Java world nearly everyone uses Log4j instead 
of the official JEE API.

In practive, you really want a powerful logging facility.
Another feature I used once in a project, was to log to RAM. We 
decided to log TRACE logs in production in order to catch a rare 
bug, but of course, it would output too many logs. So we decided 
to log everything in RAM and keep the last 1000 logs. Whenever 
there would be an exception or a CRITICAL log, the whole 1000 
logs would be dumped on disk. That feature proved very useful.
Oct 20 2013
next sibling parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Monday, 21 October 2013 at 06:27:39 UTC, SomeDude wrote:

 On Thursday, 17 October 2013 at 07:34:28 UTC, qznc wrote:

 On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton 
 wrote:

 The strength of this is that it would allow us to freely 
 integrate D libraries that use std.logger, yet filter their 
 log output from *outside* the library through the std.logger 
 API.


 This is one of the most important aspects in my opinion.
 Std.logger should be easy to use, so library writers are
 encouraged to use it. Compare this with the "unittest" keyword,
 which makes it easy to write some simple tests. Of course,
 flexibility to use complex machinery for using the 
 messages/tests
 is necessary. Just like we can hook up more advanced unit 
 testing
 frameworks, we should be able to hook up more advanced logging
 machinery. The advanced stuff is not for Phobos though. 
 Advanced
 stuff for unittests is for example, parallel execution and
 graphical reports. Advanced stuff for logging is for example 
 log
 rotation and networking.


 There is no contradiction. Complex log libraries become 
 (relatively) complex when one wants to use their advanced 
 features, but are as simple as the others when one wants to use 
 them simply.
 That's why in the Java world nearly everyone uses Log4j instead 
 of the official JEE API.

 In practive, you really want a powerful logging facility.
 Another feature I used once in a project, was to log to RAM. We 
 decided to log TRACE logs in production in order to catch a 
 rare bug, but of course, it would output too many logs. So we 
 decided to log everything in RAM and keep the last 1000 logs. 
 Whenever there would be an exception or a CRITICAL log, the 
 whole 1000 logs would be dumped on disk. That feature proved 
 very useful.


+1
Also, it would be useful to use buffered asynchronous logger 
(mentioned above).
It will help to trace and save all logs, but should be fast for 
production usage.
Oct 20 2013
prev sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/21/2013 08:27 AM, SomeDude wrote:

 In practive, you really want a powerful logging facility.
 Another feature I used once in a project, was to log to RAM. We
 decided to log TRACE logs in production in order to catch a rare bug,
 but of course, it would output too many logs. So we decided to log
 everything in RAM and keep the last 1000 logs. Whenever there would be
 an exception or a CRITICAL log, the whole 1000 logs would be dumped on
 disk. That feature proved very useful. 


That was a feature you added or was it part of the logging library?
Oct 21 2013
parent reply "SomeDude" <lovelydear mailmetrash.com> writes:
On Monday, 21 October 2013 at 08:50:50 UTC, Robert Schadek wrote:

 disk. That feature proved very useful.


 That was a feature you added or was it part of the logging 
 library?


We added it. I don't know of any logging library doing that.
Nov 04 2013
parent "Kagamin" <spam here.lot> writes:
On Tuesday, 5 November 2013 at 02:51:54 UTC, SomeDude wrote:

 We added it. I don't know of any logging library doing that.


It's BufferingForwardingAppender in log4net. We use it to improve 
logging performance.
Nov 12 2013
prev sibling next sibling parent "Arjan" <arjan ask.me.to> writes:
On Thursday, 17 October 2013 at 02:13:12 UTC, Eric Anderton wrote:

 On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei 
 Alexandrescu wrote:

 Eric, could you please enumerate a short list of features of 
 log4j that you think would be really missed if absent?


 Certainly.  Here's my top 3, with some background to explain 
 why I think they'd be missed.

 - Hierarchical logging with a centralized (singleton) logging 
 facility.  The strength of this is that it would allow us to 
 freely integrate D libraries that use std.logger, yet filter 
 their log output from *outside* the library through the 
 std.logger API. This can be accomplished by tagging each log 
 event with a category name.  Log events are then filtered by 
 prefix matching of the category name, as well as by log level.  
 Without this feature, library authors would have to provide 
 explicit API calls to manipulate their library's logging, or 
 require API users to pass logging contexts forward to the 
 library.

 - Configurable output strategies for each log category.  In 
 log4cpp, these are known as "appenders."  Appenders need not be 
 registered explicitly for each category, but can be registered 
 by category name prefix match, just like the filtering for the 
 hierarchical system (above). The idea is to allow for different 
 formatting strategies and output targets, including the 
 logrotate issue I mentioned earlier.  This provides a nice 
 integration point to tackle basic capabilities today, like file 
 logging and syslog support, and more advanced features by 3rd 
 party authors.

 - Nested Diagnostic Context support ("Mapped Diagnostic 
 Context" in log4j).  The NDC facility in log4cpp/log4cxx is 
 incredibly handy for cutting down on the amount of data one 
 usually puts into a given log event.  The gist of an NDC is 
 just a way to bracket a series of log calls with a prefix that 
 is emitted with the rest of the log line.  These contexts are 
 maintained on a thread-specific stack, such that each log event 
 is prefixed with all the information in the entire stack, at 
 the time of the event. Without this, one winds up re-inventing 
 the concept (usually poorly) to forward the same information to 
 each call to emit a log message.  It also eliminates the need 
 for a stack trace in a log message in most cases, which is 
 something that people who use SIEM software (e.g. Splunk) would 
 appreciate.

 There are other things that would be nice - configuration file 
 support, lazy evaluation of log event arguments, custom output 
 formats - but I think the above is really core of what's needed.

 For what it's worth: my opinions mostly come from my experience 
 in integrating with log4j and log4cpp.  Log4j is a very 
 heavyweight library - I don't think we need anything anywhere 
 close to that nuanced.  In contrast, log4cpp is very small 
 library and worth a look as something that may be a good model 
 for what std.logger could accomplish.

 - Eric


+1

I have used log4net some years ago, really liked the the 'context 
logging' feature. (Nested Diagnostic Context support).

I've also used www.panteios.org logging api, liked that approach 
too.
Oct 17 2013
prev sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/17/2013 04:13 AM, Eric Anderton wrote:

 On Tuesday, 15 October 2013 at 15:16:44 UTC, Andrei Alexandrescu wrote:

 Eric, could you please enumerate a short list of features of log4j
 that you think would be really missed if absent?


 Certainly.  Here's my top 3, with some background to explain why I
 think they'd be missed.

 - Hierarchical logging with a centralized (singleton) logging
 facility.  The strength of this is that it would allow us to freely
 integrate D libraries that use std.logger, yet filter their log output
 from *outside* the library through the std.logger API. This can be
 accomplished by tagging each log event with a category name.  Log
 events are then filtered by prefix matching of the category name, as
 well as by log level.  Without this feature, library authors would
 have to provide explicit API calls to manipulate their library's
 logging, or require API users to pass logging contexts forward to the
 library.


With the new MultiLogger and names for Logger, I consider this done.

 - Configurable output strategies for each log category.  In log4cpp,
 these are known as "appenders."  Appenders need not be registered
 explicitly for each category, but can be registered by category name
 prefix match, just like the filtering for the hierarchical system
 (above). The idea is to allow for different formatting strategies and
 output targets, including the logrotate issue I mentioned earlier. 
 This provides a nice integration point to tackle basic capabilities
 today, like file logging and syslog support, and more advanced
 features by 3rd party authors.

 - Nested Diagnostic Context support ("Mapped Diagnostic Context" in
 log4j).  The NDC facility in log4cpp/log4cxx is incredibly handy for
 cutting down on the amount of data one usually puts into a given log
 event.  The gist of an NDC is just a way to bracket a series of log
 calls with a prefix that is emitted with the rest of the log line. 
 These contexts are maintained on a thread-specific stack, such that
 each log event is prefixed with all the information in the entire
 stack, at the time of the event. Without this, one winds up
 re-inventing the concept (usually poorly) to forward the same
 information to each call to emit a log message.  It also eliminates
 the need for a stack trace in a log message in most cases, which is
 something that people who use SIEM software (e.g. Splunk) would
 appreciate.

 There are other things that would be nice - configuration file
 support, lazy evaluation of log event arguments, custom output formats
 - but I think the above is really core of what's needed.


IMO, this is to heavyweight and would water the current design to much.
That being said, I think it is easy to subclass Logger or MultiLogger to
achieve this. But I don't think that this should be part of the default
feature set.
Oct 17 2013
parent reply "Dicebot" <public dicebot.lv> writes:
Can you please re-generate the documentation after all recent 
updates?
Oct 18 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/18/2013 02:55 PM, Dicebot wrote:

 Can you please re-generate the documentation after all recent updates?


I usually do that. The only documentation missing is for MultiLogger, as
I'm not sure if the current implementation is done.
Oct 18 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 18 October 2013 at 13:35:19 UTC, Robert Schadek wrote:

 On 10/18/2013 02:55 PM, Dicebot wrote:

 Can you please re-generate the documentation after all recent 
 updates?


 I usually do that. The only documentation missing is for 
 MultiLogger, as
 I'm not sure if the current implementation is done.


Hm. I don't see anything related to Logger names there. It is 
supposed to be part of MultiLogger?

As discussion was slowly fading I wanted to make some sort of 
summary for this thread judging by personal observations - having 
quick overview of current state in form of docs will be helpful.
Oct 18 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/18/2013 03:49 PM, Dicebot wrote:

 On Friday, 18 October 2013 at 13:35:19 UTC, Robert Schadek wrote:

 On 10/18/2013 02:55 PM, Dicebot wrote:

 Can you please re-generate the documentation after all recent updates?


 I usually do that. The only documentation missing is for MultiLogger, as
 I'm not sure if the current implementation is done.


 Hm. I don't see anything related to Logger names there. It is supposed
 to be part of MultiLogger?

 As discussion was slowly fading I wanted to make some sort of summary
 for this thread judging by personal observations - having quick
 overview of current state in form of docs will be helpful.


I rebuild the docs and pushed them. MultiLogger docu is just a stub.
Oct 18 2013
prev sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 07:52:28 UTC, Robert Schadek wrote:

 On 10/15/2013 04:06 AM, Eric Anderton wrote:

 Here's what I think is missing:
 - System log support (as others have mentioned).  This would 
 be syslog
 or WEL, depending on environment.


 This is sort of the idea of the design, I can't anticipate your 
 needs
 therefor I should not try. I should try to give you guidelines 
 or a
 framework to work against.

 - Guarantees or options for working with log rotation 
 (logrotate.d). It's nice to either know that you must restart 
 your daemon once logs
 are rotated, or can configure logging to re-open handles 
 automatically
 or when it detects rotation has occurred.


 See previous point


Disagree. We need a log rotation support.
As I can see, available options could be:

* rotating conditions
  - by date (rotate every hour, day (default), week, month, year)
  - by file size (rotate if file size more than ... Mb)
  - by count log lines (rotate if log contains more than ... log 
lines)
  - combination of previous conditions (for example, rotate every 
day or rotate if file size more than 100 Mb)

* file names after rotation
  - by numbers (my.log, my.log.0, my.log.1, ...)
  - by ISO date and time (my-2013-10-16-00-00-00.log)

* ability to use system log rotation utility
Oct 16 2013
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 16 October 2013 at 07:18:39 UTC, ilya-stromberg 
wrote:

 Disagree. We need a log rotation support.
 As I can see, available options could be:

 * rotating conditions
  - by date (rotate every hour, day (default), week, month, year)
  - by file size (rotate if file size more than ... Mb)
  - by count log lines (rotate if log contains more than ... log 
 lines)
  - combination of previous conditions (for example, rotate 
 every day or rotate if file size more than 100 Mb)

 * file names after rotation
  - by numbers (my.log, my.log.0, my.log.1, ...)
  - by ISO date and time (my-2013-10-16-00-00-00.log)

 * ability to use system log rotation utility


I really like that Boost::Log also provides an option to replace 
old log files (instead of creating new ones) when disk capacity 
reaches a certain threshold.  It might also be worth providing an 
option to simply limit the max log file count to N, which then 
replaces oldest first.  In some unexpected situations I've seen 
the disk fill from a badly behaved program, and these are a nice 
safeguard.
Oct 20 2013
parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Sunday, 20 October 2013 at 15:34:50 UTC, Sean Kelly wrote:

 On Wednesday, 16 October 2013 at 07:18:39 UTC, ilya-stromberg 
 wrote:

 Disagree. We need a log rotation support.
 As I can see, available options could be:

 * rotating conditions
 - by date (rotate every hour, day (default), week, month, year)
 - by file size (rotate if file size more than ... Mb)
 - by count log lines (rotate if log contains more than ... log 
 lines)
 - combination of previous conditions (for example, rotate 
 every day or rotate if file size more than 100 Mb)

 * file names after rotation
 - by numbers (my.log, my.log.0, my.log.1, ...)
 - by ISO date and time (my-2013-10-16-00-00-00.log)

 * ability to use system log rotation utility


 I really like that Boost::Log also provides an option to 
 replace old log files (instead of creating new ones) when disk 
 capacity reaches a certain threshold.  It might also be worth 
 providing an option to simply limit the max log file count to 
 N, which then replaces oldest first.  In some unexpected 
 situations I've seen the disk fill from a badly behaved 
 program, and these are a nice safeguard.


+1
* delete old files when:
  - you have more than N files
  - size of all log files more than ... Mb

* support compress rotated files (for examle, in zip or gzip 
format)
Oct 20 2013
parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Sunday, 20 October 2013 at 16:05:47 UTC, ilya-stromberg wrote:

 On Sunday, 20 October 2013 at 15:34:50 UTC, Sean Kelly wrote:

 On Wednesday, 16 October 2013 at 07:18:39 UTC, ilya-stromberg 
 wrote:

 Disagree. We need a log rotation support.
 As I can see, available options could be:

 * rotating conditions
 - by date (rotate every hour, day (default), week, month, 
 year)
 - by file size (rotate if file size more than ... Mb)
 - by count log lines (rotate if log contains more than ... 
 log lines)
 - combination of previous conditions (for example, rotate 
 every day or rotate if file size more than 100 Mb)

 * file names after rotation
 - by numbers (my.log, my.log.0, my.log.1, ...)
 - by ISO date and time (my-2013-10-16-00-00-00.log)

 * ability to use system log rotation utility


 I really like that Boost::Log also provides an option to 
 replace old log files (instead of creating new ones) when disk 
 capacity reaches a certain threshold.  It might also be worth 
 providing an option to simply limit the max log file count to 
 N, which then replaces oldest first.  In some unexpected 
 situations I've seen the disk fill from a badly behaved 
 program, and these are a nice safeguard.


 +1
 * delete old files when:
  - you have more than N files
  - size of all log files more than ... Mb


Addition:
* delete old files afrer ... days (hours)


 * support compress rotated files (for examle, in zip or gzip 
 format)
Oct 20 2013
prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr> writes:
What are the philosophy behind errors vs fatal errors vs critical 
errors?
When should we use each of these?
Oct 15 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 10:49 AM, ponce wrote:

 What are the philosophy behind errors vs fatal errors vs critical errors?
 When should we use each of these?


fatal = the application is going down, I'm just letting you know
critical = the application is maybe going down, I'm not sure yet, but
this is a problem
error = something went wrong, I'm sure if this is problem
Oct 15 2013
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
I think one increasingly important point for std.log is 'structured
logging'.

Structured logging is basically not simply logging textual messages, but
also logging additional KEY/VALUE pairs of data. The idea is that logs
should not only be readable by humans but also easy to parse and
analyze. Structured logging often also includes UUIDs to simplify
finding similar errors/problems in a log file.

For example: Instead of logging
logf("Couldn't login as %s on server %s : Reason: %s", user, server,
      errorCode);

we'd do:
log(LOGIN_FAILED_UUID, "Couldn't log in", ["User": user, "Server":
    server, "ErrorCode": errorCode]); 

The log can then be queried for all events with 'LOGIN_FAILED_UUID'
uuid, Server="..." ErrorCode=42, etc.

As a nice benefit structured logging can also log the function, module
and line of the source file that logged the message, Exceptions can be
written to the log in a nice way, etc.

SystemDs journal [1] is a structured replacement for syslog and is
already being used on some linux distributions (ArchLinux, Fedora).
It's likely that the journal will replace syslog on linux so at least
for server-side software structured logging support is becoming
important. The GNOME guys are also working on a log viewer for
systemd's journal [2]. 

std.log certainly doesn't need to provide a backend for the journal
right now. But some questions regarding structured logging need to
taken into account:

* Will structured logging be integrated with the normal logging at all?
* Will the public API somehow collide with the proposed std.log API?
* Should the API for log backends already support structured logging or
  is it possible to add this later on?

Most important:
* What to do about structured logging calls when logging to a simple
  text logger? Ignore the additional fields or serialize them? What
  about the UUID?

Even if we don't have a backend implementation it'd be nice to have the
public API available. Otherwise new projects cannot use structured
logging with std.log right now and the user code needs to be
rewritten once std.log does support structured logging.

[1]
http://0pointer.de/blog/projects/journalctl.html

[2]
https://mail.gnome.org/archives/gnome-announce-list/2013-September/msg00097.html
Oct 15 2013
next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 10/15/2013 05:20 PM, Johannes Pfau wrote:

 I think one increasingly important point for std.log is 'structured
 logging'.

 Structured logging is basically not simply logging textual messages, but
 also logging additional KEY/VALUE pairs of data. The idea is that logs
 should not only be readable by humans but also easy to parse and
 analyze. Structured logging often also includes UUIDs to simplify
 finding similar errors/problems in a log file.

 For example: Instead of logging
 logf("Couldn't login as %s on server %s : Reason: %s", user, server,
       errorCode);

 we'd do:
 log(LOGIN_FAILED_UUID, "Couldn't log in", ["User": user, "Server":
     server, "ErrorCode": errorCode]); 

 The log can then be queried for all events with 'LOGIN_FAILED_UUID'
 uuid, Server="..." ErrorCode=42, etc.

 As a nice benefit structured logging can also log the function, module
 and line of the source file that logged the message, Exceptions can be
 written to the log in a nice way, etc.

 SystemDs journal [1] is a structured replacement for syslog and is
 already being used on some linux distributions (ArchLinux, Fedora).
 It's likely that the journal will replace syslog on linux so at least
 for server-side software structured logging support is becoming
 important. The GNOME guys are also working on a log viewer for
 systemd's journal [2]. 

 std.log certainly doesn't need to provide a backend for the journal
 right now. But some questions regarding structured logging need to
 taken into account:

 * Will structured logging be integrated with the normal logging at all?
 * Will the public API somehow collide with the proposed std.log API?
 * Should the API for log backends already support structured logging or
   is it possible to add this later on?

 Most important:
 * What to do about structured logging calls when logging to a simple
   text logger? Ignore the additional fields or serialize them? What
   about the UUID?

 Even if we don't have a backend implementation it'd be nice to have the
 public API available. Otherwise new projects cannot use structured
 logging with std.log right now and the user code needs to be
 rewritten once std.log does support structured logging.

 [1]
 http://0pointer.de/blog/projects/journalctl.html

 [2]
 https://mail.gnome.org/archives/gnome-announce-list/2013-September/msg00097.html


Currently there is no structured logging implemented in std.logger.

That been said, you can add it. The method Logger.logf is a variadic
template. You can simple create your own Logger Class and overwrite that
method and implemented your structured logging approach there. The only
pitfall is that you need a ref of type WhateverYourLoggerClassIsCalled
and call logf on that or you fall back to the default logf
implementation provided by the Logger class. Another way would be to
parse the message passed to logMessage and go from there.

Long story short. Global default Logger means structured logging
requires parsing. All other logger require correct reference type of logger.
Oct 15 2013
parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 October 2013 at 15:53:34 UTC, Robert Schadek wrote:

 That been said, you can add it. The method Logger.logf is a 
 variadic
 template. You can simple create your own Logger Class and 
 overwrite that
 method and implemented your structured logging approach there. 
 The only
 pitfall is that you need a ref of type 
 WhateverYourLoggerClassIsCalled
 and call logf on that or you fall back to the default logf
 implementation provided by the Logger class. Another way would 
 be to
 parse the message passed to logMessage and go from there.

 Long story short. Global default Logger means structured logging
 requires parsing. All other logger require correct reference 
 type of logger.


I don't think it is a good design to conflate logging mode with 
output mode in a single logger class. I'd prefer to see log() as 
variadic structured logger and logf() as it is now (making 
conditional logger `logIf()`)
Oct 15 2013
prev sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Tuesday, 15 October 2013 at 15:21:52 UTC, Johannes Pfau wrote:

 I think one increasingly important point for std.log is 
 'structured
 logging'.

 Structured logging is basically not simply logging textual 
 messages, but
 also logging additional KEY/VALUE pairs of data. The idea is 
 that logs
 should not only be readable by humans but also easy to parse and
 analyze. Structured logging often also includes UUIDs to 
 simplify
 finding similar errors/problems in a log file.

 For example: Instead of logging
 logf("Couldn't login as %s on server %s : Reason: %s", user, 
 server,
       errorCode);

 we'd do:
 log(LOGIN_FAILED_UUID, "Couldn't log in", ["User": user, 
 "Server":
     server, "ErrorCode": errorCode]);

 The log can then be queried for all events with 
 'LOGIN_FAILED_UUID'
 uuid, Server="..." ErrorCode=42, etc.


+1
I like the idea of structured logging.
I really need to log some complex data, not only string. I 
usually use `to!string(value)` to convert complex data to string.

About syntax - we can use something like this:
log!(user, server, errorCode)(LOGIN_FAILED_UUID, "Couldn't log 
in");

user, server and errorCode is variables passed to template. We 
can get variable value as usually and variable name via 
`__traits(identifier, variable)`, it should work after DMD 2.064.

For example, Vide.d use this idea, see `render` function:
http://vibed.org/docs

We can store complex data as string via `to!string(value)` cast 
until we can use real backend implementation like SystemDs.
Oct 18 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 18 October 2013 at 15:29:09 UTC, ilya-stromberg wrote:

 About syntax - we can use something like this:
 log!(user, server, errorCode)(LOGIN_FAILED_UUID, "Couldn't log 
 in");


That will be template instance bloat disaster for something used 
as commonly as log function (comparing to a typical few dozen 
`render` calls per web application at most)
Oct 18 2013
parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Friday, 18 October 2013 at 16:01:30 UTC, Dicebot wrote:

 On Friday, 18 October 2013 at 15:29:09 UTC, ilya-stromberg 
 wrote:

 About syntax - we can use something like this:
 log!(user, server, errorCode)(LOGIN_FAILED_UUID, "Couldn't log 
 in");


 That will be template instance bloat disaster for something 
 used as commonly as log function (comparing to a typical few 
 dozen `render` calls per web application at most)


Yes, you are right. I forgot about this.
Oct 18 2013
prev sibling next sibling parent reply "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
Also, Tango have log module:
https://github.com/SiegeLord/Tango-D2/blob/d2port/tango/util/log/Log.d

For example, "Funkwerk IT Karlsfeld" use it because Phobos 
haven't got logger:
http://dconf.org/talks/rohe.html
Oct 19 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 10/20/2013 08:52 AM, ilya-stromberg wrote:

 Also, Tango have log module:
 https://github.com/SiegeLord/Tango-D2/blob/d2port/tango/util/log/Log.d


I looked through the source and IMO the tango logger is my logger + (
configuration and more default logger) or the other way round. And this
thread had that discussion.
Oct 20 2013
parent "ilya-stromberg" <ilya-stromberg-2009 yandex.ru> writes:
On Sunday, 20 October 2013 at 10:56:44 UTC, Robert Schadek wrote:

 On 10/20/2013 08:52 AM, ilya-stromberg wrote:

 Also, Tango have log module:
 https://github.com/SiegeLord/Tango-D2/blob/d2port/tango/util/log/Log.d


 I looked through the source and IMO the tango logger is my 
 logger + (
 configuration and more default logger) or the other way round. 
 And this
 thread had that discussion.


I just put attention on it.
Link to the documentation:
http://siegelord.github.io/Tango-D2/
Oct 20 2013
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Mon, 14 Oct 2013 13:39:51 +0200, Dicebot wrote:


 As `std.logger` is still marked as "work in progress" this thread is
 less formal that typical pre-voting review. Goal is to provide as much
 input about desirable `std.logger` functionality and current state and
 let module author to use this information for preparing proposal for
 actual review / voting.
 
 Lets unleash the forces of constructive destruction.
 
 ===== Meta =====
 
 Code:
 https://github.com/D-Programming-Language/phobos/pull/1500/files Docs:
 http://burner.github.io/phobos/phobos-prerelease/std_logger.html
 
 First announcement / discussion thread :
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-


d puremagic.com

std.logger is a good start, but I must admit I do not like the way it is 
architectured. I use Java's Logging and Log4J for years so naturally I 
compare std.logger with these two frameworks (that are heavily used in 
production).

std.logger is a bad name, IMHO - it should be std.logging as there will 
be many Logger implementations there I presume...
Oct 21 2013
parent Robert Schadek <realburner gmx.de> writes:
On 10/21/2013 10:19 PM, Dejan Lekic wrote:

 On Mon, 14 Oct 2013 13:39:51 +0200, Dicebot wrote:


 As `std.logger` is still marked as "work in progress" this thread is
 less formal that typical pre-voting review. Goal is to provide as much
 input about desirable `std.logger` functionality and current state and
 let module author to use this information for preparing proposal for
 actual review / voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code:
 https://github.com/D-Programming-Language/phobos/pull/1500/files Docs:
 http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread :
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-


 d puremagic.com

 std.logger is a good start, but I must admit I do not like the way it is 
 architectured. I use Java's Logging and Log4J for years so naturally I 
 compare std.logger with these two frameworks (that are heavily used in 
 production).


unfamiliar or bad (details please).
Oct 21 2013
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this 
 thread is less formal that typical pre-voting review. Goal is 
 to provide as much input about desirable `std.logger` 
 functionality and current state and let module author to use 
 this information for preparing proposal for actual review / 
 voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: 
 https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: 
 http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread : 
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


Ok, finally making some conclusions.

===================
As a review manager
===================

This what needs to be done before moving to next review stage:

1.

Judging by review comments it is clear that some more efforts 
need to be put into demonstrating how existing platform can be 
used as a standard API for more complex logging functionality. 
Addition of `MultiLogger` is good thing here, but providing 
either separate article on topic or extra module that 
demonstrates enhancing existing system feels like a good way to 
avoid confusion.

That said, I do agree that std.logger itself should not be "all 
batteries included" module as it does contradict overall Phobos 
style. It may be extended in future once we start using 
sub-packages much more but right now having good standard API is 
much more important.

2.

Standard logger must be made thread-safe by default. Out of the 
box safety is very important for standard library.

3.

Making default `log` a conditional logger does not seem to be 
gladly accepted decision. I'd recommend to swap it with `logf` 
and rename to `logIf`.

Naming overall should be reviewed to make sure there is no word 
duplication in typical usage pattern - Phobos does favor plain 
procedural/functional approach as default and D module system 
makes it unnecessary to encode namespaces into names.

====================
Personal preferences
====================

It is a matter of personal taste but any single thing in that 
list will make me vote "No" on logger proposal:

1.

API it too verbose.
I want stuff as short and simple as this:
```
import std.logger;
warn("1");
inform("2");
die("3");
```
I don't want to refer logger class instances explicitly for any 
typical task.

Stuff like `auto logger = new MultiLogger(); 
logger.insertLogger(someOtherLogger);` is redundant beyond absurd 
- just how many times one may insert word "logger" in a single 
sentence? :)

2.

Logging payload needs to include fully qualified module name. 
This is necessary for enabling module-local output.

3.

I am not sure if proper global vs local log separation can be 
done without providing two distinct default instances in 
std.logger (as I have already said, I don't consider solutions 
which imply tracking class instance manually).

4.

Static "namespace" classes are redundant and should be replaced 
with module-scope globals.
Nov 04 2013
next sibling parent "Dicebot" <public dicebot.lv> writes:
At any moment when you feel you are ready to convince everyone 
that the proposal should be accepted as is - just let me know and 
a more formal review will be scheduled ;)
Nov 04 2013
prev sibling next sibling parent reply Robert Schadek <realburner gmx.de> writes:
On 11/04/2013 02:46 PM, Dicebot wrote:

 On Monday, 14 October 2013 at 11:39:52 UTC, Dicebot wrote:

 As `std.logger` is still marked as "work in progress" this thread is
 less formal that typical pre-voting review. Goal is to provide as
 much input about desirable `std.logger` functionality and current
 state and let module author to use this information for preparing
 proposal for actual review / voting.

 Lets unleash the forces of constructive destruction.

 ===== Meta =====

 Code: https://github.com/D-Programming-Language/phobos/pull/1500/files
 Docs: http://burner.github.io/phobos/phobos-prerelease/std_logger.html

 First announcement / discussion thread :
 http://forum.dlang.org/post/mailman.313.1377180809.1719.digitalmars-d puremagic.com


 Ok, finally making some conclusions.

 ===================
 As a review manager
 ===================

 This what needs to be done before moving to next review stage:

 1.

 Judging by review comments it is clear that some more efforts need to
 be put into demonstrating how existing platform can be used as a
 standard API for more complex logging functionality. Addition of
 `MultiLogger` is good thing here, but providing either separate
 article on topic or extra module that demonstrates enhancing existing
 system feels like a good way to avoid confusion.

 That said, I do agree that std.logger itself should not be "all
 batteries included" module as it does contradict overall Phobos style.
 It may be extended in future once we start using sub-packages much
 more but right now having good standard API is much more important.


I looked at journalctl, it looks promising any small enough to fit in
the docu as an example.

 2.

 Standard logger must be made thread-safe by default. Out of the box
 safety is very important for standard library.


on my todo list: My first idea is to make the global logger shared and
StdIoLogger and FileLogger synchronized in the right places.

 3.

 Making default `log` a conditional logger does not seem to be gladly
 accepted decision. I'd recommend to swap it with `logf` and rename to
 `logIf`.


I don't see the fuss. the bool in front is just a transparent. But If
people really want to write the 2 extra chars, be my guest.

 Naming overall should be reviewed to make sure there is no word
 duplication in typical usage pattern - Phobos does favor plain
 procedural/functional approach as default and D module system makes it
 unnecessary to encode namespaces into names.

 ====================
 Personal preferences
 ====================

 It is a matter of personal taste but any single thing in that list
 will make me vote "No" on logger proposal:

 1.

 API it too verbose.
 I want stuff as short and simple as this:
 ```
 import std.logger;
 warn("1");
 inform("2");
 die("3");
 ```
 I don't want to refer logger class instances explicitly for any
 typical task.

 Stuff like `auto logger = new MultiLogger();
 logger.insertLogger(someOtherLogger);` is redundant beyond absurd -
 just how many times one may insert word "logger" in a single sentence? :)


no and yes, the warn inform, maybe.
The MultiLogger part ... it needs a better constructor. Put it on my
todo list

 2.

 Logging payload needs to include fully qualified module name. This is
 necessary for enabling module-local output.


Easy fix, put it on my todo list.

 3.

 I am not sure if proper global vs local log separation can be done
 without providing two distinct default instances in std.logger (as I
 have already said, I don't consider solutions which imply tracking
 class instance manually).


I'm not sure as well

 4.

 Static "namespace" classes are redundant and should be replaced with
 module-scope globals.


not sure either


Thank you for taking effort to work on this "soon to be" review.
Robert
Nov 04 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 4 November 2013 at 17:08:21 UTC, Robert Schadek wrote:

 4.

 Static "namespace" classes are redundant and should be 
 replaced with
 module-scope globals.


 not sure either


My concern here is that if we need explicit additional symbol 
qualification to avoid collisions, D module system has failed. 
But I believe it actually works just fine, the problem is in 
habit of using imports in a way similar to C++ includes. Phobos 
should do more about endorsing power usage of D modules, not 
resort to demands of unad(a/o)pted ones :)
Nov 04 2013
parent reply Robert Schadek <realburner gmx.de> writes:
On 11/04/2013 06:27 PM, Dicebot wrote:

 On Monday, 4 November 2013 at 17:08:21 UTC, Robert Schadek wrote:

 4.

 Static "namespace" classes are redundant and should be replaced with
 module-scope globals.


 not sure either


 My concern here is that if we need explicit additional symbol
 qualification to avoid collisions, D module system has failed. But I
 believe it actually works just fine, the problem is in habit of using
 imports in a way similar to C++ includes. Phobos should do more about
 endorsing power usage of D modules, not resort to demands of
 unad(a/o)pted ones :)


Can you give an example, I'm not sure if I know what you mean.
Nov 05 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 5 November 2013 at 08:38:34 UTC, Robert Schadek wrote:

 Can you give an example, I'm not sure if I know what you mean.


Currently common approach is just adding top-level module imports 
like this:

```
module app;
import std.algorithm;
```

It has several issues:

1) Name clash if two modules define same symbol
2) Difficulties with tracking unused imports
3) Compilation speed penalty because of eager import in case it 
is only used in uninstantiated code branch.

However, D has 3 totally awesome import features that help to 
address all of this issues:

1) aliasing imports
```
import algo = std.algorithm;
```

It creates the very same namespace one is trying to achieve with 
namespace struct but without extra unused symbols and with full 
user control. Benefits both readability of application (obvious 
where namespace comes from) and readability of imported module 
(less redundant nesting)

2) importing specific symbols
```
import std.algorithm : remove;
```

Often module is used only for one or two functions. Mentioning 
those explicitly will avoid accidental clashes with irrelevant 
module symbols and acts as a nice self-documentation tool.

3) scope-local imports
```
template tmpl(T)
{
     import std.algorithm : remove;
}
```

scope local imports are just awesome. Paired with explicit symbol 
mentioning they almost completely eliminate risk of symbol 
clashes and make it easy to remove unused imports as you change 
the code. Also they don't happen unless scope is actually 
instantiated and thus can save some compilation times.

If all these nice tools D provides are used, any concern about 
symbol naming becomes irrelevant and having module-scope global 
names with no extra qualification becomes most KISS thing to do.
Nov 06 2013
next sibling parent Robert Schadek <realburner gmx.de> writes:
On 11/06/2013 02:21 PM, Dicebot wrote:

I see
Nov 06 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-11-06 14:21, Dicebot wrote:

 On Tuesday, 5 November 2013 at 08:38:34 UTC, Robert Schadek wrote:

 Can you give an example, I'm not sure if I know what you mean.


 Currently common approach is just adding top-level module imports like
 this:

 ```
 module app;
 import std.algorithm;
 ```

 It has several issues:

 1) Name clash if two modules define same symbol
 2) Difficulties with tracking unused imports
 3) Compilation speed penalty because of eager import in case it is only
 used in uninstantiated code branch.

 However, D has 3 totally awesome import features that help to address
 all of this issues:

 1) aliasing imports
 ```
 import algo = std.algorithm;
 ```

 It creates the very same namespace one is trying to achieve with
 namespace struct but without extra unused symbols and with full user
 control. Benefits both readability of application (obvious where
 namespace comes from) and readability of imported module (less redundant
 nesting)

 2) importing specific symbols
 ```
 import std.algorithm : remove;
 ```

 Often module is used only for one or two functions. Mentioning those
 explicitly will avoid accidental clashes with irrelevant module symbols
 and acts as a nice self-documentation tool.

 3) scope-local imports
 ```
 template tmpl(T)
 {
      import std.algorithm : remove;
 }
 ```


4) static imports

static import std.stdio;

void main ()
{
     std.stdio.writeln("foo"); // fully qualified name is required
}

2 + 3) combining selective and renamed imports

import io = std.stdio : println = writeln;

void main ()
{
     println("foo");
     io.writeln("bar");
}

6) aliases

import std.stdio;
alias println = writeln;

void main ()
{
     println("foo");
}

-- 
/Jacob Carlborg
Nov 07 2013
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 7 November 2013 at 09:09:13 UTC, Jacob Carlborg 
wrote:

 4) static imports

 static import std.stdio;

 void main ()
 {
     std.stdio.writeln("foo"); // fully qualified name is 
 required
 }

 2 + 3) combining selective and renamed imports

 import io = std.stdio : println = writeln;

 void main ()
 {
     println("foo");
     io.writeln("bar");
 }

 6) aliases

 import std.stdio;
 alias println = writeln;

 void main ()
 {
     println("foo");
 }


Yeah, also good additions! I do favor renamed imports over static 
ones in most code because fully qualified names tend to be rather 
long for anything other than Phobos. It is still extremely useful 
for code generation stuff though.
Nov 07 2013
prev sibling parent reply "Tavi Cacina" <octavian.cacina outlook.com> writes:
On Monday, 4 November 2013 at 13:46:58 UTC, Dicebot wrote:

 Ok, finally making some conclusions.


Sorry if late or already discussed: I think would be useful to 
provide a way to define the default logger module-wise. Something 
like:

// myapp/feature.d
module myapp.feature;
import std.logging;

// define the log category for this module
mixin LogCategory!("myapp.feature");

/* The LogCategory would inject something like:
   ref Logger logger(LogLevel level)
   {
     find the first in this order
       LogManager.logger("myapp.feature"),
       LogManager.logger("myapp"),
       LogManager.defaultLogger();
   }
*/

void fun()
{
   warn("ups");
   // the std.logging.warn() should get the Logger from
   // the current module's function 'logger' or a default one.
}
Nov 04 2013
parent "Dicebot" <public dicebot.lv> writes:
On Monday, 4 November 2013 at 21:31:52 UTC, Tavi Cacina wrote:

 On Monday, 4 November 2013 at 13:46:58 UTC, Dicebot wrote:

 Ok, finally making some conclusions.


 Sorry if late or already discussed: I think would be useful to 
 provide a way to define the default logger module-wise. 
 Something like:


In general it should be possible to implement by inspecting a 
fully qualified module name passed in logging payload and 
multiplexing based on it. Only thing that concerns me is choosing 
between module-filtered and global output for any given log 
statement, which is what I have mentioned in my overview.
Nov 04 2013