www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Throwable.TraceInfo

reply "david" <david some.org> writes:
I have a question regarding Throwable.TraceInfo.


=== Context ===
The application we're developing can have a high throughput of
Exceptions.
Whenever an exception occurs we log it.
Currently, this is done via the Throwable.toString() method.

The problem is this takes ages - around 2-3ms per exception!
If we don't toString it, the whole process takes a mere few usecs
(as it should).
That's 3 orders of magnitude difference.

What I want to do, is just dump the stack-trace itself (the
pointers) - and produce the strings later on, when reading the
logs.


=== To The Point ===
TraceInfo is an interface with unspecified compile-time
implementation.
During runtime, the DefaultTraceInfo is set as the runtime Trace
handler.
This lovely class, has a private member
           void*[MAXFRAMES]  callstack
I want to access this member. However I can't.
I can't even do this with compile time reflection, as it's
runtime assigned and the type itslef is hidden in the assigning
method the returns an interface implementation.
Why make this member private?
Why not make the interface expose a getCallstack() method - which
implementors can either implement, or return empty?

If we don't make this change to the interface - I'm left with
pointless code duplication from druntime.
Jan 28 2015
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
AFAIK it is exactly generation of stack trace that takes so long. 
Recently it was changes to be generated lazily which is why 
version without `toString` is much faster. I don't think it can 
be any better if you want stack trace. For getting just the 
exception data one can directly access `e.msg` / `e.file` / 
`e.line`
Jan 28 2015
next sibling parent reply "david" <david some.org> writes:
On Wednesday, 28 January 2015 at 14:17:36 UTC, Dicebot wrote:

 AFAIK it is exactly generation of stack trace that takes so 
 long. Recently it was changes to be generated lazily which is 
 why version without `toString` is much faster. I don't think it 
 can be any better if you want stack trace. For getting just the 
 exception data one can directly access `e.msg` / `e.file` / 
 `e.line`


I'll try to hack away a version with my own handler - and check 
what's actually taking time there.
However, I wouldn't think a simple stack traversal (basically ~15 
indirections) takes a 2-3ms - that's a million cycles for 15 
simple actions.
I would assume something in the order of 100 cycles per frame 
(say) - so you're left with handling in the usecs
Jan 28 2015
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 28 January 2015 at 14:41:43 UTC, david wrote:

 On Wednesday, 28 January 2015 at 14:17:36 UTC, Dicebot wrote:

 AFAIK it is exactly generation of stack trace that takes so 
 long. Recently it was changes to be generated lazily which is 
 why version without `toString` is much faster. I don't think 
 it can be any better if you want stack trace. For getting just 
 the exception data one can directly access `e.msg` / `e.file` 
 / `e.line`


 I'll try to hack away a version with my own handler - and check 
 what's actually taking time there.
 However, I wouldn't think a simple stack traversal (basically 
 ~15 indirections) takes a 2-3ms - that's a million cycles for 
 15 simple actions.
 I would assume something in the order of 100 cycles per frame 
 (say) - so you're left with handling in the usecs


Yeah, sorry, I wasn't reading carefully the first time - it is 
generation of actual stack trace strings with all symbols that is 
very costly, not raw traversal.

About the rationale for not exposing it - official position is 
that exceptions are for exceptional cases and if those happen 
often, then program is wrong. Yes, I do know that this is not 
very useful official position :)
Jan 28 2015
prev sibling parent reply "david" <david some.org> writes:
On Wednesday, 28 January 2015 at 14:17:36 UTC, Dicebot wrote:

 AFAIK it is exactly generation of stack trace that takes so 
 long. Recently it was changes to be generated lazily which is 
 why version without `toString` is much faster. I don't think it 
 can be any better if you want stack trace. For getting just the 
 exception data one can directly access `e.msg` / `e.file` / 
 `e.line`


BTW: The code of the 'ctor generates the calltrace - so it's not 
lazy!
 From the druntime:

auto  stackTop    = getBasePtr();
                     auto  stackBottom = cast(void**) 
thread_stackBottom();
                     void* dummy;

                     if( stackTop && &dummy < stackTop && stackTop 
< stackBottom )
                     {
                         auto stackPtr = stackTop;

                         for( numframes = 0; stackTop <= stackPtr 
&&
                                             stackPtr < 
stackBottom &&
                                             numframes < 
MAXFRAMES; )
                         {
                             callstack[numframes++] = *(stackPtr + 
1);
                             stackPtr = cast(void**) *stackPtr;
                         }
                     }
Jan 28 2015
parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Wednesday, 28 January 2015 at 14:45:04 UTC, david wrote:

 BTW: The code of the 'ctor generates the calltrace - so it's 
 not lazy!


No, getting the pointers of the stack trace is not lazy. 
Translating them into a string via debug symbols is lazy. Btw 
what platform are you on?
Jan 28 2015
parent reply "david" <david some.org> writes:
On Wednesday, 28 January 2015 at 14:59:36 UTC, Benjamin Thaut 
wrote:

 On Wednesday, 28 January 2015 at 14:45:04 UTC, david wrote:

 BTW: The code of the 'ctor generates the calltrace - so it's 
 not lazy!


 No, getting the pointers of the stack trace is not lazy. 
 Translating them into a string via debug symbols is lazy. Btw 
 what platform are you on?


We're exclusively on Linux.
And no - I don't want the strings during runtime.
I understand generating strings can be time consuming.
I'm willing to offload this to offline processing - during 
runtime, I just want access to the calltrace - i.e.    
void*[NUM_FRAMES].
I feel like I need to resort to code duplication from druntime :(
Jan 28 2015
next sibling parent "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Wednesday, 28 January 2015 at 15:20:57 UTC, david wrote:

 We're exclusively on Linux.
 And no - I don't want the strings during runtime.
 I understand generating strings can be time consuming.
 I'm willing to offload this to offline processing - during 
 runtime, I just want access to the calltrace - i.e.    
 void*[NUM_FRAMES].
 I feel like I need to resort to code duplication from druntime 
 :(


Yes, you will have to write your own TraceHandler to get this 
easily done. If you want to do a PR for druntime it will take 
some time until it will go through. And if a PR is done it would 
be neccessary to do it in a way so that it works on all 
plattforms.
Jan 28 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wed, 28 Jan 2015 15:20:56 +0000, david wrote:


 We're exclusively on Linux.
 And no - I don't want the strings during runtime.
 I understand generating strings can be time consuming.
 I'm willing to offload this to offline processing - during runtime, I
 just want access to the calltrace - i.e. void*[NUM_FRAMES].
 I feel like I need to resort to code duplication from druntime :(


there is nothing wrong in such code duplication.=
Jan 28 2015
prev sibling next sibling parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
Why don't you simply store references to the TraceInfo objects 
until you want to print them?
Jan 28 2015
parent "david" <david some.org> writes:
On Wednesday, 28 January 2015 at 14:19:32 UTC, Benjamin Thaut 
wrote:

 Why don't you simply store references to the TraceInfo objects 
 until you want to print them?


I can't store theses references.
The whole data is blited away to a shared-memory (from which
it is periodically dumped to disk).
This bliting needs to be fast - as (obviously) when talking about 
high
throughput exceptions - we're on the "hot" path.
Jan 28 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-28 15:09, david wrote:


 This lovely class, has a private member
            void*[MAXFRAMES]  callstack
 I want to access this member. However I can't.


Taking the address of the variable should bypass the protection. Or 
using .tupleof.

-- 
/Jacob Carlborg
Jan 28 2015