www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Profiling after exit()

reply Eugene Wissner <belka caraus.de> writes:
I have a multi-threaded application, whose threads normally run 
forever. But I need to profile this program, so I compile the 
code with -profile, send a SIGTERM and call exit(0) from my 
signal handler to exit the program. The problem is that I get the 
profiling information only from the main thread, but not from the 
other ones.

Is there a way to get the profiling information from all threads 
before terminating the program? Maybe some way to finish the 
threads gracefully? or manully call "write trace.log"-function 
for a thread?

Here is a small example that demonstrates the problem:

import core.thread;
import core.stdc.stdlib;

shared bool done = false;

void run()
{
     while (!done)
     {
         foo;
     }
}

void foo()
{
     new Object;
}

void main()
{
     auto thread = new Thread(&run);
     thread.start;
     Thread.sleep(3.seconds);

     exit(0); // Replace with "done = true;" to get the expected 
behaviour.
}

There is already an issue: 
https://issues.dlang.org/show_bug.cgi?id=971
The hack was to call trace_term() in internal/trace. call_term() 
doesn't exist anymore, I tried to export the static destructor 
from druntime/src/rt/trace.d with:

extern (C) void _staticDtor449()  nogc nothrow;

(on my system) and call it manually. I get some more information 
this way, but the numbers in the profiling report are still wrong.
Jul 27 2017
next sibling parent reply Temtaime <temtaime gmail.com> writes:
Exit is not "normal exit" for D programs so, do not use it.
Your threads should stop at some point to make able the app exit 
successfully.
There's a "join" method. You can use it with your "done" variable.
Jul 27 2017
parent reply Temtaime <temtaime gmail.com> writes:
Also there was an issue that profiling doesn't work with 
multi-threaded apps and leads to a crash.
Don't know if it is fixed.
Jul 27 2017
parent Mario =?UTF-8?B?S3LDtnBsaW4=?= <linkrope github.com> writes:
On Thursday, 27 July 2017 at 14:44:31 UTC, Temtaime wrote:

 Also there was an issue that profiling doesn't work with 
 multi-threaded apps and leads to a crash.
 Don't know if it is fixed.


Was fixed two years ago:
http://forum.dlang.org/post/mia2kf$djb$1 digitalmars.com
Jul 27 2017
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 27 July 2017 at 14:30:33 UTC, Eugene Wissner wrote:

 I have a multi-threaded application, whose threads normally run 
 forever. But I need to profile this program, so I compile the 
 code with -profile, send a SIGTERM and call exit(0) from my 
 signal handler to exit the program. The problem is that I get 
 the profiling information only from the main thread, but not 
 from the other ones.

 [...]


You will need to run it single threaded.
If you want to use the builtin-profiler.
Jul 27 2017
parent reply Eugene Wissner <belka caraus.de> writes:
On Thursday, 27 July 2017 at 14:52:18 UTC, Stefan Koch wrote:

 On Thursday, 27 July 2017 at 14:30:33 UTC, Eugene Wissner wrote:

 I have a multi-threaded application, whose threads normally 
 run forever. But I need to profile this program, so I compile 
 the code with -profile, send a SIGTERM and call exit(0) from 
 my signal handler to exit the program. The problem is that I 
 get the profiling information only from the main thread, but 
 not from the other ones.

 [...]


 You will need to run it single threaded.
 If you want to use the builtin-profiler.


Are there profilers that work well with dmd? valgrind? OProfile?
Jul 27 2017
parent Martin Nowak <code dawg.eu> writes:
On Thursday, 27 July 2017 at 15:16:35 UTC, Eugene Wissner wrote:

 Are there profilers that work well with dmd? valgrind? OProfile?


Yes, any sampling profiler works fine, e.g. perf on linux, Intel 
VTune/AMD CodeXL on Windows.
Those directly monitor CPU performance counters and have a 
negligible performance overhead compared with dmd's instrumenting 
profiler, also they don't require rebuilding of binaries.
Mar 06 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-27 16:30, Eugene Wissner wrote:

 I have a multi-threaded application, whose threads normally run forever. 
 But I need to profile this program, so I compile the code with -profile, 
 send a SIGTERM and call exit(0) from my signal handler to exit the 
 program. The problem is that I get the profiling information only from 
 the main thread, but not from the other ones.
 
 Is there a way to get the profiling information from all threads before 
 terminating the program? Maybe some way to finish the threads 
 gracefully? or manully call "write trace.log"-function for a thread?


As others have mentioned, you should in general avoid calling "exit" in 
a D program. There's a C function called "atexit" that allows to 
register a callback that is called after calling "exit". You could 
perhaps join the threads there. I don't know if that helps with the 
profiling though.

-- 
/Jacob Carlborg
Jul 27 2017
parent reply Eugene Wissner <belka caraus.de> writes:
On Friday, 28 July 2017 at 06:32:59 UTC, Jacob Carlborg wrote:

 On 2017-07-27 16:30, Eugene Wissner wrote:

 I have a multi-threaded application, whose threads normally 
 run forever. But I need to profile this program, so I compile 
 the code with -profile, send a SIGTERM and call exit(0) from 
 my signal handler to exit the program. The problem is that I 
 get the profiling information only from the main thread, but 
 not from the other ones.
 
 Is there a way to get the profiling information from all 
 threads before terminating the program? Maybe some way to 
 finish the threads gracefully? or manully call "write 
 trace.log"-function for a thread?


 As others have mentioned, you should in general avoid calling 
 "exit" in a D program. There's a C function called "atexit" 
 that allows to register a callback that is called after calling 
 "exit". You could perhaps join the threads there. I don't know 
 if that helps with the profiling though.


Unfortunately I can't join threads, because the program wouldn't 
exit then, the threads run forever normally. I thought maybe 
there is some way to kill a thread gracefully in linux, so it can 
write its profiling information; or another way to get profiling.
Thanks anyway.
Jul 28 2017
parent reply Temtaime <temtaime gmail.com> writes:
On Friday, 28 July 2017 at 08:06:33 UTC, Eugene Wissner wrote:

 On Friday, 28 July 2017 at 06:32:59 UTC, Jacob Carlborg wrote:

 On 2017-07-27 16:30, Eugene Wissner wrote:

 I have a multi-threaded application, whose threads normally 
 run forever. But I need to profile this program, so I compile 
 the code with -profile, send a SIGTERM and call exit(0) from 
 my signal handler to exit the program. The problem is that I 
 get the profiling information only from the main thread, but 
 not from the other ones.
 
 Is there a way to get the profiling information from all 
 threads before terminating the program? Maybe some way to 
 finish the threads gracefully? or manully call "write 
 trace.log"-function for a thread?


 As others have mentioned, you should in general avoid calling 
 "exit" in a D program. There's a C function called "atexit" 
 that allows to register a callback that is called after 
 calling "exit". You could perhaps join the threads there. I 
 don't know if that helps with the profiling though.


 Unfortunately I can't join threads, because the program 
 wouldn't exit then, the threads run forever normally. I thought 
 maybe there is some way to kill a thread gracefully in linux, 
 so it can write its profiling information; or another way to 
 get profiling.
 Thanks anyway.


There's no "gracefully" way to kill a thread.
If your thread cannot join, then you're doing something wrong
Jul 28 2017
parent reply Mario =?UTF-8?B?S3LDtnBsaW4=?= <linkrope github.com> writes:
On Friday, 28 July 2017 at 09:02:10 UTC, Temtaime wrote:

 There's no "gracefully" way to kill a thread.
 If your thread cannot join, then you're doing something wrong


Our programs are intended to run "forever". 24/7 servers.
Jul 28 2017
parent Jacob Carlborg <doob me.com> writes:
On 2017-07-28 11:30, Mario Kröplin wrote:


 Our programs are intended to run "forever". 24/7 servers.


What's wrong with having a bool that determines if the loop should 
continue running?

-- 
/Jacob Carlborg
Jul 28 2017