digitalmars.D.learn - Hunting down rogue memory allocations?
- Gary Willoughby (6/6) Oct 02 2014 Say i have created a program written in D, what tools are
- Kiith-Sa (23/29) Oct 02 2014 If *time* spent by allocations is a problem, profile with `perf
- Gary Willoughby (2/32) Oct 02 2014 Great thanks, I'll look into those.
- Jacob Carlborg (4/8) Oct 03 2014 If you have OS X, you can use Instruments that comes installed with Xcod...
- thedeemon (13/15) Oct 03 2014 I wrote a tiny module trackallocs.d that inserts a GC proxy and
Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them?
Oct 02 2014
On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote:Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them?If *time* spent by allocations is a problem, profile with `perf top` (assuming you have Linux): Look for 'gc', 'malloc', 'calloc', etc. (Plain perf record will also work, but not be as quick/interactive. CodeXL works too.) See https://perf.wiki.kernel.org/index.php/Tutorial - you probably need some specific arguments to get caller info, didn't use it for a while so I don't remember. If e.g. the GC is an issue, it should be immediately evident with some GC function taking e.g. over 5% of time. Usually the actual overhead will be much higher but divided into multiple smaller functions that will take little time individually. Drill down into one of these and look at its callers. Eliminate the most common source of calls, look again, repeat. Usually removing a source of alloc calls will result in better speedup than the profiler suggests. If *space* is a problem, Valgrind doesn't work with most D programs for some reason (probably D's fault), so, good luck with that. But eliminating the biggest time wasters usually helps space as well (or rather, it makes it more controllable as you know better where you allocate memory).
Oct 02 2014
"Gary Willoughby" <dev nomad.so> On Thursday, 2 October 2014 at 20:31:29 UTC, Kiith-Sa wrote:On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote:Great thanks, I'll look into those.Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them?If *time* spent by allocations is a problem, profile with `perf top` (assuming you have Linux): Look for 'gc', 'malloc', 'calloc', etc. (Plain perf record will also work, but not be as quick/interactive. CodeXL works too.) See https://perf.wiki.kernel.org/index.php/Tutorial - you probably need some specific arguments to get caller info, didn't use it for a while so I don't remember. If e.g. the GC is an issue, it should be immediately evident with some GC function taking e.g. over 5% of time. Usually the actual overhead will be much higher but divided into multiple smaller functions that will take little time individually. Drill down into one of these and look at its callers. Eliminate the most common source of calls, look again, repeat. Usually removing a source of alloc calls will result in better speedup than the profiler suggests. If *space* is a problem, Valgrind doesn't work with most D programs for some reason (probably D's fault), so, good luck with that. But eliminating the biggest time wasters usually helps space as well (or rather, it makes it more controllable as you know better where you allocate memory).
Oct 02 2014
On 02/10/14 22:31, Kiith-Sa wrote:If *time* spent by allocations is a problem, profile with `perf top` (assuming you have Linux): Look for 'gc', 'malloc', 'calloc', etc. (Plain perf record will also work, but not be as quick/interactive. CodeXL works too.)If you have OS X, you can use Instruments that comes installed with Xcode. -- /Jacob Carlborg
Oct 03 2014
On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote:Say i have created a program written in D, what tools are available for me to track memory allocations?I wrote a tiny module trackallocs.d that inserts a GC proxy and outputs to log file (or stdout) all the allocations, gathering some stats. You can turn tracking on and off in any place of your code. Used it before 2.066 where nogc appeared to check whether my code was allocation-free. Current version works in both 2.066 and earlier versions. Find it here: https://bitbucket.org/infognition/dstuff/src/ Inside it uses one hack based on knowledge of D runtime internals, so in future versions of D runtime, if Proxy struct changes again, it may need to be changed as well.
Oct 03 2014
On Friday, 3 October 2014 at 08:18:45 UTC, thedeemon wrote:On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote:This is really cool stuff, I needed something like this but didn't know it was possible. Mind if I use some parts of it in my profiler? (there's no license)Say i have created a program written in D, what tools are available for me to track memory allocations?I wrote a tiny module trackallocs.d that inserts a GC proxy and outputs to log file (or stdout) all the allocations, gathering some stats. You can turn tracking on and off in any place of your code. Used it before 2.066 where nogc appeared to check whether my code was allocation-free. Current version works in both 2.066 and earlier versions. Find it here: https://bitbucket.org/infognition/dstuff/src/ Inside it uses one hack based on knowledge of D runtime internals, so in future versions of D runtime, if Proxy struct changes again, it may need to be changed as well.
Oct 03 2014
On Friday, 3 October 2014 at 09:27:50 UTC, Kiith-Sa wrote:https://bitbucket.org/infognition/dstuff/src/Mind if I use some parts of it in my profiler? (there's no license)Sure, it's in public domain (as noted in readme).
Oct 03 2014