www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Crash handler with stack trace

reply Jeremie Pelletier <jeremiep gmail.com> writes:
I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:

http://jump.fm/UVYHG

It includes the runtime handler, a PE reader, CodeView reader and crash report
window.

So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.

Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.
Sep 19 2009
next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Jeremie Pelletier wrote:

 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG
 
 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.
 
 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.
 
 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.


Thanks a lot, Jeremie! The code looks really good. I think I'll be 
stealing the PE && CV parsing parts for Tango :)

I'm curious about the runtime project of yours :o Can you shed a bit of 
the secret? How will it compare to druntime? What's the target audience? 
Does it have anything to do with 'meshes and textures'? ;)


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Sep 19 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Tom S Wrote:


 Jeremie Pelletier wrote:

 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG
 
 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.
 
 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.
 
 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.


 
 Thanks a lot, Jeremie! The code looks really good. I think I'll be 
 stealing the PE && CV parsing parts for Tango :)


I'm glad to hear that! Just keep my name in the credits and I'll be happy :)


 I'm curious about the runtime project of yours :o Can you shed a bit of 
 the secret? How will it compare to druntime? What's the target audience? 
 Does it have anything to do with 'meshes and textures'? ;)


Well it started a bit over two years ago, back then there was no druntime, I
was digging into phobos to learn more about the language and decided it would
be a fun and challenging task to write my own runtime. Then once I had it
running stable I found out druntime had been created in the meantime!

In any ways, I learned quite a bit of the language's internals and ABI while
coding and debugging it; I began by removing all default paths and libraries
from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and
made an empty object.d and built my way up until there was no more compiler
crashes (dmd is quite whiny about not having its "intrinsic" symbols declared,
mostly the typeinfo's) and all unresolved symbols were added.

Since then the runtime was ported to D2 and kept up to date with every new
release, I've been using it for these two years and it works great.

I don't really know how it will compare to druntime, I'm not out there to
compete with them really. I use DMD on both windows and posix platforms (Win7
and Ubuntu respectively) so its the only supported compiler right now, haven't
tested 64-bit support either. It would be able to support libraries such as
phobos and tango with a few changes to the places that call into the compiler
runtime or memory manager.

Right now the target audience is me. I haven't really considered who the target
audience might be since I touch a bit of everything. I try and make it as
generic and as modular as I can, so if someone doesn't need a package they can
remove it from their build without getting tons of compile errors. I myself am
focusing towards GUI application development with unified 2D/3D rendering and
custom controls on top of that rendering as part of the runtime library, among
many other things.

So you could say it has to do with meshes and textures, although I haven't
gotten to making interfaces for that part yet. The most I did on that level
with my runtime was to write a reader for Blizzard's MPQ archives and extract
model and texture data to render it in opengl, but apart from the MPQ reader it
was all pretty ugly mixing win32 and gl code along with other stuff in the same
routine to get the model on screen as fast as possible ;)

I really am just beginning to notice people are interested in my project, I
don't know when I'll put it on dsource and/or sourceforge. And given that D2 is
still under heavy development it wouldn't be practical right now. I don't mind
releasing bits and pieces of it but I would much rather see the first public
release of it to be an usable one, although the compiler implementation part is
pretty much stable at this point, if there's enough demand for it I might begin
with that.

Jeremie
Sep 19 2009
next sibling parent reply "Vladimir Panteleev" <thecybershadow gmail.com> writes:
On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep gmail.com>  
wrote:


 I'm curious about the runtime project of yours :o Can you shed a bit of
 the secret? How will it compare to druntime? What's the target audience?
 Does it have anything to do with 'meshes and textures'? ;)


 [snip]


This is cool stuff. I'd be very interested in finding out how you handled  
memory management / garbage collection. Or have you used the standard GC  
that comes with Phobos/Tango?

-- 
Best regards,
  Vladimir                          mailto:thecybershadow gmail.com
Sep 20 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Vladimir Panteleev Wrote:


 On Sun, 20 Sep 2009 08:25:30 +0300, Jeremie Pelletier <jeremiep gmail.com>  
 wrote:
 

 I'm curious about the runtime project of yours :o Can you shed a bit of
 the secret? How will it compare to druntime? What's the target audience?
 Does it have anything to do with 'meshes and textures'? ;)


 [snip]


 
 This is cool stuff. I'd be very interested in finding out how you handled  
 memory management / garbage collection. Or have you used the standard GC  
 that comes with Phobos/Tango?


I forked the original D1 gc and rewrote it almost entirely. I made numerous
speed improvements over it, such as greatly reducing the number of pool
lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan'
bitfields from pools and rewrote the garbage collector to only use the 'mark'
one, I also merged the sweep and freelist rebuild phases together to save on
the overhead of yet another loop over the entire working set, also added
support for the gc to free memory pools if they're entirely free.

I have it on pastebin if you want to take a look:
http://pastebin.com/f7a3b4c4a

Jeremie
Sep 20 2009
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:


I forked the original D1 gc and rewrote it almost entirely. I made numerous
speed improvements over it, such as greatly reducing the number of pool
lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan'
bitfields from pools and rewrote the garbage collector to only use the 'mark'
one, I also merged the sweep and freelist rebuild phases together to save on
the overhead of yet another loop over the entire working set, also added
support for the gc to free memory pools if they're entirely free.<


That looks like lot of work. Too bad we don't have a suite of GC-heavy
benchmarks to test it. (Maybe Lucarella has collected something in the
meantime).

Bye,
bearophile
Sep 20 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
bearophile wrote:

 Jeremie Pelletier:
 

 I forked the original D1 gc and rewrote it almost entirely. I made numerous
speed improvements over it, such as greatly reducing the number of pool
lookups, smarter alloc/realloc algorithms, removed the 'freebits' and 'scan'
bitfields from pools and rewrote the garbage collector to only use the 'mark'
one, I also merged the sweep and freelist rebuild phases together to save on
the overhead of yet another loop over the entire working set, also added
support for the gc to free memory pools if they're entirely free.<


 
 That looks like lot of work. Too bad we don't have a suite of GC-heavy
benchmarks to test it. (Maybe Lucarella has collected something in the
meantime).
 
 Bye,
 bearophile


I couldn't agree more about the need for a GC benchmark suite, writing 
this module involved some of the hardest debugging work I ever did, even 
with printfs, memory dumps and windbg.

I do believe it to be stable enough for general usage in its current 
state. Its been some time now since I last had any issues with it.

If you come across such a benchmark suite, let me know, I would love to 
see how it performs against druntime's GC.

Jeremie
Sep 20 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:


 I couldn't agree more about the need for a GC benchmark suite,


I am slowly creating one (a general one) that later may be used for LDC :-)



 If you come across such a benchmark suite, let me know, I would love to 
 see how it performs against druntime's GC.


Similar GC benchmark suites don't grow on trees. I don't think one exists yet
for D (ask to Lucarella too).
You can start testing your runtime against programs that you can find in the
"binarytrees" and "gc" subdirectories here:
http://www.fantascienza.net/leonardo/js/slow_d.zip
And I'd like to know the results with both runtimes :-)

Bye,
bearophile
Sep 20 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
bearophile, el 20 de septiembre a las 22:31 me escribiste:

 Jeremie Pelletier:
 

 I couldn't agree more about the need for a GC benchmark suite,


 
 I am slowly creating one (a general one) that later may be used for LDC
 :-)
 
 

 If you come across such a benchmark suite, let me know, I would love
 to see how it performs against druntime's GC.


 
 Similar GC benchmark suites don't grow on trees. I don't think one
 exists yet for D (ask to Lucarella too).


Unfortunately I had not much success at making a benchmark suite. I think
the main reason is the... "diversity" you find in D. You have D1/Phobos,
D1/Tango and D2/Druntime. Doing a benchmark suite that works with all is
very hard, you have to "port" the benchmark to 3 almost different
languages. And any decent GC benchmark test *real* programs. I was unable
to find any real programs (suitable for a GC benchmarks at least) for
D that are properly maintained except Dil, which is the only program
I plan to use in my benchmarks (I had a lot of problems lately with a bug
in my Naive collector that I'm still trying to find). Picking Dil means to
stay in the D1 Tango world, because I don't have the time to port Dil to
D1/Phobos nor D2/Druntime, so there it goes the "cross-D-version"
benchmark.

I only have that, and a few micro-benchmarks I collected in the 3 or
4 years I've been lurking this newsgroup (there are very few though). I'll
have to make at least a few more micro-benchmarks for multi-threaded
applications, since my plan is to add concurrency to the GC, so I'll have
to measure that scenario.

You can find my benchmark sources in this git repo (is not very
interesting for now, really):
http://git.llucax.com.ar/w/software/dgc/dgcbench.git

In my blog are a few (meaningless but nice) graphs for the Naive GC =)
http://proj.llucax.com.ar/blog/dgc/blog

I'll posting the results of the tests using the Tango basic collector
soon.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
El día que falten los niños, que sobren las mujeres y que se prenda
fuego el último árbol, será el Apocalípsis.
	-- Ricardo Vaporeso. Camino Negro, 1916.
Sep 21 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:


 I was unable
 to find any real programs (suitable for a GC benchmarks at least) for
 D that are properly maintained except Dil,


Have you tried Blaze?

Bye,
bearophile
Sep 22 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
bearophile, el 22 de septiembre a las 07:28 me escribiste:

 Leandro Lucarella:
 

 I was unable
 to find any real programs (suitable for a GC benchmarks at least) for
 D that are properly maintained except Dil,


 
 Have you tried Blaze?


Blaze is an engine AFAIK, not a program. And an engine to build games
AFAIK, and game aren't usually a good candidate for benchmarking since
they can't be automated.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
VECINOS RESCATARON A CABALLITO ATROPELLADO
	-- Crónica TV
Sep 23 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


Blaze is a program that simulates 2D physics written in D.
http://www.dsource.org/projects/blaze
It's an engine, but it comes with an extensive demo that is and can be used as
benchmark. So I think it can be used for your purposes (but I don't know how
many memory allocations it performs).

Bye,
bearophile
Sep 23 2009
parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com> wrote:

 Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


 Blaze is a program that simulates 2D physics written in D.
 http://www.dsource.org/projects/blaze
 It's an engine, but it comes with an extensive demo that is and can be used as
benchmark. So I think it can be used for your purposes (but I don't know how
many memory allocations it performs).


It performs *way too fucking many*. ;) Which, I suppose, would make it
a good GC benchmark.
Sep 23 2009
next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Jarrett Billingsley, el 23 de septiembre a las 12:03 me escribiste:

 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com> wrote:

 Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


 Blaze is a program that simulates 2D physics written in D.
 http://www.dsource.org/projects/blaze
 It's an engine, but it comes with an extensive demo that is and can be
 used as benchmark. So I think it can be used for your purposes (but
 I don't know how many memory allocations it performs).


 
 It performs *way too fucking many*. ;) Which, I suppose, would make it
 a good GC benchmark.


I'll take a look at the demos then, thanks for the tips.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Aprendan de la primavera, que antecede al verano, precede al inverno y
no lo anda diciendo por ahí.
	-- Ricardo Vaporeso. Llanos de Luzuriaga, 1914.
Sep 23 2009
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley  
<jarrett.billingsley gmail.com> wrote:


 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com>  
 wrote:

 Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


 Blaze is a program that simulates 2D physics written in D.
 http://www.dsource.org/projects/blaze
 It's an engine, but it comes with an extensive demo that is and can be  
 used as benchmark. So I think it can be used for your purposes (but I  
 don't know how many memory allocations it performs).


 It performs *way too fucking many*. ;) Which, I suppose, would make it
 a good GC benchmark.


A game engine that does *any* allocations per frame?
My employee would kill me for writing something like that!
Sep 23 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Denis Koroskin wrote:

 On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley 
 <jarrett.billingsley gmail.com> wrote:
 

 On Wed, Sep 23, 2009 at 11:58 AM, bearophile 
 <bearophileHUGS lycos.com> wrote:

 Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


 Blaze is a program that simulates 2D physics written in D.
 http://www.dsource.org/projects/blaze
 It's an engine, but it comes with an extensive demo that is and can 
 be used as benchmark. So I think it can be used for your purposes 
 (but I don't know how many memory allocations it performs).


 It performs *way too fucking many*. ;) Which, I suppose, would make it
 a good GC benchmark.


 
 A game engine that does *any* allocations per frame?
 My employee would kill me for writing something like that!


You have no idea how many professional game companies write sloppy code, 
they're not all over-optimizing code zealots like Carmack :)

EA is notoriously good for that, but they do spend 3x as much on 
marketing as they do on development so go figure. Their games run way 
slower at medium settings than games by, say Blizzard, would at high 
settings.
Sep 23 2009
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 23 Sep 2009 20:03:47 +0400, Jarrett Billingsley  
<jarrett.billingsley gmail.com> wrote:


 On Wed, Sep 23, 2009 at 11:58 AM, bearophile <bearophileHUGS lycos.com>  
 wrote:

 Leandro Lucarella:


 Blaze is an engine AFAIK, not a program. And an engine to build games
 AFAIK, and game aren't usually a good candidate for benchmarking since
 they can't be automated.


 Blaze is a program that simulates 2D physics written in D.
 http://www.dsource.org/projects/blaze
 It's an engine, but it comes with an extensive demo that is and can be  
 used as benchmark. So I think it can be used for your purposes (but I  
 don't know how many memory allocations it performs).


 It performs *way too fucking many*. ;) Which, I suppose, would make it
 a good GC benchmark.



A game engine that does *any* allocations per frame?
My employer would kill me for writing something like that!
Sep 23 2009
prev sibling parent reply Kagamin <spam here.lot> writes:
Jeremie Pelletier Wrote:


 I forked the original D1 gc and rewrote it almost entirely.


Did you try to address bug 2093?
Sep 25 2009
parent Jeremie Pelletier <jeremiep gmail.com> writes:
Kagamin wrote:

 Jeremie Pelletier Wrote:
 

 I forked the original D1 gc and rewrote it almost entirely.


 
 Did you try to address bug 2093?


I just did a quick compile of the bug's test case and it worked correctly.

I wasn't even aware of that bug to tell the truth.
Sep 25 2009
prev sibling parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Jeremie Pelletier wrote:

 Tom S Wrote:
 

 Jeremie Pelletier wrote:

 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:

 http://jump.fm/UVYHG

 It includes the runtime handler, a PE reader, CodeView reader and crash report
window.

 So far it supports Windows and some bits are already done for posix platforms.
It does a register dump, stack trace with resolved symbol names with their
declaration file and line, as well as a listing of all loaded modules.

 Sorry for the lack of phobos/tango support, this is part of a runtime project
I'm building which should hopefully be released to dsource in a few months. I'm
releasing it now so if someone wants to implement a phobos or tango port
they're free to do it.


 Thanks a lot, Jeremie! The code looks really good. I think I'll be 
 stealing the PE && CV parsing parts for Tango :)


 
 I'm glad to hear that! Just keep my name in the credits and I'll be happy :)


Sure thing, I generally stick to the same philosophy :)



 I'm curious about the runtime project of yours :o Can you shed a bit of 
 the secret? How will it compare to druntime? What's the target audience? 
 Does it have anything to do with 'meshes and textures'? ;)


 
 Well it started a bit over two years ago, back then there was no druntime, I
was digging into phobos to learn more about the language and decided it would
be a fun and challenging task to write my own runtime. Then once I had it
running stable I found out druntime had been created in the meantime!
 
 In any ways, I learned quite a bit of the language's internals and ABI while
coding and debugging it; I began by removing all default paths and libraries
from dmd (both in sc.ini and compiling with "--defaultlib= --debuglib=--") and
made an empty object.d and built my way up until there was no more compiler
crashes (dmd is quite whiny about not having its "intrinsic" symbols declared,
mostly the typeinfo's) and all unresolved symbols were added.
 
 Since then the runtime was ported to D2 and kept up to date with every new
release, I've been using it for these two years and it works great.
 
 I don't really know how it will compare to druntime, I'm not out there to
compete with them really. I use DMD on both windows and posix platforms (Win7
and Ubuntu respectively) so its the only supported compiler right now, haven't
tested 64-bit support either. It would be able to support libraries such as
phobos and tango with a few changes to the places that call into the compiler
runtime or memory manager.


Thanks for the in-depth background story! Looks like you went a similar 
route that folks trying to do kernels in D choose. Perhaps if you had 
mentioned the development of this runtime earlier, druntime could be a 
joint project... But maybe it's not too late and druntime could still 
benefit from your expertise? There are scarcely few guys who understand 
the guts of writing D runtimes - I could probably count them all on the 
fingers of one palm ;) And I don't necessarily mean the concrete 
implementations but also the top-level structure.



 Right now the target audience is me. I haven't really considered who the
target audience might be since I touch a bit of everything. I try and make it
as generic and as modular as I can, so if someone doesn't need a package they
can remove it from their build without getting tons of compile errors. I myself
am focusing towards GUI application development with unified 2D/3D rendering
and custom controls on top of that rendering as part of the runtime library,
among many other things.


So it's floor wax *and* dessert topping? Sounds like you're set on doing 
  _everything_ from scratch :o I don't think nowadays I'd be willing to 
live without Tango's streams...



 So you could say it has to do with meshes and textures, although I haven't
gotten to making interfaces for that part yet. The most I did on that level
with my runtime was to write a reader for Blizzard's MPQ archives and extract
model and texture data to render it in opengl, but apart from the MPQ reader it
was all pretty ugly mixing win32 and gl code along with other stuff in the same
routine to get the model on screen as fast as possible ;)


Haha, well that's a noble goal. I'll have to get myself some readers for 
  third-party formats as well. In retrospect, writing a custom 3ds max 
exporter and supporting only it was a bad idea :P



 I really am just beginning to notice people are interested in my project, I
don't know when I'll put it on dsource and/or sourceforge. And given that D2 is
still under heavy development it wouldn't be practical right now. I don't mind
releasing bits and pieces of it but I would much rather see the first public
release of it to be an usable one, although the compiler implementation part is
pretty much stable at this point, if there's enough demand for it I might begin
with that.


I'm sure people would become interested way earlier, had you mentioned 
the project anywhere :P I'm still in D1 land, but there are folks trying 
to use D2 and craving for some code to look into and build upon. The 
more code we release to the general public, the bigger D feels like and 
the greater chances it has of attracting new users... at least that's 
what I like to believe :P


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Sep 20 2009
prev sibling parent reply grauzone <none example.net> writes:
Jeremie Pelletier wrote:

 I've been asked for my runtime crash handler with it's CodeView reader earlier
in digitalmars.D so here it is:
 
 http://jump.fm/UVYHG


Wanted to take a look at it again, but the site is defunct.
Sep 27 2009
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
grauzone wrote:

 Jeremie Pelletier wrote:

 I've been asked for my runtime crash handler with it's CodeView reader 
 earlier in digitalmars.D so here it is:

 http://jump.fm/UVYHG


 
 Wanted to take a look at it again, but the site is defunct.


http://www.mediafire.com/download.php?wy2m2hrgzdo


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Sep 27 2009