www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - stub out your gc without hacking on druntime

reply "Adam D. Ruppe" <destructionator gmail.com> writes:
I decided to boil going without gc down to one simple move: put 
this code into a file called nogc.d and then add it to your dmd 
command line:

===

import core.stdc.stdio;
import core.stdc.stdlib;

extern(C):

void* gc_malloc() {
         fprintf(stderr, "GC allocations are disabled\nProgram 
terminated\n");
         asm { hlt; }
         assert(0);
}

// druntime calls these, but we can just stub them out
void gc_init() { }
void gc_addRange() { }
void gc_term() { }

// druntime makes some classes too. we'll malloc them. This 
technically
// leaks but since it is startup code I'm pretty sure it doesn't 
matter.

// this also makes new class available to user code, but remember 
to free your classes and call the destructor:
/*
void free(Object object) {
         auto dtor = cast(void function(Object o)) 
object.classinfo.destructor;
         if(dtor)
                 dtor(object);
         free(cast(void*) object);
}
*/
extern(C) Object _d_newclass(const ClassInfo ci) {
         void* memory = malloc(ci.init.length);
         if(memory is null) {
                 fprintf(stderr, "Out of memory to allocate 
class\n");
                 asm { hlt; }
                 assert(0);
         }
         (cast(byte*) memory)[0 .. ci.init.length] = ci.init[];
         return cast(Object) memory;
}

===



You shouldn't have to modify your code nor druntime/phobos 
(though you'll probably find them being killed by hidden 
allocations!), unlike the minimal D stuff I've been talking about 
the last few weeks which replaces them entirely.

The reason it works is the gc functions come from a library file. 
.lib functions are overridden by functions with the same name in 
an object file.

So this redefines crucial gc functions, and then the linker uses 
them instead of the ones druntime provides.Thereby stubbing out 
the garbage collector in this individual exe. I tried it on both 
Windows and Linux and it seemed to work as I expected.

The resulting executable is slightly smaller too, since the 
linker can dump more functions that are never called by the stubs:

$ dmd test2.d
You have mail in /var/spool/mail/me
$ ls -lh test2
-rwxr-xr-x 1 me users 683K 2013-06-10 15:06 test2
$ dmd test2.d nogc.d
$ ls -lh test2
-rwxr-xr-x 1 me users 626K 2013-06-10 15:06 test2

(test2.d is just a random program that does writeln("ctor") and 
writeln("dtor") on a few classes to see when/if they are still 
running, nothing special there)


On IRC someone suggested an even simpler solution to me too: set 
a breakpoint at gc_malloc in your debugger. Then you can see 
where it is called and continue/stop at any time.



I found a hidden allocation in druntime using this instantly and 
already filed to bugzilla. If you are on an AMD processor you'll 
probably see it too if you try to run a program
http://d.puremagic.com/issues/show_bug.cgi?id=10323

so you won't get far with nogc.d! But if you fix that up and try 
again I was able to get my test to run.
Jun 10 2013
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jun 10, 2013 at 09:16:04PM +0200, Adam D. Ruppe wrote:

 I decided to boil going without gc down to one simple move: put this
 code into a file called nogc.d and then add it to your dmd command
 line:
 
 ===
 
 import core.stdc.stdio;
 import core.stdc.stdlib;
 
 extern(C):
 
 void* gc_malloc() {
         fprintf(stderr, "GC allocations are disabled\nProgram
 terminated\n");
         asm { hlt; }
         assert(0);
 }
 
 // druntime calls these, but we can just stub them out
 void gc_init() { }
 void gc_addRange() { }
 void gc_term() { }


Should these return errors too? Or is this just a quick-n-dirty way to
get a no-GC environment without needing hack druntime?


[...]

 extern(C) Object _d_newclass(const ClassInfo ci) {
         void* memory = malloc(ci.init.length);
         if(memory is null) {
                 fprintf(stderr, "Out of memory to allocate
 class\n");
                 asm { hlt; }
                 assert(0);
         }
         (cast(byte*) memory)[0 .. ci.init.length] = ci.init[];
         return cast(Object) memory;
 }


Hmm. What if the user code instantiates classes? Will they leak memory
then? Though I suppose the point is really just to find *hidden*
allocations, and 'new' is a pretty blatant use of the GC.


[...]

 You shouldn't have to modify your code nor druntime/phobos (though
 you'll probably find them being killed by hidden allocations!),
 unlike the minimal D stuff I've been talking about the last few
 weeks which replaces them entirely.
 
 The reason it works is the gc functions come from a library file.
 .lib functions are overridden by functions with the same name in an
 object file.
 
 So this redefines crucial gc functions, and then the linker uses
 them instead of the ones druntime provides.Thereby stubbing out the
 garbage collector in this individual exe. I tried it on both Windows
 and Linux and it seemed to work as I expected.


[...]

Very nice! Clever way of quickly testing for GC-dependence in a piece of
code.


T

-- 
Questions are the beginning of intelligence, but the fear of God is the
beginning of wisdom.
Jun 10 2013
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 11 June 2013 at 03:08:36 UTC, H. S. Teoh wrote:

 Or is this just a quick-n-dirty way to
 get a no-GC environment without needing hack druntime?


Yea, that's the idea here. Like Simen said, if these were errors, 
your program wouldn't even start, so noop is better. Though 
gc_init should call thread_init, which I didn't do here. Without 
that call, stack traces on exceptions will segfault on linux.

So extern (C) void thread_init(); extern(C) void gc_init() { 
thread_init(); } will be better than doing nothing.


 Hmm. What if the user code instantiates classes? Will they leak 
 memory then?


Unless you free() it! But like with the stub functions, druntime 
uses new in the startup code, so if this was removed entirely, 
you wouldn't get far. Exceptions are also useful and you gotta 
new them.


 Though I suppose the point is really just to find *hidden*
 allocations, and 'new' is a pretty blatant use of the GC.


Exactly, it isn't that hard to match new with free since it is 
clear that you used it and can follow the object's lifetime 
manually. A hidden allocation though, even though you might be 
able to free it, can slip past you.
Jun 11 2013
parent reply Hiemlick Hiemlicker <HH reign.com> writes:
On Tuesday, 11 June 2013 at 12:13:05 UTC, Adam D. Ruppe wrote:

 On Tuesday, 11 June 2013 at 03:08:36 UTC, H. S. Teoh wrote:

 [...]


 Yea, that's the idea here. Like Simen said, if these were 
 errors, your program wouldn't even start, so noop is better. 
 Though gc_init should call thread_init, which I didn't do here. 
 Without that call, stack traces on exceptions will segfault on 
 linux.

 [...]




Can one use this to hook in to any phobo's lib functions? What 
about simply "sniffing" the functions? Can we call the original 
functions somehow?
Jun 29 2016
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 30 June 2016 at 04:07:03 UTC, Hiemlick Hiemlicker 
wrote:

 Can one use this to hook in to any phobo's lib functions?


Yeah, you could. Easier though is to just copy the phobos lib 
file, modify it, then compile it into your program explicitly:

cp dmd2/src/phobos/std/file.d .

dmd yourprogram.d your_other_modules.d file.d


notice that I added the copied Phobos file to the compile command 
line. Then the compiler will prefer the one from your file to the 
one in Phobos itself. Nothing else needs to change, you still 
import std.file;


 Can we call the original functions somehow?


No, with this technique the linker will remove the original 
functions from the program.
Jun 29 2016
prev sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Tue, 11 Jun 2013 05:06:53 +0200, H. S. Teoh <hsteoh quickfur.ath.cx>  
wrote:


 void gc_init() { }
 void gc_addRange() { }
 void gc_term() { }


 Should these return errors too? Or is this just a quick-n-dirty way to
 get a no-GC environment without needing hack druntime?


They should not return errors. Basically, these turn gc initialization
(which druntime does at startup) and gc termination (which druntime does
at close time) into no-ops. Stubbing out gc_addRange makes sure the GC
does not try to track anything.



 Hmm. What if the user code instantiates classes? Will they leak memory
 then? Though I suppose the point is really just to find *hidden*
 allocations, and 'new' is a pretty blatant use of the GC.


They will leak, yes:

On Mon, 10 Jun 2013 21:16:04 +0200, Adam D. Ruppe  
<destructionator gmail.com> wrote:


 // this also makes new class available to user code, but remember to  
 free your classes and call the destructor:


-- 
Simen
Jun 11 2013