www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Runtime reflection costs

reply Christopher Wright <dhasenan gmail.com> writes:
Hey all,

A lot of my code uses a large amount of compile-time reflection. I was 
working with my dependency injection library, coming up with speed 
tests, and noticed that one sample (373 classes) took several minutes to 
compile and resulted in a 4MB executable. The version with direct 
constructor calls was 380KB -- dconstructor had an overhead of 8KB per 
class! Templates can't be anywhere near as efficient as runtime 
reflection in that case. There's pretty much no code reuse.

Now I'm thinking about runtime reflection. D doesn't support significant 
runtime reflection -- you can invoke a default constructor, and that's it.

For my purposes, it would be convenient to have something of this nature:
class ConstructorInfo
{
	void* functionPtr;
	TypeInfo[] arguments;
	Object invoke (void*[] args)
	{
		_d_invoke (arguments, args, functionPtr);
	}
}

The runtime would have to be able to take a void*[] and a TypeInfo[] and 
unroll that into a method call. That should be reasonable, if D has a 
consistent and not overly complicated ABI with decent documentation.

What's the overhead for this?
  * It adds a function to the runtime -- one for each supported 
platform, at least, or possibly one for each platform/compiler tuple.
  * It adds a number of ConstructorInfo objects. In this case, they will 
be sizeof(Object) + 3 * sizeof(size_t) (20 bytes on x86).
  * You have to store pointers to the parameter types.
  * It adds an array of ConstructorInfo in classinfo. This costs 2 * 
sizeof(size_t) plus N * sizeof(size_t), where N is the number of 
constructors.

Overall cost: 2D + (6 + R)N, where D is the number of classes, N is the 
number of constructors, and R is the average number of parameters per 
constructor; plus some inline assembly in the runtime, plus the cost of 
documenting the ABI. The ABI documentation is required in other places, 
and the runtime functions should not need updating once written.

For my 373-class sample, runtime reflection would add 16KB to the 
executable. After optimization, templates cost 1.7MB.

Walter, would you accept patches to add runtime reflection to D? If so, 
would you add them to D1?
Sep 28 2008
next sibling parent Sergey Gromov <snake.scaly gmail.com> writes:
Sun, 28 Sep 2008 09:25:38 -0400,
Christopher Wright wrote:

 Now I'm thinking about runtime reflection. D doesn't support significant 
 runtime reflection -- you can invoke a default constructor, and that's it.


D2's ClassInfo (2.019) has getMembers() which returns an array of 
MemberInfo, each is an instance of either MemberInfo_field or 
MemberInfo_function, containing all the info you need.  Although it 
doesn't work:

module test;
import std.stdio: writefln;
class A
{
    int i;
    this() {i = 3;}
    this(int f) {i = f;}
    int foo() {return i;}
}
void main() {
    auto ci = ClassInfo.find("test.A");
    writefln("found ", ci.name);            // prints "found test.A"
    auto m = ci.getMembers(null);
    writefln("has ", m.length, " members"); // prints "has 0 members"
}

But the principal mechanism is in place, it just needs to be 
fixed/implemented.
Sep 28 2008
prev sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Christopher Wright wrote:

 Hey all,
 
 A lot of my code uses a large amount of compile-time reflection. I was 
 working with my dependency injection library, coming up with speed 
 tests, and noticed that one sample (373 classes) took several minutes to 
 compile and resulted in a 4MB executable. The version with direct 
 constructor calls was 380KB -- dconstructor had an overhead of 8KB per 
 class! Templates can't be anywhere near as efficient as runtime 
 reflection in that case. There's pretty much no code reuse.
 
 Now I'm thinking about runtime reflection. D doesn't support significant 
 runtime reflection -- you can invoke a default constructor, and that's it.
 
 For my purposes, it would be convenient to have something of this nature:
 class ConstructorInfo
 {
     void* functionPtr;
     TypeInfo[] arguments;
     Object invoke (void*[] args)
     {
         _d_invoke (arguments, args, functionPtr);
     }
 }
 
 The runtime would have to be able to take a void*[] and a TypeInfo[] and 
 unroll that into a method call. That should be reasonable, if D has a 
 consistent and not overly complicated ABI with decent documentation.
 
 What's the overhead for this?
  * It adds a function to the runtime -- one for each supported platform, 
 at least, or possibly one for each platform/compiler tuple.
  * It adds a number of ConstructorInfo objects. In this case, they will 
 be sizeof(Object) + 3 * sizeof(size_t) (20 bytes on x86).
  * You have to store pointers to the parameter types.
  * It adds an array of ConstructorInfo in classinfo. This costs 2 * 
 sizeof(size_t) plus N * sizeof(size_t), where N is the number of 
 constructors.
 
 Overall cost: 2D + (6 + R)N, where D is the number of classes, N is the 
 number of constructors, and R is the average number of parameters per 
 constructor; plus some inline assembly in the runtime, plus the cost of 
 documenting the ABI. The ABI documentation is required in other places, 
 and the runtime functions should not need updating once written.
 
 For my 373-class sample, runtime reflection would add 16KB to the 
 executable. After optimization, templates cost 1.7MB.
 
 Walter, would you accept patches to add runtime reflection to D? If so, 
 would you add them to D1?


FlectionedCall at one point could do this... it hasn't been updated in 
over a year, though. However, it shows that it is possible to do by 
scanning the symbols in the binary (i.e. no need for explicit runtime 
support).
Sep 28 2008
parent Christopher Wright <dhasenan gmail.com> writes:
Robert Fraser wrote:

 FlectionedCall at one point could do this... it hasn't been updated in 
 over a year, though. However, it shows that it is possible to do by 
 scanning the symbols in the binary (i.e. no need for explicit runtime 
 support).


I had a look at that code. And then I threw it into /dev/null and ran 
the other way.

I guess I'll wait for D2 to get decent runtime reflection.
Sep 29 2008