• Bruno Medeiros (61/61) Mar 02 2006 Ok, I know many people have asked about reflection before, and surely
• pragma (29/38) Mar 02 2006 This is as good as any of the other proposals for Reflection this NG has...
• Bruno Medeiros (11/57) Mar 03 2006 With the RTTI info we already have, I don't think reflection would be
• Bruno Medeiros (8/17) Mar 02 2006 Strike that, we already have it. Upon further investigation I came to
• Sean Kelly (8/21) Mar 02 2006 And there are more in std/typeinfo. The problem is that, as far as I
• Bruno Medeiros (7/16) Mar 03 2006 Uh, I think the idea in D/Phobos is the same, to use typeid(sometype)
• Sean Kelly (3/14) Mar 03 2006 Then they shouldn't be in std. That was my only point.
• Bruno Medeiros (6/23) Mar 04 2006 Ah then, got it. And I agree.
• Bruno Medeiros (9/24) Mar 03 2006 Hum, perhaps the member reflection could be in the the TypeInfo class
• AgentOrange (11/72) Mar 02 2006 Yeah I think this is generally how walter had planned on doing it also. ...

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

Ok, I know many people have asked about reflection before, and surely 
everyone agrees it's a useful feature.
So what I want to know is what are the issues and challenges to design 
and implement such feature? Because now that I think of it, it seems to 
me it would be quite easy to implement runtime reflection, at least for 
field members of class objects. (method members have some problems, 
because of D's limited type information about function types)

So let's see, what do we need for reflection? We need to be able to 
determine the class of an anonymous object (already done with RTTI). And 
we need to be able do determine and access the field members of such 
object. That could be done, I believe, with a list of each field's type 
and offset relative to the object reference (this pointer).
Since we already have a TypeInfo structure, this could be done easily. 
Imagine that the ClassInfo class, has two additional members:
   TypeInfo[] memberids;
an array of the TypeInfo of each field of the class.
   int[] memberoffsets;
a respective array of the offsets relative to the 'this' pointer where 
the member is located. (should actually be size_t[] or something)

So now with this one can use reflection on an object, similarly to how 
one reads argument info from a variadic function with TypeInfo (which is 
where I got my idea). Here's a raw example of usage of such feature, a 
generic object serialization(writing to disk or somewhere else) function:


// Runtime reflection
void serialize(Object obj)
{
   TypeInfo[] memberids = obj.classinfo.memberids;
   int[] memberoffsets = obj.classinfo.memberoffsets;

   writefln(memberids.length, " members");

   foreach(int ix, TypeInfo memberid; memberids)
   {
     if (memberid == typeid(int)) {
       int m = * cast(int*) (cast(byte*)obj + memberoffsets[ix]);
       // Note that the previous could be abbreviate with something like:
       // int m = * obj.getMember!(ix, int);
       writefln("Member ",ix,", type int, value: ", m);
     }
     else if (memberid == typeid(char)) {
       char m = * cast(char*) (cast(byte*)obj + memberoffsets[ix]);
       writefln("Member ",ix,", type char, value: ", m);
     }
     // ... etc. for long, unsigneds and other primitive types

     else if (memberid.isClass()) {
       Object m = cast(Object) (cast(byte*)obj + memberoffsets[ix]);
       serialize(m);
     }
   }
}


Ok, this shows the usefulness of such feature, however there are some 
problems/limitations. It only works with primitive types (there is no 
isClass method of TypeInfo). For this feature to reach the full 
potential, then the TypeInfo class would have to be extended with info 
about the type's type, (i.e., the metatype, like struct, class, 
function, primitive, etc.), so that one could use reflection with 
non-primitive types, especially classes and arrays.

None of this seems to be hard or long to implement, so how about it? (I 
know many details need to be fleshed out, but the base idea seems good)

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

pragma <pragma_member pathlink.com> writes:

In article <du6vsh$1cep$1 digitaldaemon.com>, Bruno Medeiros says...
Ok, I know many people have asked about reflection before, and surely 
everyone agrees it's a useful feature.
So what I want to know is what are the issues and challenges to design 
and implement such feature? Because now that I think of it, it seems to 
me it would be quite easy to implement runtime reflection, at least for 
field members of class objects. (method members have some problems, 
because of D's limited type information about function types)

[...snip...]
None of this seems to be hard or long to implement, so how about it? (I 
know many details need to be fleshed out, but the base idea seems good)

This is as good as any of the other proposals for Reflection this NG has seen.
Given that, I don't think the problem has ever been with the design of any given
proposal over the other. :)

If I had to guess, the major reason that's holding this kind of thing back from
the D specification, is how to justify adding all of the reflection symbol data
into the executable/binary/whatever.  I've learned that it can take up a very
serious amount of space, most of which is usually wasted thanks to stuff that
never comes up. For example: you're not likely to attempt to serialze something
in the std.stdio namespace.

But I also think nobody would argue that runtime reflection is a tradeoff
between bloat and a whole range of algorithms that require it.  The problem lies
more on the side of systems programmers and folks who shouldn't have to contend
with such baggage.

IMO, the the best way towoard a reflection API would be the addition of a
compiler switch that toggles the existance of an internal symbol lookup table*.
This way *any* reflection API proposal, Typeinfo extensions included, would
simply be built on top of it; plus you get a nice low-level rendition (just the
table) to work with too.

Should the developer not want that bloat, they simply turn off the lookup
generation, which lets the respective D reflection code simply fall flat on its
face (symbol searches and lookups will simply return nothing, or throw).  This
way, you can shift D toward whatever development mode you have in mind, just
like we do now with other constructs.

(*Alternatively, this could be done via code introspection much like how
inlining works - but its just nicer to be in charge of what gets added and what
gets tossed out)

- EricAnderton at yahoo

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

pragma wrote:
 In article <du6vsh$1cep$1 digitaldaemon.com>, Bruno Medeiros says...
 Ok, I know many people have asked about reflection before, and surely 
 everyone agrees it's a useful feature.
 So what I want to know is what are the issues and challenges to design 
 and implement such feature? Because now that I think of it, it seems to 
 me it would be quite easy to implement runtime reflection, at least for 
 field members of class objects. (method members have some problems, 
 because of D's limited type information about function types)

 [...snip...]
 None of this seems to be hard or long to implement, so how about it? (I 
 know many details need to be fleshed out, but the base idea seems good)

 
 This is as good as any of the other proposals for Reflection this NG has seen.
 Given that, I don't think the problem has ever been with the design of any
given
 proposal over the other. :)
 
 If I had to guess, the major reason that's holding this kind of thing back from
 the D specification, is how to justify adding all of the reflection symbol data
 into the executable/binary/whatever.  I've learned that it can take up a very
 serious amount of space, most of which is usually wasted thanks to stuff that
 never comes up. For example: you're not likely to attempt to serialze something
 in the std.stdio namespace.
 
 But I also think nobody would argue that runtime reflection is a tradeoff
 between bloat and a whole range of algorithms that require it.  The problem
lies
 more on the side of systems programmers and folks who shouldn't have to contend
 with such baggage.
 
 IMO, the the best way towoard a reflection API would be the addition of a
 compiler switch that toggles the existance of an internal symbol lookup table*.
 This way *any* reflection API proposal, Typeinfo extensions included, would
 simply be built on top of it; plus you get a nice low-level rendition (just the
 table) to work with too.
 
 Should the developer not want that bloat, they simply turn off the lookup
 generation, which lets the respective D reflection code simply fall flat on its
 face (symbol searches and lookups will simply return nothing, or throw).  This
 way, you can shift D toward whatever development mode you have in mind, just
 like we do now with other constructs.
 
 (*Alternatively, this could be done via code introspection much like how
 inlining works - but its just nicer to be in charge of what gets added and what
 gets tossed out)
 
 - EricAnderton at yahoo

With the RTTI info we already have, I don't think reflection would be 
that much more information. (especially if we don't put reflection info 
about the member's names).
But you are right, there should be eventually an option to control 
whether this info is added or not. If I'm not mistaken, most C++ 
compilers have options like this, with regards to C++ RTTI.

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

Bruno Medeiros wrote:
 
 Ok, this shows the usefulness of such feature, however there are some 
 problems/limitations. It only works with primitive types (there is no 
 isClass method of TypeInfo). For this feature to reach the full 
 potential, then the TypeInfo class would have to be extended with info 
 about the type's type, (i.e., the metatype, like struct, class, 
 function, primitive, etc.), so that one could use reflection with 
 non-primitive types, especially classes and arrays.
 

Strike that, we already have it. Upon further investigation I came to 
object.d where there are children classes of TypeInfo for each of the 
metatypes. Nice! :D


-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

Sean Kelly <sean f4.ca> writes:

Bruno Medeiros wrote:
 Bruno Medeiros wrote:
 Ok, this shows the usefulness of such feature, however there are some 
 problems/limitations. It only works with primitive types (there is no 
 isClass method of TypeInfo). For this feature to reach the full 
 potential, then the TypeInfo class would have to be extended with info 
 about the type's type, (i.e., the metatype, like struct, class, 
 function, primitive, etc.), so that one could use reflection with 
 non-primitive types, especially classes and arrays.

 Strike that, we already have it. Upon further investigation I came to 
 object.d where there are children classes of TypeInfo for each of the 
 metatypes. Nice! :D

And there are more in std/typeinfo.  The problem is that, as far as I 
know, the names of these classes are not standardized, nor is the output 
returned by the name method, so they're of limited direct use.  For Ares 
I actually moved std/typeinfo into the runtime and don't expect the 
classes to be used directly, but rather for the user to do typeid(int) 
or whatever to return a reference to the appropriate TypeInfo object.


Sean

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

Sean Kelly wrote:
 And there are more in std/typeinfo.  The problem is that, as far as I 
 know, the names of these classes are not standardized, nor is the output 
 returned by the name method, so they're of limited direct use.  For Ares 
 I actually moved std/typeinfo into the runtime and don't expect the 
 classes to be used directly, but rather for the user to do typeid(int) 
 or whatever to return a reference to the appropriate TypeInfo object.
 
 
 Sean

Uh, I think the idea in D/Phobos is the same, to use typeid(sometype) 
and not the use the classes directly.

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

Sean Kelly <sean f4.ca> writes:

Bruno Medeiros wrote:
 Sean Kelly wrote:
 And there are more in std/typeinfo.  The problem is that, as far as I 
 know, the names of these classes are not standardized, nor is the 
 output returned by the name method, so they're of limited direct use.  
 For Ares I actually moved std/typeinfo into the runtime and don't 
 expect the classes to be used directly, but rather for the user to do 
 typeid(int) or whatever to return a reference to the appropriate 
 TypeInfo object.

 
 Uh, I think the idea in D/Phobos is the same, to use typeid(sometype) 
 and not the use the classes directly.

Then they shouldn't be in std.  That was my only point.


Sean

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

Sean Kelly wrote:
 Bruno Medeiros wrote:
 Sean Kelly wrote:
 And there are more in std/typeinfo.  The problem is that, as far as I 
 know, the names of these classes are not standardized, nor is the 
 output returned by the name method, so they're of limited direct 
 use.  For Ares I actually moved std/typeinfo into the runtime and 
 don't expect the classes to be used directly, but rather for the user 
 to do typeid(int) or whatever to return a reference to the 
 appropriate TypeInfo object.

 Uh, I think the idea in D/Phobos is the same, to use typeid(sometype) 
 and not the use the classes directly.

 
 Then they shouldn't be in std.  That was my only point.
 
 
 Sean

Ah then, got it. And I agree.

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

Bruno Medeiros <daiphoenixNO SPAMlycos.com> writes:

Bruno Medeiros wrote:
 Bruno Medeiros wrote:
 Ok, this shows the usefulness of such feature, however there are some 
 problems/limitations. It only works with primitive types (there is no 
 isClass method of TypeInfo). For this feature to reach the full 
 potential, then the TypeInfo class would have to be extended with info 
 about the type's type, (i.e., the metatype, like struct, class, 
 function, primitive, etc.), so that one could use reflection with 
 non-primitive types, especially classes and arrays.

 Strike that, we already have it. Upon further investigation I came to 
 object.d where there are children classes of TypeInfo for each of the 
 metatypes. Nice! :D
 
 

Hum, perhaps the member reflection could be in the the TypeInfo class 
itself. This way, not only structs could have reflection too, but maybe 
functions could also have a "members" like functionality with regards to 
their parameters! (thus achieving full type info for functions/delegates)



-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."

AgentOrange <AgentOrange_member pathlink.com> writes:

Yeah I think this is generally how walter had planned on doing it also. I dont
see it currently, but in an old version of phobos he had a commented out section
inside ClassInfo that said something to the effect // BUG: field typeinfo list
goes here

its a good idea, and we can spit out reflection info all day long, as ive been
experimenting in my dparser project, we just need to come up with a clean way of
doing it.... I think somewhere along the way we are going to want a customized
Object module - ala something like ares....

it would be nice to hear walters ideas, plans, etc. for reflection, just so we
have his take on the matter, but oh well.....




In article <du6vsh$1cep$1 digitaldaemon.com>, Bruno Medeiros says...
Ok, I know many people have asked about reflection before, and surely 
everyone agrees it's a useful feature.
So what I want to know is what are the issues and challenges to design 
and implement such feature? Because now that I think of it, it seems to 
me it would be quite easy to implement runtime reflection, at least for 
field members of class objects. (method members have some problems, 
because of D's limited type information about function types)

So let's see, what do we need for reflection? We need to be able to 
determine the class of an anonymous object (already done with RTTI). And 
we need to be able do determine and access the field members of such 
object. That could be done, I believe, with a list of each field's type 
and offset relative to the object reference (this pointer).
Since we already have a TypeInfo structure, this could be done easily. 
Imagine that the ClassInfo class, has two additional members:
   TypeInfo[] memberids;
an array of the TypeInfo of each field of the class.
   int[] memberoffsets;
a respective array of the offsets relative to the 'this' pointer where 
the member is located. (should actually be size_t[] or something)

So now with this one can use reflection on an object, similarly to how 
one reads argument info from a variadic function with TypeInfo (which is 
where I got my idea). Here's a raw example of usage of such feature, a 
generic object serialization(writing to disk or somewhere else) function:


// Runtime reflection
void serialize(Object obj)
{
   TypeInfo[] memberids = obj.classinfo.memberids;
   int[] memberoffsets = obj.classinfo.memberoffsets;

   writefln(memberids.length, " members");

   foreach(int ix, TypeInfo memberid; memberids)
   {
     if (memberid == typeid(int)) {
       int m = * cast(int*) (cast(byte*)obj + memberoffsets[ix]);
       // Note that the previous could be abbreviate with something like:
       // int m = * obj.getMember!(ix, int);
       writefln("Member ",ix,", type int, value: ", m);
     }
     else if (memberid == typeid(char)) {
       char m = * cast(char*) (cast(byte*)obj + memberoffsets[ix]);
       writefln("Member ",ix,", type char, value: ", m);
     }
     // ... etc. for long, unsigneds and other primitive types

     else if (memberid.isClass()) {
       Object m = cast(Object) (cast(byte*)obj + memberoffsets[ix]);
       serialize(m);
     }
   }
}


Ok, this shows the usefulness of such feature, however there are some 
problems/limitations. It only works with primitive types (there is no 
isClass method of TypeInfo). For this feature to reach the full 
potential, then the TypeInfo class would have to be extended with info 
about the type's type, (i.e., the metatype, like struct, class, 
function, primitive, etc.), so that one could use reflection with 
non-primitive types, especially classes and arrays.

None of this seems to be hard or long to implement, so how about it? (I 
know many details need to be fleshed out, but the base idea seems good)

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to 
be... unnatural."