digitalmars.D - Runtime reflection idea
- Adam D. Ruppe (71/71) May 30 2013 The compiler instantiates a template called RTInfo!T for every
- Steven Schveighoffer (15/28) May 30 2013 http://forum.dlang.org/post/op.wcvlmg03eav7ka@localhost.localdomain
- Adam D. Ruppe (7/12) May 30 2013 I probably read that thread originally too....
- Steven Schveighoffer (10/18) May 30 2013 If you wanted to add interfaces, would you have to modify object.d?
- Adam D. Ruppe (21/23) May 30 2013 Yeah, but that's not something you'd have to do since you can
- Adam D. Ruppe (58/58) Jun 01 2013 I got a better thing implemented, what do you think of this?
- Adam D. Ruppe (4/4) Jun 01 2013 also this post:
- Adam D. Ruppe (13/16) May 30 2013 oh man, I just had a realization: such could be (ab?)used to add
- Adam D. Ruppe (4/8) Jun 21 2013 Eureka:
- Steven Schveighoffer (4/11) Jun 21 2013 Nice :)
- Benjamin Thaut (12/12) Jun 02 2013 I think this is a very good idea. The only question is how complete this...
- Rainer Schuetze (26/36) Jun 02 2013 As a number of use cases show up for the RTInfo template (the precise GC...
- Adam D. Ruppe (118/126) Jun 02 2013 I think I have something figured out:
- Rainer Schuetze (9/15) Jun 03 2013 Interesting stuff to implement custom type info declared with the class....
- Benjamin Thaut (6/44) Jun 02 2013 That is the obvious solution to the problem. Maybe we should get
- Adam D. Ruppe (29/36) Jun 02 2013 For now, I'm going for just a UDA based thing, so it is almost
The compiler instantiates a template called RTInfo!T for every
custom type T that is made available through the TypeInfo class
(.classinfo, typeid()).
The first idea with this was to do precise GC, but it can do more
than just that. Currently, it is totally unused in druntime. I
propose we put it to use to allow some runtime reflection.
First, here's a taste of what it can look like (or whatever else,
I did name/value strings since that's easy to write for an
example and access at runtime, even without phobos, but there's
no reason we couldn't try something else, maybe a class or
something would be good):
CustomTypeInfoExtension("name", "value")
CustomTypeInfoExtension("name2", "value2")
class Test { }
int main(string[] args) {
auto rtInfo = typeid(Test).rtInfo();
import std.stdio;
foreach(ci; rtInfo.customInfo) {
writeln(ci.name, " = ", ci.value);
}
}
// prints:
// name = value
// name2 = value2
Here's some example code I wrote up as a proof of concept in my
custom druntime:
struct MoreTypeInfo {
hash_t hash;
string stringOf;
/* whatever else we want, this is extensible by druntime */
// and finally the custom info
CustomTypeInfoExtension[] customInfo;
}
struct CustomTypeInfoExtension {
// we can do this however too, I just do name/value pairs
// as something simple to get us started
string name;
string value;
string meta;
}
// fetch the data from a type using UDA capabilities
CustomTypeInfoExtension[] getCustomInfo(T)() {
CustomTypeInfoExtension[] ext;
foreach(attr; __traits(getAttributes, T))
static if(is(typeof(attr) ==
CustomTypeInfoExtension))
ext ~= attr;
return ext;
}
// and the thing dmd calls:
template RTInfo(T) {
__gshared static immutable minfo =
// the first few args are just to show druntime still
// has special access... and the last one is to fetch
// the custom stuff
MoreTypeInfo(typehash!T, T.stringof, getCustomInfo!T);
enum RTInfo = &minfo;
}
Since the template is already instantiated by the compiler - this
exists today and works if you wanna modify your object.d in
druntime - we don't have to do like a mixin registerModule to
parse this UDA. Of course, you still have that option and it
rocks, this just gives a simple runtime thing we can use out of
the box....
and ModuleInfo even has a list of all classes, and all modules in
a program, so we could go pretty nuts with runtime stuff here.
What do you all think? It is a pretty minor change to druntime to
enable this kind of thing, it wouldn't get in the way of anything
else - druntime can still use RTInfo for its own thing when the
time comes without a problem - and it seems to me like it could
be easily useful and convenient for some tasks.
May 30 2013
On Thu, 30 May 2013 10:59:52 -0400, Adam D. Ruppe <destructionator gmail.com> wrote:The compiler instantiates a template called RTInfo!T for every custom type T that is made available through the TypeInfo class (.classinfo, typeid()). The first idea with this was to do precise GC, but it can do more than just that. Currently, it is totally unused in druntime. I propose we put it to use to allow some runtime reflection.http://forum.dlang.org/post/op.wcvlmg03eav7ka localhost.localdomainand ModuleInfo even has a list of all classes, and all modules in a program, so we could go pretty nuts with runtime stuff here.I also want ModuleInfo to have an equivalent RTInfo member. Walter agreed at the conference that it was a good idea and should be done. See bug report here: http://d.puremagic.com/issues/show_bug.cgi?id=10023What do you all think? It is a pretty minor change to druntime to enable this kind of thing, it wouldn't get in the way of anything else - druntime can still use RTInfo for its own thing when the time comes without a problem - and it seems to me like it could be easily useful and convenient for some tasks.I would like to see the necessary framework for RTInfo to allow expansion WITHOUT having to modify object.d. I think this can be done, and then it allows anyone to insert whatever they want for RTInfo based on their own needs. This would set D apart from existing runtime information frameworks, like runtime information might be as simple as adding UDAs). -Steve
May 30 2013
On Thursday, 30 May 2013 at 15:12:57 UTC, Steven Schveighoffer wrote:http://forum.dlang.org/post/op.wcvlmg03eav7ka localhost.localdomainI probably read that thread originally too....I would like to see the necessary framework for RTInfo to allow expansion WITHOUT having to modify object.d. I think this can be done, and then it allows anyone to insert whatever they want for RTInfo based on their own needs.What's the difference between what you're thinking and what I outlined here? I used strings, but it could just as well be interfaces (casted back to a custom class at use time), or something to allow custom types to be added and fetched too.
May 30 2013
On Thu, 30 May 2013 13:10:35 -0400, Adam D. Ruppe <destructionator gmail.com> wrote:If you wanted to add interfaces, would you have to modify object.d? What I'm saying is, make RTInfo extendable without having to modify object.d. Each project could define how the compiler generates the RTInfo for it's specific needs, and everyone can use the same runtime. I think your idea can do this if we just move MoreTypeInfo to be instantiated via a type member, so each type could decide how it's type info is generated. -SteveI would like to see the necessary framework for RTInfo to allow expansion WITHOUT having to modify object.d. I think this can be done, and then it allows anyone to insert whatever they want for RTInfo based on their own needs.What's the difference between what you're thinking and what I outlined here? I used strings, but it could just as well be interfaces (casted back to a custom class at use time), or something to allow custom types to be added and fetched too.
May 30 2013
On Thursday, 30 May 2013 at 17:24:15 UTC, Steven Schveighoffer wrote:If you wanted to add interfaces, would you have to modify object.d?Yeah, but that's not something you'd have to do since you can just subclass it and add whatever you want there. class MyThing: Reflectable { blah whatever; } CustomTypeInfoExtension(new MyThing()) class Test {} auto info = typeid(Test).rtInfo(); foreach(item; info.customItems) if(auto c = cast(MyThing) item) { // use c } Or maybe we could implement it like: MyThing c = typeid(Test).rtInfo().getCustomItem!MyThing(); which could perhaps be used with any type. You could also get compile time info on it easily enough: class MyThing(T) : Reflectable {} CustomTypeInfoExtension(new MyThing!Test()) class Test {}
May 30 2013
I got a better thing implemented, what do you think of this?
// usage
CustomInfo!("hey") // can be anything in there, but don't repeat
a type
CustomInfo!(1)
class Test {}
auto info = typeid(Test).rtInfo();
immutable(string)* str = rtInfo.getCustomData!string;
writeln(str is null ? "not there" : *str); // prints "hey"
immutable(int)* i = rtInfo.getCustomData!int;
writeln(i is null ? "not there" : *i); // prints 1.
// implementation
struct MoreTypeInfo {
// the first two fields just show druntime can
still do its thing
hash_t hash;
string stringOf;
// and here's the extensible part
immutable(CustomTypeInfoExtension)[] customInfo;
immutable(T)* getCustomData(T)() immutable {
auto hash = typeid(T); // typehash!T;
foreach(ci; customInfo) {
if(ci.typeOfData == hash)
return cast(immutable(T)*) ci.data();
}
return null;
}
}
struct CustomTypeInfoExtension {
TypeInfo typeOfData;
void* function() data;
}
immutable(CustomTypeInfoExtension)[] getCustomInfo(T)() {
if(__ctfe) {
//bool[hash_t] seen;
immutable(CustomTypeInfoExtension)[] ext;
foreach(attr; __traits(getAttributes, T))
static if(is(typeof(attr) == CustomTypeInfoExtension)) {
//auto hash = attr.typeOfData.rtInfo.hash;
//if(hash in seen)
//assert(0, "repeated data");
//seen[hash] = true;
ext ~= cast(immutable) attr;
}
return ext;
} else return null;
}
template CustomInfo(alias T) {
__gshared static data = T;
void* getRaw() { return cast(void*) &data; }
enum CustomInfo = CustomTypeInfoExtension(
typeid(typeof(data)), &getRaw);
}
template RTInfo(T) {
__gshared static immutable minfo = MoreTypeInfo(typehash!T,
T.stringof, getCustomInfo!T);
enum RTInfo = &minfo;
}
Jun 01 2013
"Adam D. Ruppe" <destructionator gmail.com> also this post: http://forum.dlang.org/thread/tftjtzmfuauxwcgcolct forum.dlang.org#post-hautysdywtyfigsoasgv:40forum.dlang.org I'm just playing with my custom druntime and there's so much we could add if we wanted, pretty easily too thanks to ctfe.
Jun 01 2013
On Thursday, 30 May 2013 at 15:12:57 UTC, Steven Schveighoffer wrote:I also want ModuleInfo to have an equivalent RTInfo member. Walter agreed at the conference that it was a good idea and should be done. See bug report here:oh man, I just had a realization: such could be (ab?)used to add new language checks. I'm playing with a bare metal D right now, and there's a few rules I've determined to keep the memory situation sane, including "don't use non-scoped, non-refcounted delegates". An rtInfo type thing run over every module in the program could actually use traits to find any function that breaks the rules and throw a static assert error. It wouldn't just be a documented warning anymore, it'd be like expanding the compiler's own semantic checks. That's kinda awesome.
May 30 2013
On Thursday, 30 May 2013 at 15:12:57 UTC, Steven Schveighoffer wrote:I would like to see the necessary framework for RTInfo to allow expansion WITHOUT having to modify object.d. I think this can be done, and then it allows anyone to insert whatever they want for RTInfo based on their own needs.Eureka: http://forum.dlang.org/thread/zgjgfprbkeiapjipldwf forum.dlang.org#post-fnxfhvhvmhuweyxfricb:40forum.dlang.org
Jun 21 2013
On Fri, 21 Jun 2013 08:43:15 -0400, Adam D. Ruppe <destructionator gmail.com> wrote:On Thursday, 30 May 2013 at 15:12:57 UTC, Steven Schveighoffer wrote:Nice :) -SteveI would like to see the necessary framework for RTInfo to allow expansion WITHOUT having to modify object.d. I think this can be done, and then it allows anyone to insert whatever they want for RTInfo based on their own needs.Eureka: http://forum.dlang.org/thread/zgjgfprbkeiapjipldwf forum.dlang.org#post-fnxfhvhvmhuweyxfricb:40forum.dlang.org
Jun 21 2013
I think this is a very good idea. The only question is how complete this RTTI should be. This again highly depends on what it will be used for. For some users it might be ok to do a full RTTI, which will increase the executable size significantly. Other users might prefer a minimal RTTI or even no RTTI at all depending on the use case. The RTInfo template is very usefull for many different tasks, the question is if we shouldn't make the concept more generic so that you can have multiple templates which behave like the RTInfo template. I'm currently also using it for RTTI info see: http://3d.benjamin-thaut.de/?p=25 -- Kind Regards Benjamin Thaut
Jun 02 2013
On 02.06.2013 11:33, Benjamin Thaut wrote:I think this is a very good idea. The only question is how complete this RTTI should be. This again highly depends on what it will be used for. For some users it might be ok to do a full RTTI, which will increase the executable size significantly. Other users might prefer a minimal RTTI or even no RTTI at all depending on the use case. The RTInfo template is very usefull for many different tasks, the question is if we shouldn't make the concept more generic so that you can have multiple templates which behave like the RTInfo template. I'm currently also using it for RTTI info see: http://3d.benjamin-thaut.de/?p=25As a number of use cases show up for the RTInfo template (the precise GC also uses it), I think a simple approach for not stepping on each other toes would be to declare a struct type RTInfoData, and add each implementation as a member to this struct (not checked whether this actually compiles): struct RTInfoData { immutable(RuntimeReflection)* rr; immutable(thMemberInfo)* th; immutable(PreciseGCData)* gc; } template RTInfo(T) { immutable(RTInfoData) data = RTInfoData(genRuntimeReflection!T, genMemberInfo!T, genGCData!T); enum RTInfo = &data; } and TypeInfo.rtInfo() would then return a pointer to RTInfoData instead of void*. This doesn't make it modifiable from outside object.di, but I have no idea how that could be possible to begin with (without recompiling the runtime library). Unfortunately the compiler sometimes doesn't force the generation of RTInfo, but sets the m_rtInfo to 0 or 1 depending on whether the Type contains pointers or not. I always wanted to figure out why that happens...
Jun 02 2013
On Sunday, 2 June 2013 at 13:28:28 UTC, Rainer Schuetze wrote:and TypeInfo.rtInfo() would then return a pointer to RTInfoData instead of void*. This doesn't make it modifiable from outside object.di, but I have no idea how that could be possible to begin with (without recompiling the runtime library).I think I have something figured out: use side: // i'll use this as a UDA struct InfoStruct { string info; } // and this as the template extension struct MoreRtInfo { string amazing; } CustomInfo!(InfoStruct("info struct data")) // UDA style, goes into an array class RTTest { // template style, goes in a special member static template userRtInfo(This) { static __gshared i = MoreRtInfo("amazingness"); enum userRtInfo = &i; } } Retrieve: auto info = typeid(RTTest).rtInfo(); if(info is null) throw new Exception("wtf"); auto urtInfo = info.userRtInfo; // this is from the template.. if(urtInfo is null) write("urtInfo == null\n"); else { // it comes out as a void*, so we have to cast it back. auto urt = cast(immutable(MoreRtInfo)*) urtInfo; write("cool ",urt.amazing,"\n"); } // and this is fetching the CustomInfo UDA auto cd = info.getCustomInfo!InfoStruct; if(cd is null) // it returns a InfoStruct*, like AA's in operator write("InfoStruct is null\n"); else write(cd.info, "\n"); And the implementation side in object.d: struct MoreTypeInfo { // other stuff druntime is free to declare hash_t hash; string stringOf; // holder for the template thing immutable(void)* userRtInfo; // holder for the UDA thing immutable(CustomTypeInfoExtension)[] customInfo; // helper for the UDA get immutable(T)* getCustomInfo(T)() immutable { auto hash = typeid(T); // typehash!T; foreach(ci; customInfo) { if(ci.typeOfData == hash) return cast(immutable(T)*) ci.data(); } return null; } } // instantiates the T.userRtInfo, if possible template urtInfo(T) { static if (__traits(compiles, { auto a = cast(immutable(void)*) T.userRtInfo!T; })) enum urtInfo = cast(immutable(void)*) T.userRtInfo!T; else enum urtInfo = null; } template RTInfo(T) { __gshared static immutable minfo = MoreTypeInfo( // stuff druntime is free to add typehash!T, T.stringof, // the user defined template urtInfo!T, // getting the UDA stuff getCustomInfoInternal!T); enum RTInfo = &minfo; } // finally, helpers in getting the UDAs, I posted this before too struct CustomTypeInfoExtension { TypeInfo typeOfData; // ctfe complained when I tried to just do &static_data // so instead this little function helper does it for us void* function() data; } immutable(CustomTypeInfoExtension)[] getCustomInfoInternal(T)() { if(__ctfe) { //bool[hash_t] seen; immutable(CustomTypeInfoExtension)[] ext; foreach(attr; __traits(getAttributes, T)) static if(is(typeof(attr) == CustomTypeInfoExtension)) { //auto hash = attr.typeOfData.rtInfo.hash; //if(hash in seen) //assert(0, "repeated data"); //seen[hash] = true; ext ~= cast(immutable) attr; } return ext; } else return null; } // this is the user uses: CustomInfo!(something) template CustomInfo(alias T) { __gshared static data = T; void* getRaw() { return cast(void*) &data; } enum CustomInfo = CustomTypeInfoExtension( typeid(typeof(data))/*typehash!(typeof(data))*/, &getRaw); }Unfortunately the compiler sometimes doesn't force the generation of RTInfo, but sets the m_rtInfo to 0 or 1 depending on whether the Type contains pointers or not. I always wanted to figure out why that happens...weird
Jun 02 2013
On 02.06.2013 17:10, Adam D. Ruppe wrote:On Sunday, 2 June 2013 at 13:28:28 UTC, Rainer Schuetze wrote:Interesting stuff to implement custom type info declared with the class. But type info as needed for the GC has to be generated for every Type without additional annotation or declaration. Assuming a recompilation of the runtime library, but no modification to the library source, I currently can only imagine having some string import switched to some other file through a different search path. This import would then contain declarations to be mixed into the RTInfoData (or MoreRtInfo in your implementation).and TypeInfo.rtInfo() would then return a pointer to RTInfoData instead of void*. This doesn't make it modifiable from outside object.di, but I have no idea how that could be possible to begin with (without recompiling the runtime library).I think I have something figured out:
Jun 03 2013
Am 02.06.2013 15:28, schrieb Rainer Schuetze:On 02.06.2013 11:33, Benjamin Thaut wrote:That is the obvious solution to the problem. Maybe we should get something like this into drutime ASAP so that the RTInfo template remains extendable even if a percise GC is used. Kind Regards Benjamin ThautI think this is a very good idea. The only question is how complete this RTTI should be. This again highly depends on what it will be used for. For some users it might be ok to do a full RTTI, which will increase the executable size significantly. Other users might prefer a minimal RTTI or even no RTTI at all depending on the use case. The RTInfo template is very usefull for many different tasks, the question is if we shouldn't make the concept more generic so that you can have multiple templates which behave like the RTInfo template. I'm currently also using it for RTTI info see: http://3d.benjamin-thaut.de/?p=25As a number of use cases show up for the RTInfo template (the precise GC also uses it), I think a simple approach for not stepping on each other toes would be to declare a struct type RTInfoData, and add each implementation as a member to this struct (not checked whether this actually compiles): struct RTInfoData { immutable(RuntimeReflection)* rr; immutable(thMemberInfo)* th; immutable(PreciseGCData)* gc; } template RTInfo(T) { immutable(RTInfoData) data = RTInfoData(genRuntimeReflection!T, genMemberInfo!T, genGCData!T); enum RTInfo = &data; } and TypeInfo.rtInfo() would then return a pointer to RTInfoData instead of void*. This doesn't make it modifiable from outside object.di, but I have no idea how that could be possible to begin with (without recompiling the runtime library). Unfortunately the compiler sometimes doesn't force the generation of RTInfo, but sets the m_rtInfo to 0 or 1 depending on whether the Type contains pointers or not. I always wanted to figure out why that happens...
Jun 02 2013
On Sunday, 2 June 2013 at 09:33:27 UTC, Benjamin Thaut wrote:I think this is a very good idea. The only question is how complete this RTTI should be.For now, I'm going for just a UDA based thing, so it is almost all opt-in. Though one downside of my idea here is it may be slow, since it pulls info from a list (I'm doing a linear array, but it could be a binary search of AA or something without much difficulty, but my guess is most entries will only be a handful of elements long anyway). Actually adding members to the rtinfo struct would avoid this cost, but I can't think of any way to do that without modifying object.d for each one. Perhaps we could have a userRtInfo template that works the same way, but that one is defined in the module that defines the type, instead of in object.d. The module would be responsible for casting the void* in typeinfo (a new method, urtInfo to complement rtInfo) back to the type they defined. I think that would be doable at least. The other thing I'm pondering is changing TypeInfo to be generated at compile time and include all kinds of functions like std.traits offers. This would only be builtin types, user structs and classes have RTInfo instead, so that limits the size somewhat, but it nevertheless does increase exe size and isn't easy to opt out of, since you'd have to recompile at least part of druntime to do it. Expanding typeinfo is arguably less useful since if you want that info, you can do it yourself in a template like std.variant, but one use I can see is the D style variadics. Instead of a list of types with code right there, you could just do typeid().is/getNumeric() or whatever.The RTInfo template is very usefull for many different tasks, the question is if we shouldn't make the concept more generic so that you can have multiple templates which behave like the RTInfo template. I'm currently also using it for RTTI info see: http://3d.benjamin-thaut.de/?p=25very cool.
Jun 02 2013