digitalmars.D.learn - Runtime introspection, or how to get class members at runtime Fin D
- Arafel (32/32) Jun 06 2018 Hi,
- rikki cattermole (4/28) Jun 06 2018 You don't want TypeInfo.
- Arafel (10/32) Jun 06 2018 Why not (genuine question)? There's even myObject.classinfo, and I can
- rikki cattermole (5/41) Jun 06 2018 It is not an easy task building a reflection API from scratch. I'm one
- Adam D. Ruppe (16/21) Jun 06 2018 It holds just barely enough info for dynamic casting, GC, and
- Arafel (35/46) Jun 06 2018 If I wanted to add it myself, would I need to create a personalised D
- Steven Schveighoffer (11/16) Jun 06 2018 There is very little runtime reflection capability. The position has
- Jacob Carlborg (9/44) Jun 06 2018 The simplest, in my opinion would be to for the subclasses to register
- evilrat (9/25) Jun 07 2018 There is a library that creates reflection metadata for you. [1]
Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class? I have had a look as TypeInfo_Class [1], but apparently I can only get a list of types and offsets... which would be almost good enough, if not because the names of the members are missing, or at least I haven't been able to find them. In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Also, I know I could add some UDA or even crawl the modules and have this information generated automatically at compilation time and added to the type itself in a member, and I might even end up doing it, but honestly, I think it's something that the language should provide in a kind of easy / accessible way. Powerful as compile-time introspection is, I think runtime shouldn't be forgotten either :-) Thanks, A. [1]: https://dlang.org/library/object/type_info__class.html [2]: https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getFields--
Jun 06 2018
On 07/06/2018 1:28 AM, Arafel wrote:Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class?No.I have had a look as TypeInfo_Class [1], but apparently I can only get a list of types and offsets... which would be almost good enough, if not because the names of the members are missing, or at least I haven't been able to find them.You don't want TypeInfo.In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2].Doesn't exist.
Jun 06 2018
On 06/06/2018 03:30 PM, rikki cattermole wrote:You don't want TypeInfo.Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there...Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed? (I tried searching the forum, but didn't find anything relevant) I know it's got a runtime penalty, but realistically speaking, spending some bytes for the field names in the TypeInfo of a class shouldn't be that much of a problem?In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2].Doesn't exist.
Jun 06 2018
rikki cattermole <rikki cattermole.co.nz> On 07/06/2018 1:44 AM, Arafel wrote:On 06/06/2018 03:30 PM, rikki cattermole wrote:It is not an easy task building a reflection API from scratch. I'm one of the many that have tried. There is also push back from those who consider it "bloat" and don't need it. You can't just extend TypeInfo, it was never designed for it.You don't want TypeInfo.Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there...Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed? (I tried searching the forum, but didn't find anything relevant) I know it's got a runtime penalty, but realistically speaking, spending some bytes for the field names in the TypeInfo of a class shouldn't be that much of a problem?In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2].Doesn't exist.
Jun 06 2018
On Wednesday, 6 June 2018 at 13:44:39 UTC, Arafel wrote:Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there...It holds just barely enough info for dynamic casting, GC, and other language implementation stuff. (and then Object.factory for some weird reason, which actually causes bloat for so little benefit)Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed?It is possible to add it to the runtime library right now (there's a thing called rtInfo in there made to hold it, it is just null right now), just people fight over even a few bytes of added metadata. So if it is added, it would surely be some opt-in thing that will require your thing be recompiled anyway. If you can recompile the library, you can add a parallel extended info thing (MyReflectionInfo[TypeInfo] array perhaps, populated by a static this() ctor created via compile time reflection) that gives what you need. But if you can't recompile the library, the field names are simply not there....
Jun 06 2018
On 06/06/2018 03:52 PM, Adam D. Ruppe wrote:It is possible to add it to the runtime library right now (there's a thing called rtInfo in there made to hold it, it is just null right now), just people fight over even a few bytes of added metadata. So if it is added, it would surely be some opt-in thing that will require your thing be recompiled anyway.If I wanted to add it myself, would I need to create a personalised D compiler and/or D Runtime? That would be probably way too much for me :) Also, it would have to be distributed and used to create the plugins...If you can recompile the library, you can add a parallel extended info thing (MyReflectionInfo[TypeInfo] array perhaps, populated by a static this() ctor created via compile time reflection) that gives what you need.Yeah, I had some ideas in this regard, still I would like it to be transparent from the plugin creator point of view, but I don't know if it would be possible to contain everything in the base class... so far I had though about a base class like this: ``` import std.traits; import std.meta; TypeInfo[string][TypeInfo_Class] RTInfo; class Base { this(this C)() { if (typeid(C) in RTInfo) return; RTInfo[typeid(C)] = (TypeInfo[string]).init; static foreach_reverse(Type; AliasSeq!(C, BaseClassesTuple!C)) { static foreach(string field; FieldNameTuple!Type) { RTInfo[typeid(Type)][field] = typeid(typeof(__traits(getMember, Type, field))); } } } } ``` But I think children classes can bypass this constructor, so I guess it's not so easy, will have to keep trying :-) A templated static this would be cool, though: class Base { static this(this C)() { // ... } } Apparently it's not possible :-(
Jun 06 2018
On 6/6/18 9:28 AM, Arafel wrote:Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class?There is very little runtime reflection capability. The position has always been that you can *build* runtime reflection given compile-time reflection. object has rtInfo, as Adam says, which is built to be able to extend TypeInfo. But at the moment, it's not used for anything, and likely will stay that way for a while. Note, you CAN build runtime reflection using mixins, but other than rtInfo, there's no way to instrument modules you don't control. Even that's a high bar, since you would then have to have a modified druntime. -Steve
Jun 06 2018
On 2018-06-06 15:28, Arafel wrote:Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class? I have had a look as TypeInfo_Class [1], but apparently I can only get a list of types and offsets... which would be almost good enough, if not because the names of the members are missing, or at least I haven't been able to find them. In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Also, I know I could add some UDA or even crawl the modules and have this information generated automatically at compilation time and added to the type itself in a member, and I might even end up doing it, but honestly, I think it's something that the language should provide in a kind of easy / accessible way. Powerful as compile-time introspection is, I think runtime shouldn't be forgotten either :-)The simplest, in my opinion would be to for the subclasses to register themselves with the serializer. This is how Orange works to allow serializing through base class references [1]. The use compile time introspection on the subclass and serialize that as usual. [1] https://github.com/jacob-carlborg/orange/blob/master/tests/BaseClass.d#L73 -- /Jacob Carlborg
Jun 06 2018
On Wednesday, 6 June 2018 at 13:28:02 UTC, Arafel wrote:I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Also, I know I could add some UDA or even crawl the modules and have this information generated automatically at compilation time and added to the type itself in a member, and I might even end up doing it, but honestly, I think it's something that the language should provide in a kind of easy / accessible way. Powerful as compile-time introspection is, I think runtime shouldn't be forgotten either :-) Thanks, A. [1]: https://dlang.org/library/object/type_info__class.html [2]: https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getFields--There is a library that creates reflection metadata for you. [1] It seems a bit outdated and has some not-that-obvious compilation errors(for example getting ctor and callling it with runtime known type, or some other non template stuff), but other than that seems to be working (note that I didn't thorougly tested it, but its unittests succeeds on DMD 2.080 for both Windows x86 mscoff & x64 ) [1] https://code.dlang.org/packages/witchcraft
Jun 07 2018
Thanks for all the answers! Is it possible to register, say, a base class, and have all the subclasses then registered automatically? My idea would be to make it as transparent as possible for the plugin implementation, and also not having to depend on it. A.There is a library that creates reflection metadata for you. [1] It seems a bit outdated and has some not-that-obvious compilation errors(for example getting ctor and callling it with runtime known type, or some other non template stuff), but other than that seems to be working (note that I didn't thorougly tested it, but its unittests succeeds on DMD 2.080 for both Windows x86 mscoff & x64 ) [1] https://code.dlang.org/packages/witchcraft
Jun 07 2018
On Thursday, 7 June 2018 at 12:32:26 UTC, Arafel wrote:Thanks for all the answers! Is it possible to register, say, a base class, and have all the subclasses then registered automatically? My idea would be to make it as transparent as possible for the plugin implementation, and also not having to depend on it. A.I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements. Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation.
Jun 07 2018
On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote:I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements.Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens.Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation.Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A.
Jun 08 2018
On Friday, 8 June 2018 at 08:06:27 UTC, Arafel wrote:On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote:Yep. Like I said probably the easiest to use way is to place single call in each module. And there probably no other solution, because modules creates sort of isolated graph via imports. And I am not aware of any way to get list of modules passed in with compiler invocation to perform some sort of centralized one-liner registration. But anyway look at this, might give some tips on how it can be done mixin https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/component.d#L208 (note that it uses static this() - module constructor. I think this behavior was changed around 2015-2016 and now it will cause cyclic dependency errors when modules with ctors import each other) usage https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/lights.d#L12I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements.Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens.Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation.Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A.
Jun 08 2018
On Friday, 8 June 2018 at 08:21:39 UTC, evilrat wrote:On Friday, 8 June 2018 at 08:06:27 UTC, Arafel wrote:On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote:Yep. Like I said probably the easiest to use way is to place single call in each module. And there probably no other solution, because modules creates sort of isolated graph via imports. And I am not aware of any way to get list of modules passed in with compiler invocation to perform some sort of centralized one-liner registration. But anyway look at this, might give some tips on how it can be done mixin https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/component.d#L208I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements.Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens.Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation.Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A.(note that it uses static this() - module constructor. I think this behavior was changed around 2015-2016 and now it will cause cyclic dependency errors when modules with ctors import each other) usage https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/lights.d#L12Thanks very much for these links! I'm currently also trying to get a crack at runtime introspection for enabling richer serialization capabilities. It is nice to have compile time code generation, but it really sucks when dealing with object hierarchies and API interfaces. I'm doing kind of the same thing as witchcraft with explicit mixins (putting a "mixin reflect" into every stuff I want to reflect on.) But I'd like to have selective reflection/introspection, with a stuff. Also I need to inject static this(). A serious drawback. On that note, you can pass: --DRT-oncycle=ignore to your compiled app to instruct the runtime to ignore cycle warnings. linux ex.: "./app --DRT-oncycle=ignore" It is ugly as hell to disable this check, but I would accept it gladly if this would be the only impediment of getting runtime reflection. Sadly it is not, and I don't want to ramble right now :)
Jun 13 2018