www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Find symbol in unknown module at compile time?

reply "bitwise" <bitwise.pvt gmail.com> writes:
Hi,
   I have a mixin, which can mix a class into any module. At 
compile time, I will know the name of the class, it's base class, 
and that it is at global scope in some module.

Is there any way to find this class from a different module at 
compile time?

  Thanks
Dec 06 2014
next sibling parent "bitwise" <bitwise.pvt gmail.com> writes:
On Saturday, 6 December 2014 at 20:22:13 UTC, bitwise wrote:

 Hi,
   I have a mixin, which can mix a class into any module. At 
 compile time, I will know the name of the class, it's base 
 class, and that it is at global scope in some module.

 Is there any way to find this class from a different module at 
 compile time?

  Thanks



Any thoughts on having something like this?

symbol __traits(find, symbolName);
Dec 06 2014
prev sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 06 Dec 2014 20:22:12 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Hi,
    I have a mixin, which can mix a class into any module. At=20
 compile time, I will know the name of the class, it's base class,=20
 and that it is at global scope in some module.
=20
 Is there any way to find this class from a different module at=20
 compile time?


why do you need that?
Dec 06 2014
parent reply "bitwise" <bitwise.pvt gmail.com> writes:
 why do you need that?


I've made a reflection library which creates metadata on a 
per-module basis.

usage would be like this:

[code]

module test;
// classes, methods, etc..
mixin(reflect!test); // generates reflection here so private 
members are accessible

[/code]

-or-

[code]

module main;
import test;
mixin(reflect!test); // oops, test is a static library, must 
create reflection here instead

[/code]


If the reflection mixin is in the same module that it's 
reflecting, it can be easily found, but in the case of a static 
library where reflection information has to be created in a 
separate module, other modules won't be able to locate the 
reflection.
Dec 06 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 07 Dec 2014 01:12:52 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 If the reflection mixin is in the same module that it's=20
 reflecting, it can be easily found, but in the case of a static=20
 library where reflection information has to be created in a=20
 separate module, other modules won't be able to locate the=20
 reflection.


why don't create a dummy module which only keeps list of all known
objects, and create `shared static this ()` constructors which will
just register everything in that module?

in compile-time other modules can inspect things with traits, and in
run-time you'll have one central place with all known metadata, and
with autoregistration. and it's doesn't matter where you call your
metadata generation mixin, all data will end registered in one place.
Dec 06 2014
parent reply "bitwise" <bitwise.pvt gmail.com> writes:
 why don't create a dummy module which only keeps list of all 
 known
 objects, and create `shared static this ()` constructors which 
 will
 just register everything in that module?


I am actually doing this already, but this only solves for 
finding information at runtime.

One or more of these are created in a module by the mixin above:

class testReflection : Reflection {
	override void register(){ ... }
	static const(ModuleDecl) info = ... ;
}

abstract class Reflection {
	void register();
}

static this() {
	foreach(m; ModuleInfo) {
		foreach(c; m.localClasses()) {
			if(c.base is Reflection.classinfo) {
				Reflection refl = cast(Reflection)c.create();
				refl.register();
			}
		}
	}
}

So as long as 'Test_Reflection' is in the module that it's 
reflecting, I can get the info I need like this:

__traits(getMember, __traits(getMember, test, "test" ~ 
"Reflection"), "info");

But, this doesn't work if the mixin is instantiated in a separate 
module.

Now when the reflection actually can be found, the entire 
reflection data hierarchy is accessible at compile time.

Either one of these could work:

Tuple(symbols) __traits(find, string name)
Tuple(alias or string) __traits(allModules)

I have minimal experience with compiler technology right now, and 
I've just recently gotten my own fork of DMD set up, but wouldn't 
the first option be as simple as looking the name up in a symbol 
table and returning it?
Dec 06 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 07 Dec 2014 05:42:31 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

what you actually want is some cross-module compile-time data storage.
this is impossible to implement. at least to make it reliable.

with separate compilation, for example, you can't get list of "all
modules", 'cause we can link other unimported modules to the final
executable. this is somethimes useful with "extern", for example.

and there is no such thing as "main module" too, so there is no "root"
for symbols.

besides, some templates can be not instantiated at the time you doing
compile-time reflection, for example. that is why my console-ng
requires that "register all" mixin must be the last line in the module.

with some efforts and hackery you can walk all imported modules, but
this can fail at unexpected places.

tl;dr: things aren't that easy when it comes to compile-time
reflection. either live that to runtime or write compile-time wrappers
and restrict compile-time reflection usage to well-defined places and
patterns.
Dec 06 2014
parent reply "bitwise" <bitwise.pvt gmail.com> writes:
On Sunday, 7 December 2014 at 06:25:48 UTC, ketmar via 
Digitalmars-d wrote:

 On Sun, 07 Dec 2014 05:42:31 +0000
 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 what you actually want is some cross-module compile-time data 
 storage.
 this is impossible to implement. at least to make it reliable.

 with separate compilation, for example, you can't get list of 
 "all
 modules", 'cause we can link other unimported modules to the 
 final
 executable. this is somethimes useful with "extern", for 
 example.

 and there is no such thing as "main module" too, so there is no 
 "root"
 for symbols.

 besides, some templates can be not instantiated at the time you 
 doing
 compile-time reflection, for example. that is why my console-ng
 requires that "register all" mixin must be the last line in the 
 module.

 with some efforts and hackery you can walk all imported 
 modules, but
 this can fail at unexpected places.

 tl;dr: things aren't that easy when it comes to compile-time
 reflection. either live that to runtime or write compile-time 
 wrappers
 and restrict compile-time reflection usage to well-defined 
 places and
 patterns.


I think I understand what you mean. I've been surprised with the
way D is able to handle circular references, but I guess there
still has to be some order of initialization at the end of the
day.

Thinking about my current design, I believe there are places
where circular references could cause problems, but that I have
yet to find them because I haven't properly tested against a
large/complicated codebase.

I think the best plan for me would be to store all reflection
information in the reflection module itself as local static
variables of templates, i.e. Meyers singleton style. This would
disallow me from reflecting private members, but would greatly
simplify my design. Also, the mixin could then be used
exclusively for registering runtime reflection, and be placed
anywhere.

I would like to be able to reflect private members though... Is
there any way to give a module private access to an unrelated
module?

I'm thinking of something like placing a pragma at the top of a
module or adding a qualifier to a module declaration.

#pragma privileged
or
module reflection privileged;

I understand that packages are meant to solve this problem for 
regular code, but given the fact that D has such a comprehensive 
reflection system, it would be nice to have this special case.
Dec 07 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 07 Dec 2014 21:44:51 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I would like to be able to reflect private members though... Is
 there any way to give a module private access to an unrelated
 module?


nope. and i hope there will be no such thing. ;-)


 I understand that packages are meant to solve this problem for=20
 regular code, but given the fact that D has such a comprehensive=20
 reflection system, it would be nice to have this special case.


you either want to inspect something from the outside and then it's not
private, or it's private and invisible to the strangers.

by introducing hacks you killing the protection idiom altogether. if
you need to modify the module you want to inspect to insert pragma...
well, make the necessary things 'package'-protected and and inspector
to the package: you modified the module anyway.

if you want to allow external pragmas that allows poking private module
data... well, just make everything in that module public, you just
killed the whole protection thing. ;-)

what i really want to say is that if you need to even know about
private members from the outside of the module, something is very-very
wrong. dump the idea that reflection can do at least *something* with
private parts. that's bad. that's dangerous. that's dirty. *nobody* can
know anything about object internals except the object and his close
friends living in the same module.

either don't make it private or pretend that it's non-existant for
the outside world.
Dec 07 2014
parent reply "bitwise" <bitwise.pvt gmail.com> writes:
 if you want to allow external pragmas that allows poking 
 private module
 data... well, just make everything in that module public, you 
 just
 killed the whole protection thing. ;-)


This is what I mean, but I don't think it would 'kill' anything. 
It's not like I'm suggesting that cast(public) be added to the 
language. I'm suggesting a special case pragma or something that 
can be added to a module that is responsible for reflection, to 
allow that module to do it's job.

I don't think what I am suggesting is that radical. It's 
basically the same thing as casting away const. Also, I don't 
think the problem is the feature, so much as the programmers that 
would start adding '#pragma privileged' to all their files 
without a good reason.


 what i really want to say is that if you need to even know about
 private members from the outside of the module, something is 
 very-very
 wrong.


For a publicly released library, I would agree, but if you wanted 
to use runtime reflection to tweak some variables during the 
development process, this would save you the trouble of having to 
modify your interface to allow it.
Dec 07 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 08 Dec 2014 02:58:45 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 if you want to allow external pragmas that allows poking=20
 private module
 data... well, just make everything in that module public, you=20
 just
 killed the whole protection thing. ;-)


=20
 This is what I mean, but I don't think it would 'kill' anything.=20
 It's not like I'm suggesting that cast(public) be added to the=20
 language. I'm suggesting a special case pragma or something that=20
 can be added to a module that is responsible for reflection, to=20
 allow that module to do it's job.


i believe that if you need to access some private module thingys, that
thingys are made private erroneously.


 I don't think what I am suggesting is that radical. It's=20
 basically the same thing as casting away const.


i'm still sure that compiler should forbid such casts. there should be
no way to remove 'const' after it's sticked.


 Also, I don't=20
 think the problem is the feature, so much as the programmers that=20
 would start adding '#pragma privileged' to all their files=20
 without a good reason.


but they will. ;-) you know, "just in case" or "to speedup the things".

btw, i think that default module protection must be "private", not
"public". having module imports as private by default but everything
else as public is confusing.


 what i really want to say is that if you need to even know about
 private members from the outside of the module, something is=20
 very-very
 wrong.


=20
 For a publicly released library, I would agree, but if you wanted=20
 to use runtime reflection to tweak some variables during the=20
 development process, this would save you the trouble of having to=20
 modify your interface to allow it.


if you need to tweak something private... well, it's not private
anymore, there must be public API to do that. that API can be version'd
out later (or documented).
Dec 07 2014
prev sibling parent "bitwise" <bitwise.pvt gmail.com> writes:
While I hear a lot of experienced programmers take this point of 
view, I still don't really understand or agree with it. I believe 
a good language should facilitate good design, but I don't think 
it should force it. I imagine this type of principal may simplify 
code review for large projects, but haven't experienced it first 
hand.

Anyways, Thanks for the input ;)
Dec 07 2014