www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Meta information parser

reply Sjoerd van Leent <svanleent gmail.com> writes:
Hello,

I've been thinking about this, but if we want to use D for let's say 
webservices, Corba, Dynamic linking and so forth, we have to be able to 
store meta information.

Now, I don't like to have this around:

[WebMethod]
Foo();

nor do I like it the Javaish way, with  interface attributes.

So, rather than using that, wouldn't it be possible to, at compile time, 
be able to link another program to the parser, being able to generate 
additional code.

This way one doesn't have to deal with all kinds of fuzzy 
meta-declarations, but one can still have the possibility of accessing 
module/class/struct/enum/template/method information at compile time.

So I'd like to have the possibility:

foo.d
-----
module foo;
import std.stdio;

int main(char[][] args) {
	writefln("%s", "Hello World!");
	return 0;
}
-----

then:

dmd foo.d -offoo.exe -parsethrough daspect

daspect.exe, generated from daspect.d could for example look like:

daspect.d:
-----
module daspect;

import std.meta;

void main(char[][] args) {
     // From args or file location in args[1]
     Meta meta = Meta.deserialize(args);

     MetaModule module = meta.module;
     writefln("Injecting statements for: %s", module.name);

     foreach(Method method; module.methods) {
         // Perhaps using bare asm or some semi asm
         // or plain d code, parsed by dmd parser, perhaps
         // just parsing again the generated file
         method.start.inject(
             "writefln(\"%s\", \"Enter [" ~
             Meta.escape(module.name ~ "]" ~
             method.name) ~ "]\");");
         method.end.inject(
             "writefln(\"%s\", \"Leave [" ~
             Meta.escape(module.name ~ "]" ~
             method.name) ~ "]\");");
     }
     return 0;
}

any ideas on this?

Regards,
Sjoerd
May 29 2006
parent reply pragma <pragma_member pathlink.com> writes:
In article <e5fgqn$vrs$1 digitaldaemon.com>, Sjoerd van Leent says...

any ideas on this?


I think you already touched on the high-points, but I'll throw some more wood on
the fire as you covered a lot of very interesting ground.

In-code metadata is only really as useful as one can access it.  I've used it in

make standards like CORBA easy to integrate with.  I agree with you completely.

Sadly, without a completely reflective runtime, an implementation backed on what
D is right now wouldn't hold a candle to other languages and platforms.  With
that in mind, it's kind of premature to even suggest a particular way to go
about this as D simply isn't ready for it yet. :(

But were that superior reflection interface here, I'd advocate the use of
pragmas for denoting attributes, as it would make parsing the attributes out
easier to implement.

Now I do like your idea behind code injection - I've thought about this one too.
Personally, I think a runtime "code emit" interface would make a very nice
complement to a fully-fledged reflection interface.  AFAIK, there is *nothing*
keeping anyone from coding one up right now as a standalone lib; provided they
understand enough about x86 ASM, the unerpinnings should be the least
problematic part.  Desigining things to be portable between Unix and Windows,
and finding somone to provide PPC support would be the biggest set hurdles to

interface may be easier to implement than the more sourcecode heavy example you
cited earlier simply because it eliminates the need for a parser.

I'll also add that a runtime emit interface is a step in the right direction
toward creating a self-hosted "D compiler toolkit" of sorts.  The kind of thing
anyone could use to create quick and simple code generators for all kinds of
purposes, without need for the DMD toolchain to be installed along with your
application.


- EricAnderton at yahoo
May 29 2006
parent reply Sjoerd van Leent <svanleent gmail.com> writes:
I would be in agreement with all this. Yet I also consider that we 
shouldn't make code bloat when it isn't asked for. For an example, a 
device driver developer might not want to have access to runtime 
information and code emission, only perhaps for a small part of the 
software. The same goes for the GC. However, a business application 
developer does want to have such features.

The suggestion of using the pragma statement seems good enough. You 
could just write:

class Foo {
     void bar() {
     }
}
pragma(EnableReflection, Foo);


Which could then be used to enable reflection on the class Foo, 
generating another info part, within the global application scope. Same 
goes for making SOAP/CORBA or whatever kind of interop object. Just do:

class Foo {
     void bar() {
     }
}
pragma(EnableReflection, Foo);

Then do somewhere in a main routine or some general subroutine:

.
.
SoapRegister soap = new SoapRegister(80);
soap.add(Foo.reflectinfo);
soap.listen();
.
.

Same goes for CORBA, though be it a bit more complex. One could even 
emulate Java RMI, or eventually make a D Interface Registry, perhaps 
looking like a combination of COM, RMI and .NET GAC.

Besides the pragma statement, it might also be useful with some things 
to use the extern keyword:

extern (Reflective) class Foo {
     void bar();
}

Regards,
Sjoerd
Jun 02 2006
parent reply pragma <pragma_member pathlink.com> writes:
In article <e5q51e$1te3$1 digitaldaemon.com>, Sjoerd van Leent says...

I would be in agreement with all this. Yet I also consider that we 
shouldn't make code bloat when it isn't asked for. For an example, a 
device driver developer might not want to have access to runtime 
information and code emission, only perhaps for a small part of the 
software. The same goes for the GC. However, a business application 
developer does want to have such features.

The suggestion of using the pragma statement seems good enough. You 
could just write:

 [snip]


Actually, with the pragma() statement, I was thinking more along the lines of
using D's syntax more directly.  Pragma statements are defined as taking the
following forms:

pragma(foo);
pragma(foo): /* statements */
pragma(foo){ /* statements */ }

The second two forms are of particular interest for reflection support:

pragma(reflect): class Foo{ /*...*/ }
pragma(reflect){
class Bar{
}
}

Incedentally, as Walter has outlined the use of unsupported pragmas, this opens
the door for specialized compiler/preprocessor support, as the 'reflect' pragma
is currently just an idea or reccomendation.  So again, there's not a whole lot
keeping someone from drafting a reference implementation now. ;)


Same goes for CORBA, though be it a bit more complex. One could even 
emulate Java RMI, or eventually make a D Interface Registry, perhaps 
looking like a combination of COM, RMI and .NET GAC.


I like how you think, and I completely agree with you.  If you don't mind,
here's my $0.02 on all this:

I really think that the killer feature to make all of the above work seamlessly
is runtime proxy /generation/. For example, the following would work nicely:

IFoobar foo = RMIConduit.getObjectProxy!(IFoobar)(someUUID);

.. if the call to getObject() can return an object instance that supports
IFoobar, that is a proxy for a remote object.  Said object instance would have
all of its vtable entries point to methods that use the associated conduit.
This way, the RMI portion of things is totally seamless beyond the getObject()
call.  Of course, this would depend on some yet-to-be-determined reflection
mechanism (not to mention some form of runtime code-gen).  As this example uses
defined interfaces, it could even be a compile-time template setup of sorts.

And of course, to make the above work with types that participate in class
hierarchies and more, a compile-time reflection setup just won't cut it:

Foobar foo = cast(Foobar)getObjectProxy.getObject(someUUID);

Object registration would probably work in reverse, having the RMI API digest
the object's particulars by reflection.  Thankfully, this step doesn't involve
any codegen and is very straightforward:

UUID id = RMIServer.registerObject(foo);

I guess what I'm getting at is there's more to the picture than just reflection:
having it cooperate with a code-generation API is an important consideration
too.

- EricAnderton at yahoo
Jun 02 2006
parent reply Sjoerd van Leent <svanleent gmail.com> writes:
I've been sleeping with my eyes wide open! I was trying to do this, but 
didn't know about this spec. This way, one only needs to preprocess 
somehow the class. Automatically generate a lookup table entry in the 
reflection API.

The only thing one needs is a decent D front-end parser. These could be 
both DMD-frontend as well as GCD-frontend.

Regards,
Sjoerd
Jun 02 2006
parent Rémy Mouëza <Rémy_member pathlink.com> writes:
Have you heard about Dflect ? Ben Hinkle modified dmd front end to make it
generate meta data information. I don't know if the front end used is up to date
with the last dmd version. You can find it here :
http://home.comcast.net/~benhinkle/dflect
As far as I know, it does not support code injection.
D runtime uses TypeInfo and ClassInfo class instances. These classes could be
extended to allow reflection, or could be added a meta object protocol. That
latter could open the door for some aspect oriented programming stuffs.
Jun 04 2006