www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - dflect alpha

reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
I've started poking around with a reflection API for D called "dflect" and 
here's what I have after a day (the zip of the files is 
http://home.comcast.net/~benhinkle/dflect.zip). Comments welcome.
The basic idea is that to generate metadata about a module you run 
dflect.exe (based not surprisingly on dmdfe) which takes a module foo and 
generates another D source file fooinfo with auto-generated classes 
describing the original module. You then compile and link in these metadata 
modules into the final binary. Right now it only supports fields but I'll 
start poking around with methods next. I haven't even started thinking about 
templates and more complex modules. For example given the module

module test;
class Foo {
  int x;
  char[] y;
}

running dflect gives something like

// Auto-generated reflection data for module test
import dflect;
import test;
class FooInfo : MetaClass
{
    static MetaClass makeInfo(){return new FooInfo;}
    Object construct(){return new Foo;}
    this() {
      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);
    }
}
static this() {
  dflect.register("test.Foo",&FooInfo.makeInfo);
}

Here's a sample app using the resulting metadata:

import dflect;
import test;
int main() {
  MetaClass cls = getInfo("test.Foo");
  Foo u = new Foo;
  MetaField field = cls.fields["x"];
  field.set(u,100);
  field = cls.fields["y"];
  field.set(u,"hello");
  printf("setter %d %.*s\n",u.x,u.y);
  char[] res;
  field.get(u,&res);
  printf("getter %.*s\n",res);
  return 0;
}

C:\dflect\dflect>dflect test.d
C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
C:\dflect\dflect>dtest
setter 100 hello
getter hello
Jul 04 2005
next sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Cool!

One question: as far as I remember order
of fields in class instances may be different from
declaration order.

How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


I mean that dflect.exe must be always used with the
same set of command line switches as used with dmd.exe
and version of dflect.exe must be always in sync with dmd.exe.
Am I right?

BTW: does D class layout depends on command line switches?

Andrew.


"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dab8av$s2q$1 digitaldaemon.com...

 I've started poking around with a reflection API for D called "dflect" and 
 here's what I have after a day (the zip of the files is 
 http://home.comcast.net/~benhinkle/dflect.zip). Comments welcome.
 The basic idea is that to generate metadata about a module you run 
 dflect.exe (based not surprisingly on dmdfe) which takes a module foo and 
 generates another D source file fooinfo with auto-generated classes 
 describing the original module. You then compile and link in these 
 metadata modules into the final binary. Right now it only supports fields 
 but I'll start poking around with methods next. I haven't even started 
 thinking about templates and more complex modules. For example given the 
 module

 module test;
 class Foo {
  int x;
  char[] y;
 }

 running dflect gives something like

 // Auto-generated reflection data for module test
 import dflect;
 import test;
 class FooInfo : MetaClass
 {
    static MetaClass makeInfo(){return new FooInfo;}
    Object construct(){return new Foo;}
    this() {
      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);
    }
 }
 static this() {
  dflect.register("test.Foo",&FooInfo.makeInfo);
 }

 Here's a sample app using the resulting metadata:

 import dflect;
 import test;
 int main() {
  MetaClass cls = getInfo("test.Foo");
  Foo u = new Foo;
  MetaField field = cls.fields["x"];
  field.set(u,100);
  field = cls.fields["y"];
  field.set(u,"hello");
  printf("setter %d %.*s\n",u.x,u.y);
  char[] res;
  field.get(u,&res);
  printf("getter %.*s\n",res);
  return 0;
 }

 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello
Jul 04 2005
next sibling parent reply "Uwe Salomon" <post uwesalomon.de> writes:
 How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


 I mean that dflect.exe must be always used with the
 same set of command line switches as used with dmd.exe
 and version of dflect.exe must be always in sync with dmd.exe.
 Am I right?


I stumbled over that one, too. Isn't it possible to include that like this:

fields["x"] = new MetaField("x", typeid(int), Foo.x.offsetof,  
Foo.x.sizeof);

That should always work as expected?

Ciao
uwe
Jul 04 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Uwe Salomon" <post uwesalomon.de> wrote in message 
news:op.stejkmuh6yjbe6 sandmann.maerchenwald.net...

 How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


 I mean that dflect.exe must be always used with the
 same set of command line switches as used with dmd.exe
 and version of dflect.exe must be always in sync with dmd.exe.
 Am I right?


 I stumbled over that one, too. Isn't it possible to include that like 
 this:

 fields["x"] = new MetaField("x", typeid(int), Foo.x.offsetof, 
 Foo.x.sizeof);

 That should always work as expected?

 Ciao
 uwe


You're both right. When I tried Foo.x.offsetof I got a compiler error so I 
just slapped in the number. I'll post something to D.bugs so that Walter can 
fix it - assuming what I saw was a bug.
I'm also thinking of not importing the original module and in that case the 
fixed numbers would be the way to go.
Jul 04 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dac0na$1hg5$1 digitaldaemon.com...

 "Uwe Salomon" <post uwesalomon.de> wrote in message 
 news:op.stejkmuh6yjbe6 sandmann.maerchenwald.net...

 How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


 I mean that dflect.exe must be always used with the
 same set of command line switches as used with dmd.exe
 and version of dflect.exe must be always in sync with dmd.exe.
 Am I right?


 I stumbled over that one, too. Isn't it possible to include that like 
 this:

 fields["x"] = new MetaField("x", typeid(int), Foo.x.offsetof, 
 Foo.x.sizeof);

 That should always work as expected?

 Ciao
 uwe


 You're both right. When I tried Foo.x.offsetof I got a compiler error so I 
 just slapped in the number. I'll post something to D.bugs so that Walter 
 can fix it - assuming what I saw was a bug.
 I'm also thinking of not importing the original module and in that case 
 the fixed numbers would be the way to go.


I got method dispatching working and when doing so I needed the slot number 
in the vtable for each method so there are definitely going to be hard-coded 
values in the metadata code. I decided to remove the import of the original 
module so I'm leaving the field offset numbers. The size might get removed 
and computed from the TypeInfo but for now I'll let it be.
I've also written a C header file for all this so now C/C++ code can get/set 
fields and call methods on D objects using an API that roughly looks like 
JNI. I'll post an update soon.
Jul 05 2005
parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
 I've also written a C header file for all this so now C/C++ code can 
 get/set
 fields and call methods on D objects using an API that roughly looks like 
 JNI. I'll post an update soon.


Cool. I mean extremely cool.

Probably it will be possible to make transparent interop DMDScript <-> D ...
Jul 05 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
news:dabv5q$1gci$1 digitaldaemon.com...

 Cool!

 One question: as far as I remember order
 of fields in class instances may be different from
 declaration order.

 How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


 I mean that dflect.exe must be always used with the
 same set of command line switches as used with dmd.exe
 and version of dflect.exe must be always in sync with dmd.exe.
 Am I right?

 BTW: does D class layout depends on command line switches?


The only dependency I'm aware of would be version switches. Debugging and 
DbC switches shouldn't matter. Although to be safe I'd pass the same 
switches anyway.
Jul 04 2005
parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dac14g$1ht0$1 digitaldaemon.com...

 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
 news:dabv5q$1gci$1 digitaldaemon.com...

 Cool!

 One question: as far as I remember order
 of fields in class instances may be different from
 declaration order.

 How then this will reliably work for classes?:


      fields["x"] = new MetaField("x",typeid(int),8,4);
      fields["y"] = new MetaField("y",typeid(char[]),12,8);


 I mean that dflect.exe must be always used with the
 same set of command line switches as used with dmd.exe
 and version of dflect.exe must be always in sync with dmd.exe.
 Am I right?

 BTW: does D class layout depends on command line switches?


 The only dependency I'm aware of would be version switches. Debugging and 
 DbC switches shouldn't matter. Although to be safe I'd pass the same 
 switches anyway.


I see, thanks, Ben.
Jul 04 2005
prev sibling next sibling parent reply Markus Dangl <danglm in.tum.de> writes:
Ben Hinkle schrieb:

 
 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello 
 
 


Sorry, i can't find dflect.d in the zip file. Where can i get it?

Markus
Jul 07 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Markus Dangl" <danglm in.tum.de> wrote in message 
news:dakp99$273f$1 digitaldaemon.com...

 Ben Hinkle schrieb:

 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello


 Sorry, i can't find dflect.d in the zip file. Where can i get it?

 Markus


I forgot that part. I've updated the zip file and make a page for it 
http://home.comcast.net/~benhinkle/dflect/
You can call constructors and methods now. Return values are ignored 
currently. Also most likely lots of little things still don't work.
Jul 07 2005
parent reply pragma <pragma_member pathlink.com> writes:
In article <dakqub$2888$1 digitaldaemon.com>, Ben Hinkle says...

"Markus Dangl" <danglm in.tum.de> wrote in message 
news:dakp99$273f$1 digitaldaemon.com...

 Ben Hinkle schrieb:

 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello


 Sorry, i can't find dflect.d in the zip file. Where can i get it?

 Markus


I forgot that part. I've updated the zip file and make a page for it 
http://home.comcast.net/~benhinkle/dflect/
You can call constructors and methods now. Return values are ignored 
currently. Also most likely lots of little things still don't work. 


Ben,
I had no idea that this was a modified DMD front-end... at first glance it
looked like you had merely provided a way to spoof reflection via some other
means.  Bravo!  

- EricAnderton at yahoo
Jul 08 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"pragma" <pragma_member pathlink.com> wrote in message 
news:daltti$35e$1 digitaldaemon.com...

 In article <dakqub$2888$1 digitaldaemon.com>, Ben Hinkle says...

"Markus Dangl" <danglm in.tum.de> wrote in message
news:dakp99$273f$1 digitaldaemon.com...

 Ben Hinkle schrieb:

 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello


 Sorry, i can't find dflect.d in the zip file. Where can i get it?

 Markus


I forgot that part. I've updated the zip file and make a page for it
http://home.comcast.net/~benhinkle/dflect/
You can call constructors and methods now. Return values are ignored
currently. Also most likely lots of little things still don't work.


 Ben,
 I had no idea that this was a modified DMD front-end... at first glance it
 looked like you had merely provided a way to spoof reflection via some 
 other
 means.  Bravo!

 - EricAnderton at yahoo


dlint and this reflection generator were the two reasons I wanted dmdfe so 
I'm just getting around to doing what I wanted to do for a while. I figured 
the dmd front end does all the work of figuring out the module semantics so 
why not spit that metadata out?
It would also be interesting to spit out the meta data in another format 
like XML or something instead of spitting out D code. I was going to look 
into that eventually but if anyone wants to poke around they're certainly 
welcome to experiment.
Jul 08 2005
parent pragma <pragma_member pathlink.com> writes:
In article <dalvps$4pl$1 digitaldaemon.com>, Ben Hinkle says...

"pragma" <pragma_member pathlink.com> wrote in message 
news:daltti$35e$1 digitaldaemon.com...

 In article <dakqub$2888$1 digitaldaemon.com>, Ben Hinkle says...

"Markus Dangl" <danglm in.tum.de> wrote in message
news:dakp99$273f$1 digitaldaemon.com...

 Ben Hinkle schrieb:

 C:\dflect\dflect>dflect test.d
 C:\dflect\dflect>dmd dtest.d dflect.d testinfo.d test.d
 C:\dflect\dflect>dtest
 setter 100 hello
 getter hello


 Sorry, i can't find dflect.d in the zip file. Where can i get it?

 Markus


I forgot that part. I've updated the zip file and make a page for it
http://home.comcast.net/~benhinkle/dflect/
You can call constructors and methods now. Return values are ignored
currently. Also most likely lots of little things still don't work.


 Ben,
 I had no idea that this was a modified DMD front-end... at first glance it
 looked like you had merely provided a way to spoof reflection via some 
 other
 means.  Bravo!

 - EricAnderton at yahoo


dlint and this reflection generator were the two reasons I wanted dmdfe so 
I'm just getting around to doing what I wanted to do for a while. I figured 
the dmd front end does all the work of figuring out the module semantics so 
why not spit that metadata out?
It would also be interesting to spit out the meta data in another format 
like XML or something instead of spitting out D code. I was going to look 
into that eventually but if anyone wants to poke around they're certainly 
welcome to experiment. 


Funny you should mention that.

I already made an experiment into generating XML with the first DMDFE release
you wrote.

http://svn.dsource.org/projects/dsp/trunk/dexter/

I'm still learning how to get the most from the front-end.  I wanted to use this
to write documentation, so it captures comments and associates them with D
constructs.  Just pass the XML on to a stylesheet engine and presto: instant
HTML documentation.

I seriously think this is the way to go for producing documentation with D.

- EricAnderton at yahoo
Jul 10 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
A request for input: what (if any) metadata tags (which aren't already in 

and Java's annotations. I suppose the tags will have to be in comments in 
order to get past the D compiler. Should doxygen/javadoc comments be added 
to the metadata? For example I could imagine something very simple where any 
comment preceding a method gets added to the metadata for that method as a 
"description" property. Or it can try to pick apart tags like  doc,  author, 
 version, etc etc.

thoughts?
Jul 16 2005
parent reply pragma <pragma_member pathlink.com> writes:
In article <dbbcdh$1rv6$1 digitaldaemon.com>, Ben Hinkle says...

A request for input: what (if any) metadata tags (which aren't already in 

and Java's annotations. I suppose the tags will have to be in comments in 
order to get past the D compiler. Should doxygen/javadoc comments be added 
to the metadata? For example I could imagine something very simple where any 
comment preceding a method gets added to the metadata for that method as a 
"description" property. Or it can try to pick apart tags like  doc,  author, 
 version, etc etc.

thoughts? 


Ben, here's my opinion. :)

Personally, I think a documentation parser (similar to my Dexter prototype)
should be left to being its own tool.  From a maintenance standpoint, and a
coding standpoint it just makes good sense.  At the very least, meshing dflect
with a certain style of documentation reduces its usefulness as one may want to
use a different doc tool along with dflect.

I think dflect could use a good general-purpose attribute hook similar to what
NET has.  The most D-like way to achieve this would be to use a pragma():


 version(dflect){ pragma(attribute,<expression>); }


The above expression would be crammed into the static ctor() for the
corresponding metadata module, and would have to evaluate to a certain type: say
an Attribute class or somesuch.

- EricAnderton at yahoo
Jul 16 2005
parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
"pragma" <pragma_member pathlink.com> wrote in message 
news:dbbgfc$1uk2$1 digitaldaemon.com...

 In article <dbbcdh$1rv6$1 digitaldaemon.com>, Ben Hinkle says...

A request for input: what (if any) metadata tags (which aren't already in

syntax
and Java's annotations. I suppose the tags will have to be in comments in
order to get past the D compiler. Should doxygen/javadoc comments be added
to the metadata? For example I could imagine something very simple where 
any
comment preceding a method gets added to the metadata for that method as a
"description" property. Or it can try to pick apart tags like  doc, 
 author,
 version, etc etc.

thoughts?


 Ben, here's my opinion. :)

 Personally, I think a documentation parser (similar to my Dexter 
 prototype)
 should be left to being its own tool.  From a maintenance standpoint, and 
 a
 coding standpoint it just makes good sense.  At the very least, meshing 
 dflect
 with a certain style of documentation reduces its usefulness as one may 
 want to
 use a different doc tool along with dflect.


I agree dflect isn't meant to be used to generate doc. The reason I 
mentioned the doc string is that dflect is gathering up information about a 
class for use at run-time. Help is part of that metadata. For example in 
MATLAB we use Java reflection to let users load jar files and dynamically 
instantiate and use Java classes from the MATLAB command line. Having short 
doc strings of what the user just loaded would be very helpful. Otherwise 
they are on their own to figure out what they have.


 I think dflect could use a good general-purpose attribute hook similar to 
 what
 NET has.  The most D-like way to achieve this would be to use a pragma():


 version(dflect){ pragma(attribute,<expression>); }


 The above expression would be crammed into the static ctor() for the
 corresponding metadata module, and would have to evaluate to a certain 
 type: say
 an Attribute class or somesuch.

 - EricAnderton at yahoo


Good idea - though it's a pity the code is so verbose. I'll probably just 
skip the whole issue for now and dump the parse tree info and that's it.
Jul 16 2005