www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - describe-d: an introspection library

reply bogdan <szabobogdan3 gmail.com> writes:
Hi!

I wrote this small `describe-d` library to allow me to do more 
readable introspection in some of my projects.

Any feedback is appreciated!

Thanks,
Bogdan

[1] https://gitlab.com/szabobogdan3/describe-d
[2] https://code.dlang.org/packages/describe-d
Apr 13 2020
parent reply WebFreak001 <d.forum webfreak.org> writes:
On Monday, 13 April 2020 at 12:11:03 UTC, bogdan wrote:

 Hi!

 I wrote this small `describe-d` library to allow me to do more 
 readable introspection in some of my projects.

 Any feedback is appreciated!

 Thanks,
 Bogdan

 [1] https://gitlab.com/szabobogdan3/describe-d
 [2] https://code.dlang.org/packages/describe-d


cool library, looks really useful for writing big introspection 
projects, but on the other hand also looks like this would 
significantly increase compilation time currently. Have you done 
any benchmarks how your methods compare (RAM, CPU time) to 
traits? While I love the syntax and the idea, I think with 
current CTFE and templates it would be quite a big hit to use in 
a project.
Apr 14 2020
parent reply bogdan <szabobogdan3 gmail.com> writes:
On Tuesday, 14 April 2020 at 10:55:13 UTC, WebFreak001 wrote:

 On Monday, 13 April 2020 at 12:11:03 UTC, bogdan wrote:

 Hi!

 I wrote this small `describe-d` library to allow me to do more 
 readable introspection in some of my projects.

 Any feedback is appreciated!

 Thanks,
 Bogdan

 [1] https://gitlab.com/szabobogdan3/describe-d
 [2] https://code.dlang.org/packages/describe-d


 cool library, looks really useful for writing big introspection 
 projects, but on the other hand also looks like this would 
 significantly increase compilation time currently. Have you 
 done any benchmarks how your methods compare (RAM, CPU time) to 
 traits? While I love the syntax and the idea, I think with 
 current CTFE and templates it would be quite a big hit to use 
 in a project.


Thanks!

I did not make any performance measurements. If you know how 
could I do this at compile time, it would be awesome. I think, I 
would need to watch the compiler process for this. If there is no 
other way, I'll do it like this after I will have a more concrete 
APIs.

Usually templates take more time to compile that CTFE, that's why 
I intend to use less templates, or none, if is possible. I expect 
to see an increase in the compilation time, but It should not be 
exponential. I am doing this, because I work a lot with 
introspection, and some of my projects are hard to read because 
of all this mess around traits an templates.

That made me think that, I could give a few more seconds to the 
compiler to have a more readable code. I'll update my projects to 
use this library to see how the compilation performance will 
change.

Bogdan
Apr 15 2020
parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 15 April 2020 at 08:00:12 UTC, bogdan wrote:

I use a similar approach in openmethods. With the added twist 
that I need to re-create functions from existing functions, with 
some modifications (e.g. change the parameter types, add a 
parameter), while preserving function and parameter attributes. 
That inevitably leads to constructing mixin code (which I call 
"mixtures"), and it has to  work across module boundaries (i.e. 
use `ReturnType!F` in the mixture, not the stringified type).

See here:
https://github.com/jll63/openmethods.d/blob/master/source/openmethods.d#L503,
here: https://github.com/jll63/openmethods.d/blob/master/sourc
/openmethods.d#L568 and here:
https://github.com/jll63/openmethods.d/blob/master/source/bolts/reflection/metafunction.d

I did not shy away from using templates and indeed there is a 
significant increase in compilation time. On the other hand, the 
code is much cleaner than a previous iteration that created the 
mixtures directly.

If these techniques get traction, it will incite compiler 
developers to improve template instantiation, hopefully.

I wonder if templates are lazily expanded. I haven't looked at 
the compiler's code, my guess is: maybe not.
Apr 21 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 21 April 2020 at 14:43:04 UTC, Jean-Louis Leroy wrote:



 I wonder if templates are lazily expanded. I haven't looked at 
 the compiler's code, my guess is: maybe not.


If the template gets used it gets instantiated (and cached).
if not than not.
you can use the -vcg-ast switch to look at how your souce code 
looks "expandend".


 If these techniques get traction, it will incite compiler 
 developers to improve template instantiation, hopefully.


They already gained traction, unfortunately.
Making this system fast is not easy. And will need a few language 
changes, and some work from users (programmers) as well.
Apr 21 2020
parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 22 April 2020 at 05:20:18 UTC, Stefan Koch wrote:

 On Tuesday, 21 April 2020 at 14:43:04 UTC, Jean-Louis Leroy 
 wrote:



 I wonder if templates are lazily expanded. I haven't looked at 
 the compiler's code, my guess is: maybe not.


 If the template gets used it gets instantiated (and cached).


I did not word my question precisely, I meant: is it instantiated 
in its *entirety*.

Consider:

     import std.meta;

     template Function(Attributes...)
     {
       enum isPure = Attributes[0];
       enum isNogc = Attributes[1];
       alias parameters = Attributes[2];
     }

     template Parameter(int i)
     {
       // static assert(false);
     }

     template reflectFunction(alias Fun)
     {
       alias reflectFunction = Function!(
         true, true, staticMap!(Parameter, 1, 2, 3));
     }

     void foo();
     pragma(msg, reflectFunction!(foo).isPure);

Is `Parameter` instantiated in this example? If I uncomment the 
assert, it fires. This is what gives me the impression that more 
is instantiated than is really used. And that is probably why 
reflecting runtime entities as template meta-objects with 
properties for all aspects of them is slow.

I am way of my field of competence here, but I have the 
impression that speeding this up would not require a change in 
the language (unless it is stated that templates are greedily 
expanded, and common idioms rely on this).


 you can use the -vcg-ast switch to look at how your souce code 
 looks "expandend".


I tried compiling my example with `dmd -vcg-ast -c tmt.d` and I 
did got neither an AST nor an error, and the option is not in 
`dmd -h`, where can I read about it?


 They already gained traction, unfortunately.


Well...Two things. I ended up developing this approach because I 
needed to copy function attributes, UDAs, function parameter 
storage classes and UDAs to a generated function. If the 
language, or Phobos, provided me a more direct way, trust me, I 
would have used it. For example, storage classes are not part of 
a parameter's type. OK. Now you can use `Parameter!F[0]` to 
declare a parameter in a new function (great!), but you cannot 
say `__traits(getStorageClasses, F)[0] Parameter!F[0]`.

Also, one of D's selling points is that it does templates and 
meta-programming better. It's only natural that people use these 
features then (note that in C++, sorry for mentioning it, in my 
example Parameter would *not* be instantiated).
Apr 22 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 22 April 2020 at 13:32:03 UTC, Jean-Louis Leroy 
wrote:

 I tried compiling my example with `dmd -vcg-ast -c tmt.d` and I 
 did got neither an AST nor an error, and the option is not in 
 `dmd -h`, where can I read about it?


-vcg-ast is a debugging tool I original built to fix an bug in 
the inliner.
when you through the switch a new file of the form 
"soucefilename.d.cg" should get written out for every sourcefile 
in the same directory that the source file is in.

As for your other point.
I am working on a much more powerful and efficient meta 
programming system.
It's no where near done. So I will not commit to any release date 
... I've learned my lesson.
Apr 22 2020
parent reply Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 22 April 2020 at 17:16:28 UTC, Stefan Koch wrote:


 I am working on a much more powerful and efficient meta 
 programming system.


Great! Is it going to be in a library, or part of the compiler? 
Can we get a preview somewhere?
Apr 22 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 22 April 2020 at 17:27:48 UTC, Jean-Louis Leroy 
wrote:

 On Wednesday, 22 April 2020 at 17:16:28 UTC, Stefan Koch wrote:


 I am working on a much more powerful and efficient meta 
 programming system.


 Great! Is it going to be in a library, or part of the compiler? 
 Can we get a preview somewhere?


It's going to be part of the compiler.
You can look at the ... expression DIP.
which Manu posted in General, for taste of where my stuff is 
going.
Basically I will give CTFE the ability to modify type-lists 
directly.
Apr 22 2020
parent Jean-Louis Leroy <jl leroy.nyc> writes:
On Wednesday, 22 April 2020 at 17:32:33 UTC, Stefan Koch wrote:


 It's going to be part of the compiler.
 You can look at the ... expression DIP.
 which Manu posted in General, for taste of where my stuff is 
 going.


I think I see the point - I am familiar with C++ variadic 
templates.

Yes, speeding up `staticMap`, and often eliminating the need to 
use it, will hugely benefit meta-programming.

However, that will not solve the problem of unnecessary 
instantiations that my example above demonstrates (it has a bug 
btw, the parameters should have been wrapped in a 
bolts.meta.AliasPack, but it doesn't alter the spirit).

Thank you for the `-vcg-ast` switch! It looks like I am going to 
use it a lot.

I am rewriting my system to express the accessors in terms of 
free functions taking meta-objects, instead of meta-objects 
containing properties. I have a hunch that that will curb 
instantiation, compile time and .o size.
Apr 22 2020