www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Metaprogramming without templates

reply Stefan Koch <uplink.coder googlemail.com> writes:
Good Evening,

As you know from my previous typefunction project I am trying to 
make CTFE more usable for proper metaprogramming (as opposed to 
just string manipulation and string mixins).
Since I do believe that working with structure and rich 
data-types is more useful than working with only strings 
essentially.

The goal is to be able to write regular procedural code, to 
introspect existing or generate new code.

This I want to achieve with core.reflect and core.codegen, which 
will be part of druntime and give access to compiler/CTFE 
integrated reflection capabilities.

in core.reflect.decl, for example you would see:
```
import core.reflect.node; // Empty base class is defined in there
enum DeclarationKind
{
     FunctionDeclatation,
     /+
     StructDeclaration,
     UnionDeclarartion,
     ClassDeclarartion,
     VariableDeclaration,
     AliasDeclaration,
     TemplateDeclaration,
     +/
}
class Declaration : Node
{
     string name;
     DeclarationKind kind;
     Attribute[] attributes;
     Linkage linkage;
     string comment;
}

Declaration[] DeclarationsFromString(string s);
```

Which can then be used like so:
```
pragma(msg, DeclarationsFromString(
  q{
   int var = 3;
   void* myFunc (void*) {};
  }
)[1].name);
```
and it would output the name if the second Declarartion which is:
"myFunc"

this functionality does already work as of a few hours ago.
However it runs into forward reference issues even more than type 
functions did.
And I assume I'll first have to make advances on dealing with 
complicated forward reference graphs before I can show anything 
neat.
Jun 20 2021
next sibling parent sighoya <sighoya gmail.com> writes:
On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 As you know from my previous typefunction project I am trying 
 to make CTFE more usable for proper metaprogramming (as opposed 
 to just string manipulation and string mixins).


+1

But this sounds much like macros, which I prefer much.

However, they are unlikely.

Stefan, could you first make a dip for any of these, or would you 
present one grand dip for all of these things?

Just one point, it would be nice to retrieve also the context of 
the whole project as AST enacting macros as true language plugins.
Jun 21 2021
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 Good Evening,

 As you know from my previous typefunction project I am trying 
 to make CTFE more usable for proper metaprogramming (as opposed 
 to just string manipulation and string mixins).
 Since I do believe that working with structure and rich 
 data-types is more useful than working with only strings 
 essentially.

 The goal is to be able to write regular procedural code, to 
 introspect existing or generate new code.

 This I want to achieve with core.reflect and core.codegen, 
 which will be part of druntime and give access to compiler/CTFE 
 integrated reflection capabilities.

 in core.reflect.decl, for example you would see:
 ```
 import core.reflect.node; // Empty base class is defined in 
 there
 enum DeclarationKind
 {
     FunctionDeclatation,
     /+
     StructDeclaration,
     UnionDeclarartion,
     ClassDeclarartion,
     VariableDeclaration,
     AliasDeclaration,
     TemplateDeclaration,
     +/
 }
 class Declaration : Node
 {
     string name;
     DeclarationKind kind;
     Attribute[] attributes;
     Linkage linkage;
     string comment;
 }

 Declaration[] DeclarationsFromString(string s);
 ```

 Which can then be used like so:
 ```
 pragma(msg, DeclarationsFromString(
  q{
   int var = 3;
   void* myFunc (void*) {};
  }
 )[1].name);
 ```
 and it would output the name if the second Declarartion which 
 is:
 "myFunc"

 this functionality does already work as of a few hours ago.
 However it runs into forward reference issues even more than 
 type functions did.
 And I assume I'll first have to make advances on dealing with 
 complicated forward reference graphs before I can show anything 
 neat.


I'm looking forward to this but I thing the same technics are 
always involved.
So whavetever is the magic trick used, there's actually no magic. 
Either 1 or 2.

1. you need to parse new code to produce an ast. In D this is 
currently like mixin which goes by the lexical stage or more 
simply the .d you pass to the compiler.

2. you need to clone an ast to run semantic on it with a specific 
scope. In D this is currently like template and that requires to 
clone an ast so that the semantic will decorate the ast in the 
right ~~context~~ scope (e.g expression.type).

so eventually this would be something like something new that 
allows stage 2 without stage 1 ? AST macros ?
Jun 21 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 June 2021 at 10:36:56 UTC, Basile B. wrote:


 So whavetever is the magic trick used, there's actually no 
 magic. Either 1 or 2.

 1. you need to parse new code to produce an ast. In D this is 
 currently like mixin which goes by the lexical stage or more 
 simply the .d you pass to the compiler.

 2. you need to clone an ast to run semantic on it with a 
 specific scope. In D this is currently like template and that 
 requires to clone an ast so that the semantic will decorate the 
 ast in the right ~~context~~ scope (e.g expression.type).

 so eventually this would be something like something new that 
 allows stage 2 without stage 1 ? AST macros ?


Ideally you would not need stage 2 either, at least not always.
If all you are doing is using core.reflect (for example to get 
debug info).
Then no modifications of the AST is required.

What I am doing is essentially runtime reflection (like java and 
go).
However since the divide between runtime and compile-time is 
broken via CTFE,
We can use this information to splice code into the existing AST.
(which is what mixins do)
Whether I have to copy the AST or can solve for a resolution 
schedule that doesn't conflict is currently to be explored.
It's not easy.
It is however a great way to debug inconsistencies in DMDs AST.
Jun 21 2021
parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 21 June 2021 at 11:00:42 UTC, Stefan Koch wrote:

 On Monday, 21 June 2021 at 10:36:56 UTC, Basile B. wrote:


 [...]


 Ideally you would not need stage 2 either, at least not always.
 If all you are doing is using core.reflect (for example to get 
 debug info).
 Then no modifications of the AST is required.

 What I am doing is essentially runtime reflection (like java 
 and go).


ok I did not get that. Sorry for the off topic ;)
Jun 21 2021
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 21 June 2021 at 11:37:31 UTC, Basile B. wrote:

 [snip]

 ok I did not get that. Sorry for the off topic ;)


I'm not sure how off topic it is...I had the same question. I 
didn't really understand what was going on...still not sure I 
do...

The main question I would have for Stefan would be how exactly 
this relates to type functions.
Jun 21 2021
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 June 2021 at 11:42:47 UTC, jmh530 wrote:

 On Monday, 21 June 2021 at 11:37:31 UTC, Basile B. wrote:

 [snip]

 ok I did not get that. Sorry for the off topic ;)


 I'm not sure how off topic it is...I had the same question. I 
 didn't really understand what was going on...still not sure I 
 do...

 The main question I would have for Stefan would be how exactly 
 this relates to type functions.


For me it's a different take on type functions.
Type functions rely on a lot of un-inspectable compiler magic.
And while I think that they are intuitive you cannot easily see 
which introspection information you have access to.
Type functions are blending the syntax of types as types with 
types as values.
Creating hard to solve issues.
for example
```
type T = int;
pragma(msg, T.mangleof); // should this be the mangle of the 
Variable T ? or should it be the mangle of int?
```

`core.reflect` on the other hand declares what exactly you get 
back, what fields it has and which functions can be called on it.
Jun 21 2021
prev sibling next sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 ...


If something like this gets through, would this allow things like 
converting range queries into SQL, for example, like how Entity 
Framework allows?

e.g. You can convert `db.people.filter!(p => p.isMale)` directly 
into a prepared statement similar to `SELECT * FROM people WHERE 
isMale = TRUE`?
Jun 21 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 June 2021 at 12:39:25 UTC, SealabJaster wrote:

 On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 ...


 If something like this gets through, would this allow things 
 like converting range queries into SQL, for example, like how 
 Entity Framework allows?

 e.g. You can convert `db.people.filter!(p => p.isMale)` 
 directly into a prepared statement similar to `SELECT * FROM 
 people WHERE isMale = TRUE`?


Hmm yes I think so.
But the user-code for this would look quite horrid since you have 
to match a particular templates and so forth.
Also you cannot change the meaning of code that comes in.
So in that sense they're not actual macros.
You can however generate a new function/module which is exactly 
like the one you reflected on except that you do a SQL query 
rather than using the template.
Jun 21 2021
parent reply SealabJaster <sealabjaster gmail.com> writes:
On Monday, 21 June 2021 at 13:18:21 UTC, Stefan Koch wrote:

 ...


Exciting, even if the end code looks like a complete hack!

Another thing: Entity Framework, when you give it something like:

```csharp
int age = 20; // Any external variable
MyDbContext.People.Where(person => person.Age >= age);
```

It's able to determine that `age` is an external variable, so 
essentially a parameter, and generates a prepared statement like:

```sql
SELECT * FROM people WHERE age >= $1;
```

And it will then automatically forward whatever's in `age` into 
parameter $1.

This is of course a very niche use-case, but it's one of the 
things that makes Entity Framework so fluent to use (i.e. your 

database instead of in the application's memory). Of course it 
falls back to client-side evaluation if it can't translate things.

This gets more complex the more conditions you tack on, as well 
as things like 
[.Include](https://docs.microsoft.com/en-us/ef/core/querying/related-data/eager).

This doesn't necessarily need the ability to rewrite the AST, but 
as you said, simply generate based off of the reflection of the 
AST.

And there's likely a lot more use cases outside of this example. 
But I'm now wondering what other things might be possible  _
Jun 21 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 June 2021 at 14:15:27 UTC, SealabJaster wrote:

 On Monday, 21 June 2021 at 13:18:21 UTC, Stefan Koch wrote:

 ...


 Exciting, even if the end code looks like a complete hack!
 [snip]

 ```sql
 SELECT * FROM people WHERE age >= $1;
 ```

 And it will then automatically forward whatever's in `age` into 
 parameter $1.



You can now get the Variable declaration as reflection parameter.
also you can now pass scopes ;)
Look at the following code:

```D
import core.reflect.type;
import core.reflect.node;
import core.reflect.decl;

int[2] myArr;

static immutable arr_node = nodeFromName("myArr");
static immutable arr_type = cast(immutable TypeArray) 
(cast(immutable VariableDeclaration)arr_node).type;
static assert(
     arr_type.kind == "sarray" // this name is subject to change.
     // TypeKinds are supposed to be an enum not a string but I 
didn't feel like implementing it yet :)
     && arr_type.identifier == "int[2]"
     && arr_type.nextOf.identifier == "int"
     && arr_type.dim == 2
);

pragma(msg, arr_node);
```
this will output:
`VariableDeclaration("myArr", [], 0, "", TypeArray("sarray", 4u, 
8u, "int[2]", Type("int", 4u, 4u, "int"), 2LU))`
where VariableDeclaration is defined as
```D
class Declaration : Node
{
     string name;
     Node[] attributes;
     Linkage linkage;
     string comment;

     abstract immutable DeclarationKind kind() pure;
}

class VariableDeclaration : Declaration
{
     Type type;
     override final immutable DeclarationKind kind() pure { return 
DeclarationKind.VariableDeclaration; }
}
```
     type is defined as:
```D
class Type : Node
{
     string kind;
     uint alignSize;
     ulong size;
     string identifier = null; /// optional Types may be anonymous
}

class TypeNext : Type
{
     Type nextOf;
}
class TypeArray : TypeNext
{
     int dim;
}
```
     so this reads:
```JS
name: "myArr",
attributes: [],
Linkage: 0, //(Linkage.init) we don't capture it yet
comment: "",
type: {
   kind: "sarray",
   alignSize: 4
   size: 8,
   identifier: "int[2]",

   nextOf: { // element type
     kind: "int",
     alignSize: 4,
     size: 4,
     identifier: "int"
   },
dim: 2
}
```

I am rather happy with the structure of the output actually.
The implementation is such much cleaner than type functions are 
as well ;)

Let me know what you think.

P.S. I do apologize for this very long post, I don't really know 
how to present this work in progress stuff properly.
Jun 23 2021
next sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Wednesday, 23 June 2021 at 21:06:23 UTC, Stefan Koch wrote:


 You can now get the Variable declaration as reflection ...

 Let me know what you think.


I very much like where this is headed: "Meta programming is just 
programming.  Here are the very-nearly-self-documenting objects 
that you'll be using".
Jun 23 2021
prev sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Wednesday, 23 June 2021 at 21:06:23 UTC, Stefan Koch wrote:

 On Monday, 21 June 2021 at 14:15:27 UTC, SealabJaster wrote:

 On Monday, 21 June 2021 at 13:18:21 UTC, Stefan Koch wrote:

 ...






I like it so far. I can already foresee the arguments about 
whether capturing comments is something it should do >:D.


it's the only runtime reflection I'm used to:

see Expression trees: 
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/expression-trees/#creating-expression-trees-from-lambda-expressions

see Type reflection: 
https://docs.microsoft.com/en-us/dotnet/framework/reflection-and-codedom/reflection

Obviously D's potential ability for reflection will be different 
and have different capabilities, but I'm sadly not able to really 
think up anything unique or different.

Other people of course will have more of a better eye for 
specific details and pain points, something I'm unable to provide 
at the moment.

Keep up the good work though!
Jun 23 2021
parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Thursday, 24 June 2021 at 06:44:49 UTC, SealabJaster wrote:

 On Wednesday, 23 June 2021 at 21:06:23 UTC, Stefan Koch wrote:

 On Monday, 21 June 2021 at 14:15:27 UTC, SealabJaster wrote:

 On Monday, 21 June 2021 at 13:18:21 UTC, Stefan Koch wrote:

 ...






 I like it so far. I can already foresee the arguments about 
 whether capturing comments is something it should do >:D.


 because it's the only runtime reflection I'm used to:

 see Expression trees: 
 https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/expression-trees/#creating-expression-trees-from-lambda-expressions

 see Type reflection: 
 https://docs.microsoft.com/en-us/dotnet/framework/reflection-and-codedom/reflection

 Obviously D's potential ability for reflection will be 
 different and have different capabilities, but I'm sadly not 
 able to really think up anything unique or different.

 Other people of course will have more of a better eye for 
 specific details and pain points, something I'm unable to 
 provide at the moment.

 Keep up the good work though!


You missed out some toys,



https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/code-quotations

C++/CLI

https://docs.microsoft.com/en-us/cpp/dotnet/dotnet-programming-with-cpp-cli-visual-cpp?view=msvc-160

Code Generators,

https://github.com/dotnet/roslyn/blob/main/docs/features/source-generators.md


of having a polyglot runtime sharing library code.
Jun 24 2021
parent SealabJaster <sealabjaster gmail.com> writes:
On Thursday, 24 June 2021 at 09:17:15 UTC, Paulo Pinto wrote:



 https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/code-quotations



such an interesting language.

e.g. 
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/units-of-measure
Jun 26 2021
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 [snip]
 And I assume I'll first have to make advances on dealing with 
 complicated forward reference graphs before I can show anything 
 neat.


No more assumptions now I have proof :P

check the following example:
```D
import core.reflect.type;
import core.reflect.node;
import core.reflect.decl;

// reflection functions may ne annotated with  (core.reflect)
// to prevent code generation for them
// in this case the function itself does not call reflection 
functions
// such as nodeFromName. but if it did we would have to prevent 
codegen
// as nodeFromName does not have a function body
 (core.reflect) immutable(Type) getType(immutable Node node)
{
   if (immutable vd = cast(immutable VariableDeclaration)node)
   {
      return vd.type;
   }

   return null;
}

int[2] myArr;
static immutable arr_node = nodeFromName("myArr");

static immutable arr_type =
   cast(immutable TypeArray)
     (cast(immutable VariableDeclaration)arr_node).type;

static assert(arr_type.identifier == "int[2]"
   && arr_type.nextOf.identifier == "int"
   && arr_type.dim == 2
);

pragma(msg, "typeof(x).stringof before decl: '", 
nodeFromName("x").getType().identifier, "'");
typeof(myArr)* x;
pragma(msg, "typeof(x).stringof after decl: '", 
nodeFromName("x").getType().identifier, "'");
```
The output of this will be:
```D
typeof(x).stringof before decl: 'typeof(myArr)*'
typeof(x).stringof after decl: 'int[2]*'
```

as you can see the implicitly forward declared symbol `x` 
"changes" or rather "refines" and resolves it's type after it has 
been actually declared.
I hope I can find a way to compute a schedule that will leave a 
symbol in a stable state as reflection is not supposed to be able 
to modify anything.
Jun 24 2021
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 24 June 2021 at 11:53:00 UTC, Stefan Koch wrote:

 On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 [snip]
 And I assume I'll first have to make advances on dealing with 
 complicated forward reference graphs before I can show 
 anything neat.


 No more assumptions now I have proof :P


 I hope I can find a way to compute a schedule that will leave a 
 symbol in a stable state as reflection is not supposed to be 
 able to modify anything.


more news on this.
I am now forcing the resolution of types as part of the 
reflection.
That does get rid of the uncomplicated forward refs.

Also I checked that the data can be used at runtime without 
issues.

If you run to following program:

```D
     extern (C) void*** fnXXfn(void* p, void** pp);
     static immutable fn_x = cast(immutable FunctionDeclaration) 
nodeFromName("fnXXfn");
     void main()
     {
       import core.stdc.stdio;
       printf(StructToString(fn_x).ptr);
     }


     string StructToString(S)(S _struct, uint indent_level = 1)
     {
       import std.conv : to;

       char[] indent_m;
       indent_m.length = indent_level * 4;
       indent_m[] = ' ';
       string indent = cast(string) indent_m;

       string result = S.stringof ~ " = {\n";

       //TODO: if it's a class cast to base_type;

       foreach(i, e;_struct.tupleof)
       {
         auto name = __traits(identifier, _struct.tupleof[i]);
         result ~= indent ~ name ~ ": ";
         enum isAggregate(T) = is(T == class) || is(T == struct);
         if (e)
         {
             static if (is(typeof(e) : const(char)[]))
             {
                result ~= "\"" ~ e ~ "\"" ~ "\n";
             }
             else static if (is(typeof(e) : E[], E))
             {
                 if (!e.length)
                 {
                   result ~= "[]\n";
                 }
                 else
                 {
                     result ~= "[\n";
                     indent ~= "    ";
                     foreach(elem;e)
                     {
                         pragma(msg, "isArray: " , typeof(e));
                         static if (isAggregate!(typeof(elem)))
                         {
                             result ~= indent ~ 
StructToString(elem, indent_level + 2) ~ ",\n";
                         }
                         else
                         {
                            result ~= indent ~ to!string(elem) ~ 
"\n";
                         }
                     }
                     indent = indent[0 .. $ - 4];
                     result ~= indent ~ "]\n";
                 }
             }
             else static if (isAggregate!(typeof(e)))
             {
                 result ~= StructToString(e, indent_level + 1) ~ 
"\n";
             }
             else
             {
                result ~= to!string(e) ~ "\n";
             }
         }
         else
         {
             result ~= "null\n";
         }
       }
       result ~= indent[4 .. $] ~ "}";
       return result;
     }
```

it will give you the output:

```D
     immutable(FunctionDeclaration) = {
         type: immutable(FunctionType) = {
             returnType: immutable(Type) = {
                 kind: "pointer"
                 alignSize: 8
                 size: 8
                 identifier: "void***"
             }
             parameterTypes: [
                 immutable(FunctionParameter) = {
                     type: immutable(Type) = {
                         kind: "pointer"
                         alignSize: 8
                         size: 8
                         identifier: "void*"
                     }
                     identifier: "p"
                 },
                 immutable(FunctionParameter) = {
                     type: immutable(Type) = {
                         kind: "pointer"
                         alignSize: 8
                         size: 8
                         identifier: "void**"
                     }
                     identifier: "pp"
                 },
             ]
         }
         parameters: null
         fbody: null
     }
```

If you look closely you will see that it matches type of the 
function `fnXXfn`
Declared in the code up above.
If my printout code would recurse into super-classes you would 
also be able to see name of the declaration.
However I didn't do that yet ;)
This is just an example to show that these reflection classes can 
be used at runtime as well as at compile time.

Lastly you can play with this yourself if you do:
`git clone https://github.com/UplinkCoder/dmd -b core_reflect`
and
`git clone https://github.com/UplinkCoder/druntime -b 
core_reflect`
and
`git clone https://github.com/dlang/phobos -b v2.094.2`
then you go into build it
then you build phobos
you might have to remove '-transition=complex' from the makefiles 
in both druntime and phobos.

and now you should be able to run the exmaple posted above.

I am not quite sure why; but although this is based on ~master as 
of roughly 5 days ago
phobos only seems to compile if you checkout v2.094.2 ...
Jun 25 2021
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 25 June 2021 at 15:07:54 UTC, Stefan Koch wrote:

 On Thursday, 24 June 2021 at 11:53:00 UTC, Stefan Koch wrote:

 On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 [snip]






Hi there, I have just been able to do the first thing I wanted to 
use core reflect for.
dynamically loading function pointers for debugging functions.
In case the function cannot be loaded because it's not running in 
the debug environment I want the function pointers to be 
initialized to dummy functions which do nothing.

Let's have a look at the code:
```D
import core.reflect.reflect;
static immutable decls = declarationsFromTokenString(
q{
//  extern (C)  nogc pure nothrow __gshared {
      void __itt_sync_create (void *addr, const char *objtype, 
const char *objname, int attribute);
      void  __itt_sync_cancel (void* addr);
      void __itt_sync_acquired (void* addr);
      void  __itt_sync_releasing (void* addr);
      void __itt_sync_prepare (void* addr);
//  }
});
import core.reflect.nodeToSource;

pragma(msg, () {
     string result;

     foreach(d;decls)
     {
         if (auto fd = cast(FunctionDeclaration) d)
         {
             result ~= fd
                 .makeFunctionVariableWithDummyInit()
                 .nodeToSource()
             ;
         }
     }
     return result;
} ());

VariableDeclaration makeFunctionVariableWithDummyInit(const 
FunctionDeclaration fd) {
   auto type = new TypePointer();
   type.nextOf = cast()fd.type;
   auto name = fd.name;

   auto result = new VariableDeclaration();
   result.type = type;
   result.name = name;
   auto func = new FunctionLiteral();
   func.parameters = cast()fd.parameters;
   func.fbody = makeDummyBody(fd.type.returnType);
   result._init = func;
   return result;
}

BlockStatement makeDummyBody(const Type returnType)
{
     BlockStatement stmt = new BlockStatement();

     if (cast(TypeBasic) returnType && returnType.identifier == 
"void")
     {
         stmt.statements ~= new ReturnStatement();
     }
     else
         assert(0, "non void return not supported as of now");

     return stmt;
}
```

and the output of this will be

```D
void function (void* addr, char* objtype, char* objname, int 
attribute) __itt_sync_create = () {
     return ;
};
void function (void* addr) __itt_sync_cancel = () {
     return ;
};
void function (void* addr) __itt_sync_acquired = () {
     return ;
};
void function (void* addr) __itt_sync_releasing = () {
     return ;
};
void function (void* addr) __itt_sync_prepare = () {
     return ;
};
```

which does exactly what I want it declares function pointers with 
the same name as the functions extracted from the intel header.
and initializes them to dummy functions be default.
yay.

Creating a string is not how it's supposed to be spliced in 
though!
rather I will special case `mixin` to splice in `core.reflect` 
directly nodes when it is used on them.
Or maybe something more clever :)
Jun 29 2021
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 25 June 2021 at 15:07:54 UTC, Stefan Koch wrote:

 On Thursday, 24 June 2021 at 11:53:00 UTC, Stefan Koch wrote:

 On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

 [snip]






Hi there, I have just been able to do the first thing I wanted to 
use core reflect for.
dynamically loading function pointers for debugging functions.
In case the function cannot be loaded because it's not running in 
the debug environment I want the function pointers to be 
initialized to dummy functions which do nothing.

Let's have a look at the code:
```D
import core.reflect.reflect;
static immutable decls = declarationsFromTokenString(
q{
//  extern (C)  nogc pure nothrow __gshared {
      void __itt_sync_create (void *addr, const char *objtype, 
const char *objname, int attribute);
      void  __itt_sync_cancel (void* addr);
      void __itt_sync_acquired (void* addr);
      void  __itt_sync_releasing (void* addr);
      void __itt_sync_prepare (void* addr);
//  }
});
import core.reflect.nodeToSource;

pragma(msg, () {
     string result;

     foreach(d;decls)
     {
         if (auto fd = cast(FunctionDeclaration) d)
         {
             result ~= fd
                 .makeFunctionVariableWithDummyInit()
                 .nodeToSource()
             ;
         }
     }
     return result;
} ());

VariableDeclaration makeFunctionVariableWithDummyInit(const 
FunctionDeclaration fd) {
   auto type = new TypePointer();
   type.nextOf = cast()fd.type;
   auto name = fd.name;

   auto result = new VariableDeclaration();
   result.type = type;
   result.name = name;
   auto func = new FunctionLiteral();
   func.parameters = cast()fd.parameters;
   func.fbody = makeDummyBody(fd.type.returnType);
   result._init = func;
   return result;
}

BlockStatement makeDummyBody(const Type returnType)
{
     BlockStatement stmt = new BlockStatement();

     if (cast(TypeBasic) returnType && returnType.identifier == 
"void")
     {
         stmt.statements ~= new ReturnStatement();
     }
     else
         assert(0, "non void return not supported as of now");

     return stmt;
}
```

and the output of this will be

```D
void function (void* addr, char* objtype, char* objname, int 
attribute) __itt_sync_create = () {
     return ;
};
void function (void* addr) __itt_sync_cancel = () {
     return ;
};
void function (void* addr) __itt_sync_acquired = () {
     return ;
};
void function (void* addr) __itt_sync_releasing = () {
     return ;
};
void function (void* addr) __itt_sync_prepare = () {
     return ;
};
```

which does exactly what I want it declares function pointers with 
the same name as the functions extracted from the intel header.
and initializes them to dummy functions be default.
yay.

Creating a string is not how it's supposed to be spliced in 
though!
rather I will special case `mixin` to splice in `core.reflect` 
directly nodes when it is used on them.
Or maybe something more clever :)
Jun 29 2021
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 29 June 2021 at 14:28:36 UTC, Stefan Koch wrote:

 and the output of this will be

 ```D
 void function (void* addr, char* objtype, char* objname, int 
 attribute) __itt_sync_create = () {
     return ;
 };
 void function (void* addr) __itt_sync_cancel = () {
     return ;
 };
 void function (void* addr) __itt_sync_acquired = () {
     return ;
 };
 void function (void* addr) __itt_sync_releasing = () {
     return ;
 };
 void function (void* addr) __itt_sync_prepare = () {
     return ;
 };
 ```


Oops as you can see the output the result is not as intended.
The parameters are missing.
That's because we haven't run semantic yet and therefore we have 
to get the parameters from the type.

if you replace the `makeFunctionVariableWithDummyInit` function 
from the previous post with this one:

```D
VariableDeclaration makeFunctionVariableWithDummyInit(const 
FunctionDeclaration fd) {
   auto type = new TypePointer();
   type.nextOf = cast()fd.type;
   auto name = fd.name;

   auto result = new VariableDeclaration();
   result.type = type;
   result.name = name;
   auto func = new FunctionLiteral();

   typeof(func.parameters) parameters = [];
   parameters.length = fd.type.parameterTypes.length;

   foreach(i, p;fd.type.parameterTypes)
   {
    parameters[i] = new typeof(parameters[0])();
    parameters[i].name = p.identifier;
    parameters[i].type = cast()p.type;
   }
   func.parameters = parameters;
   func.fbody = makeDummyBody(fd.type.returnType);
   result._init = func;
   return result;
}
```
The output will be correct:

```D
void function (void* addr, char* objtype, char* objname, int 
attribute) __itt_sync_create = (void* addr, char* objtype, char* 
objname, int attribute) {
     return ;
};
void function (void* addr) __itt_sync_cancel = (void* addr) {
     return ;
};
void function (void* addr) __itt_sync_acquired = (void* addr) {
     return ;
};
void function (void* addr) __itt_sync_releasing = (void* addr) {
     return ;
};
void function (void* addr) __itt_sync_prepare = (void* addr) {
     return ;
};
```
Jun 29 2021