www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Initialising Context Pointer during Semantic Analysis

reply Teodor Dutu <teodor.dutu gmail.com> writes:
Hi,

For SAoC 2022, I want to translate DRuntime hooks to templates. I 
am working on `_d_newitem{U,T,iT}`. The compiler uses them to 
lower `new S(args)` expressions, where `S` is a `struct`.

To instantiate the new template hooks, the lowering has to be 
moved from the glue layer to the semantic analysis. However, when 
the constructed `struct` is nested, the lowering logic also needs 
to copy the `struct`'s context pointer(s). This is currently 
[handled by the glue layer in 
`e2ir.d`](https://github.com/dlang/dmd/blob/6bf60ea0eb174631ede0074a77d3898d943e0b30/compiler/src/dmd/
2ir.d#L1191-L1209). Now the same logic needs to be replicated in the semantic
analysis, where there is no machinery to find the right frame pointer, such as
[`getEthis()` in `toir.d`](https://github.com/dlang/dmd/blob/6bf60ea0eb174631ede0074a77d3898d943e0b30/compiler/src/dmd/toir.d#L212).

After discussing with my mentors, we have come up with 2 possible 
approaches:

1. Add a new AST node for copying context pointers to 
`expression.d`: `ContextPointerExp`. The new lowering of `new 
S(args)` in the semantic phase will insert a `ContextPointerExp` 
node and then the glue layer will visit it and copy the correct 
context pointer to the newly created `struct`. This approach is 
efficient from a runtime perspective, as it doesn't add any 
unnecessary computation, but has the disadvantage of requiring 
changes beyond the frontend. Thus it won't be transparent for LDC 
and GDC, who will have to handle `ContextPointerExp` in their own 
glue layers.

1. Change the lowering to something like this:

```d
S* s = new S();
// lowering:
S* s = (S tmp, _d_newitem(tmp));
```

This way, `tmp`'s context pointer will be set by default and 
`_d_newitem` will be able to simply copy it from `tmp` to `s` in 
a way similar to 
[`copyEmplace()`](https://github.com/dlang/dmd/blob/6bf60ea0eb174631ede0074a77d3898d943e0b30/druntime/src/core/life
ime.d#L1276-L1280). This solution is not without faults, though, as it requires
creating and initialising `tmp` only to get its context pointer.

`void`-initialising `tmp` from the snippet above is not a 
solution, as that also leaves the context pointer uninitialised. 
The code below either seg faults, or prints a random value for 
`s.y` because the context pointer of `s` is `null`:

```d
void main()
{
     int x = 3;

     struct S
     {
         int y;
         void bar()
         {
             y = x;
         }
     }

     S s = void;
     s.bar();

     writeln(s);
}
```

Do you see a better approach than 2? How should we handle context 
pointers in the semantic phase?

Thanks,
Teo
Nov 25 2022
next sibling parent Teodor Dutu <teodor.dutu gmail.com> writes:
On Friday, 25 November 2022 at 16:10:36 UTC, Teodor Dutu wrote:

 `void`-initialising `tmp` from the snippet above is not a 
 solution, as that also leaves the context pointer 
 uninitialised. The code below either seg faults, or prints a 
 random value for `s.y` because the context pointer of `s` is 
 `null`:

 ```d
 void main()
 {
     int x = 3;

     struct S
     {
         int y;
         void bar()
         {
             y = x;
         }
     }

     S s = void;
     s.bar();

     writeln(s);
 }
 ```


Also, is it the expected behaviour to leave the context pointer 
uninitalised as well when using `S s = void` or is it a bug? It 
seems odd to not set the context pointer as this makes `S s = 
void` risky to use for nested structs.
Nov 25 2022
prev sibling next sibling parent reply Johan <j j.nl> writes:
On Friday, 25 November 2022 at 16:10:36 UTC, Teodor Dutu wrote:

 Hi,

 For SAoC 2022, I want to translate DRuntime hooks to templates. 
 I am working on `_d_newitem{U,T,iT}`. The compiler uses them to 
 lower `new S(args)` expressions, where `S` is a `struct`.

 To instantiate the new template hooks, the lowering has to be 
 moved from the glue layer to the semantic analysis. However, 
 when the constructed `struct` is nested, the lowering logic 
 also needs to copy the `struct`'s context pointer(s). This is 
 currently [handled by the glue layer in 
 `e2ir.d`](https://github.com/dlang/dmd/blob/6bf60ea0eb174631ede0074a77d3898d943e0b30/compiler/src/dmd/
2ir.d#L1191-L1209). Now the same logic needs to be replicated in the semantic
analysis, where there is no machinery to find the right frame pointer, such as
[`getEthis()` in `toir.d`](https://github.com/dlang/dmd/blob/6bf60ea0eb174631ede0074a77d3898d943e0b30/compiler/src/dmd/toir.d#L212).

 After discussing with my mentors, we have come up with 2 
 possible approaches:

 1. Add a new AST node for copying context pointers to 
 `expression.d`: `ContextPointerExp`. The new lowering of `new 
 S(args)` in the semantic phase will insert a 
 `ContextPointerExp` node and then the glue layer will visit it 
 and copy the correct context pointer to the newly created 
 `struct`. This approach is efficient from a runtime 
 perspective, as it doesn't add any unnecessary computation, but 
 has the disadvantage of requiring changes beyond the frontend. 
 Thus it won't be transparent for LDC and GDC, who will have to 
 handle `ContextPointerExp` in their own glue layers.


This is not a deal breaker, as long as it is clearly 
communicated. So add a special section to the release notes for 
frontend-dependent projects what has changed / what was added. 
Adding AST nodes impacts more than just GDC and LDC.

Why not extend the information attached to a NewExpression node?
AST rewriting makes it much harder (or impossible) to generate 
good user diagnostics and debug information.

-Johan
Nov 26 2022
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Saturday, 26 November 2022 at 11:17:45 UTC, Johan wrote:

 Why not extend the information attached to a NewExpression node?
 AST rewriting makes it much harder (or impossible) to generate 
 good user diagnostics and debug information.


This.

We also lose optimization opportunities with every early lowering 
(LDC gc2stack, GDC call/argument/return flags, assume hints, etc)
Nov 26 2022
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/25/22 17:10, Teodor Dutu wrote:

 
 After discussing with my mentors, we have come up with 2 possible 
 approaches:


Probably the right approach is to lower the call to a local function, 
e.g., with local template instantiation.

This is a somewhat hacky way to achieve that without changing the 
compiler frontend:

auto make(T,alias dummy)(){
     return T(); // can do your emplace magic here
}

auto foo(){
     int x=123;
     struct S{
         int bar(){ return x; }
     }
     void[0] context;
     return make!(S,context)();
}


void main(){
     import std.stdio;
     writeln(foo().bar()); // 123
}


TBH, I think local template instantiation should work even without that 
dummy parameter and with an arbitrary number of contexts.


General remark: I think the whole point of this project is to get rid of 
compiler magic, so instead of introducing compiler magic and 
magic-UB-that-works-anyway, maybe propose general language features to 
cover the missing use cases. Not only compiler developers may want to 
hook into builtin behavior.
Nov 26 2022
parent reply Teodor Dutu <teodor.dutu gmail.com> writes:
On Saturday, 26 November 2022 at 18:12:44 UTC, Timon Gehr wrote:

 On 11/25/22 17:10, Teodor Dutu wrote:

 
 After discussing with my mentors, we have come up with 2 
 possible approaches:


 Probably the right approach is to lower the call to a local 
 function, e.g., with local template instantiation.

 This is a somewhat hacky way to achieve that without changing 
 the compiler frontend:

 auto make(T,alias dummy)(){
     return T(); // can do your emplace magic here
 }

 auto foo(){
     int x=123;
     struct S{
         int bar(){ return x; }
     }
     void[0] context;
     return make!(S,context)();
 }


 void main(){
     import std.stdio;
     writeln(foo().bar()); // 123
 }


I don't understand why passing `context` to `make()` is better 
than passing it to `_d_newitemT()` directly. I would still use 
`context` to copy the context pointer, so wouldn't it be the same?
Nov 28 2022
parent Teodor Dutu <teodor.dutu gmail.com> writes:
On Monday, 28 November 2022 at 17:25:06 UTC, Teodor Dutu wrote:

 I don't understand why passing `context` to `make()` is better 
 than passing it to `_d_newitemT()` directly. I would still use 
 `context` to copy the context pointer, so wouldn't it be the 
 same?


In addition, in my case, isn't the `alias dummy` parameter the 
same as a `const ref T dummy` parameter? Does the alias somehow 
allow me to get the context pointer more elegantly than copying 
it from `dummy`? If not, then I would still need to create 
`dummy`, pass it as a `ref` to a template (either the local one 
or the hook itself) and then copy the context pointer from it.
Nov 28 2022