www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proposal for new compiler built-ins: __CTX__ and __CONTEXT__

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
This isn't a DIP but I'd like to start this conversation to see 
what people think.

One pattern I often see in D libraries is the capturing of 
context of the call site, for example the file and line number 
and sometimes the function and module names.

There are many examples of this in the real world. Here's a quick 
search result of all the D modules containing `__FILE__` on 
Github: 
https://github.com/search?q=%22__FILE__%22+lang%3AD+&type=code

And here's similar results for `__FUNCTION__`: 
https://github.com/search?q=%22__FUNCTION__%22+lang%3AD+&type=code

They're often passed in pairs, or sometimes quadruplets.

I have a few things in mind:
- It's repetitive having to list out all of these context 
keywords and assign each one of them to their own free parameter.
- It's visually noisy seeing them and it can distract the reader.
- It's possible to accidentally pass the wrong argument to a 
parameter that is default initialized to `__FILE__` or 
`__LINE__`. Maybe your logging function takes a formatting string 
but you accidentally pass the formatting string to the parameter 
`string file = __FILE__` parameter.
- Once these parameters are received it's cumbersome to pass them 
around to other routines. Passing these downwards is a pretty 
common use-case of many libraries.
- It consumes stack space due to the need to pass context 
parameters on the stack, and might even eat up the EAX register 
(according to the ABI: 
https://dlang.org/spec/abi.html#parameters). This may or may not 
be a problem for you, depending on your use-case and depending on 
how many parameters you actually pass around.

We could simplify this and maybe even make it more flexible.

I'm envisioning a new type `__CTX__` which contains all of these 
different contexts which are currently separate keywords on: 
https://dlang.org/spec/expression.html#specialkeywords.

Here it is:

```D
struct __CTX__ {
     string file;
     string file_full_path;
     int line;
     string func;
     string pretty_func;
}
```

Having it defined as a struct serves a few objectives:
- It makes it easy to declare and use this type in user code.
- All library code will have one single compatible type they can 
easily pass around to each other.
- Makes it harder to confuse parameters. For example it's 
currently easy to pass a random string to a function expecting a 
`__FILE__`, because the parameter is a string.
- It makes it possible to choose whether to receive and pass this 
structure around by value or by reference.

We also need to initialize it. So perhaps we'd call this 
initialization keyword `__CONTEXT__`.

Here's how the client code might look like:

```D
// ctx passed by stack
void infoStack(string msg, __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}

// ctx passed by pointer
void infoRef(string msg, ref __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}

void main() {
     infoStack("Hello world");
     infoRef("Hello world");
}
```

Notice that the calls to `infoStack` and `infoRef` will generate 
different assembly code, as `__CTX__` is passed by value when 
calling `infoStack` and by reference when calling `infoRef`.

Here's a full example with some fake context just to give a 
clearer picture:

```D
import std.stdio : writefln;

ref __CTX__ __CONTEXT__() {
     static __CTX__ ctx = __CTX__("mymod.d", 
"/project/src/mymod.d", "mymod", 123,
         "mymod.func", "void mymod.func()");
     return ctx;
}

struct __CTX__ {
     string file;
     string file_full_path;
     string mod;
     int line;
     string func;
     string pretty_func;
}

// ctx passed by stack
void infoStack(string msg, __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}

// ctx passed by pointer
void infoRef(string msg, ref __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}

void main() {
     infoStack("Hello world");  // mymod.d(123): Hello world
     infoRef("Hello world");    // mymod.d(123): Hello world
}
```

I think it should be possible to receive `__CTX__` by reference 
from the compiler as all of the default parameters must be known 
at compile-time. So the context could be stored in `RODATA` or 
somewhere and an address taken from it (But I'm a bit out of my 
depth here).

I can think of some downsides:
- Passing the `__CTX__` by stack can make you capture more than 
you're interested in. Perhaps you're only interested in file + 
line and don't care about module name, function name, etc. That 
means you're suddenly consuming a lot more stack space.
- Counter-point: You can always continue to use `__FILE__` and 
`__LINE__`. I'm not suggesting to deprecate or remove these.
- It's possible `__CTX__` could later grow if the compiler 
developers decide to add a new context field to it. This would 
somewhat negatively impact all user-code which takes `__CTX__` by 
value as it would grow the stack usage.
- Therefore people might opt to use `ref __CTX__`, but that adds 
pointer indirection.

I believe context is often used in more expensive situations so 
passing by reference might be okay in many cases, for example:
- File and line contexts are used when throwing exceptions. 
Passing the file+line through a pointer indirection (`ref 
__CTX__`) isn't that expensive compared to throwing the exception 
itself.
- File and line contexts are used when logging messages. If this 
involves File I/O as it often does then the pointer indirection 
is a fraction of the total cost of the operation.

-----

Any other benefits / drawbacks, or unforeseen complications you 
can think of?
Jun 22 2023
next sibling parent reply Ernesto Castellotti <erny.castell gmail.com> writes:
On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic wrote:

 This isn't a DIP but I'd like to start this conversation to see 
 what people think.

 [...]


I don't think it's a good idea to add another special keyword, 
isn't it possible to rethink the proposal using traits?
Jun 22 2023
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 22 June 2023 at 17:49:00 UTC, Ernesto Castellotti 
wrote:

 On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic 
 wrote:

 This isn't a DIP but I'd like to start this conversation to 
 see what people think.

 [...]


 I don't think it's a good idea to add another special keyword, 
 isn't it possible to rethink the proposal using traits?


Something like `__traits(getContext)`?
Jun 24 2023
prev sibling next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
I like the idea, i have some code in my projects where i use 
__FILE__ and __LINE__, would make my functions much shorters

However, i feel like this is asking for something else

Like your example:

```D
void infoStack(string msg, __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}
```

This is redundant, you'd never pass a __CTX__ yourself, i reusing 
traits as suggested trait makes more sense

Never been a fan of uglyf-ing function arguments that one never 
gonna use themselves


```D
void infoStack(string msg) {
     enum file = __traits(context, calling_file);
     enum line = __traits(context, calling_line);
     writefln("%s(%s): %s", file, line, msg);
}
```
Jun 22 2023
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 22 June 2023 at 19:49:31 UTC, ryuukk_ wrote:

 I like the idea, i have some code in my projects where i use 
 __FILE__ and __LINE__, would make my functions much shorters

 However, i feel like this is asking for something else

 Like your example:

 ```D
 void infoStack(string msg, __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
 }
 ```

 This is redundant, you'd never pass a __CTX__ yourself, i 
 reusing traits as suggested trait makes more sense

 Never been a fan of uglyf-ing function arguments that one never 
 gonna use themselves


 ```D
 void infoStack(string msg) {
     enum file = __traits(context, calling_file);
     enum line = __traits(context, calling_line);
     writefln("%s(%s): %s", file, line, msg);
 }
 ```


There's reasons to pass a __CTX__ yourself. The classic is 
unittest functions, where you want to propagate the context of 
the user-facing callsite rather than any number of nested 
functions of test code.

Also your function has an ABI that depends on inspecting the 
function body. That's not a good idea. For one, it completely 
breaks .di files.
Jun 22 2023
parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Friday, 23 June 2023 at 04:46:47 UTC, FeepingCreature wrote:

 On Thursday, 22 June 2023 at 19:49:31 UTC, ryuukk_ wrote:

 I like the idea, i have some code in my projects where i use 
 __FILE__ and __LINE__, would make my functions much shorters

 However, i feel like this is asking for something else

 Like your example:

 ```D
 void infoStack(string msg, __CTX__ ctx = __CONTEXT__) {
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
 }
 ```

 This is redundant, you'd never pass a __CTX__ yourself, i 
 reusing traits as suggested trait makes more sense

 Never been a fan of uglyf-ing function arguments that one 
 never gonna use themselves


 ```D
 void infoStack(string msg) {
     enum file = __traits(context, calling_file);
     enum line = __traits(context, calling_line);
     writefln("%s(%s): %s", file, line, msg);
 }
 ```


 There's reasons to pass a __CTX__ yourself. The classic is 
 unittest functions, where you want to propagate the context of 
 the user-facing callsite rather than any number of nested 
 functions of test code.

 Also your function has an ABI that depends on inspecting the 
 function body. That's not a good idea. For one, it completely 
 breaks .di files.



I also value code aesthetics, bunch of extra function arguments 
uppercase with ugly _ everywhere is tasteless

I think there is value in what i propose, you can still pass 
arguments if your special case needs it, but by default it allows 
clean and nice functions
Jun 23 2023
parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
```D
void infoStack(string msg) {
     enum ctx = __traits(context);
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
}
```
Jun 23 2023
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 23 June 2023 at 14:39:36 UTC, ryuukk_ wrote:

 ```D
 void infoStack(string msg) {
     enum ctx = __traits(context);
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
 }
 ```


Suppose I put this function in a module by itself and compile it. 
What code should the compiler generate?

Because D is an AoT-compiled language, and functions may be 
called from multiple contexts, the only way for a function to 
know its calling context is for that information to be passed to 
it from the call site.
Jun 23 2023
parent ryuukk_ <ryuukk.dev gmail.com> writes:
On Friday, 23 June 2023 at 15:43:04 UTC, Paul Backus wrote:

 On Friday, 23 June 2023 at 14:39:36 UTC, ryuukk_ wrote:

 ```D
 void infoStack(string msg) {
     enum ctx = __traits(context);
     writefln("%s(%s): %s", ctx.file, ctx.line, msg);
 }
 ```


 Suppose I put this function in a module by itself and compile 
 it. What code should the compiler generate?

 Because D is an AoT-compiled language, and functions may be 
 called from multiple contexts, the only way for a function to 
 know its calling context is for that information to be passed 
 to it from the call site.


Oh that's a good point, i didn't thought of that, thanks
Jun 23 2023
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 22 June 2023 at 19:49:31 UTC, ryuukk_ wrote:

 ```D
 void infoStack(string msg) {
     enum file = __traits(context, calling_file);
     enum line = __traits(context, calling_line);
     writefln("%s(%s): %s", file, line, msg);
 }
 ```


I'm not sure how this could work with separate compilation?
Jun 24 2023
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Saturday, 24 June 2023 at 08:47:36 UTC, Andrej Mitrovic wrote:

 On Thursday, 22 June 2023 at 19:49:31 UTC, ryuukk_ wrote:

 ```D
 void infoStack(string msg) {
     enum file = __traits(context, calling_file);
     enum line = __traits(context, calling_line);
     writefln("%s(%s): %s", file, line, msg);
 }
 ```


 I'm not sure how this could work with separate compilation?


Apologies, others already responded to this and I didn't catch 
it. :)
Jun 24 2023
prev sibling next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic wrote:

 This isn't a DIP but I'd like to start this conversation to see 
 what people think.

 ...
 
 I'm envisioning a new type `__CTX__` which contains all of 
 these different contexts which are currently separate keywords 
 on: https://dlang.org/spec/expression.html#specialkeywords.

 Here it is:

 ```D
 struct __CTX__ {
     string file;
     string file_full_path;
     int line;
     string func;
     string pretty_func;
 }
 ```

 Having it defined as a struct serves a few objectives:
 - It makes it easy to declare and use this type in user code.
 - All library code will have one single compatible type they 
 can easily pass around to each other.
 - Makes it harder to confuse parameters. For example it's 
 currently easy to pass a random string to a function expecting 
 a `__FILE__`, because the parameter is a string.
 - It makes it possible to choose whether to receive and pass 
 this structure around by value or by reference.

 We also need to initialize it. So perhaps we'd call this 
 initialization keyword `__CONTEXT__`.


I love it!
Jun 22 2023
parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jun 23, 2023 at 04:48:01AM +0000, FeepingCreature via Digitalmars-d
wrote:

 On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic wrote:

 This isn't a DIP but I'd like to start this conversation to see what
 people think.
 
 ...
 
 I'm envisioning a new type `__CTX__` which contains all of these
 different contexts which are currently separate keywords on:
 https://dlang.org/spec/expression.html#specialkeywords.
 
 Here it is:
 
 ```D
 struct __CTX__ {
     string file;
     string file_full_path;
     int line;
     string func;
     string pretty_func;
 }
 ```
 
 Having it defined as a struct serves a few objectives:
 - It makes it easy to declare and use this type in user code.
 - All library code will have one single compatible type they can easily
 pass around to each other.
 - Makes it harder to confuse parameters. For example it's currently easy
 to pass a random string to a function expecting a `__FILE__`, because
 the parameter is a string.
 - It makes it possible to choose whether to receive and pass this
 structure around by value or by reference.
 
 We also need to initialize it. So perhaps we'd call this initialization
 keyword `__CONTEXT__`.


 I love it!


+1, seconded.


T

-- 
What do you call optometrist jokes? Vitreous humor.
Jun 22 2023
prev sibling next sibling parent Tim <tim.dlang t-online.de> writes:
On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic wrote:

 I'm envisioning a new type `__CTX__` which contains all of 
 these different contexts which are currently separate keywords 
 on: https://dlang.org/spec/expression.html#specialkeywords.


This could be implemented in a library after 
https://issues.dlang.org/show_bug.cgi?id=18919 is fixed. An 
implementation could then look like this:

```
struct Loc
{
    string file;
    size_t line;
}
Loc defaultLoc(string file = __FILE__, size_t line = __LINE__)
{
    return Loc(file, line);
}
void foo(Loc loc = defaultLoc)
{
    writeln(loc.file, " ", loc.line);
}
```

Currently it prints the wrong location, because `__FILE__` and 
`__LINE__` are not evaluated at the real call site, but that 
could be changed.

A library implementation would have the advantage that you could 
choose the members.
Jun 23 2023
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic wrote:

 This isn't a DIP but I'd like to start this conversation to see 
 what people think.


There's some interest in this so I'll start writing up a DIP.
Jun 24 2023
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Saturday, 24 June 2023 at 08:50:18 UTC, Andrej Mitrovic wrote:

 On Thursday, 22 June 2023 at 17:18:25 UTC, Andrej Mitrovic 
 wrote:

 This isn't a DIP but I'd like to start this conversation to 
 see what people think.


 There's some interest in this so I'll start writing up a DIP.


Be aware that DIP requests are closed right now while the process 
is being reevaluated. (Which I think is a mistake, but whatever.)
Jun 25 2023
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Sunday, 25 June 2023 at 09:35:11 UTC, FeepingCreature wrote:

 Be aware that DIP requests are closed right now while the 
 process is being reevaluated. (Which I think is a mistake, but 
 whatever.)


Oh thanks for reminding me! Found the announcement here: 
https://forum.dlang.org/post/hzzallunjtnzyspghezj forum.dlang.org
Jun 25 2023