www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - implicit-context v0.0.1

reply Guillaume Piolat <first.name gmail.com> writes:
Hi,

Ever had a bit of feature-envy about Odin's "context" feature 
[1]? It is something used to pass "contextual" parameters, like a 
logger, an allocator, to callees. It is akin to Scala's "implicit 
parameters", or Jai contexts [2].


So I went ahead and implement a proof-of-concept library to have 
scope globals in D with a TLS-based stack of environments. I went 
ahead and implement a minimal logger, allocator, and "user 
pointer" API on top of that context system.

It has a worse API and usability than a language solution but 
basically I think there is no big blocker, if TLS and C runtime 
are available to you.

DUB: https://code.dlang.org/packages/implicit-context

`implicit-context` is currently limited and 
suggestions/requests/forks/destroying are much welcome.

As I see it, without compiler support the pluses are:
- no hidden "context" parameter in function calls, no new ABI
- contextual parameters are a bit like "scoped globals", they 
will not change that often.

and the minuses are:
- manual push/pop
- wordy, opDispatch getter doesn't seem possible?


[1] https://odin-lang.org/docs/overview/#implicit-context-system
[2] 
https://medium.com/ christoffer_99666/a-little-context-d06dfdec79a3
Sep 28 2023
next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 28 September 2023 at 23:28:02 UTC, Guillaume Piolat 
wrote:

 Hi,

 Ever had a bit of feature-envy about Odin's "context" feature 
 [1]? It is something used to pass "contextual" parameters, like 
 a logger, an allocator, to callees. It is akin to Scala's 
 "implicit parameters", or Jai contexts [2].

 [...]


Interesting, what are the benefits of using this instead of 
global variables?
Sep 29 2023
next sibling parent reply Guillaume Piolat <first.name gmail.com> writes:
On Friday, 29 September 2023 at 08:33:56 UTC, Imperatorn wrote:

 Interesting, what are the benefits of using this instead of 
 global variables?


Thinking about this, it's more vs TLS variable. __gshared would 
require synchronization.

Changing the theAllocator (a TLS variable) in 
std.experimental.allocator looks like this:


         auto save = theAllocator;
         theAllocator = myAllocator;

         // do stuff with custom allocator

         theAllocator = save;



Changing the allocator in implicit-context looks like this


         context.push;
         context.allocator = myAlloc;

         // do stuff with custom allocator

         context.pop;


so now that I think about it I'm not sure if there is an 
substantial advantage over simply having TLS variables. I had the 
goal of allowing .alloca on that secondary stack. If there is 
many context variables, push and pop will be a bit faster to 
write than all the temporaries.
Sep 29 2023
next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 29 September 2023 at 11:00:05 UTC, Guillaume Piolat 
wrote:

 On Friday, 29 September 2023 at 08:33:56 UTC, Imperatorn wrote:

 Interesting, what are the benefits of using this instead of 
 global variables?


 Thinking about this, it's more vs TLS variable. __gshared would 
 require synchronization.

 Changing the theAllocator (a TLS variable) in 
 std.experimental.allocator looks like this:


         auto save = theAllocator;
         theAllocator = myAllocator;

         // do stuff with custom allocator

         theAllocator = save;



 Changing the allocator in implicit-context looks like this


         context.push;
         context.allocator = myAlloc;

         // do stuff with custom allocator

         context.pop;


 so now that I think about it I'm not sure if there is an 
 substantial advantage over simply having TLS variables. I had 
 the goal of allowing .alloca on that secondary stack. If there 
 is many context variables, push and pop will be a bit faster to 
 write than all the temporaries.


I understand, it's more like if you mix optional parameters and 
dependency injection?

I think for this to be truly valuable, it would require being 
part of the language.

I admit I haven't really thought about implicit parameters before 
your post, so I might be missing something.
Sep 29 2023
parent reply Guillaume Piolat <first.name gmail.com> writes:
On Friday, 29 September 2023 at 15:00:33 UTC, Imperatorn wrote:

 I think for this to be truly valuable, it would require being 
 part of the language.


Only if proven on DUB.


 I admit I haven't really thought about implicit parameters 
 before your post, so I might be missing something.


Think of it like envvars for threads. When you launch a process, 
the launcher knows to copy the environment variables. With 
scattered TLS variables, no new thread can get a copy of all the 
"context" it may have. But with a centralized place for context, 
you will be able to do that (not implemented yet), which kinda 
improves encapsulation.


**Example 1: Context variables are scoped**

   In a UI library, every new widget typically get a "UI context" 
(that, or factory functions only) polluting all the constructors 
there is.

Solution: Just append "uiContext" variable in the context => less 
parameters.

But it could be a TLS variable, right? Yes indeed, but then you 
may want to scope that, to remove the global from being accessed 
outside a particular scope. Globals can be accessed at any time, 
which doesn't improve a public API.


**Example 2**

When my "gfm" package was forked to "gfm7", the first thing that 
was done is:
- remove all the _gl members that would only there to confirm a 
particular shared library was used => it is part of the context
- and removed all logger interface passing => also ugly and kinda 
never changes
Sep 29 2023
next sibling parent drug007 <drug2004 bk.ru> writes:
On 29.09.2023 18:30, Guillaume Piolat wrote:

 On Friday, 29 September 2023 at 15:00:33 UTC, Imperatorn wrote:

 I think for this to be truly valuable, it would require being part of 
 the language.


 
 Only if proven on DUB.
 

 I admit I haven't really thought about implicit parameters before your 
 post, so I might be missing something.


 
 Think of it like envvars for threads. When you launch a process, the 
 launcher knows to copy the environment variables. With scattered TLS 
 variables, no new thread can get a copy of all the "context" it may 
 have. But with a centralized place for context, you will be able to do 
 that (not implemented yet), which kinda improves encapsulation.
 
 
 **Example 1: Context variables are scoped**
 
    In a UI library, every new widget typically get a "UI context" (that, 
 or factory functions only) polluting all the constructors there is.
 
 Solution: Just append "uiContext" variable in the context => less 
 parameters.
 
 But it could be a TLS variable, right? Yes indeed, but then you may want 
 to scope that, to remove the global from being accessed outside a 
 particular scope. Globals can be accessed at any time, which doesn't 
 improve a public API.
 
 
 **Example 2**
 
 When my "gfm" package was forked to "gfm7", the first thing that was 
 done is:
 - remove all the _gl members that would only there to confirm a 
 particular shared library was used => it is part of the context
 - and removed all logger interface passing => also ugly and kinda never 
 changes
 
 


But later I understand that the removing of gl has also disadvantages. 
In general implicit context is really the great idea
Sep 29 2023
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 29 September 2023 at 15:30:30 UTC, Guillaume Piolat 
wrote:

 On Friday, 29 September 2023 at 15:00:33 UTC, Imperatorn wrote:

 [...]


 Only if proven on DUB.


 [...]


 Think of it like envvars for threads. When you launch a 
 process, the launcher knows to copy the environment variables. 
 With scattered TLS variables, no new thread can get a copy of 
 all the "context" it may have. But with a centralized place for 
 context, you will be able to do that (not implemented yet), 
 which kinda improves encapsulation.

 [...]


Sounds a bit like dependency injection but for state
Sep 29 2023
parent reply Guillaume Piolat <first.name gmail.com> writes:
On Friday, 29 September 2023 at 16:56:47 UTC, Imperatorn wrote:

 Sounds a bit like dependency injection but for state


Possibly, I'm not familiar with dependency injection.
When is it useful?
Sep 30 2023
next sibling parent evilrat <evilrat666 gmail.com> writes:
On Saturday, 30 September 2023 at 12:40:29 UTC, Guillaume Piolat 
wrote:

 On Friday, 29 September 2023 at 16:56:47 UTC, Imperatorn wrote:

 Sounds a bit like dependency injection but for state


 Possibly, I'm not familiar with dependency injection.
 When is it useful?


Dependency injection is a principle of making your 
classes/functions self-contained and isolated, it means that when 
your code might need to create a resource (such as open a file to 
write data) it is instead up to the caller to provide that 
resource, but your code never does that by itself because a 
library can't possibly know the environment and restrictions of 
the target system/machine.

Simply put, you can't possibly know how to open a file in that 
system, you can't possibly know what allocator is used in the 
caller environment (think about very low-level or bare metal 
program), and so on, so instead caller must provide everything 
that your function/method/class might need to do the work.

In the most complex situations where the entire program graph is 
about to be created in the main function there is so called DI 
containers that configures all this stuff in one central place.
Sep 30 2023
prev sibling next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 30 September 2023 at 12:40:29 UTC, Guillaume Piolat 
wrote:

 On Friday, 29 September 2023 at 16:56:47 UTC, Imperatorn wrote:

 Sounds a bit like dependency injection but for state


 Possibly, I'm not familiar with dependency injection.
 When is it useful?


When you want to register a bunch of objects  and then just use 
it from various places, you just state in your ctor that you want 
to use it and it will be provided by the framework.
Sep 30 2023
prev sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Saturday, 30 September 2023 at 12:40:29 UTC, Guillaume Piolat 
wrote:


 When is it useful?


You can use it to troll Jonathan Blow.
Sep 30 2023
parent Guillaume Piolat <first.name gmail.com> writes:
On Saturday, 30 September 2023 at 15:02:16 UTC, Max Samukha wrote:

 When is it useful?


 You can use it to troll Jonathan Blow.


OT: Apart from being marketed more like a game (streaming videos, 
and similarly "finished" at launch?) I was striked that Jai has 
already many... perlisms in the syntax.

As for the other features that makes me pause, it's SoA.
If you have SoA types you might also need SoA slices, for example 
in Odin:
https://odin-lang.org/docs/overview/#soa-data-types

Is this composable?
Oct 03 2023
prev sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 29 September 2023 at 11:00:05 UTC, Guillaume Piolat 
wrote:

 On Friday, 29 September 2023 at 08:33:56 UTC, Imperatorn wrote:

 [...]


 Thinking about this, it's more vs TLS variable. __gshared would 
 require synchronization.

 [...]


Do you have an example of how this would be used in practice with 
allocators?
Oct 10 2023
prev sibling parent reply Antonio <antoniocabreraperez gmail.com> writes:
On Friday, 29 September 2023 at 08:33:56 UTC, Imperatorn wrote:

 On Thursday, 28 September 2023 at 23:28:02 UTC, Guillaume 
 Piolat wrote:

 Hi,

 Ever had a bit of feature-envy about Odin's "context" feature 
 [1]? It is something used to pass "contextual" parameters, 
 like a logger, an allocator, to callees. It is akin to Scala's 
 "implicit parameters", or Jai contexts [2].

 [...]


 Interesting, what are the benefits of using this instead of 
 global variables?


Context is dynamically generated/destroyed.  I developed this 

years ago)... I found out later something similar with AOP 
(Aspects Oriented Programming) when working with Spring in java

Lets see an example

```d
     long create(PersonDto person) =>
       withTransaction( (auto cnx){
         // Perform person creation stuff
         long personId = cnx.execute(
            "insert into people ... returning id",
            [...]
         ).first!long("id");
         return personId;
       });

     long create(CustomerDto customer) =>
       withTransaction( (auto cnx){
          long personId = create( customer.person );
          long customerId = cnx.execute(
            "insert into customers ... returning id",
            [personId, ... ]
          ).first!long("id");
          return customerId;
       });

     void main(){
       withContext((){
         CustomerDto customer = { code:"P001", 
person:{name:"Peter", nif:"3442543211F"}};
         long customerId = create( customer );
       })
     }
```

The "withTransaction" function, iternally, asks the context if 
there is an opened transaction.

* If not found:
     *  It creates one and registers it into the context.
     *  calls the delegate
     *  commits the transaction and removes it from the context
     *  returns the delegate result.
     *  If an exception is thrown by the delegate, then the 
transaction is rollbacked instead commited and the exception is 
passed through to the caller.

* If found:
     * Calls the delegate transparently and returns it's result


This use case of "implicit-context" works naturally in a "per 
thread context".


Stackability is nice: (this example is not so real, but a "how 
to" example):

```d
     void createPersonAction() =>
       withHttpResponse( res =>
       withAuthentifiedUser( user =>
       withHttpBody!Person( person =>
       withLogger("createPersonAction", (logger) {
         logger.info("Something to be logged");
         auto id = withTransaction( cnx => cnx.execute(...) );
         res.send(id) )
       })))));
```
It shoud be more natural this way ...

```d
     void createPersonAction() =>
       with( auto res = implicitHttpResponse())
       with( auto user = implicitAuthentifiedUser())
       with( auto person = implicitHttpBody!Person())
       with( auto logger = implicitLogger("createPersonAction") )
       {
         logger.info("Something to be logged");
         with( auto cnx = implicitTransaction() )
         {
            auto id =  cnx.execute(...);
            res.send(id); // Bad place... there is an oppened 
transaction here!!!
         }
       };
```

... but remember than we need to manage "exceptions" dependant 
behaviours implicitly:  **with(** is not an option for AOP.


As you can see, this is not an "Object oriented dependency 
injection"... Each "withX" internally interacts with the context 
to find or create the resource and, additionally,  performs some 
functional extra proccessing (before, after and exception).

i.e.: withHttpResponse:
* if res.send is called: this is the data to be serialized as a 
result (status 202)
* if res.send is not called, then "404 not found" will be 
generated when delegate ends.
* if an exception is raised by the delegate, it will be 
transformed in an "standard" http error

As a ramarkable benefit:  it is really simple to wrap with 
mockups when testing

Problems?

* It is "runtime" generated/consumed without compilation time 
verification (i.e.: you can call createPersonAction without an 
HttpRequest in the context )... but this is a dependency 
injection assumed problem.

* You are in risk to move to "implicit context" too many things 
(remember that functions have parameters :-) )

It was only a possible use of "implicit context" :-)


Best regards
Oct 05 2023
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 5 October 2023 at 22:38:35 UTC, Antonio wrote:

 On Friday, 29 September 2023 at 08:33:56 UTC, Imperatorn wrote:

 [...]


 Context is dynamically generated/destroyed.  I developed this 

 years ago)... I found out later something similar with AOP 
 (Aspects Oriented Programming) when working with Spring in java

 [...]


Oh, I remember now.

Long time since I heard anyone speak about AOP. But I think it 
was a valid idea.
Oct 06 2023
prev sibling parent reply MrSmith33 <mrsmith33 yandex.ru> writes:
On Thursday, 28 September 2023 at 23:28:02 UTC, Guillaume Piolat 
wrote:

 - manual push/pop


I wonder if `with` statement is helpful here to reduce verbosity
Oct 02 2023
parent Guillaume Piolat <first.name gmail.com> writes:
On Monday, 2 October 2023 at 19:04:19 UTC, MrSmith33 wrote:

 On Thursday, 28 September 2023 at 23:28:02 UTC, Guillaume 
 Piolat wrote:

 - manual push/pop


 I wonder if `with` statement is helpful here to reduce verbosity


Do you mean with:

     with(scopedContext())
     {
         set!int("myVar", 5);
     }

     ScopedContext scopedContext() { /* blah */ }

     struct ScopedContext
     {
         ~this() { context.pop; }
     }


Because at a point there was such a RAII context wrapper. I 
didn't realize, `with` extends its lifetime to the scope?
Oct 03 2023