digitalmars.D - `with` across function calls
- Nicholas Wilson (69/69) Jan 18 2019 So now that I finished moving LDC from my LLVM backend to an
- 12345swordy (6/76) Jan 18 2019 The main issue I see with this is unintentional function calls.
- Nicholas Wilson (9/14) Jan 18 2019 Well it'd do what is currently done for
- JN (14/30) Jan 18 2019 Not a big fan of such implicit stuff. I think it'd be very hard
- Nicholas Wilson (13/26) Jan 18 2019 Thats what a debugger is for, it would behave just like any other
- Simen =?UTF-8?B?S2rDpnLDpXM=?= (38/41) Jan 18 2019 How does this work with nested calls? e.g.
- Nicholas Wilson (3/14) Jan 18 2019 Correct.
- Steven Schveighoffer (7/91) Jan 18 2019 All you seem to be looking for is a context with specified default
- Nicholas Wilson (10/27) Jan 18 2019 So `q` is a struct with a number of methods, calls to its other
- luckoverthere (17/23) Jan 18 2019 Isn't that the usual argument between using globals and not using
- Nicholas Wilson (21/47) Jan 18 2019 How do you know? I don't think it would be all that much more
- luckoverthere (21/68) Jan 18 2019 Default arguments don't change. In every context you look the
- Nicholas Wilson (25/48) Jan 18 2019 Yes they can: https://run.dlang.io/is/IEQb0O
- luckoverthere (46/97) Jan 19 2019 That's pretty awful too, let's not propagate that any further
- Nicholas Wilson (30/46) Jan 19 2019 Globals are bad, yes, conceptually every function that takes a
So now that I finished moving LDC from my LLVM backend to an externally maintained "backend" I was thinking about how I could improve the design of the API. I was not very happy with the use of globals which basically follow the pattern: struct Global { void* handle; } Global g; void usercode() { g = ...; Foo foo; foo.foo(); bar(); Bar.baz(); } Here foo bar and baz call functions that somewhere down the call need to use `g` at some point. The value of `g.handle` is not going to be change by the library code, but it can't be immutable or const because handle is passes to other functions and it needs to be assignable by the user when they need to initialise it. I was hoping to be able to change that to something like // note no global void usercode() { auto g = ...; with (g) { Foo foo; foo.foo(); bar(); Baz.baz(); } } but then I realised that I can't pass that implicitly down the call stack even if I change foo, bar and baz. I was reminded of Martin Odersky's DConf Keynote and wondered if implicit parameters could be used to do something like: void bar(with Global g) { } or struct Bar { void baz(Global g = with) { } } such that void usercode() { { auto g = ...; with (g) // or with g: { bar(); // calls: bar(g); } } // or { with g = ...; Baz.baz(); // calls: Baz.baz(g); } } Obviously this requires a DIP, but what you do think of it? The example above is a bit simplified, the call that I'm trying to not pass `g`directly to looks like q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.
Jan 18 2019
On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:So now that I finished moving LDC from my LLVM backend to an externally maintained "backend" I was thinking about how I could improve the design of the API. I was not very happy with the use of globals which basically follow the pattern: struct Global { void* handle; } Global g; void usercode() { g = ...; Foo foo; foo.foo(); bar(); Bar.baz(); } Here foo bar and baz call functions that somewhere down the call need to use `g` at some point. The value of `g.handle` is not going to be change by the library code, but it can't be immutable or const because handle is passes to other functions and it needs to be assignable by the user when they need to initialise it. I was hoping to be able to change that to something like // note no global void usercode() { auto g = ...; with (g) { Foo foo; foo.foo(); bar(); Baz.baz(); } } but then I realised that I can't pass that implicitly down the call stack even if I change foo, bar and baz. I was reminded of Martin Odersky's DConf Keynote and wondered if implicit parameters could be used to do something like: void bar(with Global g) { } or struct Bar { void baz(Global g = with) { } } such that void usercode() { { auto g = ...; with (g) // or with g: { bar(); // calls: bar(g); } } // or { with g = ...; Baz.baz(); // calls: Baz.baz(g); } } Obviously this requires a DIP, but what you do think of it? The example above is a bit simplified, the call that I'm trying to not pass `g`directly to looks like q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.The main issue I see with this is unintentional function calls. with (g) { bar(); //Meant to call bar() but end up calling bar(g) }
Jan 18 2019
On Friday, 18 January 2019 at 13:51:53 UTC, 12345swordy wrote:The main issue I see with this is unintentional function calls. with (g) { bar(); //Meant to call bar() but end up calling bar(g) }Well it'd do what is currently done for import std.stdio; void bar() { writeln("()");} void bar(int i = 0) { writeln("(int)");} void main() { bar(); // () }
Jan 18 2019
On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:void usercode() { { auto g = ...; with (g) // or with g: { bar(); // calls: bar(g); } } // or { with g = ...; Baz.baz(); // calls: Baz.baz(g); } } Obviously this requires a DIP, but what you do think of it?Not a big fan of such implicit stuff. I think it'd be very hard to debug if something goes wrong. Your usecase reminds me a bit of Dependency Injection in OOP world, or even Service Locator pattern. With DI frameworks, you'd declare a function/class to require a "g", but you don't have to pass it to every call. Instead, you obtain your "g" from a global provider class. Is 'with' commonly used in D? I don't think I've ever seen it used in any sourcebase. I think it's main usecase is to initialize structs. I kind of like the .. pattern that Dart has: Foo f = new Foo() ..x = 10 // equivalent to f.x = 10 ..y = 20 // equivalent to f.y = 20
Jan 18 2019
On Friday, 18 January 2019 at 14:23:48 UTC, JN wrote:Not a big fan of such implicit stuff. I think it'd be very hard to debug if something goes wrong.Thats what a debugger is for, it would behave just like any other parameter. The debug workflow wouldn't really change, not anymore than of using default parameters. Think of it like a context aware default parameter.Your usecase reminds me a bit of Dependency Injection in OOP world, or even Service Locator pattern. With DI frameworks, you'd declare a function/class to require a "g", but you don't have to pass it to every call. Instead, you obtain your "g" from a global provider class.I'm trying to avoid globals and this is a structs wrapping opaque handles to OOP objects code, I don't deal in classes at all.Is 'with' commonly used in D? I don't think I've ever seen it used in any sourcebase. I think it's main usecase is to initialize structs. I kind of like the .. pattern that Dart has: Foo f = new Foo() ..x = 10 // equivalent to f.x = 10 ..y = 20 // equivalent to f.y = 20It used pretty frequently (for some value of frequently) for switching on enums to avoid retyping the prefix. mir-glas uses it for register-blocking https://github.com/libmir/mir-glas/blob/fd9adb0750c23db4c3948f79b63384b8082f3601/source/glas/internal/symm.d#L173 and more general for transparent access to config-type structs that need to be short lived (i.e. have a dtor that needs running).
Jan 18 2019
On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:void bar(with Global g) { }How does this work with nested calls? e.g. struct Global { int n; } void foo() { auto g = Global(0); with (g) { bar(); } } void bar(with Global g) { assert(g.n == 0); baz(); assert(g.n == 0); // g is not overwritten by baz's new g } void baz() { auto g = Global(1); with (g) { qux(); } } void qux(with Global g) { assert(g.n == 1); // use the nested g } If my intuition is correct, the above should compile with no asserts triggered. I further expect that attempting to call bar() or qux() outside a with(g) will fail: unittest { bar(); // Error: calling bar requires Global g in surrounding with() scope. Global g; bar(); // Error: Global g found in surrounding scope, but not used in with(). } -- Simen
Jan 18 2019
On Friday, 18 January 2019 at 14:29:59 UTC, Simen Kjærås wrote:If my intuition is correct, the above should compile with no asserts triggered.Yes.I further expect that attempting to call bar() or qux() outside a with(g) will fail: unittest { bar(); // Error: calling bar requires Global g in surrounding with() scope. Global g; bar(); // Error: Global g found in surrounding scope, but not used in with(). }Correct.
Jan 18 2019
On 1/18/19 7:35 AM, Nicholas Wilson wrote:So now that I finished moving LDC from my LLVM backend to an externally maintained "backend" I was thinking about how I could improve the design of the API. I was not very happy with the use of globals which basically follow the pattern: struct Global { void* handle; } Global g; void usercode() { g = ...; Foo foo; foo.foo(); bar(); Bar.baz(); } Here foo bar and baz call functions that somewhere down the call need to use `g` at some point. The value of `g.handle` is not going to be change by the library code, but it can't be immutable or const because handle is passes to other functions and it needs to be assignable by the user when they need to initialise it. I was hoping to be able to change that to something like // note no global void usercode() { auto g = ...; with (g) { Foo foo; foo.foo(); bar(); Baz.baz(); } } but then I realised that I can't pass that implicitly down the call stack even if I change foo, bar and baz. I was reminded of Martin Odersky's DConf Keynote and wondered if implicit parameters could be used to do something like: void bar(with Global g) { } or struct Bar { void baz(Global g = with) { } } such that void usercode() { { auto g = ...; with (g) // or with g: { bar(); // calls: bar(g); } } // or { with g = ...; Baz.baz(); // calls: Baz.baz(g); } } Obviously this requires a DIP, but what you do think of it? The example above is a bit simplified, the call that I'm trying to not pass `g`directly to looks like q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.All you seem to be looking for is a context with specified default parameters. Why not make a struct? auto context = With!g; context.bar(); With opDispatch and introspection, this should be doable. -Steve
Jan 18 2019
On Friday, 18 January 2019 at 14:51:35 UTC, Steven Schveighoffer wrote:On 1/18/19 7:35 AM, Nicholas Wilson wrote:So `q` is a struct with a number of methods, calls to its other methods (which won't need `g`) will be interspersed between calls that need `g`. I can't add `g` to `q`, because that breaks the logical objects, they are wrappers of opaque classes, and `g` has nothing to do with `q` except that the call `q.enqueue!(f!(args))(arg_set1)(arg_set2);` needs a `g` to create a object based on f and arg_set2. I'm not sure that would improve the legibility of the code.The example above is a bit simplified, the call that I'm trying to not pass `g`directly to looks like q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.All you seem to be looking for is a context with specified default parameters. Why not make a struct? auto context = With!g; context.bar(); With opDispatch and introspection, this should be doable. -Steve
Jan 18 2019
On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.Isn't that the usual argument between using globals and not using them. It's a lot cleaner when using them, especially when it is some sort of data that needs to be passed to basically everything. But then you deal with the joys of globals. The use case for this is very narrow and the implementation is error prone. Having to go through and check functions to see which one's have altered behavior in a with statement isn't going to be fun. In the general case this doesn't make any sense, eg I don't see this being used anywhere in phobos at all. It'd be the very specific case of an API that needs to pass around some sort of state-like object a bunch of functions. The rationale is also pretty weak, you want reduce the number of arguments you have to pass in this one specific use case. Honestly from the looks of that function call, it might be better just finding a better way of implementing whatever it is you are trying to implement.
Jan 18 2019
On Friday, 18 January 2019 at 23:10:12 UTC, luckoverthere wrote:On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:How do you know? I don't think it would be all that much more complex than default arguments.q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.Isn't that the usual argument between using globals and not using them. It's a lot cleaner when using them, especially when it is some sort of data that needs to be passed to basically everything. But then you deal with the joys of globals. The use case for this is very narrow and the implementation is error prone.Having to go through and check functions to see which one's have altered behavior in a with statement isn't going to be fun.grep? ^f? Also again, this isn't any worse than default arguments.In the general case this doesn't make any sense,That is a bold claim: have a watch of Martin's keynote.eg I don't see this being used anywhere in phobos at all.Of course not, phobos doesn't deal in contexts (except maybe std.concurrency / std.parallelism, IDK, I haven't looked at their implementation). That doesn't mean that lots of other code doesn't.It'd be the very specific case of an API that needs to pass around some sort of state-like object a bunch of functions. The rationale is also pretty weak, you want reduce the number of arguments you have to pass in this one specific use case.The case of dealing with contexts is hardly unique to me.Honestly from the looks of that function call, it might be better just finding a better way of implementing whatever it is you are trying to implement.Its globals or this, and I'd rather not use globals. More specifically, in q.enqueue!(f!(args))(arg_set1)(arg_set2); arg_set1 is the shape of the kernel dispatch and (if `q` is an out of order Queue) the dependency list of enqueue and other things that need to finish before the (user supplied) kernel `f` executes. arg_set2 relates to Parameters!f there is no logical room to put it, it belongs in neither of the two argument lists.
Jan 18 2019
On Saturday, 19 January 2019 at 00:23:40 UTC, Nicholas Wilson wrote:On Friday, 18 January 2019 at 23:10:12 UTC, luckoverthere wrote:Default arguments don't change. In every context you look the function will behave the same way no matter where it is used. It'd be no different than adding an overload. With the with statement though, it'd not obvious where it might be used.On Friday, 18 January 2019 at 12:35:33 UTC, Nicholas Wilson wrote:How do you know? I don't think it would be all that much more complex than default arguments.q.enqueue!(f!(args))(arg_set1)(arg_set2); and the function within that needs `g` also needs `f`. The expected usage is a whole lot of those calls all in one spot with different `f`, `args`, `arg_set1`, `arg_set2` I really don't want the user to have to repeat themselves anymore than absolutely necessary.Isn't that the usual argument between using globals and not using them. It's a lot cleaner when using them, especially when it is some sort of data that needs to be passed to basically everything. But then you deal with the joys of globals. The use case for this is very narrow and the implementation is error prone.You can't grep or find because the with statement removes it all entirely. That's sort of my point, you'd have to look at every function definition to see which ones get modified. At the call point you have no indication that the behavior is different. It's not the same as default arguments. You can tell at call point the number of parameters being passed to the function, and the behavior is the same everywhere that function is used. The two are absolutely different. The behavior of your suggested feature changes based on scope and context.Having to go through and check functions to see which one's have altered behavior in a with statement isn't going to be fun.grep? ^f? Also again, this isn't any worse than default arguments.Why did you split my sentence in half? You agree with me in the reply to the example given with this sentence.In the general case this doesn't make any sense,That is a bold claim: have a watch of Martin's keynote.Contexts is the specific use case, not you.It'd be the very specific case of an API that needs to pass around some sort of state-like object a bunch of functions. The rationale is also pretty weak, you want reduce the number of arguments you have to pass in this one specific use case.The case of dealing with contexts is hardly unique to me.Even if you were to use a global or this new proposed "with" feature, that function call is atrocious. That is what I meant by finding a better way of implementing what you are trying to do.Honestly from the looks of that function call, it might be better just finding a better way of implementing whatever it is you are trying to implement.Its globals or this, and I'd rather not use globals. More specifically, in q.enqueue!(f!(args))(arg_set1)(arg_set2); arg_set1 is the shape of the kernel dispatch and (if `q` is an out of order Queue) the dependency list of enqueue and other things that need to finish before the (user supplied) kernel `f` executes. arg_set2 relates to Parameters!f there is no logical room to put it, it belongs in neither of the two argument lists.
Jan 18 2019
On Saturday, 19 January 2019 at 00:51:28 UTC, luckoverthere wrote:On Saturday, 19 January 2019 at 00:23:40 UTC, Nicholas Wilson wrote:Yes they can: https://run.dlang.io/is/IEQb0O This would be like that, except: a) without the global, and b) you specifying that you want to use `a` where a implicit int parameter is expected.How do you know? I don't think it would be all that much more complex than default arguments.Default arguments don't change. In every context you look the function will behave the same way no matter where it is used. It'd be no different than adding an overload. With the with statement though, it'd not obvious where it might be used.You can't grep or find because the with statement removes it all entirely.Uh, yes you can: "grep with", it will show up in the function signature _and_ in the scope of the call site. Its like being able to grep for "cast" (which is literally the entire reason for having cast as a keyword).That's sort of my point, you'd have to look at every function definition to see which ones get modified. At the call point you have no indication that the behavior is different.Again this is not different to default parameters, also you have a `with (foo)` in scope.It's not the same as default arguments. You can tell at call point the number of parameters being passed to the function, and the behavior is the same everywhere that function is used. The two are absolutely different.What? Firstly with default parameters you _can't_ tell how many parameters are actually passed to the function at the call site. Secondly, this is _exactly_ the same as default parameters. But also like default parameters, you typically don't care because they are default for a reason i.e. the defaults make sense.Why did you split my sentence in half? You agree with me in the reply to the example given with this sentence.Because there were two halves to the sentence.Contexts are the general use case, yes. I'm not sure what you're point is there.The case of dealing with contexts is hardly unique to me.Contexts is the specific use case, not you.Even if you were to use a global or this new proposed "with" feature, that function call is atrocious.Welcome to the wonderful world of compute API, and believe me, this is nice. That single call in OpenCL is 4 + O(arg_set2) calls, CUDA its about 4 and they are all horrible and involve no end of casts to void* and is as type unsafe as its is possible be. The fact that it is a single call and type safe speaks wonders for D's meta programming.
Jan 18 2019
On Saturday, 19 January 2019 at 01:35:41 UTC, Nicholas Wilson wrote:On Saturday, 19 January 2019 at 00:51:28 UTC, luckoverthere wrote:That's pretty awful too, let's not propagate that any further with with. But not really what I mean, at the very least with that default value you know that function access a global for some reason, and should probably be avoided.On Saturday, 19 January 2019 at 00:23:40 UTC, Nicholas Wilson wrote:Yes they can: https://run.dlang.io/is/IEQb0O This would be like that, except: a) without the global, and b) you specifying that you want to use `a` where a implicit int parameter is expected.How do you know? I don't think it would be all that much more complex than default arguments.Default arguments don't change. In every context you look the function will behave the same way no matter where it is used. It'd be no different than adding an overload. With the with statement though, it'd not obvious where it might be used.Lol I can only imagine this, does anyone actually even grep. I wonder just how many hits doing grep __traits does with something like phobos. This is by no means scalable at all. When reading code no one wants to have to grep around the entire code base because someone was lazy.You can't grep or find because the with statement removes it all entirely.Uh, yes you can: "grep with", it will show up in the function signature _and_ in the scope of the call site. Its like being able to grep for "cast" (which is literally the entire reason for having cast as a keyword).Yes you have with in scope, but that's it. You'd have to look through every function in that with scope to see which functions are affected by it. In the event something is wrong.That's sort of my point, you'd have to look at every function definition to see which ones get modified. At the call point you have no indication that the behavior is different.Again this is not different to default parameters, also you have a `with (foo)` in scope.It is NOT _exactly_ the same, irregardless of where you use a function with default values they are going to behavior the same no matter where they. void test( int a, int b = 10 ); test( 10 ); // same behavior *everywhere* test( 10, 10 ); // same behavior *everywhere* with( codeSmell ) { test( 10 ); // same behavior *everywhere* test( 10, 10 ); // same behavior *everywhere* } static this() { test( 10 ); // same behavior *everywhere* test( 10, 10 ); // same behavior *everywhere* } Where as with your proposed DIP. void test( int a = with ); test(); // nope can't do t his int value; with( value ) { test(); // ok different behavior given context, unexpected } This is honestly why I don't use with() basically at all. And neither does anyone else really. It just makes logic harder to reason about.It's not the same as default arguments. You can tell at call point the number of parameters being passed to the function, and the behavior is the same everywhere that function is used. The two are absolutely different.What? Firstly with default parameters you _can't_ tell how many parameters are actually passed to the function at the call site. Secondly, this is _exactly_ the same as default parameters. But also like default parameters, you typically don't care because they are default for a reason i.e. the defaults make sense.And you misunderstood both halves because you didn't treat them as a whole :).Why did you split my sentence in half? You agree with me in the reply to the example given with this sentence.Because there were two halves to the sentence.They are a specific use case yes.Contexts are the general use case, yes. I'm not sure what you're point is there.The case of dealing with contexts is hardly unique to me.Contexts is the specific use case, not you.Yikes, I wouldn't be celebrating making that a single call. Especially since you are effectively passing 3 different sets of arguments in one line of code. That's not something to be celebrating. Either way good luck getting this DIP passed, you'll need it.Even if you were to use a global or this new proposed "with" feature, that function call is atrocious.Welcome to the wonderful world of compute API, and believe me, this is nice. That single call in OpenCL is 4 + O(arg_set2) calls, CUDA its about 4 and they are all horrible and involve no end of casts to void* and is as type unsafe as its is possible be. The fact that it is a single call and type safe speaks wonders for D's meta programming.
Jan 19 2019
On Saturday, 19 January 2019 at 22:55:49 UTC, luckoverthere wrote:That's pretty awful too, let's not propagate that any further with with. But not really what I mean, at the very least with that default value you know that function access a global for some reason, and should probably be avoided.Globals are bad, yes, conceptually every function that takes a global does that except you can't ever specify something different in place of the global. This is trying to get rid of them.Lol I can only imagine this, does anyone actually even grep.Thats not the point. If it works for grep, it will work for ^f, it will work for an intelligent editor etc.Yes you have with in scope, but that's it. You'd have to look through every function in that with scope to see which functions are affected by it. In the event something is wrong.And the situation is _far worse_ with a global. Thats why they are considered to be so bad. At least in this case there is one specific type you are interested, also `-vcg-ast` will tell you exactly what the code does. So will debug info.It is NOT _exactly_ the same,That was w.r.t. the mechanism (which it is), not the effect on readability, which, while not perfect by any means, is far better than using globals, which is the only other way to do it.This is honestly why I don't use with() basically at all. And neither does anyone else really. It just makes logic harder to reason about.Again the baseline for comparison is globals, which are worse.Yikes, I wouldn't be celebrating making that a single call. Especially since you are effectively passing 3 different sets of arguments in one line of code. That's not something to be celebrating.For reference, CUDA does this as: kernel<templateargs><<<shape,args, queue>>>(parameters, of, kernel); and is IMO the sole reason it is soooo much more popular than OpenCL (and therefore a significant part of the reason Nvidia is valued so much higher). I modelled the DCompute call off that. Both are completely type safe, but CUDA does the magic behind the scenes in the compiler whereas I do it in the library. Now not all kernels are templated, so most of the time that will be just two. The reason to not go yikes at it is because that is a single logical operation (launch me this function with these configuration parameters on this queue, and these function parameters). The fact that it is a while load of calls is irrelevant to the user: what they want is to launch a kernel.
Jan 19 2019