www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to provide this arg or functor for algorithm?

reply "FreeSlave" <freeslave93 gmail.com> writes:
Let's say I want to map some range using some context.
The obvious way is to do:

uint[3] arr = [1,2,3];
uint context = 2;
auto r = arr[].map!(delegate(value) { return value * context; });

The problem is that this allocates delegate, so it can't be used 
in  nogc code.
What I want to do might look like this:

static struct Caller
{
     this(uint context)  nogc {
         _context = context;
     }
     auto opCall(uint value)  nogc {
         return value * _context;
     }
     uint _context;
}

auto caller = Caller(2);
auto r = arr[].map!(&caller.opCall);

But it will not work of course since function must be a 
compile-time parameter.

So the way to go would be:

auto caller = Caller(2);
auto r = arr[].map!(Caller.opCall)(&caller);

But map and other algorithms don't support this interface.

The other way is

auto r = arr.map!(Caller(2));

But again, since it's template parameter, it can't use variables 
unknown at compile time:

uint context = ...;
auto r = arr.map!(Caller(context)); //will not work

So what's the solution? Of course besides rewriting the whole 
std.algorithm.
Aug 16 2015
next sibling parent reply "cym13" <cpicard openmailbox.org> writes:
On Sunday, 16 August 2015 at 11:53:42 UTC, FreeSlave wrote:
 Let's say I want to map some range using some context.
 The obvious way is to do:

 uint[3] arr = [1,2,3];
 uint context = 2;
 auto r = arr[].map!(delegate(value) { return value * context; 
 });
To me the obvious way is to use a lambda, not a delegate: uint[3] arr = [1,2,3]; uint context = 2; auto r = arr[].map!(value => value * context);
Aug 16 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 16 August 2015 at 12:04:51 UTC, cym13 wrote:
 To me the obvious way is to use a lambda, not a delegate:
Lambdas and delegates are the same thing, just different syntax.
Aug 16 2015
prev sibling next sibling parent reply "cym13" <cpicard openmailbox.org> writes:
On Sunday, 16 August 2015 at 11:53:42 UTC, FreeSlave wrote:
 Let's say I want to map some range using some context.
 The obvious way is to do:

 uint[3] arr = [1,2,3];
 uint context = 2;
 auto r = arr[].map!(delegate(value) { return value * context; 
 });

 The problem is that this allocates delegate, so it can't be 
 used in  nogc code.
 What I want to do might look like this:

 static struct Caller
 {
     this(uint context)  nogc {
         _context = context;
     }
     auto opCall(uint value)  nogc {
         return value * _context;
     }
     uint _context;
 }

 auto caller = Caller(2);
 auto r = arr[].map!(&caller.opCall);

 But it will not work of course since function must be a 
 compile-time parameter.

 So the way to go would be:

 auto caller = Caller(2);
 auto r = arr[].map!(Caller.opCall)(&caller);

 But map and other algorithms don't support this interface.

 The other way is

 auto r = arr.map!(Caller(2));

 But again, since it's template parameter, it can't use 
 variables unknown at compile time:

 uint context = ...;
 auto r = arr.map!(Caller(context)); //will not work

 So what's the solution? Of course besides rewriting the whole 
 std.algorithm.
Ok, so as my lambda proposition obviously doesn't work, here is one way that does using a templated function. There may be a way to make it shorter, I don't know. import std.conv; import std.stdio; template fun(uint context) { static uint withContext(uint value) { return value * context; } auto fun(uint[] arr) nogc { return arr.map!withContext; } } void main(string[] args) { [1, 2, 3].to!(uint[]) .fun!2 .writeln; }
Aug 16 2015
parent "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 16 August 2015 at 12:30:54 UTC, cym13 wrote:
 On Sunday, 16 August 2015 at 11:53:42 UTC, FreeSlave wrote:
 [...]
Ok, so as my lambda proposition obviously doesn't work, here is one way that does using a templated function. There may be a way to make it shorter, I don't know. import std.conv; import std.stdio; template fun(uint context) { static uint withContext(uint value) { return value * context; } auto fun(uint[] arr) nogc { return arr.map!withContext; } } void main(string[] args) { [1, 2, 3].to!(uint[]) .fun!2 .writeln; }
It works only because 2 is known constant at compile time.
Aug 16 2015
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 08/16/2015 04:53 AM, FreeSlave wrote:

 The problem is that this allocates delegate, so it can't be used in
  nogc code.
Would constructing the delegate by setting its .funcptr and .ptr properties work in this case? You can have a pool of context objects which become the context for the delegate. http://ddili.org/ders/d.en/lambda.html#ix_lambda..funcptr Ali
Aug 16 2015
parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 16 August 2015 at 15:29:10 UTC, Ali Çehreli wrote:
 On 08/16/2015 04:53 AM, FreeSlave wrote:

 The problem is that this allocates delegate, so it can't be
used in
  nogc code.
Would constructing the delegate by setting its .funcptr and .ptr properties work in this case? You can have a pool of context objects which become the context for the delegate. http://ddili.org/ders/d.en/lambda.html#ix_lambda..funcptr Ali
I don't see how this can solve the problem. What I tried: import std.stdio; import std.range; import std.algorithm; struct Caller { this(uint context) { _context = context; } uint method(uint value) { return _context * value; } uint _context; } nogc auto func(uint[] arr, uint function(uint) f) { return arr.map!(f); } void main(string[] args) { uint[] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; writeln(func(arr, d.funcptr)); } It still says it needs allocation: test.d(17): Error: function test.func nogc function allocates a closure with the GC funcptr does not play any role here, since passing the delegate directly leads to the same error.
Aug 16 2015
parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 16 August 2015 at 16:23:05 UTC, FreeSlave wrote:
 On Sunday, 16 August 2015 at 15:29:10 UTC, Ali Çehreli wrote:
 On 08/16/2015 04:53 AM, FreeSlave wrote:

 The problem is that this allocates delegate, so it can't be
used in
  nogc code.
Would constructing the delegate by setting its .funcptr and .ptr properties work in this case? You can have a pool of context objects which become the context for the delegate. http://ddili.org/ders/d.en/lambda.html#ix_lambda..funcptr Ali
I don't see how this can solve the problem. What I tried: import std.stdio; import std.range; import std.algorithm; struct Caller { this(uint context) { _context = context; } uint method(uint value) { return _context * value; } uint _context; } nogc auto func(uint[] arr, uint function(uint) f) { return arr.map!(f); } void main(string[] args) { uint[] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; writeln(func(arr, d.funcptr)); } It still says it needs allocation: test.d(17): Error: function test.func nogc function allocates a closure with the GC funcptr does not play any role here, since passing the delegate directly leads to the same error.
Forgot about data ptr. nogc auto func(uint[] arr, uint function(uint) f, void* data) { uint delegate(uint) d; d.funcptr = f; d.ptr = data; return arr.map!(d); } void main(string[] args) { uint[] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; writeln(func(arr, d.funcptr, d.ptr)); } Still the same error though.
Aug 16 2015
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 08/16/2015 09:26 AM, FreeSlave wrote:

 It still says it needs allocation:

 test.d(17): Error: function test.func  nogc function allocates a
 closure with the GC
I wrapped it inside a class object but it still thinks it needs to allocate: import std.stdio; import std.range; import std.algorithm; struct Caller { this(uint context) { _context = context; } uint method(uint value) { return _context * value; } uint _context; } class DelegateRef { uint delegate(uint) d; this (uint delegate(uint) d) { this.d = d; } } // HERE: // Error: function deneme.func nogc function allocates // a closure with the GC nogc auto func(uint[] arr, DelegateRef d) { return arr.map!(a => d.d(a)); } void main(string[] args) { uint[] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; writeln(func(arr, new DelegateRef(d))); } Ali
Aug 16 2015
parent reply "cym13" <cpicard openmailbox.org> writes:
On Sunday, 16 August 2015 at 22:22:07 UTC, Ali Çehreli wrote:
 // HERE:
 // Error: function deneme.func  nogc function allocates
 //        a closure with the GC
  nogc auto func(uint[] arr, DelegateRef d)
 {
     return arr.map!(a => d.d(a));
 }
Aren't you making another delegate in the map by using "=>" that needs to allocate because it uses 'd' which is out of its scope?
Aug 16 2015
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 08/16/2015 03:36 PM, cym13 wrote:
 On Sunday, 16 August 2015 at 22:22:07 UTC, Ali Çehreli wrote:
 // HERE:
 // Error: function deneme.func  nogc function allocates
 //        a closure with the GC
  nogc auto func(uint[] arr, DelegateRef d)
 {
     return arr.map!(a => d.d(a));
 }
Aren't you making another delegate in the map by using "=>" that needs to allocate because it uses 'd' which is out of its scope?
I did not see that at all. :) I've finally gotten it to work by stepping into template realm where the compiler is a master of attributes: :) import std.stdio; import std.range; import std.algorithm; struct Caller { this(uint context) { _context = context; } // ADDED pure pure uint method(uint value) { return _context * value; } uint _context; } // Now the type of d is a template parameter nogc auto func(Func)(uint[] arr, Func d) { return arr.map!(d); } void main(string[] args) { uint[] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; writeln(func(arr, d)); } Prints: [2, 4, 6] Ali
Aug 16 2015
parent reply "anonymous" <anonymous example.com> writes:
On Sunday, 16 August 2015 at 23:05:42 UTC, Ali Çehreli wrote:
 // Now the type of d is a template parameter
  nogc auto func(Func)(uint[] arr, Func d)
 {
     return arr.map!(d);
 }
Huh. I think func being a template is the key here. When the original code is put in a template, it works too (with 2.068): ---- void func()() nogc { import std.algorithm; uint[3] arr = [1,2,3]; uint context = 2; auto r = arr[].map!(delegate(value) { return value * context; }); } void main() { func(); } ----
Aug 17 2015
parent reply "thedeemon" <dlang thedeemon.com> writes:
On Monday, 17 August 2015 at 09:51:47 UTC, anonymous wrote:
 Huh. I think func being a template is the key here. When the 
 original code is put in a template, it works too (with 2.068):
Nope, it "works" only because "r" is unreferenced and gets thrown out. Just try using r.front there, for example, and the error returns.
Aug 17 2015
parent reply "anonymous" <anonymous example.com> writes:
On Monday, 17 August 2015 at 10:28:33 UTC, thedeemon wrote:
 Nope, it "works" only because "r" is unreferenced and gets 
 thrown out. Just try using r.front there, for example, and the 
 error returns.
You're right, it falls short. But I think r not being referenced is not exactly it. Using front in Ali's func breaks it in the same way. And returning r from func works. Wait, returning r from func works? Yes: ---- auto func()(uint[] arr, uint context) nogc { import std.algorithm; auto r = arr[].map!(delegate(value) { return value * context; }); return r; } void main() nogc { uint[3] arr = [1,2,3]; uint context = 2; auto r = func(arr[], context); import std.algorithm: equal; import std.range: only; assert(equal(r, only(2, 4, 6))); } ---- Is that supposed to compile? A closure is needed for the delegate, isn't it? With nogc, where is the closure stored? This looks like an accepts-invalid bug to me. It doesn't compile when func is not a template. So maybe the check is broken for templates. It also doesn't compile with 2.067, so this may be a regression. Coming back to Ali's code, here's a version that shows that func doesn't need template parameters, but it needs to be a template: ---- import std.stdio; import std.algorithm; struct Caller { uint method(uint value) pure nogc { return _context * value; } uint _context; } auto func()(uint[] arr, uint delegate(uint) pure nogc d) nogc { return arr.map!(d); } void main() nogc { uint[3] arr = [1,2,3]; uint context = 2; auto c = Caller(context); auto d = &c.method; auto r = func(arr[], d); import std.algorithm: equal; import std.range: only; assert(equal(r, only(2, 4, 6))); } ---- I think this relies on the same discrepancy as the problematic code above.
Aug 17 2015
parent reply "thedeemon" <dlang thedeemon.com> writes:
On Monday, 17 August 2015 at 12:38:05 UTC, anonymous wrote:
 auto func()(uint[] arr, uint delegate(uint) pure  nogc d)  nogc
 {
     return arr.map!(d);
 }

 void main()  nogc
 {
     uint[3] arr = [1,2,3];
     uint context = 2;
     auto c = Caller(context);
     auto d = &c.method;

     auto r = func(arr[], d);

     import std.algorithm: equal;
     import std.range: only;
     assert(equal(r, only(2, 4, 6)));
 }
I've just checked with my runtime GC hook. Here the call to func() allocates 12 bytes via gc_malloc, and it's the same for a 4-elements array, so it's not for the array itself, it's for a closure, I think.
Aug 17 2015
parent reply "thedeemon" <dlang thedeemon.com> writes:
On Monday, 17 August 2015 at 16:18:50 UTC, thedeemon wrote:
 I've just checked with my runtime GC hook. Here the call to 
 func() allocates 12 bytes via gc_malloc, and it's the same for 
 a 4-elements array, so it's not for the array itself, it's for 
 a closure, I think.
Also, compiling with -vgc says nothing, compiler (2.068) seems to miss this allocation.
Aug 17 2015
parent "anonymous" <anonymous example.com> writes:
On Monday, 17 August 2015 at 16:21:16 UTC, thedeemon wrote:
 On Monday, 17 August 2015 at 16:18:50 UTC, thedeemon wrote:
 I've just checked with my runtime GC hook. Here the call to 
 func() allocates 12 bytes via gc_malloc, and it's the same for 
 a 4-elements array, so it's not for the array itself, it's for 
 a closure, I think.
Also, compiling with -vgc says nothing, compiler (2.068) seems to miss this allocation.
Thanks for confirming. It seems to be a known issue that the compiler doesn't recognize the nogc violation by the closure: https://issues.dlang.org/show_bug.cgi?id=14771 That's not a regression, though. 2.067 rejecting the code has something to do with this assert: https://github.com/D-Programming-Language/phobos/blob/v2.067.1/std/algori hm/iteration.d#L453 (code is the same for 2.068). I don't know if that assert should trigger a nogc violation or not. But anyway, the real issue is 14771, as far as I can tell.
Aug 17 2015