www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Patterns to avoid GC with capturing closures?

reply Peter Alexander <peter.alexander.au gmail.com> writes:
Consider this code, which is used as an example only:

auto scaleAll(int[] xs, int m) {
   return xs.map!(x => m * x);
}

As m is captured, the delegate for map will rightly allocate the 
closure in the GC heap.

In C++, you would write the lambda to capture m by value, but 
this is not a facility in D.

I can write scaleAll like this:

auto scaleAll(int[] xs, int m)  nogc {
   return repeat(m).zip(xs).map!(mx => mx[0] * mx[1]);
}

So that repeat(m) stores m, but it is quite hacky to work like 
this.

I could write my own range that does this, but this is also not 
desirable.

Are there any established patterns, libraries, or language 
features that can help avoid the GC allocation in a principled 
way here?
Aug 24 2018
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/24/18 11:18 AM, Peter Alexander wrote:

 Consider this code, which is used as an example only:
 
 auto scaleAll(int[] xs, int m) {
    return xs.map!(x => m * x);
 }
 
 As m is captured, the delegate for map will rightly allocate the closure 
 in the GC heap.
 
 In C++, you would write the lambda to capture m by value, but this is 
 not a facility in D.
 
 I can write scaleAll like this:
 
 auto scaleAll(int[] xs, int m)  nogc {
    return repeat(m).zip(xs).map!(mx => mx[0] * mx[1]);
 }
 
 So that repeat(m) stores m, but it is quite hacky to work like this.
 
 I could write my own range that does this, but this is also not desirable.
 
 Are there any established patterns, libraries, or language features that 
 can help avoid the GC allocation in a principled way here?


This is somewhat related to a suggestion I had last month: 
https://forum.dlang.org/post/pjnue1$olt$1 digitalmars.com

I also hate to have such a thing allocate. The only scalable solution I 
can think of is to write your own range function which has the 
appropriate state saved by value. But then you lose all the goodies from 
Phobos.

Having a way to capture state and give that state to std.algorithm 
ranges would be really cool.

-Steve
Aug 24 2018
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 24 August 2018 at 15:18:13 UTC, Peter Alexander wrote:

 I can write scaleAll like this:

 auto scaleAll(int[] xs, int m)  nogc {
   return repeat(m).zip(xs).map!(mx => mx[0] * mx[1]);
 }

 So that repeat(m) stores m, but it is quite hacky to work like 
 this.


Here's a spoonful of sugar to help that go down easier:

https://run.dlang.io/is/8lTmZg
Aug 24 2018
parent aliak <something something.com> writes:
On Friday, 24 August 2018 at 22:51:40 UTC, Paul Backus wrote:

 On Friday, 24 August 2018 at 15:18:13 UTC, Peter Alexander 
 wrote:

 I can write scaleAll like this:

 auto scaleAll(int[] xs, int m)  nogc {
   return repeat(m).zip(xs).map!(mx => mx[0] * mx[1]);
 }

 So that repeat(m) stores m, but it is quite hacky to work like 
 this.


 Here's a spoonful of sugar to help that go down easier:

 https://run.dlang.io/is/8lTmZg


And with some mixin magic you can get syntax like:

int a = 2, b = 3;
xs.pack(a, b).map!(unpack!((x, a, b) => x * a * b));

https://run.dlang.io/is/gb5Io4

Of course it sucks that you can't just tell a delegate to capture 
by value.
Aug 26 2018
prev sibling parent reply vit <vit vit.vit> writes:
On Friday, 24 August 2018 at 15:18:13 UTC, Peter Alexander wrote:

 Consider this code, which is used as an example only:

 auto scaleAll(int[] xs, int m) {
   return xs.map!(x => m * x);
 }

 As m is captured, the delegate for map will rightly allocate 
 the closure in the GC heap.

 In C++, you would write the lambda to capture m by value, but 
 this is not a facility in D.

 I can write scaleAll like this:

 auto scaleAll(int[] xs, int m)  nogc {
   return repeat(m).zip(xs).map!(mx => mx[0] * mx[1]);
 }

 So that repeat(m) stores m, but it is quite hacky to work like 
 this.

 I could write my own range that does this, but this is also not 
 desirable.

 Are there any established patterns, libraries, or language 
 features that can help avoid the GC allocation in a principled 
 way here?


I try pack/unpack solution, but it looks horrible.
Problem is when every part of chain (filter, map, ...) need 
capture different variables.
I implement modified version of algorithm (filter, map, until, 
tee): https://dpaste.dzfl.pl/929a7af4e87f

You don't need reimplement all of std.algorithm, only parts which 
can be in the middle of chain, things like any, all, each, count 
can be ignored.


example:
void main() nogc{
     import std.algorithm : any;
     import std.range : iota;
     import util.algorithm : map, filter;

     const int
		a = 1,
		b = 2,
		c = 3;
	
     const x = iota(0, 10)
         .map!((x, i) => x*i)(a)   ///map!((x) => x*a)
         .map!((x, i) => x*i)(b)   ///map!((x) => x*b)
         .filter!((x, i) => x%i)(c)///filter!((x) => x%c)
         .any!(x => x % c);

     assert(x == true);

}
Aug 25 2018
parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 26 August 2018 at 06:08:39 UTC, vit wrote:

     const x = iota(0, 10)
         .map!((x, i) => x*i)(a)   ///map!((x) => x*a)
         .map!((x, i) => x*i)(b)   ///map!((x) => x*b)
         .filter!((x, i) => x%i)(c)///filter!((x) => x%c)
         .any!(x => x % c);


I think it's easier to just use zip than to reimplement large 
chunks of std.algorithm:

const x = iota(0, 10)
     .zip!(repeat(a), repeat(b))
     .map!(unpack!((x, a, b) => x*a*b))
     .zip!(repeat(c))
     .filter!(unpack!((x, c) => x%c)) // redundant with `any`
     .any!(unpack!((x, c) => x%c))
Aug 26 2018