digitalmars.D.learn - Variables with scoped destruction in closures
- =?UTF-8?B?Tm9yZGzDtnc=?= (53/53) Oct 14 2016 The following code
- =?UTF-8?B?Tm9yZGzDtnc=?= (3/4) Oct 14 2016 Correction, should be:
- ag0aep6g (53/103) Oct 14 2016 Your `s.move()` isn't called once when the closure is created, but every...
- =?UTF-8?B?Tm9yZGzDtnc=?= (9/55) Oct 14 2016 I'm not interested in either of these solutions.
- =?UTF-8?B?Tm9yZGzDtnc=?= (2/3) Oct 14 2016 That of course being incorrect D code that leaks memory.
- ag0aep6g (27/31) Oct 14 2016 [...]
- Steven Schveighoffer (7/21) Oct 14 2016 Note: no matter what solution you can come up with, having the GC
- Walter Bright (11/13) Oct 14 2016 https://github.com/dlang/dmd/blob/master/src/toir.d#L820
The following code
import std.algorithm.iteration : filter;
import std.algorithm.mutation : move;
import std.range : iota;
static private struct S
{
import core.memory : GC;
disable this(this);
this(int x)
{
_ptr = cast(typeof(_ptr))GC.malloc((*_ptr).sizeof);
*_ptr = x;
}
~this() { GC.free(_ptr); } // scoped destruction
property auto ref value() safe pure nothrow nogc { return
*_ptr; }
alias value this;
int* _ptr;
}
auto below1(size_t n,
S s = S.init)
{
// this could work, if the compiler could infer that
// `s` can be implicitly converted to an r-value
return 0.iota(n).filter!(_ => _ < s);
}
auto below2(size_t n,
S s = S.init)
{
// this should work, because `s` is an r-value
return 0.iota(n).filter!(_ => _ < s.move());
}
unittest
{
S s = S(42);
s.value = 43;
s.value = 42;
assert(s.value == 42);
// both these fail
100.below1(s.move());
100.below2(s.move());
}
fails to compile with DMD Git master with error message
t_scope.d(23,6): Error: variable t_scope.below.s has scoped
destruction, cannot build closure
It illustrates one the simplest cases where a container-type
cannot be used inside a lambda closure needed by D's great lazy
ranges.
Can somebody explain why not even `below2` compiles eventhough
`s.move()` is inferred to be an r-value?
Until this gets fixed in the compiler, is there something I can
do in the mean-while to make it possible to use instances of `S`
inside of range lambda closures?
Oct 14 2016
=?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> On Friday, 14 October 2016 at 10:18:19 UTC, Nordlöw wrote:t_scope.d(23,6): Error: variable t_scope.below.s ...Correction, should be: t_scope.d(23,6): Error: variable t_scope.below1.s ...
Oct 14 2016
On 10/14/2016 12:18 PM, Nordlöw wrote:import std.algorithm.iteration : filter; import std.algorithm.mutation : move; import std.range : iota; static private struct S { import core.memory : GC; disable this(this); this(int x) { _ptr = cast(typeof(_ptr))GC.malloc((*_ptr).sizeof); *_ptr = x; } ~this() { GC.free(_ptr); } // scoped destruction property auto ref value() safe pure nothrow nogc { return *_ptr; } alias value this; int* _ptr; } auto below1(size_t n, S s = S.init) { // this could work, if the compiler could infer that // `s` can be implicitly converted to an r-value return 0.iota(n).filter!(_ => _ < s); } auto below2(size_t n, S s = S.init) { // this should work, because `s` is an r-value return 0.iota(n).filter!(_ => _ < s.move()); } unittest { S s = S(42); s.value = 43; s.value = 42; assert(s.value == 42); // both these fail 100.below1(s.move()); 100.below2(s.move()); } fails to compile with DMD Git master with error message t_scope.d(23,6): Error: variable t_scope.below.s has scoped destruction, cannot build closure It illustrates one the simplest cases where a container-type cannot be used inside a lambda closure needed by D's great lazy ranges. Can somebody explain why not even `below2` compiles eventhough `s.move()` is inferred to be an r-value?Your `s.move()` isn't called once when the closure is created, but every time the lambda is called. The closure must already be set up at that point. So s vs. s.move() doesn't make a difference with regards to closure creation.Until this gets fixed in the compiler, is there something I can do in the mean-while to make it possible to use instances of `S` inside of range lambda closures?I don't see an obvious compiler bug. I'm not sure why exactly the compiler doesn't just move s to the closure, but it would at least be a bit surprising. The function would assume ownership of s without that being spelled out. There may be more serious issues which I'm not aware of. As for ways to make this work: 1) You can move s to the heap yourself: ---- auto below3(size_t n, S s = S.init) { import std.algorithm.mutation: moveEmplace; auto onHeap = cast(S*) new ubyte[S.sizeof]; moveEmplace(s, *onHeap); /* If there's a function that does allocation and moveEmplace in one go, I can't find it. */ return 0.iota(n).filter!(_ => _ < *onHeap); } ---- 2) Or you can move it into a struct that gets returned (more involved): ---- auto below4(size_t n, S s = S.init) { static struct Below4CustomFilter(R) { R range; S s; this(R range, S s) { this.range = range; this.s = s.move(); skipFiltered(); } private void skipFiltered() { while (!range.empty && range.front >= s.value) range.popFront(); } property bool empty() { return range.empty; } property auto front() { return range.front; } void popFront() { range.popFront(); skipFiltered(); } /* ... more advanced range primitives ... */ } static customFilter(R)(R range, S s) { return Below4CustomFilter!R(range, s.move()); } return customFilter(0.iota(n), s.move()); } ----
Oct 14 2016
On Friday, 14 October 2016 at 14:00:53 UTC, ag0aep6g wrote:
As for ways to make this work:
1) You can move s to the heap yourself:
----
auto below3(size_t n, S s = S.init)
{
import std.algorithm.mutation: moveEmplace;
auto onHeap = cast(S*) new ubyte[S.sizeof];
moveEmplace(s, *onHeap);
/* If there's a function that does allocation and
moveEmplace
in one go, I can't find it. */
return 0.iota(n).filter!(_ => _ < *onHeap);
}
----
2) Or you can move it into a struct that gets returned (more
involved):
----
auto below4(size_t n, S s = S.init)
{
static struct Below4CustomFilter(R)
{
R range;
S s;
this(R range, S s)
{
this.range = range;
this.s = s.move();
skipFiltered();
}
private void skipFiltered()
{
while (!range.empty && range.front >= s.value)
range.popFront();
}
property bool empty() { return range.empty; }
property auto front() { return range.front; }
void popFront() { range.popFront(); skipFiltered(); }
/* ... more advanced range primitives ... */
}
static customFilter(R)(R range, S s)
{
return Below4CustomFilter!R(range, s.move());
}
return customFilter(0.iota(n), s.move());
}
----
I'm not interested in either of these solutions.
The whole idea in the first place was to avoid extra (GC) heap
allocations together with reusing existing Phobos ranges in a
functional way.
If I remove the explicit dtor from `S` the code compiles. Can
somebody please explain why? Walter?
Thanks, anyway.
I guess I have to fix the compiler myself :)
Oct 14 2016
On Friday, 14 October 2016 at 16:37:06 UTC, Nordlöw wrote:If I remove the explicit dtor from `S` the code compiles. CanThat of course being incorrect D code that leaks memory.
Oct 14 2016
On Friday, 14 October 2016 at 14:00:53 UTC, ag0aep6g wrote:As for ways to make this work: 1) You can move s to the heap yourself:[...]2) Or you can move it into a struct that gets returned (more involved):[...] 3) Put a struct on the heap that acts as the closure: ---- auto below5(size_t n, S s = S.init) { import std.algorithm.mutation: moveEmplace; static struct ExplicitClosure { S s; bool pred(size_t _) { return _ < s; } } auto myClosure = new ExplicitClosure(s.move()); auto pred = &myClosure.pred; return 0.iota(n).filter!pred; } ---- This is the one I was trying to think of, but I got lost along the way. I'm not sure if the code is really correct, because thinking about delegates being passed in template (alias) parameters confuses me. If this is ok, it should make only the one explicit `new` allocation. No closure should be created for the `filter` call. If passing `&closure.pred` like that is not ok, I'm lost again as for how one might construct an alternative to a closure.
Oct 14 2016
On 10/14/16 6:18 AM, Nordlöw wrote:
The following code
import std.algorithm.iteration : filter;
import std.algorithm.mutation : move;
import std.range : iota;
static private struct S
{
import core.memory : GC;
disable this(this);
this(int x)
{
_ptr = cast(typeof(_ptr))GC.malloc((*_ptr).sizeof);
*_ptr = x;
}
~this() { GC.free(_ptr); } // scoped destruction
Note: no matter what solution you can come up with, having the GC
destroy this struct, and the destructor attempting to free the pointer,
is going to result in errors.
Instead, you should use C malloc/free here.
This is what RefCounted does.
-Steve
Oct 14 2016
On 10/14/2016 3:18 AM, Nordlöw wrote:t_scope.d(23,6): Error: variable t_scope.below.s has scoped destruction, cannot build closurehttps://github.com/dlang/dmd/blob/master/src/toir.d#L820 The problem is the closure is generated when it is expected that the delegate will survive past the end of the scope (it's the whole point of a closure). But with a destructor that runs at the end of the scope, it cannot survive, and so the user of the closure will be referring to a destroyed object. There is a current PR to improve the closure decision so that fewer closures are necessary, https://github.com/dlang/dmd/pull/5972 I don't know if that will resolve the issue you're having. (A smaller test case would be nice!)
Oct 14 2016
On Saturday, 15 October 2016 at 05:41:05 UTC, Walter Bright wrote:The problem is the closure is generated when it is expected that the delegate will survive past the end of the scope (it's the whole point of a closure). But with a destructor that runs at the end of the scope, it cannot survive, and so the user of the closure will be referring to a destroyed object.while this is generally true, it bites me every time i'm using my refcounted objects. i have std.stdio.File replacement, which is just an rc-struct for the outside world, and i can't write generalized templates with it, as i'm hitting this "scoped destruction" issue often enough for it to be annoying. like, for example, when i want to accept generalized i/o stream (i have template checkers for that) in write[ln] replacement, so i have to write this abominations: public void write(A...) (VFile fl, A args) { writeImpl!(false)(fl, args); } public void write(ST, A...) (auto ref ST fl, A args) if (!is(ST == VFile) && isRorWStream!ST) { writeImpl!(false, ST)(fl, args); } VFile is perfectly valid "OutputStream", but i can't use general template, 'cause VFile has dtor which decrements rc, and compiler complains. i.e. that should be just: public void write(ST, A...) (auto ref ST fl, A args) if (isRorWStream!ST) { writeImpl!(false, ST)(fl, args); } can we, please, have some way to tell the compiler that it is perfectly ok for some structs to have scoped destruction *and* be a member of closure? also, what kind of *closure* compiler tries to build here? it is a simple templated function, which calls another templated function, nothing fancy.
Oct 15 2016
On Saturday, 15 October 2016 at 07:55:30 UTC, ketmar wrote: p.s. compiler doesn't complain each time, only in some circumstances. i don't remember the exact code now, but some of it has nothing to do with closures at all -- no std.algo, no templates with lambda args, etc.
Oct 15 2016