www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Nasty corner case behaviour

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
Got bitten by this today:

	import std.algorithm : filter, map;
	import std.stdio;
	
	struct S {
		bool flag;
		auto method() {
			return [ 1 ].filter!(e => flag);
		}
	}
	
	void main() {
		auto s = S(true);
		writeln(s.method);
	
		auto arr = [ s ];
		writeln(arr[0].method);
	
		auto mappedArray = arr.map!(e => e.method);
		writeln(mappedArray.front);
		writeln(mappedArray);
	}

Expected output:

	[1]
	[1]
	[1]
	[[1]]

Actual output:

	[1]
	[1]
	[1]
	[[]]

Exercise for the reader: explain the problem.

Hint: it's not a bug in Phobos.

If method() is changed to the following, the problem goes away:

	auto method() {
		auto _flag = flag;
		return [ 1 ].filter!(e => _flag);
	}

It should be obvious by now that the problem is caused by the lambda
incorrectly closing over a member variable in a struct instance that
subsequently goes out of scope, while the returned closure lives on.
Unfortunately, neither -dip25 nor -dip1000 catches this leakage, even if
main() is marked  safe.


T

-- 
Тише едешь, дальше будешь.
Apr 17
next sibling parent Suleyman <sahmi.soulaimane gmail.com> writes:
On Thursday, 18 April 2019 at 05:01:17 UTC, H. S. Teoh wrote:
 ...
The value of flag is lost. To easily see this make flag an int. import std.algorithm : filter, map; import std.stdio; struct S { int flag; auto method() { return [ 1 ].filter!((e) { writeln("\nflag: ", flag); return flag == 10; }); } } void main() { auto s = S(10); auto arr = [ s ]; auto mappedArray = arr.map!(e => e.method); writeln(mappedArray); } Expected output: [[ flag: 10 1]] Actual output: [[ flag: 1872694912 ]]
Apr 18
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 4/18/19 1:01 AM, H. S. Teoh wrote:
 Got bitten by this today:
 
Yeeouch! Bugzilla issue #?
Apr 18
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 18, 2019 at 11:05:10PM -0400, Nick Sabalausky (Abscissa) via
Digitalmars-d wrote:
 On 4/18/19 1:01 AM, H. S. Teoh wrote:
 Got bitten by this today:
 
Yeeouch! Bugzilla issue #?
Just filed a new bug: https://issues.dlang.org/show_bug.cgi?id=19812 T -- This is not a sentence.
Apr 19
prev sibling next sibling parent reply Suleyman <sahmi.soulaimane gmail.com> writes:
On Thursday, 18 April 2019 at 05:01:17 UTC, H. S. Teoh wrote:
 ...
Sneaky enough. The problem is the value copy semantics of S which results in an address to a dead stack frame being saved. If you make it a ref parameter the issue goes away. import std.algorithm : filter, map; import std.stdio; struct S { bool flag; auto method() { return [ 1 ].filter!(e => flag); } } void main() { auto s = S(true); writeln(s.method); auto arr = [ s ]; writeln(arr[0].method); auto mappedArray = arr.map!((ref e) => e.method); writeln(mappedArray.front); writeln(mappedArray); } Output: [1] [1] [1] [[1]]
Apr 19
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Apr 19, 2019 at 10:17:08PM +0000, Suleyman via Digitalmars-d wrote:
 On Thursday, 18 April 2019 at 05:01:17 UTC, H. S. Teoh wrote:
 ...
Sneaky enough. The problem is the value copy semantics of S which results in an address to a dead stack frame being saved. If you make it a ref parameter the issue goes away.
[...] Using ref solves it for this specific case, but will still exhibit the same problem if the struct referenced by the ref is a temporary due to other factors. E.g., if you pass the struct to a function that then returns map!(S.method) back to the caller, then the ref will still be referencing a temporary that goes out of scope while the lambda still holds the reference. The underlying problem is that the lambda closes over a variable whose lifetime is not guaranteed to be at least the lifetime of the lambda. It's exactly this sort of scoping issues that -dip1000 is supposed to prevent, but it seems somehow this case was missed. T -- IBM = I'll Buy Microsoft!
Apr 19
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Thursday, 18 April 2019 at 05:01:17 UTC, H. S. Teoh wrote:
 Got bitten by this today:

 	import std.algorithm : filter, map;
 	import std.stdio;

 	struct S {
 		bool flag;
 		auto method() {
 			return [ 1 ].filter!(e => flag);
 		}
 	}

 	void main() {
 		auto s = S(true);
 		writeln(s.method);

 		auto arr = [ s ];
 		writeln(arr[0].method);

 		auto mappedArray = arr.map!(e => e.method);
 		writeln(mappedArray.front);
 		writeln(mappedArray);
 	}

 Expected output:

 	[1]
 	[1]
 	[1]
 	[[1]]

 Actual output:

 	[1]
 	[1]
 	[1]
 	[[]]

 Exercise for the reader: explain the problem.

 Hint: it's not a bug in Phobos.

 If method() is changed to the following, the problem goes away:

 	auto method() {
 		auto _flag = flag;
 		return [ 1 ].filter!(e => _flag);
 	}

 It should be obvious by now that the problem is caused by the 
 lambda incorrectly closing over a member variable in a struct 
 instance that subsequently goes out of scope, while the 
 returned closure lives on. Unfortunately, neither -dip25 nor 
 -dip1000 catches this leakage, even if main() is marked  safe.


 T
I find this one a bit confusing. Shouldn't the struct be moved to the heap when the compiler detects that it's being closed over?
Apr 19
parent Suleyman <sahmi.soulaimane gmail.com> writes:
On Friday, 19 April 2019 at 23:42:53 UTC, Meta wrote:
 I find this one a bit confusing. Shouldn't the struct be moved 
 to the heap when the compiler detects that it's being closed 
 over?
It does. the `this` parameter in the following function is copied to a heap closure space. auto method() { return [ 1 ].filter!(e => this.flag); } But `this` is a reference so the pointer is copied instead of the value.
Apr 19