www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Closures and loop scope

reply "Idan Arye" <GenericNPC gmail.com> writes:
Consider the following code. What will it print?

     auto arr=new ulong delegate()[5];

     foreach(i;0..arr.length){
         arr[i]=()=>i;
     }

     writeln(arr.map!`a()`());

It is natural to expect that it will print [0, 1, 2, 3, 4], but 
it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all 
the delegates refer to the same `i` in their closures, and at the 
end of the loop that `i` is set to `5`.

While this makes some sense when you consider how the closures 
are implemented, it is still not what you expect from that code. 
`i` is supposed to be local to each iteration of the loop, and 
here it "leaks" between the iterations.

There is an hackaround:

     auto arr=new ulong delegate()[5];

     foreach(i;0..arr.length){
         arr[i]=(j=>()=>j)(i);
     }

     writeln(arr.map!`a()`());

This prints [0, 1, 2, 3, 4] as expected, since we use another 
function to create a separate stack frame to store the value of 
`i`. A different way to implement that idea:

     auto arr=new ulong delegate()[5];

     foreach(i;0..arr.length){
         arr[i]={
             auto j=i;
             return ()=>j;
         }();
     }

     writeln(arr.map!`a()`());



Can this be fixed? *Should* this be fixed?

Discuss
Jun 04 2013
next sibling parent "angel" <andrey.gelman gmail.com> writes:
Why fix if it ain't broken ?
Your example and the 'hackaround' look like some university exam
questions. Just wait for the students, and let'em eat it.
Jun 04 2013
prev sibling next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 4 June 2013 at 19:19:57 UTC, Idan Arye wrote:

 Can this be fixed? *Should* this be fixed?


For what it's worth, Javascript works the same way. So I've come 
to the pattern of having a function return a function when it is 
looping in both languages.

So while it might not be ideal, it isn't totally unexpected since 
other languages do it too.
Jun 04 2013
parent "Idan Arye" <GenericNPC gmail.com> writes:
On Tuesday, 4 June 2013 at 19:42:39 UTC, Adam D. Ruppe wrote:

 On Tuesday, 4 June 2013 at 19:19:57 UTC, Idan Arye wrote:

 Can this be fixed? *Should* this be fixed?


 For what it's worth, Javascript works the same way. So I've 
 come to the pattern of having a function return a function when 
 it is looping in both languages.

 So while it might not be ideal, it isn't totally unexpected 
 since other languages do it too.


Ruby is the most interesting:

     arr=[]
     for i in 0...5
         arr<< lambda{i}
     end

     puts arr.map{|e|e.call}.to_s

prints [4, 4, 4, 4, 4], because it uses the built-in for loop, so 
`i` is reused in all iterations. On the other hand:

     arr=[]
     (0...5).each do|i|
         arr<< lambda{i}
     end

     puts arr.map{|e|e.call}.to_s

prints [0, 1, 2, 3, 4], because each call to the rubyblock has 
it's own stack frame and it's own `i`. This can also be done in D:


     auto arr=new ulong delegate()[5];

     auto iotaDelegate(T...)(T args){
         return (int delegate(ref ulong) dlg){
             foreach(i;iota(args)){
                 dlg(i);
             }
             return 0;
         };
     }

     foreach(i;iotaDelegate(arr.length)){
         arr[i]=()=>i;
     }

     writeln(arr.map!`a()`());

This prints [0, 1, 2, 3, 4], because each iteration of the 
foreach is actually an function call, so it has it's own stack 
frame.
Jun 04 2013
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
Huh, this is disappointing. I haven't used closures enough in D 
to notice this issue, but I know of this kind of issue very well, 
because it is a common annoyance in JavaScript. Newer, 
non-cross-browsers versions of JavaScript get around this with 
'let' instead of 'var,' which introduces block scope. So 
something like this works as expected:

let arr = Array(5);

for (let i = 0; i < 5; ++i) {
     arr[i] = function() { return i; }; // The scoped i is used 
here.
}

let anotherArr = Array(5);

for (let i = 0; i < 5; ++i) {
     anotherArr[i] = arr[i](); // Return a different i each time.
}

console.log(anotherArr); // 0, 1, 2, 3, 4, as expected

Is it possible for similar semantics to exist in D by default? I 
imagine this is harder to implement.
Jun 04 2013
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/04/2013 09:19 PM, Idan Arye wrote:

 Consider the following code. What will it print?

      auto arr=new ulong delegate()[5];

      foreach(i;0..arr.length){
          arr[i]=()=>i;
      }

      writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but it
 actually prints [5, 5, 5, 5, 5]. The reason is obvious - all the
 delegates refer to the same `i` in their closures, and at the end of the
 loop that `i` is set to `5`.

 ...


It is not that obvious. They refer to different i's that happen to 
reside at the same place in the stack frame. It's a bug.

It is more obvious that it is a bug given this code snippet:

import std.stdio, std.algorithm;

void main(){
     auto arr=new ulong delegate()[5];
     foreach(immutable i;0..arr.length){
          arr[i]={auto j=i;return {assert(j==i); return i;};}();
     }
     writeln(arr.map!`a()`());
}

As you can see, 'i' mutates even though it is declared immutable.


...


Yes. Yes.

http://d.puremagic.com/issues/show_bug.cgi?id=2043
Jun 04 2013
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 22:52:27 +0200
Timon Gehr <timon.gehr gmx.ch> wrote:


 On 06/04/2013 09:19 PM, Idan Arye wrote:

 Consider the following code. What will it print?

      auto arr=new ulong delegate()[5];

      foreach(i;0..arr.length){
          arr[i]=()=>i;
      }

      writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but it
 actually prints [5, 5, 5, 5, 5]. The reason is obvious - all the
 delegates refer to the same `i` in their closures, and at the end
 of the loop that `i` is set to `5`.

 ...


 
 It is not that obvious. They refer to different i's that happen to 
 reside at the same place in the stack frame. It's a bug.
 
 It is more obvious that it is a bug given this code snippet:
 
 import std.stdio, std.algorithm;
 
 void main(){
      auto arr=new ulong delegate()[5];
      foreach(immutable i;0..arr.length){
           arr[i]={auto j=i;return {assert(j==i); return i;};}();
      }
      writeln(arr.map!`a()`());
 }
 
 As you can see, 'i' mutates even though it is declared immutable.
 


Keep in mind that this exhibits the same "mutating immutable" behavior:

// Prints 0, 1, 2, 3, 4, even though i is immutable
foreach(immutable i; 0..5){
    writeln(i);

It's questionable as to whether that's really a problem. And even if it
is a problem, it would *only* be because you make i immutable. So
this is irrelevant to the OP's examples because:

1. He didn't use immutable, and

2. A delegate is *expected* to read the values inside its closure at
the time of delegate *invocation*, not at the time of delegate creation.


...


 
 Yes. Yes.
 
 http://d.puremagic.com/issues/show_bug.cgi?id=2043


That refers to a local defined within the loop, not the iteration
variable itself, which is different from the OP. Also, it's
questionable that there's any problem there *other* than the immutable
stuff.
Jun 04 2013
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/04/2013 11:57 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 22:52:27 +0200
 Timon Gehr <timon.gehr gmx.ch> wrote:


 On 06/04/2013 09:19 PM, Idan Arye wrote:

 Consider the following code. What will it print?

       auto arr=new ulong delegate()[5];

       foreach(i;0..arr.length){
           arr[i]=()=>i;
       }

       writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but it
 actually prints [5, 5, 5, 5, 5]. The reason is obvious - all the
 delegates refer to the same `i` in their closures, and at the end
 of the loop that `i` is set to `5`.

 ...


 It is not that obvious. They refer to different i's that happen to
 reside at the same place in the stack frame. It's a bug.

 It is more obvious that it is a bug given this code snippet:

 import std.stdio, std.algorithm;

 void main(){
       auto arr=new ulong delegate()[5];
       foreach(immutable i;0..arr.length){
            arr[i]={auto j=i;return {assert(j==i); return i;};}();
       }
       writeln(arr.map!`a()`());
 }

 As you can see, 'i' mutates even though it is declared immutable.


 Keep in mind that this exhibits the same "mutating immutable" behavior:

 // Prints 0, 1, 2, 3, 4, even though i is immutable
 foreach(immutable i; 0..5){
      writeln(i);



No, it does not! My code contains immutable variables j and i (in fact, 
5 pairs of these), where with DMD's buggy behaviour it appears that at 
one point in time j==i and at another point in time j!=i.

Your code contains five distinct immutable variables i with different 
values.


 It's questionable as to whether that's really a problem. And even if it
 is a problem, it would *only* be because you make i immutable. So
 this is irrelevant to the OP's examples because:

 1. He didn't use immutable, and
 ...


No, this point is irrelevant. It is a bug in any case. If immutable is 
used, this bug can be exploited to break the const system. Hence, I was 
using immutable to make my point more clear. Apparently it had the 
converse effect.


 2. A delegate is *expected* to read the values inside its closure at
 the time of delegate *invocation*, not at the time of delegate creation.


Obviously. What you seem to miss is that there is no aliasing between 
any of the delegate closures in the above code.

Note that I understand exactly what you think is happening.


 ...


 Yes. Yes.

 http://d.puremagic.com/issues/show_bug.cgi?id=2043


 That refers to a local defined within the loop, not the iteration
 variable itself, which is different from the OP.


No, it is not, as I have explained before. Since 2.063, the loop 
variable that is exposed from the foreach loop behaves like a for-loop 
loop body-local variable.


 Also, it's
 questionable that there's any problem there *other* than the immutable
 stuff.


Obviously there is. The only reason why the code appears to behave as 
you assume it does is because the loop-local variables happen to be 
allocated at the same place for each iteration. It's a memory safety 
issue. Allocated memory is allocated again before it is freed.


correctly allocate a closure context per loop iteration.
Jun 04 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/05/2013 12:16 AM, Timon Gehr wrote:

 On 06/04/2013 11:57 PM, Nick Sabalausky wrote:

 ...

 Keep in mind that this exhibits the same "mutating immutable" behavior:

 // Prints 0, 1, 2, 3, 4, even though i is immutable
 foreach(immutable i; 0..5){
      writeln(i);



 No, it does not! ...


You can disregard that, there was a race condition.
Jun 04 2013
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 05 Jun 2013 00:16:48 +0200
Timon Gehr <timon.gehr gmx.ch> wrote:

 
 No, it does not! [...]
 


Yea, see my other recent responses. For some reason I was
(incorrectly) thinking of closures being captured and stored at the end
of the current function call instead of (correctly) at the end of the
current lexical scope.
Jun 04 2013
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 21:19:56 +0200
"Idan Arye" <GenericNPC gmail.com> wrote:


 Consider the following code. What will it print?
 
      auto arr=new ulong delegate()[5];
 
      foreach(i;0..arr.length){
          arr[i]=()=>i;
      }
 
      writeln(arr.map!`a()`());
 
 It is natural to expect that it will print [0, 1, 2, 3, 4], but 
 it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all 
 the delegates refer to the same `i` in their closures, and at the 
 end of the loop that `i` is set to `5`.
 


I think the problem is simply a misunderstanding of closures.

Consider this:

    int a = 2;
    auto dg = () => a;
    a = 3;
    
    // This prints 3
    writeln(dg());
    
    // Now this prints 5
    a = 5;
    writeln(dg());

The thing to keep in mind is that closures are *not* evaluated at the
point of creation (otherwise they may as well not be written as a
delegate at all). They're evaluated at the point of invocation. And
that's the whole point: to say "Ok program, I want you to *hold on* to
this set of instructions for now...ignore them right now, but I'll tell
you when I want you to 'open the envelope' and look at it".

In other words, the scope captured by a delegate is *by reference*, not
by value. It's the *same* scope, not a snapshot copy of the scope.

So when you put the delegate creation in a loop:

    foreach(a; iota(0..6))
        dg = () => a;

It *is* expected that you're *not* sticking 0, 1, 2, 3, etc inside the
delegate. That's because you're not evaluating "a" *at all* here,
you're just crerates a delegate that *refers* to "a" itself. You're
just creating the exact same delegate five times. In other words:

You're just saying:
Store the following set of instructions into 'dg': "Read the value of
'a' and then return it."

You're *not* saying:
Read the value of 'a' and *then* create a delegate that returns that
value.
Jun 04 2013
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/04/2013 11:03 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 21:19:56 +0200
 "Idan Arye" <GenericNPC gmail.com> wrote:


 Consider the following code. What will it print?

       auto arr=new ulong delegate()[5];

       foreach(i;0..arr.length){
           arr[i]=()=>i;
       }

       writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but
 it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all
 the delegates refer to the same `i` in their closures, and at the
 end of the loop that `i` is set to `5`.


 I think the problem is simply a misunderstanding of closures.
 ...


I believe what you explain is exactly what he suspected is happening.


 ...

 So when you put the delegate creation in a loop:

      foreach(a; iota(0..6))
          dg = () => a;

 It *is* expected that you're *not* sticking 0, 1, 2, 3, etc inside the
 delegate. That's because you're not evaluating "a" *at all* here,
 you're just crerates a delegate that *refers* to "a" itself. You're
 just creating the exact same delegate five times. In other words:

 You're just saying:
 Store the following set of instructions into 'dg': "Read the value of
 'a' and then return it."

 You're *not* saying:
 Read the value of 'a' and *then* create a delegate that returns that
 value.


'a' refers to a different location for every loop iteration. This is a 
language change in 2.063.
Jun 04 2013
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 23:24:51 +0200
Timon Gehr <timon.gehr gmx.ch> wrote:


 On 06/04/2013 11:03 PM, Nick Sabalausky wrote:

 ...

 So when you put the delegate creation in a loop:

      foreach(a; iota(0..6))
          dg = () => a;

 It *is* expected that you're *not* sticking 0, 1, 2, 3, etc inside
 the delegate. That's because you're not evaluating "a" *at all*
 here, you're just crerates a delegate that *refers* to "a" itself.
 You're just creating the exact same delegate five times. In other
 words:

 You're just saying:
 Store the following set of instructions into 'dg': "Read the value
 of 'a' and then return it."

 You're *not* saying:
 Read the value of 'a' and *then* create a delegate that returns that
 value.


 
 'a' refers to a different location for every loop iteration. This is
 a language change in 2.063.
 


foreach(a;0..5)
    writeln(&a);

For me, that prints the same address five times in both 2.062 and
2.063. I would argue that's as it should be.
Jun 04 2013
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/04/2013 11:38 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 23:24:51 +0200
 Timon Gehr <timon.gehr gmx.ch> wrote:


 On 06/04/2013 11:03 PM, Nick Sabalausky wrote:

 ...

 So when you put the delegate creation in a loop:

       foreach(a; iota(0..6))
           dg = () => a;

 It *is* expected that you're *not* sticking 0, 1, 2, 3, etc inside
 the delegate. That's because you're not evaluating "a" *at all*
 here, you're just crerates a delegate that *refers* to "a" itself.
 You're just creating the exact same delegate five times. In other
 words:

 You're just saying:
 Store the following set of instructions into 'dg': "Read the value
 of 'a' and then return it."

 You're *not* saying:
 Read the value of 'a' and *then* create a delegate that returns that
 value.


 'a' refers to a different location for every loop iteration. This is
 a language change in 2.063.


 foreach(a;0..5)
      writeln(&a);

 For me, that prints the same address five times in both 2.062 and
 2.063.


It does not matter. The lifetimes of the different a's do not overlap.


 I would argue that's as it should be.


That argument would be mistaken. The compiler is free to eg. unroll the 
loop completely and use different addresses.

Behind the scenes, the foreach loop is rewritten to something like:

for(int __a=0;__a<5;__a++){
     int a = __a;
     writeln(&a);
}
Jun 04 2013
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 23:48:32 +0200
Timon Gehr <timon.gehr gmx.ch> wrote:


 On 06/04/2013 11:38 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 23:24:51 +0200
 Timon Gehr <timon.gehr gmx.ch> wrote:

 'a' refers to a different location for every loop iteration. This
 is a language change in 2.063.


 foreach(a;0..5)
      writeln(&a);

 For me, that prints the same address five times in both 2.062 and
 2.063.


 
 It does not matter. The lifetimes of the different a's do not overlap.
 


Ok, I see what you mean now: Closures are expected to capture the
current scope, not just the current function's scope (which wouldn't
have been right anyway as it would have prevented access to the loop
variable).

Ie:

    int delegate() dg;

    {
        int a = 2;
        dg = () => a;
    }

    int a = 100;
    writeln(dg());

That prints 2, as expected.

I hadn't thought of that. With that in mind, then yes, the OP's example
should print 0, 1, 2, 3, etc, not all 5's.
Jun 04 2013
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 04 Jun 2013 23:48:32 +0200
Timon Gehr <timon.gehr gmx.ch> wrote:

 Behind the scenes, the foreach loop is rewritten to something like:
 
 for(int __a=0;__a<5;__a++){
      int a = __a;
      writeln(&a);
 }


Yea, I think the years I spent using C has corrupted my brain into
seeing...

   foreach(int a; 0..5)

...as shorthand for:

   for(int a=0; a < 5; a++) { /* use a */ }

Which is something I wrote far too many times in the 90's ;)
Jun 04 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/05/2013 12:17 AM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 23:48:32 +0200
 Timon Gehr <timon.gehr gmx.ch> wrote:

 Behind the scenes, the foreach loop is rewritten to something like:

 for(int __a=0;__a<5;__a++){
       int a = __a;
       writeln(&a);
 }


 Yea, I think the years I spent using C has corrupted my brain into
 seeing...

     foreach(int a; 0..5)

 ...as shorthand for:

     for(int a=0; a < 5; a++) { /* use a */ }

 Which is something I wrote far too many times in the 90's ;)


It was exactly that until recently. With the language version 
implemented by DMD 2.062, the behaviour in the OP was actually correct.
Jun 04 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/4/13 5:03 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 21:19:56 +0200
 "Idan Arye"<GenericNPC gmail.com>  wrote:


 Consider the following code. What will it print?

       auto arr=new ulong delegate()[5];

       foreach(i;0..arr.length){
           arr[i]=()=>i;
       }

       writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but
 it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all
 the delegates refer to the same `i` in their closures, and at the
 end of the loop that `i` is set to `5`.


 I think the problem is simply a misunderstanding of closures.



or something. It was a sort of a big deal, and a matter in which 
reasonable people were disagreeing.

Andrei
Jun 04 2013
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/05/2013 12:42 AM, Andrei Alexandrescu wrote:

 On 6/4/13 5:03 PM, Nick Sabalausky wrote:

 On Tue, 04 Jun 2013 21:19:56 +0200
 "Idan Arye"<GenericNPC gmail.com>  wrote:


 Consider the following code. What will it print?

       auto arr=new ulong delegate()[5];

       foreach(i;0..arr.length){
           arr[i]=()=>i;
       }

       writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but
 it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all
 the delegates refer to the same `i` in their closures, and at the
 end of the loop that `i` is set to `5`.


 I think the problem is simply a misunderstanding of closures.


 FWIW this


By 'this' I assume you mean aliasing semantics of a foreach variable?



 or something. It was a sort of a big deal, and a matter in which
 reasonable people were disagreeing.
...



D has a different lowering of foreach, and hence the above is an 
instance of http://d.puremagic.com/issues/show_bug.cgi?id=2043.
Jun 04 2013
prev sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/04/2013 03:42 PM, Andrei Alexandrescu wrote:



 or something. It was a sort of a big deal, and a matter in which
 reasonable people were disagreeing.


FWIW too, here is Dart's take on the problem: "Closures inside of Dart’s 
for loops capture the value of the index, avoiding a common pitfall 
found in JavaScript."

 
https://www.dartlang.org/docs/dart-up-and-running/contents/ch02.html#for-loops

Ali
Jun 04 2013
prev sibling next sibling parent reply Max Klyga <email domain.com> writes:
On 2013-06-04 19:19:56 +0000, Idan Arye said:


 snip




Closures should capture (and in most functional languages they do) by 
value (at least primitive types) to avoid confusion.
Jun 04 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/05/2013 08:44 AM, Max Klyga wrote:

 On 2013-06-04 19:19:56 +0000, Idan Arye said:


 snip





You mean the DMD behaviour? No way.





Again, no way.


that the foreach iteration variable exposed to the user was moved into 
the loop body. This is already the case in D, and yet the behaviour is 
still the broken one.

http://d.puremagic.com/issues/show_bug.cgi?id=2043.


 Closures should capture (and in most functional languages they do)  by value


In which functional languages?


 (at least primitive types) to avoid confusion.


Maybe there is a confusion of terminology. What do you mean when you say 
capture by value?
Jun 05 2013
prev sibling next sibling parent reply "JS" <js.mdnq gmail.com> writes:
On Tuesday, 4 June 2013 at 19:19:57 UTC, Idan Arye wrote:

 Consider the following code. What will it print?

     auto arr=new ulong delegate()[5];

     foreach(i;0..arr.length){
         arr[i]=()=>i;
     }

     writeln(arr.map!`a()`());

 It is natural to expect that it will print [0, 1, 2, 3, 4], but 
 it actually prints [5, 5, 5, 5, 5]. The reason is obvious - all 
 the delegates refer to the same `i` in their closures, and at 
 the end of the loop that `i` is set to `5`.

 While this makes some sense when you consider how the closures 
 are implemented, it is still not what you expect from that 
 code. `i` is supposed to be local to each iteration of the 
 loop, and here it "leaks" between the iterations.




The issue is rather simple but I'll make it more clear:

The delegates are called AFTER the for loop(since you are storing 
them in an array, presumably to use them later). i = 5(or even 
undetermined) after the loop, hence the behavior is correct.

It is wrong to use a local variable inside a delegate when the 
delegate is called outside the scope of that variable. This 
should technically be an error as it is undefined behavior.

I imagine the compiler isn't smart enough to know that you are 
storing delegates to be used outside the scope of i.

Basically when the lifetime of a variable has ended all 
references to it(such as  in a delegate) are invalid. Only 
delegate invocation referencing live variables will be valid.
Jul 23 2013
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 23 July 2013 at 13:37:26 UTC, JS wrote:

 It is wrong to use a local variable inside a delegate when the 
 delegate is called outside the scope of that variable. This 
 should technically be an error as it is undefined behavior.


AFAIK it is defined in D, compiler should move it to heap if it 
is captured by delegate and gets out of scope.
Jul 23 2013
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jul 23, 2013 at 03:37:24PM +0200, JS wrote:
[...]

 It is wrong to use a local variable inside a delegate when the
 delegate is called outside the scope of that variable. This should
 technically be an error as it is undefined behavior.


That's not true. According to TDPL, the compiler should detect such
references, and allocate these variables on the heap instead of the
stack, so that the delegate acts like a closure. The variables will be
GC'd once the delegate is no longer used.

This is part of what makes delegates so powerful -- you can use them to
capture the local context of a function and, in effect, access that
context long after the function call has finished.



 I imagine the compiler isn't smart enough to know that you are
 storing delegates to be used outside the scope of i.


[...]

It should know (and in fact does, at least in the simple cases I tried).
If it doesn't, that's a bug that should be filed in bugzilla.


T

-- 
PNP = Plug 'N' Pray
Jul 23 2013
prev sibling parent reply "Yota" <yotaxp thatGoogleMailThing.com> writes:
I just ran into the problem myself, and it really caught me off 
guard.

import std.stdio, core.thread;
void main() {
     foreach (i; 0..3) {
         immutable ii = i; // Copy for good measure?
         writeln("Main: ", ii);
         auto t = new Thread({
             writeln("Worker Start: ", ii);
             Thread.sleep(dur!"msecs"(1));
             writeln("Worker End: ", ii);
         });
         t.start();
     }
}

  Output:
Main: 0
Main: 1
Worker Start: 1
Main: 2
Worker Start: 2
Worker Start: 2
Worker End: 2
Worker End: 2
Worker End: 2

I caught on that the problem has to do with capturing stack 
variables that are going out of scope, but this is resulting a 
mutating immutable, without subverting the type system, and 
without a complaint from the compiler. (First iteration, 'ii' is 
captured while 0.  The captured 'ii' is then 1 when the thread 
begins, and 2 when it ends.)  This seems like a major problem in 
the type system, without even being a problem with the type 
system.

The following was suggested as a means to capture by value:

On Tuesday, 4 June 2013 at 19:19:57 UTC, Idan Arye wrote:

 There is an hackaround:

     auto arr=new ulong delegate()[5];

     foreach(i;0..arr.length){
         arr[i]=(j=>()=>j)(i);
     }

     writeln(arr.map!`a()`());



This indeed appears to work, but... why?  From the looks of it, 
it is doing the exact same thing.  It is capturing a local stack 
variable that immediately goes out of scope.  Why isn't the extra 
stack frame being overwritten each iteration?  Also, that 
anonymous function really looks like something that the compiler 
would inline, so is there even an extra stack frame?  I think I 
see why Idan called it a 'hackaround'.


I gather two things from this:

1. We really need a legit way to force capture-by-value on a 
variable by variable basis.  (Unless there's already another way. 
  Anyone?)

2. Why are immutable local variables not ALWAYS captured by 
value?  It's not like they are expected to change at all in the 
first place.  This would at least fix the promise that the type 
system makes regarding immutables. (Well... until pointers to 
immutables on the stack come into play.)
Sep 19 2013
parent "Colden Cullen" <ColdenCullen gmail.com> writes:
Have there been any updates on this issue? I can't seem to find 
anything else written about it, and it's still wreaking havoc 
with my team.
Nov 16 2014