www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 6652] New: foreach parameter with number range is always ref

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652

           Summary: foreach parameter with number range is always ref
           Product: D
           Version: D2
          Platform: Other
        OS/Version: FreeBSD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: dawg dawgfoto.de



void main() {
  size_t cnt;
  foreach(ulong n; 0 .. 10)
  {
    ++n;
    ++cnt;
  }
  assert(cnt == 10);

  cnt = 0;
  foreach(ref ulong n; 0 .. 10)
  {
    ++n;
    ++cnt;
  }
  assert(cnt == 5);
}

---

As this is rewritten in terms of a for loop all writes to n will
alter the loop.
A hidden copy of n is needed for non-ref parameters to match the range
foreach semantic.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 12 2011
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc



Recently I have started a very long thread about this problem:

http://www.digitalmars.com/d/archives/digitalmars/D/About_foreach_loops_138630.html

http://www.digitalmars.com/d/archives/digitalmars/D/Re_About_foreach_loops_138666.html


An alternative and probably a bit better solution is to think of 0..10 as a
immutable entity (just like the integer number "1" is immutable, likewise a
range of numbers is immutable), so the foreach index is a const (the compiler
keeps only one index, for efficiency, and modifies this const index):


foreach (int i; 0 .. 10) {
    i++; // forbidden, i is const
}


If you want to modify the index variable inside the loop, then you use a for()
loop.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 12 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Making a const/immutable copy is not the right solution to this.
Instead a mutable copy of a hidden loop variable should be made.
Being a copy is the common behavior for non-ref foreach arguments,
to my surprise it has even become my intuitive assumption of what's happening.
The old behavior can be achieved through a ref argument.

not possible using const:
foreach(i; 1 .. 10)
  while(i--) { do some }

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 12 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652





 Making a const/immutable copy is not the right solution to this.
Keep in mind that foreach(i;0..10) must have *zero* abstraction penalty over a for loop even with non-optimizing D compilers, because it's meant to replace for loops everywhere possible. The more abstraction you put into foreach there higher the probability it will not have zero abstraction penalty (currently it has a bit of penalty for nested loops, sometimes).
 not possible using const:
 foreach(i; 1 .. 10)
   while(i--) { do some }
Use a for loop :-) -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 12 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




https://github.com/D-Programming-Language/dmd/pull/377

Foreach arguments behave like function arguments. Here they don't.
The variable can be optimized out, if no altering happens.
This will not happen in a debug build, where it is irrelevant in comparison to
every variable being accessed through the stack.

You can use ref, if you're having too expensive copies
foreach(ref const i; iter(0) .. iter(10)) as with every other foreach argument.

Most important it has an explicit rule, that one can alter the loop index
through using a ref index.
foreach(ref idx, v; [0, 1, 2, 3, 4, 5])
  idx += stride - 1;

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 13 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652





 https://github.com/D-Programming-Language/dmd/pull/377
 - I've double checked that a simple size_t index is optimized out if unaltered
I suggest you to check it fifteen more times, using 4 nested foreach, with some code inside the bodies, with other data types (ubyte, short, ulong, real, double, etc), etc. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Sep 13 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




It really is not that much an issue of performance.
The compiler should be able to eliminate dead assignments
and *& for ref parameters.

The issue is that of breaking code. I don't know any feasible
solution to attach a deprecation to this now and change it later.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 13 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Masahiro Nakagawa <repeatedly gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |repeatedly gmail.com



08:47:41 PDT ---
I hit this issue today.

Current behavior is different from foreach semantics.
So, I agree dawg opinion.
We should fix this bug!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 11 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
           Platform|Other                       |All
         OS/Version|FreeBSD                     |All



I've opened three pulls:
https://github.com/D-Programming-Language/dmd/pull/1008
https://github.com/D-Programming-Language/dmd/pull/1009
https://github.com/D-Programming-Language/dmd/pull/1010

To reduce breaking of existing codes,
1. Warn to modifying loop variable in foreach body.
   It is shown only when -w switch is specified.
2. Deprecate modifying loop variable in foreach body.
   If user specifies -d switch, it is allowed.
3. Allow modifying loop variable in foreach body, and it does not affect to
   the number of iterations of the loop.

How about?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652






 To reduce breaking of existing codes,
 1. Warn to modifying loop variable in foreach body.
    It is shown only when -w switch is specified.
 2. Deprecate modifying loop variable in foreach body.
    If user specifies -d switch, it is allowed.
 3. Allow modifying loop variable in foreach body, and it does not affect to
    the number of iterations of the loop.
This is great, you are the best Kenji Hara. I prefer the number 2. I think it breaks none of my programs. The number 3 is a trap, because it silently changes the semantics of old D code. And it's bug-prone for new D programmers too because they can change the variable by mistake. Generally immutable variables are safer. Are you able and willing to compile the whole Phobos with the option number 2? So we can see how often Phobos code change the foreach-on-range iteration variable. What's the semantics of this, now? foreach (ref int i; 0 .. 10) i++; -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code



---


 
 To reduce breaking of existing codes,
 1. Warn to modifying loop variable in foreach body.
    It is shown only when -w switch is specified.
 2. Deprecate modifying loop variable in foreach body.
    If user specifies -d switch, it is allowed.
 3. Allow modifying loop variable in foreach body, and it does not affect to
    the number of iterations of the loop.
This is great, you are the best Kenji Hara. I prefer the number 2. I think it breaks none of my programs.
They are the phases to change behavior. I think we should allow modifying loop variable in foreach body, but it should not affect to iteration.
 The number 3 is a trap, because it silently changes the semantics of old D
 code. And it's bug-prone for new D programmers too because they can change the
 variable by mistake. Generally immutable variables are safer.
 Are you able and willing to compile the whole Phobos with the option number 2?
 So we can see how often Phobos code change the foreach-on-range iteration
 variable.
http://d.puremagic.com/test-results/pulls.ghtml See auto tester. With all pull request, Phobos compile succeeds. So there is no code that changes the foreach-on-range iteration variable. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Ryuichi OHORI <r.97all gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |r.97all gmail.com



---

 I prefer the number 2. I think it breaks none of my programs.
 
 The number 3 is a trap, because it silently changes the semantics of old D
 code. And it's bug-prone for new D programmers too because they can change the
 variable by mistake. Generally immutable variables are safer.
In my point of view, as a newcomer to D, more bug-prone is the current behavior. Foreach statement provides iteration over arrays, ranges, etc, and notation of range "0..n" also *looks like* a collection. So, foreach range statements should work like foreach over collection. I have wrote to stuck in my program: foreach (i; 0..M^^n) { foreach (j; 0..n) { a[j] = i % M; i /= M; } // operations which use a but i } which I wrote in Python before: for i in range(M**n): for j in range(n): a[j] = i % M i /= M and was sad to see an infinite loop. Even a new programmer *intends* to change the value of i when changing, if not just a typo. If someone want to affect loop, s/he can write i = 0; while (i < 10) { // operations which change the value of i i += 1; } -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652






 They are the phases to change behavior.
I see.
 I think we should allow modifying loop
 variable in foreach body, but it should not affect to iteration.
Generally changing the iteration variable isnt't a very good idea. It looks bug-prone, like modifying function arguments inside functions :-) foreach (i; 0 .. 10) { i++; writeln(i); }
 So there is no
 code that changes the foreach-on-range iteration variable.
Good. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




If phase 3 will be accepted, I hope this syntax too will be accepted:

foreach (const i; 0 .. 10) {}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652






 In my point of view, as a newcomer to D, more bug-prone is the current
 behavior.
Of course. But here the comparison wasn't between the current behavour and the phase 3. It was mostly a comparison between the phase 2 and phase 3. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




---

 If phase 3 will be accepted, I hope this syntax too will be accepted:
 
 foreach (const i; 0 .. 10) {}
I think it should be allowed. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




---

 In my point of view, as a newcomer to D, more bug-prone is the current
 behavior.
 Of course. But here the comparison wasn't between the current behavour and the
 phase 3. It was mostly a comparison between the phase 2 and phase 3.
What followed it was my opinion to it:
 Even a new programmer *intends* to change the value of i when changing, if not
just a typo.
The problem is that changing the value of a variable affects loop, not changing itself. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 16 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Just a note.

void main() {
    import std.stdio;
    auto array = [10, 20, 30, 40, 50];
    foreach (i, item; array) {
        writeln(item);
        i++;
    }
}


Currently it prints:

10
30
50

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 17 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




How about?
Sounds great. It doesn't break code and allows us to fix this finally.
foreach (i, item; array)
Yeah, it should apply to the index variable as well. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 19 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




---

How about?
Sounds great. It doesn't break code and allows us to fix this finally.
foreach (i, item; array)
Yeah, it should apply to the index variable as well.
-- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 21 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




---


How about?
Sounds great. It doesn't break code and allows us to fix this finally.
foreach (i, item; array)
Yeah, it should apply to the index variable as well.
(Sorry, I accidentally pressed the "commit" button...) OK. Now, my pull requests also care the key of array in iteration. You can see the test case: https://github.com/D-Programming-Language/dmd/pull/1008/files#L5L-1 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 21 2012
prev sibling next sibling parent "IK" <istvan.karolyi rocketmail.com> writes:
 foreach parameter with number range is always ref
I noticed this by getting into infinite loops. But that, more than anything, hints of very bad programming style. :)
Jun 21 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/8ea1a0ef35b9cef3047699396487f4d2fe149a39
fix Issue 6652 - 1. Warn modifying non ref variable if -w is specified.

https://github.com/D-Programming-Language/dmd/commit/38a0a5141a3455395e8b9571a57bf85ed698c6b3


fix Issue 6652 - 1. Warn modifying non ref variable if -w is specified.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 21 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Commits pushed to master
Great. So what's the time frame until deprecation, 6 month? -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 22 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




---
*** Issue 6214 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 02 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




See also:

http://forum.dlang.org/thread/znbtczbgipqqzllafmzk forum.dlang.org

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 03 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/780ce35e8338435e5d063da2cc93360d7f0aecbd
Appendix for Issue 6652

If *implicit ref* foreach key doesn't have a mutable type, warning is not
necessary.

https://github.com/D-Programming-Language/dmd/commit/882324ed350d087949e000a78b32c75396ea003a


Appendix for Issue 6652

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 13 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/ac7fd08a9e08fb333d9341f21bdb19ba42e1ee38
fix Issue 6652 - 2. Deprecate modifying non ref variable.

https://github.com/D-Programming-Language/dmd/commit/ae95b0f935e00ec03224180a24c4396485367eb8


fix Issue 6652 - 2. Deprecate modifying non ref variable.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 24 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Andrej Mitrovic <andrej.mitrovich gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |andrej.mitrovich gmail.com
         Resolution|                            |FIXED



13:16:06 PST ---
This is now a deprecated feature. I've closed the report, but perhaps it should
stay open until the feature is completely gone?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652




Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/6e018c83c2fcd138c6596ca63a3fbf9140e9c69c
fix Issue 6652 - 3. Change modifying non ref variable allowed and not affect to
loop

https://github.com/D-Programming-Language/dmd/commit/ad74ad3bd8ae763db3f68c0a6647cb1909d477dd


fix Issue 6652 - 3. Change modifying non ref variable allowed and not affect to
loop

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 06 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugzilla digitalmars.com
           Severity|normal                      |enhancement



21:36:00 PST ---


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 06 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=6652


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |lat7h virginia.edu



*** Issue 2543 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 01 2013