www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Changing elements during foreach

reply "Krzysztof Ciebiera" <ciebie mimuw.edu.pl> writes:
Is the following compiler behavior consistent with language 
specification?

import std.stdio;

void main()
{
     int a[][] = [[1,2,3]];
     foreach(x; a)
     {
         x[0] = 0;
         x ~= 4;
     }
     writeln(a);
}

I understand why thw program could output [1,2,3] (like in C++ 
without &) or [0,2,3,4] (python, C++ ref). But [0,2,3]? It was 
unpleasant surprise.
Oct 21 2013
next sibling parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
21.10.2013 17:31, Krzysztof Ciebiera пишет:

 Is the following compiler behavior consistent with language specification?

 import std.stdio;

 void main()
 {
      int a[][] = [[1,2,3]];
      foreach(x; a)
      {
          x[0] = 0;
          x ~= 4;
      }
      writeln(a);
 }

 I understand why thw program could output [1,2,3] (like in C++ without
 &) or [0,2,3,4] (python, C++ ref). But [0,2,3]? It was unpleasant surprise.


It's expected.
foreach(ref x; a) == C++ with &
but without ref you get the first element of a - i.e. [1, 2, 3] then set 
x[0] = 0 so you get [0, 2, 3]. But when you append 4 you append it to 
local copy of x - because there is no 'ref'. So your local copy of x 
became [0, 2, 3, 4] but when scope is gone you get unmodified copy of a 
with old array [0, 2, 3]. Changing the first element without ref 
possible because you modify inner array which is reference itself.
Oct 21 2013
parent reply "Krzysztof Ciebiera" <ciebie mimuw.edu.pl> writes:
On Monday, 21 October 2013 at 10:41:38 UTC, Alexandr Druzhinin 
wrote:

 21.10.2013 17:31, Krzysztof Ciebiera пишет:

 void main()
 {
     int a[][] = [[1,2,3]];
     foreach(x; a)
     {
         x[0] = 0;
         x ~= 4;
     }
     writeln(a);
 }
 ...
 &) or [0,2,3,4] (python, C++ ref). But [0,2,3]? It was 
 unpleasant surprise.


 It's expected.
 foreach(ref x; a) == C++ with &
 but without ref you get the first element of a - i.e. [1, 2, 3] 
 then set x[0] = 0 so you get [0, 2, 3]. But when you append 4 
 you append it to local copy of x - because there is no 'ref'.


So, when exactly my local copy of data should be created? Now it 
is created during appending element to an array (when I switch 
instructions order, first append x ~= 4, then set x[0] to 0, as a 
result I get [1,2,3]). Maybe I should get a warning (like when 
hiding variable from outer scope)?
Oct 21 2013
parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
21.10.2013 17:55, Krzysztof Ciebiera пишет:

 On Monday, 21 October 2013 at 10:41:38 UTC, Alexandr Druzhinin wrote:

 21.10.2013 17:31, Krzysztof Ciebiera пишет:

 void main()
 {
     int a[][] = [[1,2,3]];
     foreach(x; a)
     {
         x[0] = 0;
         x ~= 4;
     }
     writeln(a);
 }
 ...
 &) or [0,2,3,4] (python, C++ ref). But [0,2,3]? It was unpleasant
 surprise.


 It's expected.
 foreach(ref x; a) == C++ with &
 but without ref you get the first element of a - i.e. [1, 2, 3] then
 set x[0] = 0 so you get [0, 2, 3]. But when you append 4 you append it
 to local copy of x - because there is no 'ref'.


 So, when exactly my local copy of data should be created? Now it is
 created during appending element to an array (when I switch instructions
 order, first append x ~= 4, then set x[0] to 0, as a result I get
 [1,2,3]). Maybe I should get a warning (like when hiding variable from
 outer scope)?


If you switch instruction order you create local copy and then set x[0] 
in local copy so original is unchanged. But local copy creating depends 
on several thing and happens not every appending in general. Your way is 
not D-ish one. What do you want to do?
Oct 21 2013
parent reply "Krzysztof Ciebiera" <ciebie mimuw.edu.pl> writes:
On Monday, 21 October 2013 at 14:59:54 UTC, Alexandr Druzhinin 
wrote:

 If you switch instruction order you create local copy and then 
 set x[0] in local copy so original is unchanged. But local copy 
 creating depends on several thing and happens not every 
 appending in general. Your way is not D-ish one. What do you 
 want to do?


I have an array of solutions to a problem. Each solution is an 
array of elements (not numbers, but objects of some classes). 
What I want to do is: I want to take a look at every single 
solution and extend it if it is possible (eg. if solution 
contains obbects a,b,c it can also contain d). The whole thing is 
rather big so I don't want to have any nice functional-style 
solution which will involve copying data. Especially since I have 
to perform similar extensions over and over again.

I understand slices now and I don't find it consistent with "no 
shoot in the foot by default" statement.
--
KC
Oct 21 2013
next sibling parent reply "qznc" <qznc web.de> writes:
On Monday, 21 October 2013 at 16:22:29 UTC, Krzysztof Ciebiera 
wrote:

 I understand slices now and I don't find it consistent with "no 
 shoot in the foot by default" statement.


I agree. The pitfalls are well understood, yet everybody seems to 
love them. Ok, compared to C array they are an improvement due to 
bounds checking. If the elements are const or immutable (string) 
everything is fine, but write+append is basically 
implementation-defined behavior.

Once there is a proper std.collections, I will probably adapt my 
tutorial to recommend a safe alternative (ArrayList?).
Oct 21 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, October 21, 2013 21:16:00 qznc wrote:

 On Monday, 21 October 2013 at 16:22:29 UTC, Krzysztof Ciebiera
 
 wrote:

 I understand slices now and I don't find it consistent with "no
 shoot in the foot by default" statement.


 
 I agree. The pitfalls are well understood, yet everybody seems to
 love them. Ok, compared to C array they are an improvement due to
 bounds checking. If the elements are const or immutable (string)
 everything is fine, but write+append is basically
 implementation-defined behavior.
 
 Once there is a proper std.collections, I will probably adapt my
 tutorial to recommend a safe alternative (ArrayList?).


Just don't use slices when appending. It's only an issue when you're appending 
and relying on slices continuing to refer to the same array, and that's not 
going to work. Slices are primarily for reading, not writing. Using a 
container doesn't change that. It just makes it so that you can't even attempt 
to append to a slice, because the slice is a different type than that container 
and won't support appending.

- Jonathan m Davis
Oct 21 2013
parent reply "ixid" <nuaccount gmail.com> writes:
On Monday, 21 October 2013 at 19:37:47 UTC, Jonathan M Davis 
wrote:

 On Monday, October 21, 2013 21:16:00 qznc wrote:

 On Monday, 21 October 2013 at 16:22:29 UTC, Krzysztof Ciebiera
 
 wrote:

 I understand slices now and I don't find it consistent with 
 "no
 shoot in the foot by default" statement.


 
 I agree. The pitfalls are well understood, yet everybody seems 
 to
 love them. Ok, compared to C array they are an improvement due 
 to
 bounds checking. If the elements are const or immutable 
 (string)
 everything is fine, but write+append is basically
 implementation-defined behavior.
 
 Once there is a proper std.collections, I will probably adapt 
 my
 tutorial to recommend a safe alternative (ArrayList?).


 Just don't use slices when appending. It's only an issue when 
 you're appending
 and relying on slices continuing to refer to the same array, 
 and that's not
 going to work. Slices are primarily for reading, not writing. 
 Using a
 container doesn't change that. It just makes it so that you 
 can't even attempt
 to append to a slice, because the slice is a different type 
 than that container
 and won't support appending.

 - Jonathan m Davis


What would be the issue/s with disallowing appending to slices? 
So you'd have to explicitly duplicate before you could append.
Oct 21 2013
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, October 21, 2013 22:14:08 ixid wrote:

 What would be the issue/s with disallowing appending to slices?
 So you'd have to explicitly duplicate before you could append.


All arrays are slices. There is no difference between the two. It's just that 
if you start appending to one, it will eventually no longer point to the same 
block of memory as any other arrays that pointed to the block of memory that 
it was pointing to. So, there's no way for the compiler to know whether you're 
trying to append to an array that was created by slicing another array as 
opposed to being directly allocated.

Also, disallowing appending to slices would be needlessly restrictive even if 
you could have that restriction. There's nothing wrong whatsoever with 
appending to a slice. It just means that you have to be aware that it may end 
up reallocating and not be a slice of the same memory anymore. So, if you want 
to avoid that, then don't append to the array. If you don't mind that it 
reallocates for what your code is doing, then it's not a problem. All it means 
is that you have to be aware that appending to an array may make it so that it 
will reallocate. Whether that's good or bad depends on what you're doing.

- Jonathan M Davis
Oct 21 2013
prev sibling parent "qznc" <qznc web.de> writes:
On Monday, 21 October 2013 at 20:14:09 UTC, ixid wrote:

 On Monday, 21 October 2013 at 19:37:47 UTC, Jonathan M Davis 
 wrote:

 On Monday, October 21, 2013 21:16:00 qznc wrote:

 On Monday, 21 October 2013 at 16:22:29 UTC, Krzysztof Ciebiera
 
 wrote:

 I understand slices now and I don't find it consistent with 
 "no
 shoot in the foot by default" statement.


 
 I agree. The pitfalls are well understood, yet everybody 
 seems to
 love them. Ok, compared to C array they are an improvement 
 due to
 bounds checking. If the elements are const or immutable 
 (string)
 everything is fine, but write+append is basically
 implementation-defined behavior.
 
 Once there is a proper std.collections, I will probably adapt 
 my
 tutorial to recommend a safe alternative (ArrayList?).


 Just don't use slices when appending. It's only an issue when 
 you're appending
 and relying on slices continuing to refer to the same array, 
 and that's not
 going to work. Slices are primarily for reading, not writing. 
 Using a
 container doesn't change that. It just makes it so that you 
 can't even attempt
 to append to a slice, because the slice is a different type 
 than that container
 and won't support appending.


 What would be the issue/s with disallowing appending to slices? 
 So you'd have to explicitly duplicate before you could append.


Appending itself is not the problem. Append + write is 
indeterministic.

  int[] a = [1,2,3];
  int[] b = a;
  a ~= [4,5]; // append
  a[0] = 42; // write
  assert (b[0] == 42) // indeterministic, implementation-specific

It depends on the capacity during the append. If a reallocation 
happened, then b[0]==1. Otherwise b[0]==42.

If you can forbid the write (e.g. immutable(char)[] aka string), 
you can append as much as you want. If you can disallow the 
append (convention, no type system support for this), you can 
write as much as you want.

For something like Javas ArrayList, the behavior would be a 
deterministic b[0]==42, but also a.length==b.length.
Oct 21 2013
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 21 October 2013 at 16:22:29 UTC, Krzysztof Ciebiera 
wrote:

 What I want to do is: I want to take a look at every single 
 solution and extend it if it is possible (eg. if solution 
 contains obbects a,b,c it can also contain d).


void main()
{
	import std.stdio;
	int[][] data = [ [ 0, 1, 2], [2, 3, 4] ];
	foreach(ref arr; data)
	{
		if (arr[0] == 2)
			arr ~= 5;
	}
	writeln(data);
}

?

(original arrays can be also allocated with extra capacity to 
avoid copy-allocation upon extending)
Oct 21 2013
prev sibling next sibling parent "qznc" <qznc web.de> writes:
On Monday, 21 October 2013 at 10:31:51 UTC, Krzysztof Ciebiera 
wrote:

 Is the following compiler behavior consistent with language 
 specification?

 import std.stdio;

 void main()
 {
     int a[][] = [[1,2,3]];
     foreach(x; a)
     {
         x[0] = 0;
         x ~= 4;
     }
     writeln(a);
 }

 I understand why thw program could output [1,2,3] (like in C++ 
 without &) or [0,2,3,4] (python, C++ ref). But [0,2,3]? It was 
 unpleasant surprise.


The behavior of D's slices is often unintuitive. To understand 
them, read this article: http://dlang.org/d-array-article.html
Oct 21 2013
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Krzysztof Ciebiera:


 Is the following compiler behavior consistent with language 
 specification?


Changing elements during foreach is something to avoid, perhaps 
I'd like it to be statically forbidden. If you add to this the 
reference-struct nature of arrays, you get in troubles.

Bye,
bearophile
Oct 21 2013