www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Array slice length confusion

reply Tim Matthews <tim.matthews7 gmail.com> writes:
First off if this has been asked so many times please just ignore this. 
You should be able to collapse this whole subject in most good new 
readers plus it is in d.learn out of the way so I expect no "ahh here we 
go again" sort of replies.

Anyway I like how you can get a slice of array that is just a view into 
the original. If I modify the slice, it modifies the full original array 
which can be very useful. If however I modify the length of the slice 
which I was doing in an attempt to extend the view, it becomes it's own 
array.

This example should help explain what I mean:

module test;

import std.stdio;

void main()
{
     char[5] a = "Hello";
     char[] b;
     b = a[1..3]; //b is a view into a
     writeln(a); //(Hello)
     writeln(b); //(el)
     b[0] = 'a'; //no copy on write
     writeln(a); //(Hallo)
     writeln(b); //(al)
     b.length = b.length+2; //COW on length change?
     b[0] = 'z'; //does not modify a
     writeln(a); //(Hallo)
     writeln(b); //(zl)
}


Was this a design choice, bug, undefined behavior, etc...?
Jul 07 2009
next sibling parent reply Kagamin <spam here.lot> writes:
Tim Matthews Wrote:


 Was this a design choice, bug, undefined behavior, etc...?


Design choice. In order to widen slice to should recreate it from the original
array.
Jul 08 2009
parent reply Kagamin <spam here.lot> writes:
Kagamin Wrote:


 Tim Matthews Wrote:
 

 Was this a design choice, bug, undefined behavior, etc...?


 
 Design choice. In order to widen slice to should recreate it from the original
array.


For this I use begin markers:

ubyte[] buff=new ubyte[100];
int half=buff.length/2;
ubyte[] buff1=buff[0..half];
ubyte[] buff2=buff[half..$];
Jul 08 2009
parent Tim Matthews <tim.matthews7 gmail.com> writes:
Kagamin wrote:

 For this I use begin markers:
 
 ubyte[] buff=new ubyte[100];
 int half=buff.length/2;
 ubyte[] buff1=buff[0..half];
 ubyte[] buff2=buff[half..$];


I actually think this has nothing to do with my OP.
Jul 08 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 08 Jul 2009 02:48:31 -0400, Tim Matthews <tim.matthews7 gmail.com>  
wrote:


 Was this a design choice, bug, undefined behavior, etc...?


design choice.

A slice *is* an array in the current design.  Increasing the length may or  
may not make a copy of it.  An array slice doesn't know it was originally  
part of another array (i.e. there are no references back to the original  
array) so how would it know that there is data around it?  The only  
exception is appending to a prefix slice.  If a slice starts at the front  
of an allocated array, the runtime cannot tell that it was not the  
original array, so it clobbers data that was in the original array:

char[] str = "blah".dup;
char[] sl = str[0..1];
sl ~= "r";
assert(str == "brah");

BTW, all of this is defined behavior, see  
http://www.digitalmars.com/d/2.0/arrays.html#resize

-Steve
Jul 08 2009
parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
Steven Schveighoffer wrote:

 On Wed, 08 Jul 2009 02:48:31 -0400, Tim Matthews 
 <tim.matthews7 gmail.com> wrote:
 

 Was this a design choice, bug, undefined behavior, etc...?


 
 design choice.
 
 A slice *is* an array in the current design.  Increasing the length may 
 or may not make a copy of it.  An array slice doesn't know it was 
 originally part of another array (i.e. there are no references back to 
 the original array) so how would it know that there is data around it?  
 The only exception is appending to a prefix slice.  If a slice starts at 
 the front of an allocated array, the runtime cannot tell that it was not 
 the original array, so it clobbers data that was in the original array:
 
 char[] str = "blah".dup;
 char[] sl = str[0..1];
 sl ~= "r";
 assert(str == "brah");
 
 BTW, all of this is defined behavior, see 
 http://www.digitalmars.com/d/2.0/arrays.html#resize
 
 -Steve


I thought a slice would behave slighty different due to some sort of 
meta data that is a separate area of memory so it doesn't effect D's abi.

Anyway if anyone gets here through a search engine the correct code 
assuming you can guarantee it is a slice and that doesn't go out of 
bounds. It is a systems language after all :)

module test;

import std.stdio;

void resizeView(T)(ref T[] slice, size_t newLength)
{
     (cast(size_t*)(&slice))[0] = newLength;
}

void main()
{
     char[5] a = "hello";
     char[] b = a[1..3];
     writeln(a); //(hello)
     writeln(b); //(el)
     resizeView(b, b.length+1);
     writeln(a); //(hello)
     writeln(b); //(ell)
}
Jul 08 2009
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 08 Jul 2009 23:09:52 -0400, Tim Matthews <tim.matthews7 gmail.com>  
wrote:


 Steven Schveighoffer wrote:

 On Wed, 08 Jul 2009 02:48:31 -0400, Tim Matthews  
 <tim.matthews7 gmail.com> wrote:


 Was this a design choice, bug, undefined behavior, etc...?


  design choice.
  A slice *is* an array in the current design.  Increasing the length  
 may or may not make a copy of it.  An array slice doesn't know it was  
 originally part of another array (i.e. there are no references back to  
 the original array) so how would it know that there is data around it?   
 The only exception is appending to a prefix slice.  If a slice starts  
 at the front of an allocated array, the runtime cannot tell that it was  
 not the original array, so it clobbers data that was in the original  
 array:
  char[] str = "blah".dup;
 char[] sl = str[0..1];
 sl ~= "r";
 assert(str == "brah");
  BTW, all of this is defined behavior, see  
 http://www.digitalmars.com/d/2.0/arrays.html#resize
  -Steve


 I thought a slice would behave slighty different due to some sort of  
 meta data that is a separate area of memory so it doesn't effect D's abi.

 Anyway if anyone gets here through a search engine the correct code  
 assuming you can guarantee it is a slice and that doesn't go out of  
 bounds. It is a systems language after all :)

 module test;

 import std.stdio;

 void resizeView(T)(ref T[] slice, size_t newLength)
 {
      (cast(size_t*)(&slice))[0] = newLength;
 }

 void main()
 {
      char[5] a = "hello";
      char[] b = a[1..3];
      writeln(a); //(hello)
      writeln(b); //(el)
      resizeView(b, b.length+1);
      writeln(a); //(hello)
      writeln(b); //(ell)
 }


This is really not a good idea.  You've removed one of the core features  
of the array -- memory safety.  Doing this is just asking for memory  
corruption.  You should either re-slice the original array, or create a  
type that has a reference to the original array so it can be resliced:

start with:

struct TimSlice(T)
{
   T[] slice;
   alias slice this;
   T[] srcArray;

   // override length property
   size_t length() {return slice.length;}
   void length(size_t) {/* use srcArray here to re-slice */}
}

This is assuming you are using D2, of course.  With D1, it's more  
difficult.

-Steve
Jul 08 2009
parent Tim Matthews <tim.matthews7 gmail.com> writes:
Steven Schveighoffer wrote:

 This is really not a good idea.  You've removed one of the core features 
 of the array -- memory safety.  Doing this is just asking for memory 
 corruption.  You should either re-slice the original array, or create a 
 type that has a reference to the original array so it can be resliced


I explicitly said that it was unsafe code and off course most situations 
would be better suited with the extra abstraction or re slice the original.
Jul 08 2009
prev sibling parent reply Kagamin <spam here.lot> writes:
Tim Matthews Wrote:


 I thought a slice would behave slighty different due to some sort of 
 meta data that is a separate area of memory so it doesn't effect D's abi.
 


Current plan is to introduce new type - array - into the language.
Jul 09 2009
parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
Kagamin wrote:

 Tim Matthews Wrote:
 

 I thought a slice would behave slighty different due to some sort of 
 meta data that is a separate area of memory so it doesn't effect D's abi.


 Current plan is to introduce new type - array - into the language.


Do you know the ng posts or where ever that was posted?
Jul 09 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Tim Matthews wrote:

 Kagamin wrote:

 Tim Matthews Wrote:


 I thought a slice would behave slighty different due to some sort of
 meta data that is a separate area of memory so it doesn't effect D's
 abi.


 Current plan is to introduce new type - array - into the language.


 
 Do you know the ng posts or where ever that was posted?


Best bet would be to search for "T[new]".
Jul 09 2009
parent grauzone <none example.net> writes:
Daniel Keep wrote:

 
 Tim Matthews wrote:

 Kagamin wrote:

 Tim Matthews Wrote:


 I thought a slice would behave slighty different due to some sort of
 meta data that is a separate area of memory so it doesn't effect D's
 abi.


 Current plan is to introduce new type - array - into the language.


 Do you know the ng posts or where ever that was posted?


 
 Best bet would be to search for "T[new]".


Superseded by Andrei's Array struct.
Jul 09 2009
prev sibling parent reply BCS <none anon.com> writes:
Hello Tim,


 Anyway I like how you can get a slice of array that is just a view
 into the original. If I modify the slice, it modifies the full
 original array which can be very useful. If however I modify the
 length of the slice which I was doing in an attempt to extend the
 view, it becomes it's own array.
 


[...]

 Was this a design choice, bug, undefined behavior, etc...?
 


you can grab a larger chunk like this


char[] buff1 = something;
char[] buff2 = buff1[10..20];

buff2 = buff2.ptr[0..20];

note this is unsafe and will give you no warnings if you extend past the 
end of buff1;
Jul 08 2009
parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
BCS wrote:

 Hello Tim,
 

 Anyway I like how you can get a slice of array that is just a view
 into the original. If I modify the slice, it modifies the full
 original array which can be very useful. If however I modify the
 length of the slice which I was doing in an attempt to extend the
 view, it becomes it's own array.


 [...]

 Was this a design choice, bug, undefined behavior, etc...?


 
 you can grab a larger chunk like this
 
 
 char[] buff1 = something;
 char[] buff2 = buff1[10..20];
 
 buff2 = buff2.ptr[0..20];
 
 note this is unsafe and will give you no warnings if you extend past the 
 end of buff1;
 
 


I know that but I when I said I was trying to extend the view I really 
mean that I was just trying to extend the view. Also depending on what 
news reader you use you should see that it branched off in a particular 
direction to a solution.
Jul 09 2009
parent reply BCS <none anon.com> writes:
Hello Tim,


 BCS wrote:
 

 Hello Tim,
 

 Anyway I like how you can get a slice of array that is just a view
 into the original. If I modify the slice, it modifies the full
 original array which can be very useful. If however I modify the
 length of the slice which I was doing in an attempt to extend the
 view, it becomes it's own array.
 


 [...]
 

 Was this a design choice, bug, undefined behavior, etc...?
 


 you can grab a larger chunk like this
 
 char[] buff1 = something;
 char[] buff2 = buff1[10..20];
 buff2 = buff2.ptr[0..20];
 
 note this is unsafe and will give you no warnings if you extend past
 the end of buff1;
 


 I know that but I when I said I was trying to extend the view I really
 mean that I was just trying to extend the view.


If that doesn't "extend the view", then I'm not understanding what you are 
asking for.

the above should compile to the sudocode:

t = &(buff.ptr[0]);
L = 20-0;
buff2.{ptr, length} = {t, L};

and should optimize to just an assignment to length.


 Also depending on what
 news reader you use you should see that it branched off in a
 particular direction to a solution.
 


If you are referring the your response to Steven Schveighoffer, that amounts 
to the same thing I proposed but is even less safe (I don't think the layout 
of an array reference is speced) and harder to understand.
Jul 09 2009
parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
Sorry I misread (lacking sleep). I can just re slice from the slice, 
silly me.

import std.stdio;

void extendSlice(T)(ref T[] slice, uint extendCount)
{
     slice = slice.ptr[0..slice.length+extendCount];
}

void main()
{
     char[] a = "Hello".dup;
     char[] b;
     b = a[1..3]; //b is a view into a
     writeln(a); //(Hello)
     writeln(b); //(el)
     extendSlice(b,1);
     writeln(a); //(Hello)
     writeln(b); //(ell)
}
Jul 09 2009
parent BCS <none anon.com> writes:
Hello Tim,


 Sorry I misread (lacking sleep).


Been there, done that. :oz
Jul 09 2009