www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Any way to set len and ptr in dyn arrays?

reply nobody <nobody mailinator.com> writes:
The only way I have found is to use a struct/union kludge:

// redefine the dyn array struct
struct sArrayKludge
{
   int len;
   void* ptr;
}

// use a union to allow reaching inside
union uArrayKludge
{
   byte[] myArray;
   sArrayKludge myArrayInternals;
}

byte someFunc()
{
   // declare the union where I would declare myArray
   uArrayKludge theArray;

   int[] buf = cast(byte[]) read("file.ext");

   // use the union where I would use the array
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

   // now I can use the ith byte
   return theArray.myArray[buf[0]+buf[1]];
}



Thanks in advance.
Aug 07 2006
next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
nobody wrote:

   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];


erm ...

foo = (&buf[2])[0 .. buf[0] * buf[1]];
Aug 07 2006
parent reply nobody <nobody mailinator.com> writes:
Tom S wrote:

 nobody wrote:

   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];


 erm ...

 foo = (&buf[2])[0 .. buf[0] * buf[1]];


Thanks for your answer. I understand that would ultimately have 
the same result. My question is about whether that directly sets 
the len and ptr fields or are does more stuff happen. If so what 
is it? If not then why is there no direct access to the ptr and 
len fields?
Aug 08 2006
parent reply Frank Benoit <keinfarbton nospam.xyz> writes:
nobody schrieb:

 Tom S wrote:

 nobody wrote:

   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];


 erm ...

 foo = (&buf[2])[0 .. buf[0] * buf[1]];


 
 Thanks for your answer. I understand that would ultimately have the same
 result. My question is about whether that directly sets the len and ptr
 fields or are does more stuff happen. If so what is it? If not then why
 is there no direct access to the ptr and len fields?
 


This builds a slice, which is like setting ptr/length of an array. This
is called slicing. Instead of Toms example, I think you can also write this:
foo = buf[ 2 .. buf[0] * buf[1] + 2 ];

Refering to http://www.digitalmars.com/d/arrays.html
slicing:
b = a[1..3];
array copy:
s[] = t[];
s[1..2] = t[0..1];
Aug 08 2006
parent reply nobody <nobody mailinator.com> writes:
Frank Benoit wrote:

 nobody schrieb:

 Tom S wrote:

 nobody wrote:

   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];


 erm ...

 foo = (&buf[2])[0 .. buf[0] * buf[1]];


 Thanks for your answer. I understand that would ultimately have the same
 result. My question is about whether that directly sets the len and ptr
 fields or are does more stuff happen. If so what is it? If not then why
 is there no direct access to the ptr and len fields?


 
 This builds a slice, which is like setting ptr/length of an array. This
 is called slicing. Instead of Toms example, I think you can also write this:
 foo = buf[ 2 .. buf[0] * buf[1] + 2 ];
 


Thanks to you also for your response. I was trying to suggest 
before that I understand how slicing ends up -- ie with ptr and 
len as I desired. I am worried about what happens while the 
slicing happens. I am worried about any gotchas.
Aug 08 2006
parent reply Frank Benoit <keinfarbton nospam.xyz> writes:
 This builds a slice, which is like setting ptr/length of an array. This
 is called slicing. Instead of Toms example, I think you can also write
 this:
 foo = buf[ 2 .. buf[0] * buf[1] + 2 ];


 
 Thanks to you also for your response. I was trying to suggest before
 that I understand how slicing ends up -- ie with ptr and len as I
 desired. I am worried about what happens while the slicing happens. I am
 worried about any gotchas.


I my understanding this is exactly like setting the ptr and length.
One side effect could be, that the compiler perhaps inserts array
boundary check. But I don't know that.
Aug 08 2006
parent reply BCS <BCS pathlink.com> writes:
Frank Benoit wrote:

This builds a slice, which is like setting ptr/length of an array. This
is called slicing. Instead of Toms example, I think you can also write
this:
foo = buf[ 2 .. buf[0] * buf[1] + 2 ];


Thanks to you also for your response. I was trying to suggest before
that I understand how slicing ends up -- ie with ptr and len as I
desired. I am worried about what happens while the slicing happens. I am
worried about any gotchas.


 
 
 I my understanding this is exactly like setting the ptr and length.
 One side effect could be, that the compiler perhaps inserts array
 boundary check. But I don't know that.


I don't think that any extra overhead is involved in slicing. D doesn't 
even check if a pointer is valid. (That might be a gotcha come to think 
of it)

void main()
{
	char[] foo = (cast(char*)null)[0..10];
}
Aug 08 2006
parent nobody <nobody mailinator.com> writes:
BCS wrote:

 Frank Benoit wrote:

 This builds a slice, which is like setting ptr/length of an array. This
 is called slicing. Instead of Toms example, I think you can also write
 this:
 foo = buf[ 2 .. buf[0] * buf[1] + 2 ];


 Thanks to you also for your response. I was trying to suggest before
 that I understand how slicing ends up -- ie with ptr and len as I
 desired. I am worried about what happens while the slicing happens. I am
 worried about any gotchas.



 I my understanding this is exactly like setting the ptr and length.
 One side effect could be, that the compiler perhaps inserts array
 boundary check. But I don't know that.


 
 I don't think that any extra overhead is involved in slicing. D doesn't 
 even check if a pointer is valid. (That might be a gotcha come to think 
 of it)
 
 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


Great example! I am now convinced there is nothing to worry 
about. Thanks everyone for the help!
Aug 09 2006
prev sibling parent reply James Dunne <james.jdunne gmail.com> writes:
nobody wrote:

 The only way I have found is to use a struct/union kludge:
 
 // redefine the dyn array struct
 struct sArrayKludge
 {
   int len;
   void* ptr;
 }
 
 // use a union to allow reaching inside
 union uArrayKludge
 {
   byte[] myArray;
   sArrayKludge myArrayInternals;
 }
 
 byte someFunc()
 {
   // declare the union where I would declare myArray
   uArrayKludge theArray;
 
   int[] buf = cast(byte[]) read("file.ext");
 
   // use the union where I would use the array
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];
 
   // now I can use the ith byte
   return theArray.myArray[buf[0]+buf[1]];
 }
 
 
 
 Thanks in advance.


This is very interesting and all, but why would you do such a thing?

Is the .length property on the dynamic array just not cuttin' it for you?

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O 
M--  V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e 
h>--->++ r+++ y+++
------END GEEK CODE BLOCK------

James Dunne
Aug 07 2006
parent reply nobody <nobody mailinator.com> writes:
James Dunne wrote:

 This is very interesting and all, but why would you do such a thing?
 
 Is the .length property on the dynamic array just not cuttin' it for you?
 


No it is not. The whole raison d'etre for that array is to give 
me access to the data in buf. From my reading of the docs it 
looks like any case in which the length increases then this 
array can be copied and might not point to the data in buf 
anymore. So now if I tried to save changes to buf by passing it 
to the std.file.write method it would look like nothing happened.
Aug 08 2006
parent reply James Dunne <james.jdunne gmail.com> writes:
nobody wrote:

 James Dunne wrote:
 

 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for you?


 
 No it is not. The whole raison d'etre for that array is to give me 
 access to the data in buf. From my reading of the docs it looks like any 
 case in which the length increases then this array can be copied and 
 might not point to the data in buf anymore. So now if I tried to save 
 changes to buf by passing it to the std.file.write method it would look 
 like nothing happened.


I'm still not following you.

You want to access, on the byte level, the compiler-dependent 
implementation details of how dynamic arrays work.  I get that.  But why?

Here:


// Sample program (with v0.163):
import	std.file;

int main(char[][] args) {
	char[]	myFileContents = cast(char[]) std.file.read("hello.txt");

	size_t	origLength = myFileContents.length;
	myFileContents.length = origLength + 20;
	myFileContents[origLength .. $] = "20 characters here!!";

	std.file.write("hello2.txt", cast(void[]) myFileContents);

	return 0;
}


Show me pseudo code of the larger problem you're trying to solve.  I 
think your assumptions on the way D's dynamic arrays work are getting in 
the way here.


-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O 
M--  V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e 
h>--->++ r+++ y+++
------END GEEK CODE BLOCK------

James Dunne
Aug 08 2006
parent reply nobody <nobody mailinator.com> writes:
James Dunne wrote:

 nobody wrote:

 James Dunne wrote:


 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for 
 you?


 No it is not. The whole raison d'etre for that array is to give me 
 access to the data in buf. From my reading of the docs it looks like 
 any case in which the length increases then this array can be copied 
 and might not point to the data in buf anymore. So now if I tried to 
 save changes to buf by passing it to the std.file.write method it 
 would look like nothing happened.


 
 I'm still not following you.
 
 You want to access, on the byte level, the compiler-dependent 
 implementation details of how dynamic arrays work.  I get that.  But why?
 
 Here:
 
 
 // Sample program (with v0.163):
 import    std.file;
 
 int main(char[][] args) {
     char[]    myFileContents = cast(char[]) std.file.read("hello.txt");
 
     size_t    origLength = myFileContents.length;
     myFileContents.length = origLength + 20;
     myFileContents[origLength .. $] = "20 characters here!!";
 
     std.file.write("hello2.txt", cast(void[]) myFileContents);
 
     return 0;
 }
 
 
 Show me pseudo code of the larger problem you're trying to solve.  I 
 think your assumptions on the way D's dynamic arrays work are getting in 
 the way here.
 
 


Sorry for the long delay in my reply. I was trying to work out a 
minimal example of my problem and it got way too big. Instead I 
left it for this morning hoping I might find a smaller way to 
illustrate what I meant. My original example was crap (sorry for 
that!)so here is an example that is close:


int main(char[][] args)
{
   char[] buf = "parse words";
   char[] ex = buf[0..1];
   while(ex[$-1] != ' ')
     ex.length = ex.length + 1;
   dout.writeLine(uex.ex);
   return 0;
}


Instead of writing into buf using the ex array this example 
instead tries to read from buf via the ex array. However as I 
mentioned before if you increase ex's length via .length then it 
no longer points into buf. As a result the above example never 
returns.

The way forward is changing the length by some other means. This 
is what my original example was poorly trying to illustrate. 
This version in fact works just fine:


struct sArrayKludge
{
   int len;
   void* ptr;
}
union uArrayKludge
{
   char[] ex;
   sArrayKludge meta;
}

int main(char[][] args)
{
   char[] buf = "parse words";
   uArrayKludge uex;
   uex.meta.ptr = &buf[0];
   uex.meta.len = 1;
   while(uex.ex[$-1] != ' ')
     uex.meta.len = uex.meta.len + 1;
   dout.writeLine(uex.ex);
   return 0;
}


Tom S, Frank Benoit and BCS helped me to see that slicing will 
probably not have strange side effects so I should be using the 
following version which also works fine:


int main(char[][] args)
{
   char[] buf = "parse words";
   char[] ex = buf[0..1];
   while(ex[$-1] != ' ')
     ex = buf[0..ex.length+1];
   dout.writeLine(ex);
   return 0;
}


So now with that said I am curious what incorrect assumptions 
you thought I had about dynamic arrays? I always try to pay 
attention when people say I might be missing something and even 
if what you were going to say is not applicable to me it might 
help someone else.
Aug 10 2006
parent James Dunne <james.jdunne gmail.com> writes:
nobody wrote:

 James Dunne wrote:
 

 nobody wrote:


 James Dunne wrote:


 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for 
 you?


 No it is not. The whole raison d'etre for that array is to give me 
 access to the data in buf. From my reading of the docs it looks like 
 any case in which the length increases then this array can be copied 
 and might not point to the data in buf anymore. So now if I tried to 
 save changes to buf by passing it to the std.file.write method it 
 would look like nothing happened.



 I'm still not following you.

 You want to access, on the byte level, the compiler-dependent 
 implementation details of how dynamic arrays work.  I get that.  But why?

 Here:


 // Sample program (with v0.163):
 import    std.file;

 int main(char[][] args) {
     char[]    myFileContents = cast(char[]) std.file.read("hello.txt");

     size_t    origLength = myFileContents.length;
     myFileContents.length = origLength + 20;
     myFileContents[origLength .. $] = "20 characters here!!";

     std.file.write("hello2.txt", cast(void[]) myFileContents);

     return 0;
 }


 Show me pseudo code of the larger problem you're trying to solve.  I 
 think your assumptions on the way D's dynamic arrays work are getting 
 in the way here.


 Sorry for the long delay in my reply. I was trying to work out a minimal 
 example of my problem and it got way too big. Instead I left it for this 
 morning hoping I might find a smaller way to illustrate what I meant. My 
 original example was crap (sorry for that!)so here is an example that is 
 close:
 
 
 int main(char[][] args)
 {
   char[] buf = "parse words";
   char[] ex = buf[0..1];
   while(ex[$-1] != ' ')
     ex.length = ex.length + 1;
   dout.writeLine(uex.ex);
   return 0;
 }
 
 
 Instead of writing into buf using the ex array this example instead 
 tries to read from buf via the ex array. However as I mentioned before 
 if you increase ex's length via .length then it no longer points into 
 buf. As a result the above example never returns.
 
 The way forward is changing the length by some other means. This is what 
 my original example was poorly trying to illustrate. This version in 
 fact works just fine:
 
 
 struct sArrayKludge
 {
   int len;
   void* ptr;
 }
 union uArrayKludge
 {
   char[] ex;
   sArrayKludge meta;
 }
 
 int main(char[][] args)
 {
   char[] buf = "parse words";
   uArrayKludge uex;
   uex.meta.ptr = &buf[0];
   uex.meta.len = 1;
   while(uex.ex[$-1] != ' ')
     uex.meta.len = uex.meta.len + 1;
   dout.writeLine(uex.ex);
   return 0;
 }
 
 
 Tom S, Frank Benoit and BCS helped me to see that slicing will probably 
 not have strange side effects so I should be using the following version 
 which also works fine:
 
 
 int main(char[][] args)
 {
   char[] buf = "parse words";
   char[] ex = buf[0..1];
   while(ex[$-1] != ' ')
     ex = buf[0..ex.length+1];
   dout.writeLine(ex);
   return 0;
 }
 
 
 So now with that said I am curious what incorrect assumptions you 
 thought I had about dynamic arrays? I always try to pay attention when 
 people say I might be missing something and even if what you were going 
 to say is not applicable to me it might help someone else.


Okay I think your incorrect assumption at first was that increasing the 
length field of a sliced array would extend that slice forward through 
the original array.

Ideally, you should never depend on compiler-dependent details of 
implementation unless that behavior is rigorously defined in the ABI. 
With D's basic set of primitive array operations and slicing, you can do 
just about anything you need to.

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O 
M--  V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e 
h>--->++ r+++ y+++
------END GEEK CODE BLOCK------

James Dunne
Aug 10 2006