digitalmars.D - Accessing Dynamic Arrays - best approach?
- AEon (28/28) Mar 15 2005 Playing around with dynamic arrays like:
- Regan Heath (14/49) Mar 15 2005 I get an "ArrayBoundsError" when I compile either of these examples with...
- AEon (16/38) Mar 15 2005 Just tested this again. Using any of these parameter sets in my makefile...
- Regan Heath (17/56) Mar 15 2005 Can you attach your test code. I still get "Access Violation". I suspect...
- Regan Heath (9/16) Mar 15 2005 To clarify:
-
AEon
(37/50)
Mar 15 2005
- Regan Heath (16/64) Mar 15 2005 I now get the same results as you did. I would call this 'undefined'
- Russ Lewis (9/25) Mar 15 2005 You're correct that this code is not valid.
- Derek Parnell (27/64) Mar 15 2005 By doing this, D allocates at least enough RAM to hold one element and s...
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (6/10) Mar 16 2005 As a side note, "d.length++;" is not allowed
- AEon (60/81) Mar 16 2005 Thanx for the ~= example. I have read about it but was not aware that
- Unknown W. Brackets (14/29) Mar 16 2005 I didn't see anything "scary" or "unclear". If you use a slice, you get...
- AEon (13/33) Mar 16 2005 Ah... indeed.
- Regan Heath (8/16) Mar 16 2005 At the bottom of each page, i.e.
- AEon (3/8) Mar 16 2005 Thanx... sigh, need to check my eyes ;)
- Regan Heath (32/79) Mar 16 2005 No, that was me :)
- Regan Heath (8/21) Mar 16 2005 It appears someone else agreed also:
Playing around with dynamic arrays like: int[] d; // Dynamic Array has no size! Just declared, not initialized yet?! // -> d.length = 0 d[10] = 3; // Error: Access Violation but with a "dummy call": d.length = 1; suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works! I would have expected that a "d.length = 11;" is required for "d[10] = 3;" to work? Put differently, once a dynamic array has been declared (int[] d;), how does one go about to let you access elements in the array? Let's say you have a dynamic array, and you want to copy 10 numbers (that are calculated one by one and do *not* already exist) into it. Would you do this?: int[] d; d.length = 1; // First call, dynamic array would only need to hold 1 number d[0] = 3; // 1st number next number: d.length ++; d[1] = 8; // 2nd number IOW would one resize (re-length) the dynamic array every time? Obviously one could probably upsize the array in larger steps, e.g. if you know you will have 1000 numbers, you could "d.length = d.length + 100;", but would the above be the "good" way to go? AEon
Mar 15 2005
On Tue, 15 Mar 2005 22:07:34 +0000 (UTC), AEon <AEon_member pathlink.com> wrote:Playing around with dynamic arrays like: int[] d; // Dynamic Array has no size! Just declared, not initialized yet?! // -> d.length = 0 d[10] = 3; // Error: Access Violation but with a "dummy call": d.length = 1; suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works!I get an "ArrayBoundsError" when I compile either of these examples with no flags, and "Access Violation" if I compile them with -release.I would have expected that a "d.length = 11;" is required for "d[10] = 3;" to work?It is.Put differently, once a dynamic array has been declared (int[] d;), how does one go about to let you access elements in the array?1. set the length. 2. use [],for,while,do,foreach to access elements.Let's say you have a dynamic array, and you want to copy 10 numbers (that are calculated one by one and do *not* already exist) into it. Would you do this?: int[] d; d.length = 1; // First call, dynamic array would only need to hold 1 number d[0] = 3; // 1st number next number: d.length ++; d[1] = 8; // 2nd number IOW would one resize (re-length) the dynamic array every time? Obviously one could probably upsize the array in larger steps, e.g. if you know you will have 1000 numbers, you could "d.length = d.length + 100;", but would the above be the "good" way to go?If you know you need 1000 numbers, you can write: d.length = 1000; foreach(int i, inout int v; d) { //i is the index into the array v = //calculation goes here } Regan
Mar 15 2005
Regan Heath says...Just tested this again. Using any of these parameter sets in my makefile: DFLAGS=-I. DFLAGS=-O -I. DFLAGS=-w -O -I. DFLAGS=-w -O -inline -I. yields the "Error: ArrayBoundsError test(266)" you mentioned. DFLAGS=-w -O -release -inline -I. compiles cleanly for me. And running the test program will show the proper values. No crash. This seems to mean that when developping, one should *not* use the -release switch since that "hides" certain bugs, e.g. turns off full checking?Playing around with dynamic arrays like: int[] d; d[10] = 3; // Error: Access Violation but with a "dummy call": d.length = 1; suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works!I get an "ArrayBoundsError" when I compile either of these examples with no flags, and "Access Violation" if I compile them with -release.Good... the above made me nervous, and I was starting to wonder if array boundary checks are actually done.I would have expected that a "d.length = 11;" is required for "d[10] = 3;" to work?It is.Ok... so setting the length is a *must*. AEonPut differently, once a dynamic array has been declared (int[] d;), how does one go about to let you access elements in the array?1. set the length. 2. use [],for,while,do,foreach to access elements.
Mar 15 2005
On Tue, 15 Mar 2005 22:59:14 +0000 (UTC), AEon <AEon_member pathlink.com> wrote:Can you attach your test code. I still get "Access Violation". I suspect it's a 'fluke', as in the memory referenced by d[10] just happens to be yours to modify, perhaps part of another variable.Just tested this again. Using any of these parameter sets in my makefile: DFLAGS=-I. DFLAGS=-O -I. DFLAGS=-w -O -I. DFLAGS=-w -O -inline -I. yields the "Error: ArrayBoundsError test(266)" you mentioned. DFLAGS=-w -O -release -inline -I. compiles cleanly for me. And running the test program will show the proper values. No crash.Playing around with dynamic arrays like: int[] d; d[10] = 3; // Error: Access Violation but with a "dummy call": d.length = 1; suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works!I get an "ArrayBoundsError" when I compile either of these examples with no flags, and "Access Violation" if I compile them with -release.This seems to mean that when developping, one should *not* use the -release switch since that "hides" certain bugs, e.g. turns off full checking?Yes. -release disables the runtime array bounds checks, and others.Only when -release is not used.Good... the above made me nervous, and I was starting to wonder if array boundary checks are actually done.I would have expected that a "d.length = 11;" is required for "d[10] = 3;" to work?It is.Sort of, yes.. you can get an array with a valid length without setting it explicitly, eg. int[] original = "12345"; //original.length is 5 int[] new; new = original.dup; //new.length is now 5 new = original[0..3] //new.length is now 3 //note: new[1] = 5; //will crash on linux, but not windows, as 'new' slices 'original' which is static. ReganOk... so setting the length is a *must*.Put differently, once a dynamic array has been declared (int[] d;), how does one go about to let you access elements in the array?1. set the length. 2. use [],for,while,do,foreach to access elements.
Mar 15 2005
On Wed, 16 Mar 2005 12:08:27 +1300, Regan Heath <regan netwin.co.nz> wrote:int[] original = "12345"; //original.length is 5 int[] new; new = original.dup; //new.length is now 5 new = original[0..3] //new.length is now 3 //note: new[1] = 5; //will crash on linux, but not windows, as 'new' slices 'original' which is static.To clarify: new = original.dup; new[1] = 5; will not crash. But: new = original[0..3]; new[1] = 5; might, depending on OS. Regan
Mar 15 2005
Regan Heath says...Can you attach your test code. I still get "Access Violation". I suspect it's a 'fluke', as in the memory referenced by d[10] just happens to be yours to modify, perhaps part of another variable.<code> import std.c.stdio; int main (char[][] args) { printf("\n Clearing dynamic arrays int[] d and d = null\n\n"); int[] d; printf(" d.length = %d (int[] d)\n", d.length); d.length = 1; printf(" d.length = %d (d.length = 11)\n", d.length); printf(" d[10] = %d (inits values!)\n", d[10]); d[10] = 3; printf(" d[10] = %d (d[10] = 3)\n", d[10]); d = null; // printf(" d[10] = %d (d = null)\n", d[10]); // Error: Access Violation return 0; } </code> (dmd 0.118, windows)dmd -w -release test3.dE:\d\dmd\bin\..\..\dm\bin\link.exe test3,,,user32+kernel32/noi;test3Clearing dynamic arrays int[] d and d = null d.length = 0 (int[] d) d.length = 1 (d.length = 11) d[10] = 0 (inits values!) d[10] = 3 (d[10] = 3) As you can see d[10] access works when -release is turned on.dmd -w test3.dE:\d\dmd\bin\..\..\dm\bin\link.exe test3,,,user32+kernel32/noi;test3Clearing dynamic arrays int[] d and d = null d.length = 0 (int[] d) d.length = 1 (d.length = 11) Error: ArrayBoundsError test3.d(12) A "normal" compile on the other hand shows the boundary violation.Sort of, yes.. you can get an array with a valid length without setting it explicitly, eg. int[] original = "12345"; //original.length is 5 int[] new; new = original.dup; //new.length is now 5 new = original[0..3] //new.length is now 3The length of 5 to me reads a bit strange, I would have assumed 1. And that you can assign a string literal (what that the right term) to an int array is also strange. Will test the above some more. BTW since IIRC "new" is a keyword, using it as a variable is kinda evil :) AEon
Mar 15 2005
On Tue, 15 Mar 2005 23:41:35 +0000 (UTC), AEon <AEon_member pathlink.com> wrote:Regan Heath says...I now get the same results as you did. I would call this 'undefined' behaviour. Perhaps Walter can shed some light? Perhaps arrays allocate a minimum size for efficiency?Can you attach your test code. I still get "Access Violation". I suspect it's a 'fluke', as in the memory referenced by d[10] just happens to be yours to modify, perhaps part of another variable.<code> import std.c.stdio; int main (char[][] args) { printf("\n Clearing dynamic arrays int[] d and d = null\n\n"); int[] d; printf(" d.length = %d (int[] d)\n", d.length); d.length = 1; printf(" d.length = %d (d.length = 11)\n", d.length); printf(" d[10] = %d (inits values!)\n", d[10]); d[10] = 3; printf(" d[10] = %d (d[10] = 3)\n", d[10]); d = null; // printf(" d[10] = %d (d = null)\n", d[10]); // Error: Access Violation return 0; } </code> (dmd 0.118, windows)dmd -w -release test3.dE:\d\dmd\bin\..\..\dm\bin\link.exe test3,,,user32+kernel32/noi;test3Clearing dynamic arrays int[] d and d = null d.length = 0 (int[] d) d.length = 1 (d.length = 11) d[10] = 0 (inits values!) d[10] = 3 (d[10] = 3) As you can see d[10] access works when -release is turned on.Doh! sorry, the code above will never compile. Will test the above some more. BTW since IIRC "new" is aSort of, yes.. you can get an array with a valid length without setting it explicitly, eg. int[] original = "12345"; //original.length is 5 int[] new; new = original.dup; //new.length is now 5 new = original[0..3] //new.length is now 3The length of 5 to me reads a bit strange, I would have assumed 1. And that you can assign a string literal (what that the right term) to an int array is also strange.keyword, using it as a variable is kinda evil :)Yet another reason my code is illegal. Let me try again: char[] original = "12345"; char[] a; a = original.dup; a = original[0..3]; (same comments as before) Regan
Mar 15 2005
AEon wrote:Playing around with dynamic arrays like: int[] d; // Dynamic Array has no size! Just declared, not initialized yet?! // -> d.length = 0 d[10] = 3; // Error: Access Violation but with a "dummy call": d.length = 1; suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works!You're correct that this code is not valid. You're obviously compiling with -release, because, otherwise, D would do bounds checking for you and you would get an ArrayBoundsError. My guess why it works is that D has allocated some memory for the array, and while the allocation is small, theOScan only do access protection on a page level. Thus, when you overflow the array, you are reading memory which you are not supposed to touch - but there is no way to force you not to.
Mar 15 2005
On Tue, 15 Mar 2005 22:07:34 +0000 (UTC), AEon wrote:Playing around with dynamic arrays like: int[] d; // Dynamic Array has no size! Just declared, not initialized yet?! // -> d.length = 0 d[10] = 3; // Error: Access ViolationBecause at this point your array does not point to any RAM.but with a "dummy call": d.length = 1;By doing this, D allocates at least enough RAM to hold one element and sets the array to point to that RAM.suddenly I can access elements in the array, even though I am exceeding the length limits? d[10] = 3; // Works! I would have expected that a "d.length = 11;" is required for "d[10] = 3;" to work?If you compile with -release, DMD turns off checking for bad index values. It instead assumes you know what you are doing. By luck, you got to access some RAM that your application owned, but it wasn't necessarily owned by the array itself.Put differently, once a dynamic array has been declared (int[] d;), how does one go about to let you access elements in the array? Let's say you have a dynamic array, and you want to copy 10 numbers (that are calculated one by one and do *not* already exist) into it. Would you do this?: int[] d; d.length = 1; // First call, dynamic array would only need to hold 1 number d[0] = 3; // 1st number next number: d.length ++; d[1] = 8; // 2nd number IOW would one resize (re-length) the dynamic array every time?You could, but that is not the way I'd do it. I would do either ... d.length = 10; d[0] = 3; d[1] = 8; d[3] = ... etc... or d ~= 3; d ~= 8; d ~= ... etc ...Obviously one could probably upsize the array in larger steps, e.g. if you know you will have 1000 numbers, you could "d.length = d.length + 100;", but would the above be the "good" way to go?You would probably select the method that made more sense for the application. Presetting the length is fine if you know how many items you are adding, but the concatenation method may be more useful when you don't know how many items there will be. -- Derek Melbourne, Australia 16/03/2005 11:59:08 AM
Mar 15 2005
AEon wrote:next number: d.length ++; d[1] = 8; // 2nd numberAs a side note, "d.length++;" is not allowed (since you *might* use it to hurt yourself...) One must do "d.length = d.length + 1;" instead. Or just use "d ~= 8;", which is a lot easier ? --anders
Mar 16 2005
Derek Parnell says...d ~= 3; d ~= 8; d ~= ... etc ...You would probably select the method that made more sense for the application. Presetting the length is fine if you know how many items you are adding, but the concatenation method may be more useful when you don't know how many items there will be.Thanx for the ~= example. I have read about it but was not aware that would work when adding array elements. Just continued reading the documentation, and found an "optimized" example: int[] array; array.length = 100; // guess for (i = 0; 1; i++) { c = getinput(); if (!c) break; if (i == array.length) array.length = array.length * 2; array[i] = c; } array.length = i; // Resize to actual array size! And indeed the manual frowns on, doing "inefficient" array.length++; calls. Regan,Glad, I am not totally wacking out :)Doh! sorry, the code above will never compile.int[] original = "12345"; //original.length is 5 ...Let me try again:char[] original = "12345"; char[] a;a = original.dup; a = original[0..3];I have read a scary section in the manual under Arrays, Setting Dynamic Array Length... where it seems to turn out that slicing can lead to code that will not be fully predicatble. In the above example, if you where to change elements in a a = original[0..3]; a[2] = 'd'; then original[2] could be 'd' or it could still be '3'. Because when slicing it seems to be not clear if a is actually a seperate copy of original or just a "reference". To ensure that you are manipulating a copy, the manual suggests one use .dup: a = original.dup; a[2] = 'd'; -> original[2] uneffected. This is only how I understand things from first reading, I may have missunderstood something. I had the feeling that the example in Arrays, Setting Dynamic Array Length, had a mistake, or at least I did not understand what was meant: char[] a = new char[20]; char[] b = a[0..10]; char[] c = a[10..20]; b.length = 15; // always resized in place because it is sliced // from a[] which has enough memory for 15 chars b[11] = 'x'; // a[15] and c[5] are also affected ... Should that not read?: b[11] = 'x'; // a[11] and c[1] are also affected Visualization of the memory: var element number 1111111111 a 01234567890123456789 b 0123456789 c 0123456789 With this behavior I would assume that slices are probably wonderful as convenient "aliases" to (in this example) substrings, but should not be used to manipulate the original arrays? anders,d.length ++; d[1] = 8; // 2nd numberAs a side note, "d.length++;" is not allowed (since you *might* use it to hurt yourself...)One must do "d.length = d.length + 1;" instead. Or just use "d ~= 8;", which is a lot easier ?Aha... good to know! AEon
Mar 16 2005
I have read a scary section in the manual under Arrays, Setting Dynamic Array Length... where it seems to turn out that slicing can lead to code that will not be fully predicatble. In the above example, if you where to change elements in a a = original[0..3]; a[2] = 'd'; then original[2] could be 'd' or it could still be '3'. Because when slicing it seems to be not clear if a is actually a seperate copy of original or just a "reference". To ensure that you are manipulating a copy, the manual suggests one use .dup:I didn't see anything "scary" or "unclear". If you use a slice, you get the memory, not a copy of it. If you use .dup, you get a copy. Sometimes you don't want a copy - imagine: if (str[0 .. 7] == "http://") ... Do you want to actually allocate more memory for that, or are you happy with the comparison as is?With this behavior I would assume that slices are probably wonderful as convenient "aliases" to (in this example) substrings, but should not be used to manipulate the original arrays?Why not? What if you did this: char[] str = "test<script>"; int i = find(str, "<script>"); str[i .. i + 8] = ' '; Doesn't it make sense, there, to modify in place? Faster than making a copy, modifying it, and then replacing the old, no? -[Unknown]
Mar 16 2005
Unknown W. Brackets says...I didn't see anything "scary" or "unclear". If you use a slice, you get the memory, not a copy of it. If you use .dup, you get a copy.Well it had surprised me, would have been a better way of putting it.Sometimes you don't want a copy - imagine: if (str[0 .. 7] == "http://") ... Do you want to actually allocate more memory for that, or are you happy with the comparison as is?Ah... indeed.Ahem :)... yes it does make sense. But I had only been thinking of working with "copies" and manipulating those, my mistake. When directly manipulating the "original" strings via slices, that would indeed be predicatable in outcome and useful. Thanx for the real world examples. Regan,With this behavior I would assume that slices are probably wonderful as convenient "aliases" to (in this example) substrings, but should not be used to manipulate the original arrays?Why not? What if you did this: char[] str = "test<script>"; int i = find(str, "<script>"); str[i .. i + 8] = ' '; Doesn't it make sense, there, to modify in place? Faster than making a copy, modifying it, and then replacing the old, no?It appears someone else agreed also: http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/ArraysIndeed. Thanx for the link.If you find any other possible typos you can amend them here: http://www.prowiki.org/wiki4d/wiki.cgi?DocCommentsHmmm... I have trouble finding some link/button to actually submit something new?! I use the Firefox browser, could that be the problem? AEon
Mar 16 2005
On Wed, 16 Mar 2005 12:20:24 +0000 (UTC), AEon <AEon_member pathlink.com> wrote:At the bottom of each page, i.e. http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Arrays there is and 'Edit' link and als 'Edit text of this page'. before you can do that you need to go to preferences and enter your name, so it can track who changed what. ReganIt appears someone else agreed also: http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/ArraysIndeed. Thanx for the link.If you find any other possible typos you can amend them here: http://www.prowiki.org/wiki4d/wiki.cgi?DocCommentsHmmm... I have trouble finding some link/button to actually submit something new?! I use the Firefox browser, could that be the problem?
Mar 16 2005
Regan Heath says...At the bottom of each page, i.e. http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Arrays there is and 'Edit' link and als 'Edit text of this page'. before you can do that you need to go to preferences and enter your name, so it can track who changed what.Thanx... sigh, need to check my eyes ;) AEon
Mar 16 2005
On Wed, 16 Mar 2005 10:18:26 +0000 (UTC), AEon <AEon_member pathlink.com> wrote:No, that was me :)Glad, I am not totally wacking out :)Doh! sorry, the code above will never compile.int[] original = "12345"; //original.length is 5 ...original[2] will be 'd'. This is guaranteed IIRC. When you type "a = " you always make a reference to the original data. However, if you said: char[] original = "12345"; char[] a; a.length = 3; a[0..3] = original[0..3]; it would copy the values from 'original' to 'a'.Let me try again:char[] original = "12345"; char[] a;a = original.dup; a = original[0..3];I have read a scary section in the manual under Arrays, Setting Dynamic Array Length... where it seems to turn out that slicing can lead to code that will not be fully predicatble. In the above example, if you where to change elements in a a = original[0..3]; a[2] = 'd'; then original[2] could be 'd' or it could still be '3'.Because when slicing it seems to be not clear if a is actually a seperate copy of original or just a "reference".It depends on what you say, "a = " always creates a reference, but "a[x..y] = " always copies the data.To ensure that you are manipulating a copy, the manual suggests one use .dup: a = original.dup; a[2] = 'd'; -> original[2] uneffected.Correct. dup creates a copy of the array or slice. eg. (note .dup on a slice) a = original[0..3].dup; a[2] = 'd' //original[2] unaffectedThis is only how I understand things from first reading, I may have missunderstood something.There may be a few subtle things left to learn. I've tried to show them above.I had the feeling that the example in Arrays, Setting Dynamic Array Length, had a mistake, or at least I did not understand what was meant: char[] a = new char[20]; char[] b = a[0..10]; char[] c = a[10..20]; b.length = 15; // always resized in place because it is sliced // from a[] which has enough memory for 15 chars b[11] = 'x'; // a[15] and c[5] are also affected ... Should that not read?: b[11] = 'x'; // a[11] and c[1] are also affectedI agree. I think it's a typo/mistake.Visualization of the memory: var element number 1111111111 a 01234567890123456789 b 0123456789 c 0123456789 With this behavior I would assume that slices are probably wonderful as convenient "aliases" to (in this example) substrings, but should not be used to manipulate the original arrays?You can use them to manipulate the originals, it can be quite handy. The rest of the time, make sure you use COW (Copy On Write), meaning every time you write to an array (which you're not intentionally using to manipulate the original) call "dup" first, then write to it. eg. char[] original = "12345"; char[] a; a = original; ..lots of code.. a = a.dup; a[2] = 'd'; Regan
Mar 16 2005
On Thu, 17 Mar 2005 00:20:44 +1300, Regan Heath <regan netwin.co.nz> wrote:It appears someone else agreed also: http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Arrays If you find any other possible typos you can amend them here: http://www.prowiki.org/wiki4d/wiki.cgi?DocComments I believe Walter will eventually get round to making all these changes to the real docs? Reganchar[] a = new char[20]; char[] b = a[0..10]; char[] c = a[10..20]; b.length = 15; // always resized in place because it is sliced // from a[] which has enough memory for 15 chars b[11] = 'x'; // a[15] and c[5] are also affected ... Should that not read?: b[11] = 'x'; // a[11] and c[1] are also affectedI agree. I think it's a typo/mistake.
Mar 16 2005