www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Passing dynamic arrays

reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 Hi,
 
 I do not understand what's going on behind the scene with this code. Or
 better said I have some idea but maybe I do not see the whole point.
 
 void foo(int[] array) {
 	array.length += 1000; // may copy the array
 	array[0] = 1;
 }
 
 auto a = new int[1];
 foo(a);
 assert(a[0] == 1); // fails if a needs to copied inside foo
 

 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
 I think I should use ref int[] in the example above, shouldn't I?
 
 Jens

But they are past by reference. You can modify the data all you want, but cannot reassign the reference itself. Resizing the array may cause a reassignment of that reference. It is not different from the following code except resizing does not guarantee a reference change. import std.stdio; void assignValue(A a) { a = new A(); a.value = 6; } class A { int value; } void main() { A a = new A(); assignValue(a); writeln(a.value); }
Nov 08 2010
next sibling parent reply spir <denis.spir gmail.com> writes:
On Mon, 08 Nov 2010 15:32:56 -0500
Jesse Phillips <jessekphillips+D gmail.com> wrote:

 But they are past by reference. You can modify the data all you want, but=

No, they are _not_ passed by reference. Stop saying that, this is precisely= what causes confusion. This is reference semantics: class C { int* ints; } void main () { auto c =3D new C(); auto d =3D c; auto ints =3D [1,2,3]; c.ints =3D &(ints[0]); assert(c.ints =3D=3D d.ints); assert(*(c.ints) =3D=3D *(d.ints)); } Whatever change one performs on object fields never breaks the relation wit= h other vars pointing to the same object; because the object itself is refe= renced. D arrays do not work that way: the kind of struct is *copied*, not referenc= ed; arrays themselves are *values*. So that changes to the content that req= uires reallocation breaks the relation. (Additional confusion is brought by the fact that, if a is an array, after = "b=3Da" (b is a) yields true, for any reason, but this is also wrong. I gue= ss 'is' is overloaded for arrays to compare the adresses of the contents in= stead of the ones of the array-structs, but this is misguided.) Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 08 2010
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
spir Wrote:

 No, they are _not_ passed by reference. Stop saying that, this is precisely
what causes confusion. This is reference semantics:

void main() { auto a = new int[1]; auto b = a; a[0] = 5; assert(a == b); assert(a[0] == b[0]); assert(a is b); writefln("%x == %x", &a[0], &b[0]); moreExample(a, cast(uint) &a[0]); } void moreExample(int[] a, uint aAddr) { assert(cast(uint)&a[0] == aAddr); writefln("%x == %x", &a[0], aAddr); }
 D arrays do not work that way: the kind of struct is *copied*, not referenced;
arrays themselves are *values*. So that changes to the content that requires
reallocation breaks the relation.

The struct you are referring to _is_ the reference. There is no such thing as "copy by reference," everything is copied by value. However, some values are just a reference to another location.
 (Additional confusion is brought by the fact that, if a is an array, after
"b=a" (b is a) yields true, for any reason, but this is also wrong. I guess
'is' is overloaded for arrays to compare the adresses of the contents instead
of the ones of the array-structs, but this is misguided.)

The array-struct is the reference, so it is what gets compared. That means both the internal pointer and length must be the same. Just because the reference is more than an address does not make it any less a reference. void main() { auto a = new int[2]; a[0] = 5; a[1] = 10; auto b = a.dup; auto c = a[0..1]; assert(a == b); assert(a[0] == b[0]); assert(a !is b); assert(a !is c); } The distinction is that an Array can have its reference changed by resizing (which is not an option for other reference types).
Nov 08 2010
parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Daniel Gibson Wrote:

 On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That means
both the internal pointer and length must be the same. Just because the
reference is more than an address does not make it any less a reference.

Unlike in C, a D array is more than a reference. An array is the data (or a reference to its data) + metadata (its length) - the metadata belongs to the array (and not to the reference).

If you are using the definition of reference which would include C pointers (i.e. not C++ references), then there is nothing in the definition of reference that precludes it from having meta data. "In distributed computing, the reference may contain more than an address or identifier; it may also include an embedded specification of the network protocols used to locate and access the referenced object, the way information is encoded or serialized." http://en.wikipedia.org/wiki/Reference_(computer_science)
 This means that, when you say "the array is passed by reference" one
 expects that also the arrays length is passed by reference - because
 the length belongs to the array.

This is a good point. Though if the length did stay with the array, it is still reasonable that a reallocation of the array would cause the reference to no longer point to the same array.
 The distinction is that an Array can have its reference changed by resizing
(which is not an option for other reference types).

If that is not an option for any other reference type, why should it be an option for arrays? That is just inconsistent and doesn't make any sense.

Well, if you can come up with a good definition for what "increasing the size of a class" would do, then maybe it should be added. It really doesn't matter. Arrays are their own type, the have their own semantics. It does help to think of them in terms of slices (which is what TDPL refers to them as), yet that does not remove the fact that they are in dead a reference type. Many times familiar terms are used so that a behavior is quickly understood. For example it is common to say that arrays in C is just a pointer into a memory location. But in reality that is not true. http://www.lysator.liu.se/c/c-faq/c-2.html
 
 No, Arrays should not be considered reference types when passed to a function.
 As someone else said before: Logically you don't pass the array but a
 slice that contains the whole array.
 
 Cheers,
 - Daniel

Nov 08 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 I think of an array as a hybrid between a reference and a value type.  The  
 data is passed by reference, the length is passed by value.  This mean  
 changing the length only affects the local copy, but changing the data  
 affects all arrays that point to that data.

Let's play some more :-) The data is passed by pointer value, the length is passed by value: void foo1(ref int[2] arr) { int[2] a = [10, 20]; arr = a; } void foo2(int[] arr) { int[2] a = [10, 20]; arr = a; } void main() { int[2] arr; arr = [1, 2]; foo1(arr); assert(arr == [10, 20]); arr = [1, 2]; foo2(arr); assert(arr == [1, 2]); } Bye, bearophile
Nov 09 2010
prev sibling next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Steven Schveighoffer Wrote:

 It depends on your definition of reference type.  I agree with Daniel.  If  
 you want to get into academic definitions, yes, 'technically' an array is  
 a reference, but it's confusing to someone who's not interested in  
 exploring the theoretical parts of computer science.

I think viewing a reference as a pointer is wrong. But I did already agree that since it is claimed to be passed by reference it is odd that the length is not part of it.
 I think of an array as a hybrid between a reference and a value type.  The  
 data is passed by reference, the length is passed by value.  This mean  
 changing the length only affects the local copy, but changing the data  
 affects all arrays that point to that data.

Yet this still isn't complete because if you resize the array the data may not point to that same data anymore. I do not see this as an issue, but if you understand this, the fact that the length is passed by value isn't important, because you won't be relying on whether it points to the same data or not. D arrays are done differently then other types/languages. I don't have an issue with changing the wording to clarify, but if we are going to do that suggestions should be made. But saying it isn't a reference does nothing.
 There is no more distinction between slices or arrays, they are  
 one and the same. 

Yeah, I'm glad about this. Arrays are much nice for this.
Nov 09 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.

It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.

What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.
 I think of an array as a hybrid between a reference and a value type.
 The data is passed by reference, the length is passed by value. This
 mean changing the length only affects the local copy, but changing the
 data affects all arrays that point to that data.

Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is perhaps the best way to describe them, with less misunderstanding. However, if I had to say whether dynamic arrays are value types or reference types, I would agree with Jesse and call them reference types, and I would not feel this is inaccurate. Let's try a test everyone, look at this code: void test() { int[] a = [1, 2, 3]; int[] b = a; int[] c = a; } and tell us, what you would you reply if asked "how many arrays are created during test's scope"? I would say "1", and not feel it is inaccurate. However, if asked how "many dynamic arrays are created during test's scope?", I would likely say "3". This is because what I consider an array to *be*, is its (contiguous & homogeneous) elements. With that definition, then D's static arrays are value types, because when you assign a static array value to a static array variable, the underlying _array_ (ie, the contents), get copied. Conversely, D's dynamic arrays are reference types, because when you assign a dynamic array value to a dynamic array variable, the underlying "array" (ie, the contents) are not copied, instead you get two references to exactly the same "array" (ie, same data). This viewpoint gets a bit murkier because an "array" can be contained inside another "array", but that doesn't fundamentally change the it (the viewpoint that is). But ultimately neither view is right or wrong, it just depends on how we define our terms, how we conceptualize things. However, it's probably best to stick to what the spec/TDPL says about it, if it says anything specific. (I don't think it does though :/ ) Or to avoid ambiguous designations. ... Oh man, I need to clear my head. -- Bruno Medeiros - Software Engineer
Nov 26 2010
next sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:50, Bruno Medeiros wrote:
 Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is
 perhaps the best way to describe them, with less misunderstanding.

 However, if I had to say whether dynamic arrays are value types or
 reference types, I would agree with Jesse and call them reference types,
 and I would not feel this is inaccurate.

ARRGHH, actually this is not entirely true, I forgot about something: null vs. empty arrays. Indeed on assignment (and parameter passing) dynamic arrays work just like reference types. The reallocation operations also don't go against that either. However, in D's dynamic arrays, null is the same as an empty array! Which in practice means there is no proper null "value" for dynamic arrays, like there is for other reference types, and I think that is kinda of an essential characteristic for a reference type. So actually I would not call D's dynamic arrays "reference types" and feel this is really accurate. -- Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 29/11/2010 14:13, Steven Schveighoffer wrote:
 On Fri, 26 Nov 2010 14:50:27 -0500, Bruno Medeiros
 <brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.

It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.

What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.

A class is a reference type. Every operation on a class instance operates on all aliases to that class, only assignment to another class instance of the same type decouples it. Consider these two statements: "a class is a reference type" class C { int length; } foo(C c) { c.length = 5; // because a class is a reference type, this affects the original, right? } "an array is a reference type" foo(int[] arr) { arr.length = 5; // because an array is a reference type, this affects the original, right? No?!!! } This is the major confusion that I think people see. I would say people assume a "reference type" means that the reference's members are all shared between all aliases. For an array, it is one level removed, and that confuses the hell out of people. But the power gained by doing D's way is worth way waaaay more than confusing some noobs.

Hum, I see your point, yeah, I guess I do agree somewhat. I mean, it goes back to the issue of what one thinks the "array" is. There is no misunderstanding in the code above if one considers the array to be just it's elements... However, as you point out, it would not be uncommon that a newbie would consider the "members" of the array to be part of it (and thus shared across all aliases). It's not even unreasonable to think that, in fact. But then with that mental model it would be quite inaccurate to say "an array is a reference type", yeah. (even disregarding the issues with empty vs null arrays, a different beast altogether) -- Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Jesse Phillips wrote:
 Jens Mueller Wrote:
 
 Hi,
 
 I do not understand what's going on behind the scene with this code. Or
 better said I have some idea but maybe I do not see the whole point.
 
 void foo(int[] array) {
 	array.length += 1000; // may copy the array
 	array[0] = 1;
 }
 
 auto a = new int[1];
 foo(a);
 assert(a[0] == 1); // fails if a needs to copied inside foo
 

 I find this behavior rather strange. Arrays are neither passed by value
 (copying the whole array) nor by reference. I see reasons for doing it
 like this, e.g. doing array = array[1..$] inside should not affect the
 outside.
 But I wonder whether these semantics are well enough documented?
 I think I should use ref int[] in the example above, shouldn't I?
 
 Jens

But they are past by reference. You can modify the data all you want, but cannot reassign the reference itself. Resizing the array may cause a reassignment of that reference. It is not different from the following code except resizing does not guarantee a reference change. import std.stdio; void assignValue(A a) { a = new A(); a.value = 6; } class A { int value; } void main() { A a = new A(); assignValue(a); writeln(a.value); }

I like that explanation. Jonathan is saying the same, I think. I'll guess my misunderstanding is mainly caused by figuring out that a reassign is happening and that a reassign to a reference changes the reference. In C++ you cannot change a reference (I hope I'm right here.). When using a std::vector one does not need to think about this. What's the general use of a = new A() in the above code? Where is it useful? Jens
Nov 08 2010
next sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 I like that explanation. Jonathan is saying the same, I think.

Yes, same thing.
 In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.

Don't know too much about C++ references, but as mentioned somewhere you can use the Array container which won't have this issue
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens

I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.
Nov 08 2010
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 09 November 2010 02:43:57 Jens Mueller wrote:
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens

I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.

I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects?

Why wouldn't you be able to change it? You can change any parameter as long as it's not const or immutable (or in, which implies const). The fact that it's a referenc is irrelevant. I can assign whatever I want to references and pointers in a function whether they were passed in or not. Sure, if you want to alter the original pointer or reference, that's not going to work unless it was passed as ref, but that's the same as any other parameter. Why would you expect altering a reference to alter the original? Sure, altering what it _refers to_ should alter what the original refers to because they refer to the same thing, but altering the reference itself shouldn't alter the original because they're two different references. There is no reason why parameters should have to stay the same as what they were passed in as - regardless of whether they're value types or reference types. If you want that behavior, use const, in, or immutable. - Jonathan M Davis
Nov 09 2010
prev sibling next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
 I don't really have any good use-case examples. Maybe an initialization
 function? Developed your own number object (big int) and were thinking
 in terms of it being a refrence you thought
 
 a = a + BigInt(7);
 
 would result in a being resigned in the calling function. Or maybe just a
 function that swaps two class references:
 
 void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) {
 
     auto tmp = a; a = b; b = tmp;
 
 }
 
 Actually that turned out to be a pretty good one.

I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects?

Why wouldn't you be able to change it? You can change any parameter as long as it's not const or immutable (or in, which implies const). The fact that it's a referenc is irrelevant. I can assign whatever I want to references and pointers in a function whether they were passed in or not. Sure, if you want to alter the original pointer or reference, that's not going to work unless it was passed as ref, but that's the same as any other parameter. Why would you expect altering a reference to alter the original? Sure, altering what it _refers to_ should alter what the original refers to because they refer to the same thing, but altering the reference itself shouldn't alter the original because they're two different references. There is no reason why parameters should have to stay the same as what they were passed in as - regardless of whether they're value types or reference types. If you want that behavior, use const, in, or immutable.

I see your point. You argue that the behavior is consistent. My point is that this consistency can lead to bugs. I may forget the ref. But I'll keep in mind to never forget the ref if it is needed. Jens
Nov 09 2010
parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Jens Mueller Wrote:

 I see your point. You argue that the behavior is consistent. My point is
 that this consistency can lead to bugs. I may forget the ref. But I'll
 keep in mind to never forget the ref if it is needed.
 
 Jens

Well, in the case of classes, I don't think it would be very common. For arrays it can be nice since you can assign back a slice of the array that you want to work with. void main(string args) { args = args[1..$]; } Otherwise I suggest you start labeling all parameters with in. Then you are prevented from modifying the reference, and can decide if it should be a ref parameter. Who knows, maybe you'll find a reason to leave it off.
Nov 09 2010
prev sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
 I see your point. You argue that the behavior is consistent. My point is
 that this consistency can lead to bugs. I may forget the ref. But I'll
 keep in mind to never forget the ref if it is needed.
 
 Jens

Well, in the case of classes, I don't think it would be very common. For arrays it can be nice since you can assign back a slice of the array that you want to work with. void main(string args) { args = args[1..$]; } Otherwise I suggest you start labeling all parameters with in. Then you are prevented from modifying the reference, and can decide if it should be a ref parameter. Who knows, maybe you'll find a reason to leave it off.

With dynamic arrays I totally agree. Slicing is very useful. I just want a safe rule to work with it. Maybe that's not needed anymore because by now I spend enough time on this that probably I'll never forget. With in I also disallow changing the object itself, right? I want to only forbid the changing of the reference of an argument inside the function. With in/const I disallow every change. I may want to change data of an array. Jens PS The more we talk about it, the more I come to the conclusion that this is just something you need to know. I can live with that.
Nov 10 2010
prev sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 11/8/2010 17:43, Jonathan M Davis wrote:
 D references are more like Java references.

That's true for class references. D also supports pass-by-reference through the 'ref' keyword, which works like C++ references. -- Rainer Deyke - rainerd eldwood.com
Nov 12 2010
prev sibling next sibling parent so <so so.do> writes:
 I like that explanation. Jonathan is saying the same, I think. I'll
 guess my misunderstanding is mainly caused by figuring out that a
 reassign is happening and that a reassign to a reference changes the
 reference. In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.
 What's the general use of a = new A() in the above code? Where is it
 useful?

 Jens

Yes in C++, you can't redirect a reference after initialization. And also can't have a std::vector of references which is mainly by this reason. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 08 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 16:11:46 so wrote:
 I like that explanation. Jonathan is saying the same, I think. I'll
 guess my misunderstanding is mainly caused by figuring out that a
 reassign is happening and that a reassign to a reference changes the
 reference. In C++ you cannot change a reference (I hope I'm right
 here.). When using a std::vector one does not need to think about this.
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens

Yes in C++, you can't redirect a reference after initialization. And also can't have a std::vector of references which is mainly by this reason.

D references are more like Java references. They're really pointers, but you don't have to dereference them, so you can reassign them to something else. You can't do that in C++, because references are treated more like name aliases of variables than pointers. So, if you think in terms of C++ pointers vs D references, then C++ and D function the same in this respect. - Jonathan M Davis
Nov 08 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 08, 2010 13:54:28 spir wrote:
 On Mon, 08 Nov 2010 15:32:56 -0500
 
 Jesse Phillips <jessekphillips+D gmail.com> wrote:
 But they are past by reference. You can modify the data all you want, but
 cannot reassign the reference itself.

No, they are _not_ passed by reference. Stop saying that, this is precisely what causes confusion. This is reference semantics:

As Jesse says, they _are_ passed by reference. The struct itself _is_ the reference. What makes arrays odd is that anything that resizes it returns a _new_ reference or alters the current one. So, using ~= or length can result in a reference with a new ptr value and a new length value, and the reference then refers to a different block of memory. In the case of an object reference, you could implement ~= to return a new reference in exactly the same manner and get behavior similar to an array, but no one does that normally. _Everything_ which is not passed as ref or out is passed by value in D. And technically, ref and out probably wrap the value being passed in a pointer and that pointer is passed by value. Really, this is easier to understand when dealing with pointers than references, since references hide the dereferencing process. Take this program for instance: import std.stdio; void func1(int* b) { *b = 12; } void func2(int* c) { c = new int; *c = 7; } void func3(int** d) { *d = new int; **d = 3; } void main() { int* i = new int; int* j = i; *i = 5; writefln("%s %s", *i, *j); func1(i); writefln("%s %s", *i, *j); func2(i); writefln("%s %s", *i, *j); func3(&i); writefln("%s %s", *i, *j); } It prints out 5 5 12 12 12 12 3 12 i and j both point to the same memory, so when that memory is altered, the value that you get when you dereference them is altered. When func1() is called, the value of i is passed to func1() - that is the address that i points to - so b points to the same memory that i and j do, so altering the value in that memory, alters the memory for all three pointers. When calling func2(), c is a copy of i and holds the same address. However, setting c to a new address means that it no longer points to the same memory and any changes made to the memory that it points to does not alter the memory that i and j point to. So, they still print out the same value when dereferenced. func3() takes the address of i. That means that d the holds the address of i and can alter i itself rather than just the memory that it points to. c points to the memory that holds i itself. So, when assigning the dereferenced c to a new address, i itself receives a new address. Then when altering the memory that's pointed to by the address that c holds, you alter the memory that i points to and change that value. However, while j held the same address as i originally and altering the memory at that address therefore altered what both i and j pointed to, once i was given a new address by c, they i and j held different addresses and altering the memory that one pointed to did not alter the one that the other pointed to. I expect that you've gone over this sort of thing before, but you need to realize that references are _exactly_ the same. The only difference is the lack of dereferencing step when accessing what it points to. So, whether you're passing object reference or an array reference, you're passing that reference by value, and any changes to the copy won't change the original. Changes to what the reference refers to _will_ change what the original reference points to as well since they point to the same thing, but changes to the copied reference won't change the original. So, if make the copied reference point to something else, then it won't change what the original reference pointed to anymore. Having an array that you use ~= on is like doing objRef = new Foo(); on a reference that was passed into the function. It now points to something else, so it's not going to affect the original. What gets somewhat weird about it is that ~= doesn't necessarily return a new reference, and if it doesn't then the length is different, so it still hasn't affected the original reference (if it was going to, it would have resulted in a reallocation). The value of the copied reference has changed. I grant you that if you're expecting ~= to return the same reference every time and thereby somehow alter the original reference, then you're going to be surprised. But it's completely consistent with how pointers and references work. You can alter the memory that they point to, but since the pointer or reference itself was passed by value, altering it will not alter the original. - Jonathan M Davis
Nov 08 2010
prev sibling next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
<jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That means
both the internal pointer and length must be the same. Just because the
reference is more than an address does not make it any less a reference.

Unlike in C, a D array is more than a reference. An array is the data (or a reference to its data) + metadata (its length) - the metadata belongs to the array (and not to the reference). This means that, when you say "the array is passed by reference" one expects that also the arrays length is passed by reference - because the length belongs to the array.
 The distinction is that an Array can have its reference changed by resizing
(which is not an option for other reference types).

If that is not an option for any other reference type, why should it be an option for arrays? That is just inconsistent and doesn't make any sense. No, Arrays should not be considered reference types when passed to a function. As someone else said before: Logically you don't pass the array but a slice that contains the whole array. Cheers, - Daniel
Nov 08 2010
prev sibling next sibling parent reply spir <denis.spir gmail.com> writes:
On Mon, 8 Nov 2010 17:08:32 -0800
Jonathan M Davis <jmdavisProg gmx.com> wrote:

 As Jesse says, they _are_ passed by reference. The struct itself _is_ the=

 reference.=20

(Well, that is a sensible redefinition of "reference"; but it is simply _no= t_ what the word means in any other context.) It is true that the inner, hidden, memory area (static array) containing th= e elements is indeed referenced, actually "pointed", from the dynamic array= struct: struct ValueArray(Element) { Element* elements; uint length; } (Well, actually, this may not be a struct, but it's easier to imagine it so= .) But: the dyn array itself, meaning the struct, is not referenced: "a2 =3D a= 1" copies it, as well as parameter passing. And the fact that the internal = memory is referenced is an implementation detail that should *not* affect s= emantics. The inner pointer is there because we need some kind of indirecti= on to implement variable-size thingies, and the means for this is pointers. This is precisely where & why people get bitten: implementation leaks out i= nto semantics. Actually, one could conceptually replace the (pointer,length) pair by a sin= gle field of type MemoryArea -- which would be a plain value. Then, there w= ould be no more (visible) pointer in the dyn array, right? (Actually, it wo= uld just be hidden deeper inside the MemoryArea field... but that is again = implementation detail!) We should not mess up pointers used for implementation mechanics, like in t= he case of dyn arrays, or more generally variable size data structure, with= pointers used as true references carrying semantics, like in the case of t= he difference between struct and class. And precisely, replacing array struct by a class, or explicitely referencin= g the struct, would make a *reference* dyn array type. See below an example= of a primitive sort of such an array type (you can only put new elements i= n it ;-), implemented as class. After "a2 =3D a1", every change to one of the vars affects the other var; w= hether the change requires reallocation is irrelevant; this detail belongs = to implementation, not to semantics. Now, replace class with struct and you have a type for *value* dyn arrays. = Which works exactly like D ones. The assertion will fail; and output should be interesting ;-) Hope it's clear, because I cannot do better. I do not mean that D arrays are bad in any way. They work perfectly and are= very efficient. Enforcing a true interface between implementation and sema= ntics would certainly have a relevant cost in terms of space & time. But pl= ease, stop stating D arrays are referenced if you want newcomers to have a = chance & understand the actual behaviour, to use them without beeing consta= ntly bitten, and to stop & complain. Denis class RefArray(Element) { Element* elements; uint length; private uint capacity; this () { this.elements =3D cast(Element*) malloc(Element.sizeof); this.capacity =3D 1; this.length =3D 0; } void reAlloc() { writeln("realloc"); this.capacity *=3D 2; size_t memSize =3D this.capacity * Element.sizeof; realloc(this.elements, memSize); } void put(Element element) { if (this.length >=3D this.capacity) this.reAlloc(); this.elements[this.length] =3D element; ++ this.length; } void opBinary(string op) (Element element) if (op =3D=3D "+") { this.put(element); } } void main () { auto a1 =3D new RefArray!int(); auto a2 =3D a1; foreach (int i ; 1..8) {a2.put(i);} assert(a1.length =3D=3D a2.length); foreach (int i ; 0 .. a2.length) writef("%s=3D%s ", a1.elements[i], a2.elements[i]); writeln(); a1 + 8; a1 + 9; foreach (int i ; 0 .. a2.length) writef("%s=3D%s ", a1.elements[i], a2.elements[i]); writeln(); } -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
parent =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
On 11/09/2010 09:36 AM, spir wrote:
 On Mon, 8 Nov 2010 17:08:32 -0800
 Jonathan M Davis<jmdavisProg gmx.com>  wrote:

 As Jesse says, they _are_ passed by reference. The struct itself _is_ the
 reference.

(Well, that is a sensible redefinition of "reference"; but it is simply _not_ what the word means in any other context.) It is true that the inner, hidden, memory area (static array) containing the elements is indeed referenced, actually "pointed", from the dynamic array struct: struct ValueArray(Element) { Element* elements; uint length; } (Well, actually, this may not be a struct, but it's easier to imagine it so.) But: the dyn array itself, meaning the struct, is not referenced: "a2 = a1" copies it, as well as parameter passing. And the fact that the internal memory is referenced is an implementation detail that should *not* affect semantics. The inner pointer is there because we need some kind of indirection to implement variable-size thingies, and the means for this is pointers. This is precisely where& why people get bitten: implementation leaks out into semantics. Actually, one could conceptually replace the (pointer,length) pair by a single field of type MemoryArea -- which would be a plain value. Then, there would be no more (visible) pointer in the dyn array, right? (Actually, it would just be hidden deeper inside the MemoryArea field... but that is again implementation detail!) We should not mess up pointers used for implementation mechanics, like in the case of dyn arrays, or more generally variable size data structure, with pointers used as true references carrying semantics, like in the case of the difference between struct and class. And precisely, replacing array struct by a class, or explicitely referencing the struct, would make a *reference* dyn array type. See below an example of a primitive sort of such an array type (you can only put new elements in it ;-), implemented as class. After "a2 = a1", every change to one of the vars affects the other var; whether the change requires reallocation is irrelevant; this detail belongs to implementation, not to semantics. Now, replace class with struct and you have a type for *value* dyn arrays. Which works exactly like D ones. The assertion will fail; and output should be interesting ;-) Hope it's clear, because I cannot do better. I do not mean that D arrays are bad in any way. They work perfectly and are very efficient. Enforcing a true interface between implementation and semantics would certainly have a relevant cost in terms of space& time. But please, stop stating D arrays are referenced if you want newcomers to have a chance& understand the actual behaviour, to use them without beeing constantly bitten, and to stop& complain. Denis class RefArray(Element) { Element* elements; uint length; private uint capacity; this () { this.elements = cast(Element*) malloc(Element.sizeof); this.capacity = 1; this.length = 0; } void reAlloc() { writeln("realloc"); this.capacity *= 2; size_t memSize = this.capacity * Element.sizeof; realloc(this.elements, memSize); } void put(Element element) { if (this.length>= this.capacity) this.reAlloc(); this.elements[this.length] = element; ++ this.length; } void opBinary(string op) (Element element) if (op == "+") {

...wait! Did you just overload binary operator + to mean append?
Nov 09 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Tue, 9 Nov 2010 02:14:17 +0100
Daniel Gibson <metalcaedes gmail.com> wrote:

 On Tue, Nov 9, 2010 at 1:24 AM, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:
 The array-struct is the reference, so it is what gets compared. That me=


reference is more than an address does not make it any less a reference.

Unlike in C, a D array is more than a reference. An array is the data (or a reference to its data) + metadata (its length) - the metadata belongs to the array (and not to the reference). This means that, when you say "the array is passed by reference" one expects that also the arrays length is passed by reference - because the length belongs to the array.

Exactly. When a D object (class instance) holds value or reference fields, changing = any of them affects other variables denoting the same object, right? This i= s not true for D dyn arrays. D object implement reference semantics, while = D arrays implement value semantics. Or rather, they do it superficially (sh= allow copy). To have true value semantics, one would need a kind of this(th= is) copy constructor that also copies the target memory area adressed by th= e array's internal pointer.
 The distinction is that an Array can have its reference changed by resi=



If that is not an option for any other reference type, why should it be an option for arrays? That is just inconsistent and doesn't make any sense. =20 No, Arrays should not be considered reference types when passed to a func=

 As someone else said before: Logically you don't pass the array but a
 slice that contains the whole array.

Exactly, again. The comparison with slices makes sense. If I don't mess up = everything after all those discussions, "b =3D a;" has the same semantics a= s "b =3D a[0..$];". A new array struct is built with equal pointer & length= as the original one. Meaning both arrays _initially_ address the same memo= ry area. If D arrays were referenced instead, then no copy would happen at all, inst= ead a reference to the same struct would be created; so that later changes = would be shared -- however they internally happen, including reallocation. = Like for D objects. For people used to manually implement variable-size data structures (eg in = plain C), saying that a D dyn array is a kind of (pointer,length) tuple (a = fat pointer) defining an internal memory area (static array) is also useful= . They can imagine the internal mechanics and from there infer actual behav= iour. But saying that D dyn arrays are referenced can only bring confusion. And s= tating a parallel with D objects even more: they do _not_ behave the same w= ay. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
prev sibling next sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
 What's the general use of a = new A() in the above code? Where is it
 useful?
 
 Jens

I don't really have any good use-case examples. Maybe an initialization function? Developed your own number object (big int) and were thinking in terms of it being a refrence you thought a = a + BigInt(7); would result in a being resigned in the calling function. Or maybe just a function that swaps two class references: void swap(T)(T a, T b) { // Correct void swap(T)(ref T a, ref T b) { auto tmp = a; a = b; b = tmp; } Actually that turned out to be a pretty good one.

I do see why a = new A() is useful. But it makes only sense if I passed a/b as ref a/b. Basically I wonder why I do not get a warning when changing the reference in that situation. So my question is more why am I allowed to change the reference even though I didn't pass it as ref a. I was looking for a use of that. Assuming there is good use then there is no reason to forbid it. But if there is no good use I'd like to be warned when compiling the above swap. Because it's an error. For dynamic arrays slicing is a good example to allow it. But why should it be allowed for objects? Jens
Nov 09 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips  
<jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing the  
 size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their own  
 semantics. It does help to think of them in terms of slices (which is  
 what TDPL refers to them as), yet that does not remove the fact that  
 they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly  
 understood. For example it is common to say that arrays in C is just a  
 pointer into a memory location. But in reality that is not true.

It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science. I think of an array as a hybrid between a reference and a value type. The data is passed by reference, the length is passed by value. This mean changing the length only affects the local copy, but changing the data affects all arrays that point to that data. I think it also helps to think of arrays as slices (that's what they are anyways). There is no more distinction between slices or arrays, they are one and the same. An array does not own its data, and that is a major point of confusion. An array always just references data, it never owns it. -Steve
Nov 09 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Tue, 09 Nov 2010 07:42:13 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 I think of an array as a hybrid between a reference and a value type.  Th=

 data is passed by reference, the length is passed by value.  This mean =20
 changing the length only affects the local copy, but changing the data =20
 affects all arrays that point to that data.

The pointer is passed by value as well. Thinking at an array as a fat point= er, a (pointer,length) tuple, well, the whole tuple is a plain value. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 09 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 12 November 2010 17:55:31 Rainer Deyke wrote:
 On 11/8/2010 17:43, Jonathan M Davis wrote:
 D references are more like Java references.

That's true for class references. D also supports pass-by-reference through the 'ref' keyword, which works like C++ references.

True. But generally when talking about references, class references are what is being referred to. Personally, I would say that in the case of a ref parameter, the argument is being passed _by_ reference but not that it _is_ a reference. I suppose that the whole issue could get pretty confusing though if we're not clear on what we're referring to. - Jonathan M Davis
Nov 12 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 26 Nov 2010 19:50:27 +0000
Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.

It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.

What do you mean "depends on your definition of reference type" ? I=20 think that what a reference type is, is generally understood fairly well=

 by the majority of developers, even if understood implicitly. I think=20
 our confusion here has more to do to what we consider an "array", rather=

 than what we consider to "reference type to be. See below.
=20
=20
 I think of an array as a hybrid between a reference and a value type.
 The data is passed by reference, the length is passed by value. This
 mean changing the length only affects the local copy, but changing the
 data affects all arrays that point to that data.

Well, saying _dynamic arrays_ are a hybrid, like you mentioned, is=20 perhaps the best way to describe them, with less misunderstanding. =20 However, if I had to say whether dynamic arrays are value types or=20 reference types, I would agree with Jesse and call them reference types,=

 and I would not feel this is inaccurate. Let's try a test everyone, look=

 at this code:
=20
 void test() {
    int[] a =3D [1, 2, 3];
    int[] b =3D a;
    int[] c =3D a;
 }
=20
 and tell us, what you would you reply if asked "how many arrays are=20
 created during test's scope"? I would say "1", and not feel it is=20
 inaccurate. However, if asked how "many dynamic arrays are created=20
 during test's scope?", I would likely say "3".
=20
 This is because what I consider an array to *be*, is its (contiguous &=20
 homogeneous) elements. With that definition, then D's static arrays are=20
 value types, because when you assign a static array value to a static=20
 array variable, the underlying _array_ (ie, the contents), get copied.
 Conversely, D's dynamic arrays are reference types, because when you=20
 assign a dynamic array value to a dynamic array variable, the underlying=

 "array" (ie, the contents) are not copied, instead you get two=20
 references to exactly the same "array" (ie, same data). This viewpoint=20
 gets a bit murkier because an "array" can be contained inside another=20
 "array", but that doesn't fundamentally change the it (the viewpoint=20
 that is).

I think your explanation helps & clarify why there are misunderstandings (I= mean between people in the D community). Someone used to playing with _sta= tic_ arrays in a low-level language is used to identify an array with a mem= ory area. But at a slightly higher level, an array is seen by other people = as an element in a program like an int, a struct, a set or whatever; this a= pplies to any kind of array, not only static. The thin interface implemented by D [the (pointer,length) tuple] creates a = minimal abstraction that causes these 2 points of view to diverge in practi= ce. The structure, the array element is not referenced, but the memory area= is, well, "pointed".
 But ultimately neither view is right or wrong, it just depends on how we=

 define our terms, how we conceptualize things. However, it's probably=20
 best to stick to what the spec/TDPL says about it, if it says anything=20
 specific. (I don't think it does though :/ ) Or to avoid ambiguous=20
 designations.

In many languages, there are data structures very similar to D's dynamic ar= rays, but where the element is referenced as well. So that when newcomers r= ead that D arrays are referenced, they can only think "yes, of course", and= fall into the trap. Note that the "pointing" here has no semantic value, unlike what is usually= meant by the notion of reference type; it is instead plain internal mechan= ics, a necessity to implement a value of variable length. Either copying the area on assignment (real value type), or on the contrary= referencing the element (real ref type), would both result in array types = in which these internal mechanics are opaque to the user. Note that I don't= advocate for this; I have finally understood the efficiency of D's choice.= But it's hard to get.
 ... Oh man, I need to clear my head.

... Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 26 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 26 Nov 2010 14:50:27 -0500, Bruno Medeiros  
<brunodomedeiros+spam com.gmail> wrote:

 On 09/11/2010 12:42, Steven Schveighoffer wrote:
 On Mon, 08 Nov 2010 21:29:42 -0500, Jesse Phillips
 <jessekphillips+D gmail.com> wrote:

 Well, if you can come up with a good definition for what "increasing
 the size of a class" would do, then maybe it should be added.

 It really doesn't matter. Arrays are their own type, the have their
 own semantics. It does help to think of them in terms of slices (which
 is what TDPL refers to them as), yet that does not remove the fact
 that they are in dead a reference type.

 Many times familiar terms are used so that a behavior is quickly
 understood. For example it is common to say that arrays in C is just a
 pointer into a memory location. But in reality that is not true.

It depends on your definition of reference type. I agree with Daniel. If you want to get into academic definitions, yes, 'technically' an array is a reference, but it's confusing to someone who's not interested in exploring the theoretical parts of computer science.

What do you mean "depends on your definition of reference type" ? I think that what a reference type is, is generally understood fairly well by the majority of developers, even if understood implicitly. I think our confusion here has more to do to what we consider an "array", rather than what we consider to "reference type to be. See below.

A class is a reference type. Every operation on a class instance operates on all aliases to that class, only assignment to another class instance of the same type decouples it. Consider these two statements: "a class is a reference type" class C { int length; } foo(C c) { c.length = 5; // because a class is a reference type, this affects the original, right? } "an array is a reference type" foo(int[] arr) { arr.length = 5; // because an array is a reference type, this affects the original, right? No?!!! } This is the major confusion that I think people see. I would say people assume a "reference type" means that the reference's members are all shared between all aliases. For an array, it is one level removed, and that confuses the hell out of people. But the power gained by doing D's way is worth way waaaay more than confusing some noobs. Note that many people are used to object-based languages such as C# and Java where arrays actually *are* class instances, and therefore full reference types. -Steve
Nov 29 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Mon, 29 Nov 2010 09:13:17 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:

 A class is a reference type.  Every operation on a class instance operate=

 on all aliases to that class, only assignment to another class instance o=

 the same type decouples it.
=20
 Consider these two statements:
=20
 "a class is a reference type"
=20
 class C
 {
     int length;
 }
=20
 foo(C c)
 {
    c.length =3D 5; // because a class is a reference type, this affects t=

 original, right?
 }
=20
 "an array is a reference type"
=20
 foo(int[] arr)
 {
     arr.length =3D 5; // because an array is a reference type, this affec=

 the original, right?  No?!!!
 }
=20
 This is the major confusion that I think people see.  I would say people =

 assume a "reference type" means that the reference's members are all =20
 shared between all aliases.  For an array, it is one level removed, and =

 that confuses the hell out of people.  But the power gained by doing D's =

 way is worth way waaaay more than confusing some noobs.
=20
 Note that many people are used to object-based languages such as C# and =

 Java where arrays actually *are* class instances, and therefore full =20
 reference types.

Very good explanation of the mental issue for newcomers. Note that this doe= s not apply only to people coming from languages similar to Java or C#, but= also from most, if not all, dynamic languages (including one that are not = "officially" OO, like Lua). Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 30 2010