www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Arrays, garbage collection

reply "bearophile" <bearophileHUGS lycos.com> writes:
The D type inference for array literals is now more flexible:

void main() {
     auto[$][$] m1 = [[1, 2], [3, 4], [5, 6]];
     pragma(msg, typeof(m1));  // int[2][3]
     const auto[string] aa1 = ["red": 1, "blue": 2];
     pragma(msg, typeof(aa1)); // const(int)[string]
}


It helps avoid bugs like:

int[5] a = [1,2,4,5];
void main() {}

And it makes the usage of value arrays (fixed size arrays) more 
handy. We need to minimize the work that the garbage collector 
has to do. Value arrays help in this. So making value arrays more 
handy and natural to use is good (regardless how the current 
Phobos hates them).

- - - - - - - - -

What's missing is a handy and compiler-efficient syntax to create 
value array literals. Some persons have proposed the "[]s" syntax:


void main() {
     // Some imports here.
     foo([[1, 2]s, [3, 4]]s);
     auto t1 = tuple([1, 2]s, "red");
     auto aa1 = ["key": [1, 2]s];
     auto pairs = 10.iota.map!(i => [i, i + 10]s);
}


To do those same things without the []s syntax you have to write 
longer code.

- - - - - - - - -

Another missing handy feature regards dynamic/associative arrays 
(this syntax is currently accepted):

void main() {
     scope int[] a = [1, 2];
     scope int[int] aa = [1: 2];
}


A good management of memory ownership is needed to make "scope 
dynamic/associative arrays" fully safe. They should be more 
efficient for the GC because they can be deallocated safely when 
the scope ends, without no need for the GC to track their usage 
(if they contain references to objects that contain other 
references, then the GC has to track and manage those other 
references).

- - - - - - - - -

Sometimes I have suggested that it's useful to have in Phobos a 
kind of variable-length value arrays that are handled in a 
special way by the compiler when they are allocated in a stack 
frame, usable in  nogc functions (so it's a hybrid Phobos/D 
feature). This should give the advantages of C99 Variable Length 
Arrays without most of their disadvantages and complexities (the 
main disadvantage that's left is the risk of stack overflow. 
Currently on Windows D doesn't give a nice error message when the 
stack overflows, as it used to do).

- - - - - - - - -

With those features, with a good static management of memory 
ownership, D can become safer and reduce the work for the GC. I 
think only a limited percentage of arrays need to be fully 
handled by the GC.

Bye,
bearophile
Jan 29 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/29/15 5:08 PM, bearophile wrote:

 What's missing is a handy and compiler-efficient syntax to create value
 array literals. Some persons have proposed the "[]s" syntax:


No, we need to define a function for that - please file an enhancement 
request, thanks. -- Andrei
Jan 29 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Friday, 30 January 2015 at 02:08:23 UTC, Andrei Alexandrescu 
wrote:

 On 1/29/15 5:08 PM, bearophile wrote:

 What's missing is a handy and compiler-efficient syntax to 
 create value
 array literals. Some persons have proposed the "[]s" syntax:


 No, we need to define a function for that - please file an 
 enhancement request, thanks. -- Andrei


I don't think it is wize to add a new array literal syntax before 
:
  - added optimization to promote them on heap. Thing like int[2] 
= [1, 2] can trivially avoid GC allocation.
  - added some support for ownership/lifetime. This will allow the 
compiler to promote even more on heap.

If don't expect the new syntax to be that useful if these point 
are addressed. If I'm turn out to be wrong, we can reconsider at 
that point.
Jan 29 2015
parent "bearophile" <bearophileHUGS lycos.com> writes:
deadalnix:


  - added optimization to promote them on heap. Thing like 
 int[2] = [1, 2] can trivially avoid GC allocation.


See my usage examples (think of the examples as not having the 
"s" suffix):

void main() {
     // Some imports here.
     foo([[1, 2]s, [3, 4]]s);
     auto t1 = tuple([1, 2]s, "red");
     auto aa1 = ["key": [1, 2]s];
     auto pairs = 10.iota.map!(i => [i, i + 10]s);
}

A smart system can promote only the first one to value array. The 
compiler has to leave the other cases to the judgement of the 
programmer.



  - added some support for ownership/lifetime. This will allow 
 the compiler to promote even more on heap.


This is what I too have said in my original post. For those 
features to work well you need first a good static management of 
memory ownership.



 If don't expect the new syntax to be that useful if these point 
 are addressed. If I'm turn out to be wrong, we can reconsider 
 at that point.


I can show code similar to the examples above where the []s 
syntax is useful.

And using a function as Andrei suggests is not a good idea:

foo(staticArray(staticArray(1, 2), staticArray(3, 4)));

I don't think lot of people is going to write code like that.

And if you are about to suggest this:

alias S = staticArray;

Well, do you like to see aliases in your code?

Bye,
bearophile
Jan 30 2015
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


 No, we need to define a function for that - please file an 
 enhancement request, thanks. -- Andrei


No, the enhancement is in Bugzilla since lot of time. But it's 
for a built-in syntax. A function name is too much long, and when 
the inlining is switched off it makes the code bad. Thanks.

Bye,
bearophile
Jan 30 2015
prev sibling next sibling parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d writes:
V Fri, 30 Jan 2015 01:08:30 +0000
bearophile via Digitalmars-d <digitalmars-d puremagic.com> napsáno:

 
 It helps avoid bugs like:
 
 int[5] a = [1,2,4,5];
 


No, it makes more bugs possible

// this is compile error so its ok
int[MUST_HAVE_FIVE_ITEMS] a=[1,2,3,5];
// Error: mismatched array lengths, 5 and 4


// now this is compile error so its ok
int[$] a = [1,2,3,5]; // oops I miss one value
// but there is no error at compile time
Jan 29 2015
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Daniel Kozák:


 No, it makes more bugs possible

 // this is compile error so its ok
 int[MUST_HAVE_FIVE_ITEMS] a=[1,2,3,5];
 // Error: mismatched array lengths, 5 and 4


Have you tried to compile my code? It doesn't give an error.

Bye,
bearophile
Jan 30 2015
parent "Daniel Kozak" <kozzi11 gmail.com> writes:
On Friday, 30 January 2015 at 08:54:55 UTC, bearophile wrote:

 Daniel Kozák:


 No, it makes more bugs possible

 // this is compile error so its ok
 int[MUST_HAVE_FIVE_ITEMS] a=[1,2,3,5];
 // Error: mismatched array lengths, 5 and 4


 Have you tried to compile my code? It doesn't give an error.

 Bye,
 bearophile


Ohh I misread it. Ok it really does not trigger any error.

But still would not be better just fix this issue
Jan 30 2015
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 30 January 2015 at 07:37:07 UTC, Daniel Kozák wrote:

 V Fri, 30 Jan 2015 01:08:30 +0000
 bearophile via Digitalmars-d <digitalmars-d puremagic.com> 
 napsáno:

 
 It helps avoid bugs like:
 
 int[5] a = [1,2,4,5];
 


 No, it makes more bugs possible

 // this is compile error so its ok
 int[MUST_HAVE_FIVE_ITEMS] a=[1,2,3,5];
 // Error: mismatched array lengths, 5 and 4


 // now this is compile error so its ok
 int[$] a = [1,2,3,5]; // oops I miss one value
 // but there is no error at compile time


If you already know that it needs to have 5 elements, don't use 
`$`. Just like you wouldn't use `auto` if you want your variable 
to be `long`.
Jan 30 2015
prev sibling parent reply "Foo" <Foo test.de> writes:
On Friday, 30 January 2015 at 01:08:31 UTC, bearophile wrote:

 The D type inference for array literals is now more flexible:

 void main() {
     auto[$][$] m1 = [[1, 2], [3, 4], [5, 6]];
     pragma(msg, typeof(m1));  // int[2][3]
     const auto[string] aa1 = ["red": 1, "blue": 2];
     pragma(msg, typeof(aa1)); // const(int)[string]
 }


 It helps avoid bugs like:

 int[5] a = [1,2,4,5];
 void main() {}

 And it makes the usage of value arrays (fixed size arrays) more 
 handy. We need to minimize the work that the garbage collector 
 has to do. Value arrays help in this. So making value arrays 
 more handy and natural to use is good (regardless how the 
 current Phobos hates them).

 - - - - - - - - -

 What's missing is a handy and compiler-efficient syntax to 
 create value array literals. Some persons have proposed the 
 "[]s" syntax:


 void main() {
     // Some imports here.
     foo([[1, 2]s, [3, 4]]s);
     auto t1 = tuple([1, 2]s, "red");
     auto aa1 = ["key": [1, 2]s];
     auto pairs = 10.iota.map!(i => [i, i + 10]s);
 }


 To do those same things without the []s syntax you have to 
 write longer code.


----
 nogc
 safe
T[n] s(T = Args[0], size_t n = Args.length, Args...)(auto ref 
Args args) pure nothrow {
	return [args];
}

 nogc
 safe
T[n] s(T, size_t n)(auto ref T[n] values) pure nothrow {
	return values;
}
	
void main() {
	pragma(msg, typeof(s(1, 2, 3)));
	pragma(msg, typeof([1, 2, 3].s));
}
----

Compilation output:
int[3]
int[3]

You only have to type a dot between the array and the 's'.
Because of pure and nothrow and the low cost of the function call 
even such a lousy thing like the DMD optimizer should be capable 
of inlining such a function every time.
Jan 30 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/30/15 1:29 AM, Foo wrote:

  nogc
  safe
 T[n] s(T = Args[0], size_t n = Args.length, Args...)(auto ref Args args)
 pure nothrow {
      return [args];
 }

  nogc
  safe
 T[n] s(T, size_t n)(auto ref T[n] values) pure nothrow {
      return values;
 }

 void main() {
      pragma(msg, typeof(s(1, 2, 3)));
      pragma(msg, typeof([1, 2, 3].s));
 }
 ----

 Compilation output:
 int[3]
 int[3]

 You only have to type a dot between the array and the 's'.
 Because of pure and nothrow and the low cost of the function call even
 such a lousy thing like the DMD optimizer should be capable of inlining
 such a function every time.


Thanks for this code, it's a lot nicer and simpler than mine. -- Andrei
Jan 30 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/30/2015 6:54 AM, Andrei Alexandrescu wrote:

 On 1/30/15 1:29 AM, Foo wrote:
 Thanks for this code, it's a lot nicer and simpler than mine. -- Andrei



I'm going to swipe this for an upcoming presentation I'm doing. Thanks, Foo!
Jan 30 2015
parent "Namespace" <rswhite4 gmail.com> writes:
On Friday, 30 January 2015 at 22:21:15 UTC, Walter Bright wrote:

 On 1/30/2015 6:54 AM, Andrei Alexandrescu wrote:

 On 1/30/15 1:29 AM, Foo wrote:
 Thanks for this code, it's a lot nicer and simpler than mine. 
 -- Andrei



 I'm going to swipe this for an upcoming presentation I'm doing. 
 Thanks, Foo!


My pleasure.
Jan 30 2015
prev sibling parent reply "Martin Nowak" <code dawg.eu> writes:
 Thanks for this code, it's a lot nicer and simpler than mine. 
 -- Andrei


So, pull request for std.array and reverting the dollar thingie?
Problem solved?
Jan 31 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/15 1:10 AM, Martin Nowak wrote:

 Thanks for this code, it's a lot nicer and simpler than mine. -- Andrei


 So, pull request for std.array and reverting the dollar thingie?
 Problem solved?


This looks like the way to go. I apologize to the community for the 
indecision shown. -- Andrei
Jan 31 2015
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/31/2015 8:08 AM, Andrei Alexandrescu wrote:

 On 1/31/15 1:10 AM, Martin Nowak wrote:

 Thanks for this code, it's a lot nicer and simpler than mine. -- Andrei


 So, pull request for std.array and reverting the dollar thingie?
 Problem solved?


 This looks like the way to go. I apologize to the community for the indecision
 shown. -- Andrei


I'm on board with this, too.
Jan 31 2015