bearophile <bearophileHUGS lycos.com> writes:

Do you remember if this bug is in Bugzilla?


import std.algorithm;
void main() {
    enum a = [3, 1, 2];
    enum s = sort(a);
    assert(equal(a, [3, 1, 2]));
    assert(equal(s, [1, 2, 3]));
}


Bye,
bearophile

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Sunday, November 13, 2011 17:54:14 bearophile wrote:
 Do you remember if this bug is in Bugzilla?
 
 
 import std.algorithm;
 void main() {
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever 
code you used them in. So,

enum a = [3, 1, 2];
enum s = sort(a);

is equivalent to

enum a = [3, 1, 2];
enum s = sort([3, 1, 2]);

a isn't altered at all, because a doesn't really exist. Notice that

assert(equal(a, [3, 1, 2]));

still passes. It's equivalent to

assert(equal([3, 1, 2], [3, 1, 2]));

- Jonathan M Davis

bearophile <bearophileHUGS lycos.com> writes:

Jonathan M Davis:

 import std.algorithm;
 void main() {
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 }

 
 It's not a bug. Those an manifest constants. They're copy-pasted into whatever 
 code you used them in. So,
 
 enum a = [3, 1, 2];
 enum s = sort(a);
 
 is equivalent to
 
 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort
in-place a "constant". I have to stop thinking of them as constants. I don't
like this design of enums...

On the other hand this gives the error message I was looking for, until today I
didn't even think about const enums:

import std.algorithm;
const enum a = [1, 2];
void main() {
    sort(a);
}


So I guess I'll start using "cont enum" and "immutable enum" instead of enums
:-)

Bye,
bearophile

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Sunday, November 13, 2011 19:02:01 bearophile wrote:
 Jonathan M Davis:
 import std.algorithm;
 void main() {
 
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 
 }

 
 It's not a bug. Those an manifest constants. They're copy-pasted into
 whatever code you used them in. So,
 
 enum a = [3, 1, 2];
 enum s = sort(a);
 
 is equivalent to
 
 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 
 You are right, there's no DMD bug here. Yet, it's a bit surprising to sort
 in-place a "constant". I have to stop thinking of them as constants. I
 don't like this design of enums...
 
 On the other hand this gives the error message I was looking for, until
 today I didn't even think about const enums:
 
 import std.algorithm;
 const enum a = [1, 2];
 void main() {
     sort(a);
 }
 
 
 So I guess I'll start using "cont enum" and "immutable enum" instead of
 enums :-)

It depends entirely on what you're trying to do. If you understand how 
manifest constants work, then they can be quite advantageous. What you 
probably really want for arrays though is not an enum but just a const or 
immutable module variable.

immutable a = [3, 1, 2];

Otherwise, you're allocating a new array every time you use the enum. So, use 
a manifest constant when you want to avoid having it take up any memory but 
don't care about whatever allocations may occur when it's used (primitive 
types such as ints being a prime example), whereas if you want an actual 
memory location for the constant and/or want it to be allocated only once, 
then use variable rather than an enum.

- Jonathan M Davis

so <so so.so> writes:

On Mon, 14 Nov 2011 02:09:40 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 It depends entirely on what you're trying to do. If you understand how
 manifest constants work, then they can be quite advantageous. What you
 probably really want for arrays though is not an enum but just a const or
 immutable module variable.

 immutable a = [3, 1, 2];

 Otherwise, you're allocating a new array every time you use the enum.  
 So, use
 a manifest constant when you want to avoid having it take up any memory  
 but
 don't care about whatever allocations may occur when it's used (primitive
 types such as ints being a prime example), whereas if you want an actual
 memory location for the constant and/or want it to be allocated only  
 once,
 then use variable rather than an enum.

 - Jonathan M Davis

Wait a second, it is definitely a bug. You can't modify an enum.

"immutable a = [3, 1, 2];" is practically "enum a = [3, 1, 2];".

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Monday, November 14, 2011 02:16:57 so wrote:
 On Mon, 14 Nov 2011 02:09:40 +0200, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 It depends entirely on what you're trying to do. If you understand how
 manifest constants work, then they can be quite advantageous. What you
 probably really want for arrays though is not an enum but just a const
 or
 immutable module variable.
 
 immutable a = [3, 1, 2];
 
 Otherwise, you're allocating a new array every time you use the enum.
 So, use
 a manifest constant when you want to avoid having it take up any memory
 but
 don't care about whatever allocations may occur when it's used
 (primitive
 types such as ints being a prime example), whereas if you want an actual
 memory location for the constant and/or want it to be allocated only
 once,
 then use variable rather than an enum.
 
 - Jonathan M Davis

 
 Wait a second, it is definitely a bug. You can't modify an enum.
 
 "immutable a = [3, 1, 2];" is practically "enum a = [3, 1, 2];".

No. Those are two _very_ different things.

immutable a = [3, 1, 2];

creates a variable of type immutable(int[]). You takes up memory. You can take 
its address - e.g. &a. It exists even if you don't use it at all.

enum a = [3, 1, 2];

on the other hand, doesn't create any variable at all. It's what's called a 
manifest constant. It's a placeholder. It takes up no memory. You can't take 
its address. If you never use it, it doesn't end up in the generated program 
at all. Every use of a effectively copies the value of a to wherever its used. 
So,

enum a = [3, 1, 2];
sort(a);

is _identical_ to

sort([3, 1, 2]);

That means that if you use a in any code, it's copied in that code such that 
you could end up allocating a lot of memory that you didn't intend to if you 
heavily use manifest constants which are strings or arrays. But it also means 
that you get a new copy to use at each location, which can also be useful. And 
for types that don't end up on the heap (e.g. int), there's really no cost to 
it, and it actually is _more_ efficient (albeit marginally), since it avoids 
having to allocate any memory for the enum itself, since it's not a variable.

- Jonathan M Davis

so <so so.so> writes:

On Mon, 14 Nov 2011 02:50:50 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 No. Those are two _very_ different things.

 immutable a = [3, 1, 2];

 creates a variable of type immutable(int[]). You takes up memory. You  
 can take
 its address - e.g. &a. It exists even if you don't use it at all.

 enum a = [3, 1, 2];

 on the other hand, doesn't create any variable at all. It's what's  
 called a
 manifest constant. It's a placeholder. It takes up no memory. You can't  
 take
 its address. If you never use it, it doesn't end up in the generated  
 program
 at all. Every use of a effectively copies the value of a to wherever its  
 used.
 So,

 enum a = [3, 1, 2];
 sort(a);

 is _identical_ to

 sort([3, 1, 2]);

 That means that if you use a in any code, it's copied in that code such  
 that
 you could end up allocating a lot of memory that you didn't intend to if  
 you
 heavily use manifest constants which are strings or arrays. But it also  
 means
 that you get a new copy to use at each location, which can also be  
 useful. And
 for types that don't end up on the heap (e.g. int), there's really no  
 cost to
 it, and it actually is _more_ efficient (albeit marginally), since it  
 avoids
 having to allocate any memory for the enum itself, since it's not a  
 variable.

 - Jonathan M Davis

Trying to remember the discussions on digitalmars.D regarding  
enum/immutable. There were contradicting opinions.
You ask the same question at two different times, you get at least two  
different answers, especially on this matter.
There was/is not a consensus on how it works or should work.

While i think enum/immutable are not same thing. Because with enum you  
mean it is a compile time value and compile time only, but immutable might  
rely on runtime initializers. So it differs here. This just means enum  
gives you more guaranties, not less.

enum a = 5;
immutable b = 6;
a = 3;
b = 3;

If the above code is invalid for both enum and immutable, so should the  
"in-place" sort case. I understand how it works, but it doesn't make any  
sense why it should work that way.

Jonathan M Davis <jmdavisProg gmx.com> writes:

On Monday, November 14, 2011 04:03:54 so wrote:
 Trying to remember the discussions on digitalmars.D regarding
 enum/immutable. There were contradicting opinions.
 You ask the same question at two different times, you get at least two
 different answers, especially on this matter.
 There was/is not a consensus on how it works or should work.
 
 While i think enum/immutable are not same thing. Because with enum you
 mean it is a compile time value and compile time only, but immutable might
 rely on runtime initializers. So it differs here. This just means enum
 gives you more guaranties, not less.
 
 enum a = 5;
 immutable b = 6;
 a = 3;
 b = 3;
 
 If the above code is invalid for both enum and immutable, so should the
 "in-place" sort case. I understand how it works, but it doesn't make any
 sense why it should work that way.

It works exactly as it's designed to work. There are people who disagree with 
how it's designed, but it works exactly as designed. The above is invalid, 
because you can't assign to an enum - it's not a variable - and you can't 
assign to an immutable variable. sort works with an enum because you're 
passing it a valid array which is _not_ immutable. It's a new array just as if 
you'd passed it the enum's value directly. That may be surprising to some, but 
it's exactly as designed and is not a bug. If you think that it should work 
differently, then you're arguing against the current design, which is a 
completely different discussion.

- Jonathan M Davis

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
      enum a = [3, 1, 2];
      enum s = sort(a);
      assert(equal(a, [3, 1, 2]));
      assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to sort
in-place a "constant". I have to stop thinking of them as constants. I don't
like this design of enums...

It is the right design. Why should enum imply const or immutable? (or 
inout, for that matter). They are completely orthogonal.

enum Enum{
     opt1,
     opt2,
}

void main(){
     auto moo = Enum.opt1;
     moo = Enum.opt2; // who would seriously want an error here???
}


enum a = [1,2,3];

void main(){
     auto x = a;
     x = [2,1,3]; // ditto
}


 On the other hand this gives the error message I was looking for, until today
I didn't even think about const enums:

 import std.algorithm;
 const enum a = [1, 2];
 void main() {
      sort(a);
 }


 So I guess I'll start using "cont enum" and "immutable enum" instead of enums
:-)

You can do that, but they are not a full replacement. How would you get 
a sorted version of such an enum, for instance? =)

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 09:27 AM, Timon Gehr wrote:
 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted into
 whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 enum Enum{
 opt1,
 opt2,
 }

 void main(){
 auto moo = Enum.opt1;
 moo = Enum.opt2; // who would seriously want an error here???
 }


 enum a = [1,2,3];

 void main(){
 auto x = a;
 x = [2,1,3]; // ditto
 }


 On the other hand this gives the error message I was looking for,
 until today I didn't even think about const enums:

 import std.algorithm;
 const enum a = [1, 2];
 void main() {
 sort(a);
 }


 So I guess I'll start using "cont enum" and "immutable enum" instead
 of enums :-)

 You can do that, but they are not a full replacement. How would you get
 a sorted version of such an enum, for instance? =)

(.dup, obviously.)

so <so so.so> writes:

On Mon, 14 Nov 2011 10:27:21 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 It is the right design. Why should enum imply const or immutable? (or  
 inout, for that matter). They are completely orthogonal.

 enum Enum{
      opt1,
      opt2,
 }

 void main(){
      auto moo = Enum.opt1;
      moo = Enum.opt2; // who would seriously want an error here???
 }

You are missing the point, nobody asked that. You are assigning it to  
auto, a runtime variable.
Which was asked was about modifying a constant, sort(a) means sort a  
in-place. So you cant do:

immutable a;
sort(a);

But with current design you can do:

enum a;
sort(a);

Which is to me, quite wrong.

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 10:20 AM, so wrote:
 On Mon, 14 Nov 2011 10:27:21 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 enum Enum{
 opt1,
 opt2,
 }

 void main(){
 auto moo = Enum.opt1;
 moo = Enum.opt2; // who would seriously want an error here???
 }

 You are missing the point, nobody asked that.

I think you are missing the point. What else are you asking for?

 You are assigning it to
 auto, a runtime variable.
 Which was asked was about modifying a constant, sort(a) means sort a
 in-place. So you cant do:

 immutable a;
 sort(a);

 But with current design you can do:

 enum a;
 sort(a);

 Which is to me, quite wrong.

It is just as right or wrong as doing

enum a;
sort([1,2,3]);

The design of enums is probably even irrelevant for this discussion.

Dejan Lekic <dejan.lekic gmail.com> writes:

so wrote:

 
 immutable a;
 sort(a);
 
 But with current design you can do:
 
 enum a;
 sort(a);
 
 Which is to me, quite wrong.

It is not wrong at all.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
      enum a = [3, 1, 2];
      enum s = sort(a);
      assert(equal(a, [3, 1, 2]));
      assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted into  
 whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to  
 sort in-place a "constant". I have to stop thinking of them as  
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or  
 inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used  
on an array.

Consider that:

enum a = "hello";

foo(a);

Does not allocate heap memory, even though "hello" is a reference type.

However:

enum a = ['h', 'e', 'l', 'l', 'o'];

foo(a);

Allocates heap memory every time a is *used*.  This is counter-intuitive,  
one uses enum to define things using the compiler, not during runtime.   
It's used to invoke CTFE, to avoid heap allocation.  It's not a glorified  
#define macro.

The deep issue here is not that enum is used as a manifest constant, but  
rather the fact that enum can map to a *function call* rather than the  
*result* of that function call.

Would you say this should be acceptable?

enum a = malloc(5);

foo(a); // calls malloc(5) and passes the result to foo.

If the [...] form is an acceptable enum, I contend that malloc should be  
acceptable as well.

My view is that enum should only be acceptable on data that is immutable,  
or implicitly cast to immutable, and should *never* map to an expression  
that calls a function during runtime.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted
 into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 There is definitely some debatable practice here for wherever enum is
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*. This is counter-intuitive,
 one uses enum to define things using the compiler, not during runtime.
 It's used to invoke CTFE, to avoid heap allocation. It's not a glorified
 #define macro.

 The deep issue here is not that enum is used as a manifest constant, but
 rather the fact that enum can map to a *function call* rather than the
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should be
 acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 
'o'].

enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on 
every use

But malloc is not CTFE-able, that is why it fails.


 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.

 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all.
But being stored as enum should not imply restrictions on type qualifiers.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted
 into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 There is definitely some debatable practice here for wherever enum is
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*. This is counter-intuitive,
 one uses enum to define things using the compiler, not during runtime.
 It's used to invoke CTFE, to avoid heap allocation. It's not a glorified
 #define macro.

 The deep issue here is not that enum is used as a manifest constant, but
 rather the fact that enum can map to a *function call* rather than the
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should be
 acceptable as well.

 a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l',  
 'o'].

 enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on  
 every use

 But malloc is not CTFE-able, that is why it fails.

You are comparing apples to oranges here.  Whether it's CTFE able or not  
has nothing to do with it, since the code is executed at runtime, not  
compile time.

 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

 Too restrictive imho.

It allows the compiler to evaluate the enum at compile time, and store any  
referenced data in ROM, avoiding frequent heap allocations (similar to  
string literals).

IMO, type freedom is lower on the priority list than performance.

You can already define a symbol that calls arbitrary code at runtime:

 property int[] a() { return [3, 1, 2];}

Why should we muddy enum's goals with also being able to call functions  
during runtime?

 and should *never* map to an
 expression that calls a function during runtime.

 Well, I would not miss that at all.
 But being stored as enum should not imply restrictions on type  
 qualifiers.

The restrictions are required in order to avoid calling runtime functions  
for enum usage.  Without the restrictions, you must necessarily call  
runtime functions for any reference-based types (to avoid modifying the  
original).

Note that I'm not saying literals in general should not trigger heap  
allocations, I'm saying assigning such literals to enums should require  
unrestricted copying without runtime function calls.

I don't think you would miss this as much as you think.  Assigning a  
non-immutable array from an immutable one is as easy as adding a .dup, and  
then the code is more clear that an allocation is taking place.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted
 into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 There is definitely some debatable practice here for wherever enum is
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*. This is counter-intuitive,
 one uses enum to define things using the compiler, not during runtime.
 It's used to invoke CTFE, to avoid heap allocation. It's not a glorified
 #define macro.

 The deep issue here is not that enum is used as a manifest constant, but
 rather the fact that enum can map to a *function call* rather than the
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should be
 acceptable as well.

 a indeed refers to the result of the evaluation of ['h', 'e', 'l',
 'l', 'o'].

 enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on
 every use

 But malloc is not CTFE-able, that is why it fails.

 You are comparing apples to oranges here. Whether it's CTFE able or not
 has nothing to do with it, since the code is executed at runtime, not
 compile time.

The code is executed at compile time. It is just that the value is later 
created by allocating at runtime.

enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because 
writeln is not ctfe-able.


 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

 Too restrictive imho.

 It allows the compiler to evaluate the enum at compile time, and store
 any referenced data in ROM, avoiding frequent heap allocations (similar
 to string literals).

 IMO, type freedom is lower on the priority list than performance.

 You can already define a symbol that calls arbitrary code at runtime:

  property int[] a() { return [3, 1, 2];}

 Why should we muddy enum's goals with also being able to call functions
 during runtime?

As I said, I would not miss the capability of enums to create mutable 
arrays a lot. Usually you don't want that behavior, and explicitly 
.dup-ing is just fine.

But I think it is a bit exaggerated to say enums can call functions at 
runtime. It is up to the compiler how to implement the array allocation.

 and should *never* map to an
 expression that calls a function during runtime.

 Well, I would not miss that at all.
 But being stored as enum should not imply restrictions on type
 qualifiers.

 The restrictions are required in order to avoid calling runtime
 functions for enum usage. Without the restrictions, you must necessarily
 call runtime functions for any reference-based types (to avoid modifying
 the original).

Yes, I don't need that. But I don't really want compile time 
capabilities hampered.

enum a = [2,1,4];
enum b = sort(a); // should be fine.

auto c = a;
// sort(c); // don't care a lot if this works

 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

Yes, I get that. And I think it makes sense. But I am not (yet?) 
convinced that the solution to make all enums non-assignable, 
head-mutable and tail-immutable is satisfying.

 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

It would be somewhat odd.

enum a = [2,1,4];
enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

enum c = a.dup;   // does this implicitly convert to immutable, or what 
happens here?
enum d = sort(c); // does not work?

enum e = foo(a.dup, b.dup, c.dup, d.dup);

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Mon, 14 Nov 2011 14:59:50 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted
 into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit surprising  
 to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 There is definitely some debatable practice here for wherever enum is
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference  
 type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*. This is  
 counter-intuitive,
 one uses enum to define things using the compiler, not during runtime.
 It's used to invoke CTFE, to avoid heap allocation. It's not a  
 glorified
 #define macro.

 The deep issue here is not that enum is used as a manifest constant,  
 but
 rather the fact that enum can map to a *function call* rather than the
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should  
 be
 acceptable as well.

 a indeed refers to the result of the evaluation of ['h', 'e', 'l',
 'l', 'o'].

 enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on
 every use

 But malloc is not CTFE-able, that is why it fails.

 You are comparing apples to oranges here. Whether it's CTFE able or not
 has nothing to do with it, since the code is executed at runtime, not
 compile time.

 The code is executed at compile time. It is just that the value is later  
 created by allocating at runtime.

 enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because  
 writeln is not ctfe-able.

Look at the code generated for enum a = [1, 2, 3].  using a is replaced  
with a call to _d_arrayliteral.  There is no CTFE going on.

 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

 Too restrictive imho.

 It allows the compiler to evaluate the enum at compile time, and store
 any referenced data in ROM, avoiding frequent heap allocations (similar
 to string literals).

 IMO, type freedom is lower on the priority list than performance.

 You can already define a symbol that calls arbitrary code at runtime:

  property int[] a() { return [3, 1, 2];}

 Why should we muddy enum's goals with also being able to call functions
 during runtime?

 As I said, I would not miss the capability of enums to create mutable  
 arrays a lot. Usually you don't want that behavior, and explicitly  
 .dup-ing is just fine.

 But I think it is a bit exaggerated to say enums can call functions at  
 runtime. It is up to the compiler how to implement the array allocation.

The compiler has no choice.  It must develop the array at runtime, or else  
the type allows one to modify the source value (just like in D1 how you  
could modify string literals).  In essence, the compiler is creating a new  
copy for every usage (and building it from scratch).

 and should *never* map to an
 expression that calls a function during runtime.

 Well, I would not miss that at all.
 But being stored as enum should not imply restrictions on type
 qualifiers.

 The restrictions are required in order to avoid calling runtime
 functions for enum usage. Without the restrictions, you must necessarily
 call runtime functions for any reference-based types (to avoid modifying
 the original).

 Yes, I don't need that. But I don't really want compile time  
 capabilities hampered.

 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

I was actually surprised that this compiles.  But this should not be a  
problem even if a was immutable(int)[].  sort should be able to create a  
copy of an immutable array in order to sort it.  It doesn't matter the  
performance hit, because this should all be done at compile time.

 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

 Yes, I get that. And I think it makes sense. But I am not (yet?)  
 convinced that the solution to make all enums non-assignable,  
 head-mutable and tail-immutable is satisfying.

When I see an enum, I think "evaluated at compile time".  No matter how  
complex it is to build that value, it should be built at compile-time and  
*used* at runtime.  No complex function calls should be done at runtime,  
an enum is a value.

I did an interesting little test:

import std.algorithm;
import std.stdio;

int[] foo(int[] x)
{
     return x ~ x;
}
enum a = [3, 1, 2];
enum b = sort(foo(foo(foo(a))));

void main()
{
     writeln(b);
}

Want to see the assembly generated for the writeln call?

                 push    018h
                 mov     EAX,offset FLAT:_D11TypeInfo_Ai6__initZ SYM32
                 push    EAX
                 call      _d_arrayliteralTX PC32
                 add     ESP,8
                 mov     ECX,1
                 mov     [EAX],ECX
                 mov     4[EAX],ECX
                 mov     8[EAX],ECX
                 mov     0Ch[EAX],ECX
                 mov     010h[EAX],ECX
                 mov     014h[EAX],ECX
                 mov     018h[EAX],ECX
                 mov     01Ch[EAX],ECX
                 mov     EDX,2
                 mov     020h[EAX],EDX
                 mov     024h[EAX],EDX
                 mov     028h[EAX],EDX
                 mov     02Ch[EAX],EDX
                 mov     030h[EAX],EDX
                 mov     034h[EAX],EDX
                 mov     038h[EAX],EDX
                 mov     03Ch[EAX],EDX
                 mov     EBX,3
                 mov     040h[EAX],EBX
                 mov     044h[EAX],EBX
                 mov     048h[EAX],EBX
                 mov     04Ch[EAX],EBX
                 mov     050h[EAX],EBX
                 mov     054h[EAX],EBX
                 mov     058h[EAX],EBX
                 mov     05Ch[EAX],EBX
                 mov     ECX,EAX
                 mov     EAX,018h
                 mov     -8[EBP],EAX
                 mov     -4[EBP],ECX
                 mov     EDX,-4[EBP]
                 mov     EAX,-8[EBP]
                 push    EDX
                 push    EAX
                 call       
_D3std5stdio76__T7writelnTS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZ7writelnFS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZv PC32


Really?  That's a better solution than using ROM space to store the result  
of the expression as evaluated at compile time?  The worst part is that  
this will be used *EVERY TIME* I use the enum b (even if I pass it as a  
const array).

 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

 It would be somewhat odd.

 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

I don't think .dup should be necessary at compile time.  Creating a sorted  
copy of an immutable array should be quite doable.

 enum c = a.dup;   // does this implicitly convert to immutable, or what  
 happens here?

Either a compile error (cannot store mutable reference data as an enum),  
or an implicit conversion back to immutable.

 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

Again, I don't think .dup would be used for dependent enums, I was rather  
thinking dup would be used where you need a mutable copy of an array  
during enum usage in normal code.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 14:59:50 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

 It's not a bug. Those an manifest constants. They're copy-pasted
 into whatever
 code you used them in. So,

 enum a = [3, 1, 2];
 enum s = sort(a);

 is equivalent to

 enum a = [3, 1, 2];
 enum s = sort([3, 1, 2]);

 You are right, there's no DMD bug here. Yet, it's a bit
 surprising to
 sort in-place a "constant". I have to stop thinking of them as
 constants. I don't like this design of enums...

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 There is definitely some debatable practice here for wherever enum is
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference
 type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*. This is
 counter-intuitive,
 one uses enum to define things using the compiler, not during runtime.
 It's used to invoke CTFE, to avoid heap allocation. It's not a
 glorified
 #define macro.

 The deep issue here is not that enum is used as a manifest
 constant, but
 rather the fact that enum can map to a *function call* rather than the
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc
 should be
 acceptable as well.

 a indeed refers to the result of the evaluation of ['h', 'e', 'l',
 'l', 'o'].

 enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on
 every use

 But malloc is not CTFE-able, that is why it fails.

 You are comparing apples to oranges here. Whether it's CTFE able or not
 has nothing to do with it, since the code is executed at runtime, not
 compile time.

 The code is executed at compile time. It is just that the value is
 later created by allocating at runtime.

 enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because
 writeln is not ctfe-able.

 Look at the code generated for enum a = [1, 2, 3]. using a is replaced
 with a call to _d_arrayliteral. There is no CTFE going on.

There is some ctfe going on, but the compiler has to allocate the result 
anew every time it is used. So there is also some runtime overhead.

To make my point clearer:

int foo(){return 100;}
enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100, 100];

void main(){
     auto x = a; // this does *not* call foo. But it allocates a new 
array literal
}


 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

 Too restrictive imho.

 It allows the compiler to evaluate the enum at compile time, and store
 any referenced data in ROM, avoiding frequent heap allocations (similar
 to string literals).

 IMO, type freedom is lower on the priority list than performance.

 You can already define a symbol that calls arbitrary code at runtime:

  property int[] a() { return [3, 1, 2];}

 Why should we muddy enum's goals with also being able to call functions
 during runtime?

 As I said, I would not miss the capability of enums to create mutable
 arrays a lot. Usually you don't want that behavior, and explicitly
 .dup-ing is just fine.

 But I think it is a bit exaggerated to say enums can call functions at
 runtime. It is up to the compiler how to implement the array allocation.

 The compiler has no choice. It must develop the array at runtime, or
 else the type allows one to modify the source value (just like in D1 how
 you could modify string literals). In essence, the compiler is creating
 a new copy for every usage (and building it from scratch).

That is a quality of implementation issue. The language semantics do not 
require that.

 and should *never* map to an
 expression that calls a function during runtime.

 Well, I would not miss that at all.
 But being stored as enum should not imply restrictions on type
 qualifiers.

 The restrictions are required in order to avoid calling runtime
 functions for enum usage. Without the restrictions, you must necessarily
 call runtime functions for any reference-based types (to avoid modifying
 the original).

 Yes, I don't need that. But I don't really want compile time
 capabilities hampered.

 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

 I was actually surprised that this compiles. But this should not be a
 problem even if a was immutable(int)[]. sort should be able to create a
 copy of an immutable array in order to sort it. It doesn't matter the
 performance hit, because this should all be done at compile time.

It does not, but explicitly calling .dup works
immutable x = [3,2,1];
immutable y = sort(x.dup);

 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

 Yes, I get that. And I think it makes sense. But I am not (yet?)
 convinced that the solution to make all enums non-assignable,
 head-mutable and tail-immutable is satisfying.

 When I see an enum, I think "evaluated at compile time". No matter how
 complex it is to build that value, it should be built at compile-time
 and *used* at runtime. No complex function calls should be done at
 runtime, an enum is a value.

Exactly. Therefore you assign from it by copying it.

Compare to static array.

int[10] x = [1,2,3,4,5,6,7,8,9,0];

x still needs to be initialized at runtime.

 I did an interesting little test:

 import std.algorithm;
 import std.stdio;

 int[] foo(int[] x)
 {
 return x ~ x;
 }
 enum a = [3, 1, 2];
 enum b = sort(foo(foo(foo(a))));

 void main()
 {
 writeln(b);
 }

 Want to see the assembly generated for the writeln call?

 push 018h
 mov EAX,offset FLAT:_D11TypeInfo_Ai6__initZ SYM32
 push EAX
 call _d_arrayliteralTX PC32
 add ESP,8
 mov ECX,1
 mov [EAX],ECX
 mov 4[EAX],ECX
 mov 8[EAX],ECX
 mov 0Ch[EAX],ECX
 mov 010h[EAX],ECX
 mov 014h[EAX],ECX
 mov 018h[EAX],ECX
 mov 01Ch[EAX],ECX
 mov EDX,2
 mov 020h[EAX],EDX
 mov 024h[EAX],EDX
 mov 028h[EAX],EDX
 mov 02Ch[EAX],EDX
 mov 030h[EAX],EDX
 mov 034h[EAX],EDX
 mov 038h[EAX],EDX
 mov 03Ch[EAX],EDX
 mov EBX,3
 mov 040h[EAX],EBX
 mov 044h[EAX],EBX
 mov 048h[EAX],EBX
 mov 04Ch[EAX],EBX
 mov 050h[EAX],EBX
 mov 054h[EAX],EBX
 mov 058h[EAX],EBX
 mov 05Ch[EAX],EBX
 mov ECX,EAX
 mov EAX,018h
 mov -8[EBP],EAX
 mov -4[EBP],ECX
 mov EDX,-4[EBP]
 mov EAX,-8[EBP]
 push EDX
 push EAX
 call
 _D3std5stdio76__T7writelnTS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZ7writelnFS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZv PC32



 Really? That's a better solution than using ROM space to store the
 result of the expression as evaluated at compile time? The worst part is
 that this will be used *EVERY TIME* I use the enum b (even if I pass it
 as a const array).

That just tells us that DMD sucks at generating code for array literals.

This generates identical code:

import std.stdio;

void main() {
     writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 
3, 3, 3, 3, 3]);
}

You don't need enums for that.


What it actually should for both our examples is more like the following:

import std.stdio;

immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 
2, 2, 3, 3, 3, 3, 3, 3, 3, 3];

void main() {
     writeln(_somewhereinrom.dup);
}

push   %ebp
mov    %esp,%ebp
pushl  0x8097184
pushl  0x8097180
mov    $0x80975c8,%eax
push   %eax
call   8079470 <_adDupT>
add    $0xc,%esp
push   %edx
push   %eax
call   807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
xor    %eax,%eax
pop    %ebp
ret


If writeln would actually be const correct, the compiler could even get 
rid of the allocation.

This is not about enums that much, it is about array literals.

The fact that stack static array initialization allocates is one of DMDs 
bigger warts.

Look at the ridiculous code generated for the following example:

void main() {
     int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 
3, 3, 3, 3, 3, 3];
     writeln(x);
}


 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

 It would be somewhat odd.

 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

 I don't think .dup should be necessary at compile time. Creating a
 sorted copy of an immutable array should be quite doable.

I agree, phobos won't currently do it though.

 enum c = a.dup; // does this implicitly convert to immutable, or what
 happens here?

 Either a compile error (cannot store mutable reference data as an enum),
 or an implicit conversion back to immutable.

 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

 Again, I don't think .dup would be used for dependent enums, I was
 rather thinking dup would be used where you need a mutable copy of an
 array during enum usage in normal code.

But if the type of a,b,c,d is immutable(int)[] and foo is a function 
that takes 4 int[]s then the .dup's are necessary to pass type checking.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 Look at the code generated for enum a = [1, 2, 3]. using a is replaced
 with a call to _d_arrayliteral. There is no CTFE going on.

 There is some ctfe going on, but the compiler has to allocate the result  
 anew every time it is used. So there is also some runtime overhead.

 To make my point clearer:

 int foo(){return 100;}
 enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100,  
 100];

 void main(){
      auto x = a; // this does *not* call foo. But it allocates a new  
 array literal
 }

Yes, you are right.  The issue is that the resulting array is initialized  
at runtime, not that CTFE is being avoided.  After doing some of these  
tests, I have a better understanding of the issues.

 The compiler has no choice. It must develop the array at runtime, or
 else the type allows one to modify the source value (just like in D1 how
 you could modify string literals). In essence, the compiler is creating
 a new copy for every usage (and building it from scratch).

 That is a quality of implementation issue. The language semantics do not  
 require that.

The language semantics require that if an enum type points at mutable  
data, a runtime allocation *must* occur to avoid corruption of literals.   
I think a rule requiring an enum to be immutable or implicitly cast to  
immutable puts the burden of runtime allocation on the coder, making it  
clear what's going on.

In C++, novice coders typically pass classes by value not knowing what a  
horrible thing this is doing.  Then they are puzzled why the code is so  
slow, the syntax is so short!  This is another case of a hidden allocation  
which can be either avoided or made visible.

 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

 I was actually surprised that this compiles. But this should not be a
 problem even if a was immutable(int)[]. sort should be able to create a
 copy of an immutable array in order to sort it. It doesn't matter the
 performance hit, because this should all be done at compile time.

 It does not, but explicitly calling .dup works
 immutable x = [3,2,1];
 immutable y = sort(x.dup);

I'm saying sort (or another symbol, ctfesort?) can be made to do the dup  
automatically for you so you don't have to have it when using ctfe.  Extra  
allocations during CTFE cost nothing (well, with a properly GC'd compiler,  
that is).

Update: I have a better idea, see below.

 When I see an enum, I think "evaluated at compile time". No matter how
 complex it is to build that value, it should be built at compile-time
 and *used* at runtime. No complex function calls should be done at
 runtime, an enum is a value.

 Exactly. Therefore you assign from it by copying it.

 Compare to static array.

 int[10] x = [1,2,3,4,5,6,7,8,9,0];

 x still needs to be initialized at runtime.

Yes, but this is spelled out because copying a static array requires  
moving data.  However, this does *not* require a hidden allocation (even  
though it does do a hidden allocation currently).

I'm not worried about copying data as much as I am about hidden  
allocations.  Hidden allocations are a huge drag on performance.  Every  
time you allocate, you need to take a global GC lock, and it's an  
unbounded operation (doing one allocation could run a collection cycle).

 I did an interesting little test:


[snip]

 That just tells us that DMD sucks at generating code for array literals.

 This generates identical code:

 import std.stdio;

 void main() {
      writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3,  
 3, 3, 3, 3, 3]);
 }

 You don't need enums for that.


 What it actually should for both our examples is more like the following:

 import std.stdio;

 immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,  
 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];

 void main() {
      writeln(_somewhereinrom.dup);
 }

 push   %ebp
 mov    %esp,%ebp
 pushl  0x8097184
 pushl  0x8097180
 mov    $0x80975c8,%eax
 push   %eax
 call   8079470 <_adDupT>
 add    $0xc,%esp
 push   %edx
 push   %eax
 call   807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
 xor    %eax,%eax
 pop    %ebp
 ret


 If writeln would actually be const correct, the compiler could even get  
 rid of the allocation.

That is the idea.  Get rid of the hidden allocation.  Writeln *is* const  
correct, it can certainly print immutable(int)[].  The issue is not  
writeln, it's what the type of the array  literal/enum is.

Technically, an array literal is equivalent to an enum, and should follow  
the same rules.

 This is not about enums that much, it is about array literals.

 The fact that stack static array initialization allocates is one of DMDs  
 bigger warts.

 Look at the ridiculous code generated for the following example:

 void main() {
      int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3,  
 3, 3, 3, 3, 3, 3];
      writeln(x);
 }

Yes, these are all cases of the same issue.

 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

 I don't think .dup should be necessary at compile time. Creating a
 sorted copy of an immutable array should be quite doable.

 I agree, phobos won't currently do it though.

This is easily fixed.  But maybe there is a better way (see below).

 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

 Again, I don't think .dup would be used for dependent enums, I was
 rather thinking dup would be used where you need a mutable copy of an
 array during enum usage in normal code.

 But if the type of a,b,c,d is immutable(int)[] and foo is a function  
 that takes 4 int[]s then the .dup's are necessary to pass type checking.

What about this idea:

At a global level, expressions that result in CTFE being triggered, can be  
implicitly cast from mutable to immutable and vice versa via a deep-dup.   
This allows you to use enums as parameters to functions accepting mutable  
references.  Then enums that are derived from other enums do not need to  
follow the same rules as runtime code that uses the enums.

This of course, only happens at the global-expressions level, as function  
internals must compile at runtime as well.

What I thought of as a solution was to create CTFE only functions that  
wrap other functions to do a dup.  But you wouldn't want to do this during  
runtime, because dup is expensive.  During compile time, dup costs  
nothing.  This idea essentially takes the place of that boilerplate code.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 Look at the code generated for enum a = [1, 2, 3]. using a is replaced
 with a call to _d_arrayliteral. There is no CTFE going on.

 There is some ctfe going on, but the compiler has to allocate the
 result anew every time it is used. So there is also some runtime
 overhead.

 To make my point clearer:

 int foo(){return 100;}
 enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100,
 100];

 void main(){
 auto x = a; // this does *not* call foo. But it allocates a new array
 literal
 }

 Yes, you are right. The issue is that the resulting array is initialized
 at runtime, not that CTFE is being avoided. After doing some of these
 tests, I have a better understanding of the issues.

 The compiler has no choice. It must develop the array at runtime, or
 else the type allows one to modify the source value (just like in D1 how
 you could modify string literals). In essence, the compiler is creating
 a new copy for every usage (and building it from scratch).

 That is a quality of implementation issue. The language semantics do
 not require that.

 The language semantics require that if an enum type points at mutable
 data, a runtime allocation *must* occur to avoid corruption of literals.
 I think a rule requiring an enum to be immutable or implicitly cast to
 immutable puts the burden of runtime allocation on the coder, making it
 clear what's going on.

 In C++, novice coders typically pass classes by value not knowing what a
 horrible thing this is doing. Then they are puzzled why the code is so
 slow, the syntax is so short! This is another case of a hidden
 allocation which can be either avoided or made visible.

 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

 I was actually surprised that this compiles. But this should not be a
 problem even if a was immutable(int)[]. sort should be able to create a
 copy of an immutable array in order to sort it. It doesn't matter the
 performance hit, because this should all be done at compile time.

 It does not, but explicitly calling .dup works
 immutable x = [3,2,1];
 immutable y = sort(x.dup);

 I'm saying sort (or another symbol, ctfesort?) can be made to do the dup
 automatically for you so you don't have to have it when using ctfe.
 Extra allocations during CTFE cost nothing (well, with a properly GC'd
 compiler, that is).

 Update: I have a better idea, see below.

 When I see an enum, I think "evaluated at compile time". No matter how
 complex it is to build that value, it should be built at compile-time
 and *used* at runtime. No complex function calls should be done at
 runtime, an enum is a value.

 Exactly. Therefore you assign from it by copying it.

 Compare to static array.

 int[10] x = [1,2,3,4,5,6,7,8,9,0];

 x still needs to be initialized at runtime.

 Yes, but this is spelled out because copying a static array requires
 moving data. However, this does *not* require a hidden allocation (even
 though it does do a hidden allocation currently).

 I'm not worried about copying data as much as I am about hidden
 allocations. Hidden allocations are a huge drag on performance. Every
 time you allocate, you need to take a global GC lock, and it's an
 unbounded operation (doing one allocation could run a collection cycle).

You don't actually _need_ a global GC lock. It is just how it is 
implemented in this case. Note that this is an explicit allocation:

int[] a = [1,2,3]; // just as explicit as a NewExpression

Only the enums "hide" it sometimes, but actually you should know the 
involved types.

 I did an interesting little test:


 [snip]

 That just tells us that DMD sucks at generating code for array literals.

 This generates identical code:

 import std.stdio;

 void main() {
 writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3,
 3, 3, 3, 3]);
 }

 You don't need enums for that.


 What it actually should for both our examples is more like the following:

 import std.stdio;

 immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,
 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];

 void main() {
 writeln(_somewhereinrom.dup);
 }

 push %ebp
 mov %esp,%ebp
 pushl 0x8097184
 pushl 0x8097180
 mov $0x80975c8,%eax
 push %eax
 call 8079470 <_adDupT>
 add $0xc,%esp
 push %edx
 push %eax
 call 807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
 xor %eax,%eax
 pop %ebp
 ret


 If writeln would actually be const correct, the compiler could even
 get rid of the allocation.

 That is the idea. Get rid of the hidden allocation. Writeln *is* const
 correct, it can certainly print immutable(int)[].

Well, there is a function called writeln that can do that. That is a 
different function. But the one that gets actually called is not const 
correct as well.


This is writeln:

// Most general instance
void writeln(T...)(T args)
if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[]))
{
     stdout.write(args, '\n');
}


=>
writeln([1,2,3]);
// modulo IFTY:
writeln!(int[])([1,2,3]);
// const correct?
writeln!(int[])([1,2,3].idup); // nope!

Error: cannot implicitly convert expression (_adDupT(& 
_D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]


Now if there was a const there, the compiler could infer that writeln 
will not change the .duped array. Ergo it could pass the reference to 
immutable data directly. It would maybe help against template code bloat 
a bit, but not that much because const(immutable(int)[]) and the like 
are distinct types to const(int[]).




 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

Remember that immutable is transitive. That can really get in your way 
in this case.



 This is not about enums that much, it is about array literals.

 The fact that stack static array initialization allocates is one of
 DMDs bigger warts.

 Look at the ridiculous code generated for the following example:

 void main() {
 int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3,
 3, 3, 3, 3, 3];
 writeln(x);
 }

 Yes, these are all cases of the same issue.

 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

 I don't think .dup should be necessary at compile time. Creating a
 sorted copy of an immutable array should be quite doable.

 I agree, phobos won't currently do it though.

 This is easily fixed. But maybe there is a better way (see below).

 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

 Again, I don't think .dup would be used for dependent enums, I was
 rather thinking dup would be used where you need a mutable copy of an
 array during enum usage in normal code.

 But if the type of a,b,c,d is immutable(int)[] and foo is a function
 that takes 4 int[]s then the .dup's are necessary to pass type checking.

 What about this idea:

 At a global level, expressions that result in CTFE being triggered, can
 be implicitly cast from mutable to immutable and vice versa via a
 deep-dup. This allows you to use enums as parameters to functions
 accepting mutable references. Then enums that are derived from other
 enums do not need to follow the same rules as runtime code that uses the
 enums.

 This of course, only happens at the global-expressions level, as
 function internals must compile at runtime as well.

 What I thought of as a solution was to create CTFE only functions that
 wrap other functions to do a dup. But you wouldn't want to do this
 during runtime, because dup is expensive. During compile time, dup costs
 nothing. This idea essentially takes the place of that boilerplate code.

 -Steve


That could work, but I think this is cluttering up the semantics a bit.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:

 Yes, but this is spelled out because copying a static array requires
 moving data. However, this does *not* require a hidden allocation (even
 though it does do a hidden allocation currently).

 I'm not worried about copying data as much as I am about hidden
 allocations. Hidden allocations are a huge drag on performance. Every
 time you allocate, you need to take a global GC lock, and it's an
 unbounded operation (doing one allocation could run a collection cycle).

 You don't actually _need_ a global GC lock. It is just how it is  
 implemented in this case.

This is all fine in theory.  I haven't seen any implementations.  But  
memory allocations are not cheap, even without a GC lock.

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the  
 involved types.

As I've said, there are already ways to explicitly allocate memory.  A  
suggested replacement for this is:

int[] a = array(1, 2, 3);

And you could always do:

int[] a = [1, 2, 3].dup;

Nobody complains about having to do:

char[] a = "hello".dup;

I don't see why we couldn't do the same for all array literals.

 That is the idea. Get rid of the hidden allocation. Writeln *is* const
 correct, it can certainly print immutable(int)[].

 Well, there is a function called writeln that can do that. That is a  
 different function. But the one that gets actually called is not const  
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :  
 const(char)[]))
 {
      stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&  
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means.  If [1, 2, 3] is typed as immtuable(int)[],  
it will compile.

Simple test:

immutable(int)[] a = [1, 2, 3];
writeln(a);

works with 2.056.

 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

 Remember that immutable is transitive. That can really get in your way  
 in this case.

If the data is stored in ROM, it should be typed as immutable.  If it's  
not, then it could be modified.  The only other option is heap allocation  
and construction on use, which I've already said is undesirable.

If we start from "all array literals and enums that contain references  
store the referenced data in ROM," then we will find ways to deal with any  
inconveniences.  It's all a matter of what's more important in a system  
language, performance or convenience.

 What about this idea:

 At a global level, expressions that result in CTFE being triggered, can
 be implicitly cast from mutable to immutable and vice versa via a
 deep-dup. This allows you to use enums as parameters to functions
 accepting mutable references. Then enums that are derived from other
 enums do not need to follow the same rules as runtime code that uses the
 enums.

 This of course, only happens at the global-expressions level, as
 function internals must compile at runtime as well.

 What I thought of as a solution was to create CTFE only functions that
 wrap other functions to do a dup. But you wouldn't want to do this
 during runtime, because dup is expensive. During compile time, dup costs
 nothing. This idea essentially takes the place of that boilerplate code.


 That could work, but I think this is cluttering up the semantics a bit.

It's just a shortcut.  In essence, during compile time execution, .dups  
are added for you because they are free.  During runtime, they are not  
free, so you must add them.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:

 Yes, but this is spelled out because copying a static array requires
 moving data. However, this does *not* require a hidden allocation (even
 though it does do a hidden allocation currently).

 I'm not worried about copying data as much as I am about hidden
 allocations. Hidden allocations are a huge drag on performance. Every
 time you allocate, you need to take a global GC lock, and it's an
 unbounded operation (doing one allocation could run a collection cycle).

 You don't actually _need_ a global GC lock. It is just how it is
 implemented in this case.

 This is all fine in theory. I haven't seen any implementations. But
 memory allocations are not cheap, even without a GC lock.

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on 
arbitrary reference types.


 That is the idea. Get rid of the hidden allocation. Writeln *is* const
 correct, it can certainly print immutable(int)[].

 Well, there is a function called writeln that can do that. That is a
 different function. But the one that gets actually called is not const
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
 const(char)[]))
 {
 stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

 I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
 it will compile.

 Simple test:

 immutable(int)[] a = [1, 2, 3];
 writeln(a);

 works with 2.056.

That is like saying

void foo(int[] a){...}             // prints a
void bar(immutable(int)[] a){...}  // prints a

int[] a = [1,2,3];
immutable(int)[] b = [1, 2, 3];
foo(a);
bar(b);

"=> Oh, bar is const correct because I can call foo with mutable data 
and bar with immutable data."

foo is writeln!(int[])
bar is writeln!(immutable(int)[])

in effect, if you do:

writeln("hello".dup);

The compiler cannot assume that the IFTI'd writeln!(char[]) will not 
change the argument, ergo it cannot optimize away the .dup, even though 
it would be a valid transformation as long as the programmer does not 
make the program semantics depend on the identity relation on immutable 
references (doing so defeats the purpose of immutability).


 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

 Remember that immutable is transitive. That can really get in your way
 in this case.

 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

Both are more important. It is D.
Everything you need to do is make the enum static immutable instead.
D is also a scripting language and an application programming language.

auto a = [new Foo, new Bar, new Qux]; // I want that to work.

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/16/2011 08:26 PM, Timon Gehr wrote:
 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

(It currently does, if that was unclear)

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not on  
 arbitrary reference types.

string is a reference type.  We hear no complaints about strings being  
stored in ROM and the type of literals being immutable.

Make no mistake, this doesn't cover every current instance array literals,  
such as ones which contain necessarily-heap-allocated entities.  Those  
would be covered by a library function (e.g. array(...)).  But those are  
not array literals anyways, they are constructors.

 Well, there is a function called writeln that can do that. That is a
 different function. But the one that gets actually called is not const
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
 const(char)[]))
 {
 stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

 I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
 it will compile.

 Simple test:

 immutable(int)[] a = [1, 2, 3];
 writeln(a);

 works with 2.056.

 That is like saying

 void foo(int[] a){...}             // prints a
 void bar(immutable(int)[] a){...}  // prints a

 int[] a = [1,2,3];
 immutable(int)[] b = [1, 2, 3];
 foo(a);
 bar(b);

 "=> Oh, bar is const correct because I can call foo with mutable data  
 and bar with immutable data."

 foo is writeln!(int[])
 bar is writeln!(immutable(int)[])

 in effect, if you do:

 writeln("hello".dup);

 The compiler cannot assume that the IFTI'd writeln!(char[]) will not  
 change the argument, ergo it cannot optimize away the .dup, even though  
 it would be a valid transformation as long as the programmer does not  
 make the program semantics depend on the identity relation on immutable  
 references (doing so defeats the purpose of immutability).

writeln is not contractually const correct, but that's only because most  
of D is not const correct either.  For example, Object.toString is not  
const, so if you const-ified all writeln args, you couldn't print  
objects.  But writeln itself does not change any data (a rogue toString  
might change data, but that is not common practice).

Once D becomes const correct, writeln *can* apply const to all it's  
parameters.

 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

 Both are more important. It is D.
 Everything you need to do is make the enum static immutable instead.
 D is also a scripting language and an application programming language.

 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux);

The one case which is difficult to do is initializing a fixed-size array  
with a literal that uses runtime data.  I suppose we'd need a function  
that returns a fixed-sized array made of its arguments, and doing the init  
builds it in place.  i.e.:

Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

would not do any moving of references, it would construct the fixed sized  
array in-place.  Initializing fixed sized arrays with array literals  
already needs attention anyway.

-Steve

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 15:00:16 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 The one case which is difficult to do is initializing a fixed-size array  
 with a literal that uses runtime data.  I suppose we'd need a function  
 that returns a fixed-sized array made of its arguments, and doing the  
 init builds it in place.  i.e.:

 Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

 would not do any moving of references, it would construct the fixed  
 sized array in-place.  Initializing fixed sized arrays with array  
 literals already needs attention anyway.

one benefit here, we could use auto:

auto objs = array_fixed(new Foo, new Bar, new Qux);

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not on
 arbitrary reference types.

 string is a reference type. We hear no complaints about strings being
 stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a 
built-in value type.

static assert(isMutable!string);

 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

It is true that they are constructors, but they are currently also 
called array literals.

 Well, there is a function called writeln that can do that. That is a
 different function. But the one that gets actually called is not const
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
 const(char)[]))
 {
 stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

 I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
 it will compile.

 Simple test:

 immutable(int)[] a = [1, 2, 3];
 writeln(a);

 works with 2.056.

 That is like saying

 void foo(int[] a){...} // prints a
 void bar(immutable(int)[] a){...} // prints a

 int[] a = [1,2,3];
 immutable(int)[] b = [1, 2, 3];
 foo(a);
 bar(b);

 "=> Oh, bar is const correct because I can call foo with mutable data
 and bar with immutable data."

 foo is writeln!(int[])
 bar is writeln!(immutable(int)[])

 in effect, if you do:

 writeln("hello".dup);

 The compiler cannot assume that the IFTI'd writeln!(char[]) will not
 change the argument, ergo it cannot optimize away the .dup, even
 though it would be a valid transformation as long as the programmer
 does not make the program semantics depend on the identity relation on
 immutable references (doing so defeats the purpose of immutability).

 writeln is not contractually const correct, but that's only because most
 of D is not const correct either. For example, Object.toString is not
 const, so if you const-ified all writeln args, you couldn't print
 objects. But writeln itself does not change any data (a rogue toString
 might change data, but that is not common practice).

 Once D becomes const correct, writeln *can* apply const to all it's
 parameters.

Ok, I see. writeln still could be const-correct in principle. It would 
need to apply const to the parameters that have a const toString method. 
But the language is not powerful enough to do that. There is no way to 
intercept IFTI... I am commonly missing that feature.

 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

 Both are more important. It is D.
 Everything you need to do is make the enum static immutable instead.
 D is also a scripting language and an application programming language.

 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

 auto a = array(new Foo, new Bar, new Qux);

Requiring that works modulo template bloat and breakage of existing 
code. Also, it does not really buy us anything imho.

 The one case which is difficult to do is initializing a fixed-size array
 with a literal that uses runtime data. I suppose we'd need a function
 that returns a fixed-sized array made of its arguments, and doing the
 init builds it in place. i.e.:

 Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

 would not do any moving of references, it would construct the fixed
 sized array in-place. Initializing fixed sized arrays with array
 literals already needs attention anyway.

 From the compiler side. Not necessarily from the std lib side.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not on
 arbitrary reference types.

 string is a reference type. We hear no complaints about strings being
 stored in ROM and the type of literals being immutable.

 string is not an immutable type. It is immutable(char)[] and char is a  
 built-in value type.

 static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or  
implicitly castable to immutable).  The point is that the data referenced  
is stored in ROM and therefore a) immutable and b) fully defined at  
compile-time.

 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

 It is true that they are constructors, but they are currently also  
 called array literals.

I'm looking to redefine what a literal means in D.

 writeln is not contractually const correct, but that's only because most
 of D is not const correct either. For example, Object.toString is not
 const, so if you const-ified all writeln args, you couldn't print
 objects. But writeln itself does not change any data (a rogue toString
 might change data, but that is not common practice).

 Once D becomes const correct, writeln *can* apply const to all it's
 parameters.

 Ok, I see. writeln still could be const-correct in principle. It would  
 need to apply const to the parameters that have a const toString method.  
 But the language is not powerful enough to do that. There is no way to  
 intercept IFTI... I am commonly missing that feature.

I have an enhancement request in for intercepting IFTI (not sure if it  
applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap  
 allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

 Both are more important. It is D.
 Everything you need to do is make the enum static immutable instead.
 D is also a scripting language and an application programming language.

 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

 auto a = array(new Foo, new Bar, new Qux);

 Requiring that works modulo template bloat and breakage of existing  
 code. Also, it does not really buy us anything imho.

I think the bloat is a wash.  Every time I use an array literal, there is  
a complete instantiation of what would be in an array template inline in  
the calling function.

Yes, it breaks code.  It's worth it.  To avoid a heap allocation for [1,  
2, 3] is worth breaking code that uses runtime data in an array literal.   
The compiler can be helpful in the error message:

Error somefile.d(345): array literals cannot contain runtime data.  Maybe  
you meant:
  array(new Foo, new Bar, new Qux);

I want to point out that currently there is *NO* way to make an immutable  
array literal that lives in ROM.  This is unacceptable.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not on
 arbitrary reference types.

 string is a reference type. We hear no complaints about strings being
 stored in ROM and the type of literals being immutable.

 string is not an immutable type. It is immutable(char)[] and char is a
 built-in value type.

 static assert(isMutable!string);

 It fits my definition of a valid enum reference type (immutable or
 implicitly castable to immutable). The point is that the data referenced
 is stored in ROM and therefore a) immutable and b) fully defined at
 compile-time.

Indeed. But fact is, the data that is qualified with immutable is not of 
reference type therefore it behaves nicely. And you don't get that in 
the general case.

 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

 It is true that they are constructors, but they are currently also
 called array literals.

 I'm looking to redefine what a literal means in D.

 writeln is not contractually const correct, but that's only because most
 of D is not const correct either. For example, Object.toString is not
 const, so if you const-ified all writeln args, you couldn't print
 objects. But writeln itself does not change any data (a rogue toString
 might change data, but that is not common practice).

 Once D becomes const correct, writeln *can* apply const to all it's
 parameters.

 Ok, I see. writeln still could be const-correct in principle. It would
 need to apply const to the parameters that have a const toString
 method. But the language is not powerful enough to do that. There is
 no way to intercept IFTI... I am commonly missing that feature.


I am looking for something like this:

template writeln(T...)(T){
     alias writelnImpl!(writelnInferConst!T) writeln;
}

(it is even trivial to parse, unlike normal function definitions!)

 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

It has a complexity of at least 2^numparams and probably all kinds of 
odd implications for the compiler internals that would lead to a buggy 
implementation.

I think this is a better solution:

void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

Then it is just a matter of applying proper value range propagation for 
IFTY:

void bar(T: short)(T t){...}

void main(){
     bar(1); // ok
}



 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap
 allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

 Both are more important. It is D.
 Everything you need to do is make the enum static immutable instead.
 D is also a scripting language and an application programming language.

 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

 auto a = array(new Foo, new Bar, new Qux);

 Requiring that works modulo template bloat and breakage of existing
 code. Also, it does not really buy us anything imho.

 I think the bloat is a wash. Every time I use an array literal, there is
 a complete instantiation of what would be in an array template inline in
 the calling function.

With the template function you have that too because the calling 
function builds the arguments. It is just that you also get the template 
bloat and an additional function call. Unless it is inlined, of course.

 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

There is:

void main(){
     static immutable x = [1,2,3];
}


And the compiler *really* should write this to ROM:

void main(){
     immutable(int)[] x=[1,2,3]; // should be slice to ROM, because the 
array literal contents are typed as immutable and known during compile time.
}

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/16/2011 11:39 PM, Timon Gehr wrote:
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

BTW, this already works for your use case:

void foo2(ParameterTypeTuple!foo t){foo(t);}

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 18:25:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 11:39 PM, Timon Gehr wrote:
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

 BTW, this already works for your use case:

 void foo2(ParameterTypeTuple!foo t){foo(t);}

My use case is incomplete, I minimized it too much.  I will update it.

-Steve

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 17:39:16 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know  
 the
 involved types.

 As I've said, there are already ways to explicitly allocate memory.  
 A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not  
 on
 arbitrary reference types.

 string is a reference type. We hear no complaints about strings being
 stored in ROM and the type of literals being immutable.

 string is not an immutable type. It is immutable(char)[] and char is a
 built-in value type.

 static assert(isMutable!string);

 It fits my definition of a valid enum reference type (immutable or
 implicitly castable to immutable). The point is that the data referenced
 is stored in ROM and therefore a) immutable and b) fully defined at
 compile-time.

 Indeed. But fact is, the data that is qualified with immutable is not of  
 reference type therefore it behaves nicely. And you don't get that in  
 the general case.

In the general case, there is always a library function for construction.   
In other words, what the compiler currently does for array literals can be  
done in a library.  But a library cannot create ROM space.  The  
compiler-based features (and CTFE in general) should be helping us create  
things at compile time, not at run time.  We already have the tools to  
construct arrays at runtime.

 I am looking for something like this:

 template writeln(T...)(T){
      alias writelnImpl!(writelnInferConst!T) writeln;
 }

 (it is even trivial to parse, unlike normal function definitions!)

What does writelnInferConst!T do?  I'm afraid I'm not getting what you are  
saying.

I was thinking writeln should do this:

void writeln(T...)(const T args) {...}

 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

 It has a complexity of at least 2^numparams and probably all kinds of  
 odd implications for the compiler internals that would lead to a buggy  
 implementation.

I'm not a compiler writer, but I don't see how this is the case.

 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for  
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
      bar(1); // ok
 }

The issue with all this is, IFTI doesn't work that way:

void foo(T: short)(T t) {}

void main()
{
    foo(1);
}

testifti.d(5): Error: template testifti.foo(T : short) does not match any  
function template declaration
testifti.d(5): Error: template testifti.foo(T : short) cannot deduce  
template function from argument types !()(int)

IFTI decides the types of literals before trying to find a proper template  
to instantiate.  Any possibility to intercept the decision of literal type  
or of instantiation would be useful.  I think that it's better suited to  
the constraints, because there is more power there.  But I'm not sure.  If  
you can find a more straightforward way, I'm all for it.

In any case, I need to update the bug report, because the general case is  
if foo has an overload.  For instance:

foo(short s);
foo(wstring w);

foo2 should be able to call both with 1 and "hello" without issue.

My driving use case to create the enhancement was creating a wrapper type  
that intercepted function calls.  I planned to use opDispatch, but it  
didn't quite work with literals.

 I think the bloat is a wash. Every time I use an array literal, there is
 a complete instantiation of what would be in an array template inline in
 the calling function.

 With the template function you have that too because the calling  
 function builds the arguments. It is just that you also get the template  
 bloat and an additional function call. Unless it is inlined, of course.

You are right, I hadn't thought of that.  But what about this:  most array  
literals are actually literals (made up of CTFE-decided values).  Making  
them ROM-stored would *eliminate* bloat as compared to the current  
implementation.

Also, given how template-centric D and phobos are, I think at some point  
we need to examine how to minimize template bloat in general, by  
coalescing identical code into one function, or not emitting functions  
that are always inlined, not to mention avoiding storing templates only  
used at compile-time in the code (e.g. isInputRange).

 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

 There is:

 void main(){
      static immutable x = [1,2,3];
 }

Seems rather odd you should have to jump through these hoops to get the  
compiler to use ROM space.  But I concede that I did not know of this  
trick.  It does not sway my opinion that CTFE should produce ROM-stored  
references, and library function should be used for runtime-constructed  
references.

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 17:39:16 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know
 the
 involved types.

 As I've said, there are already ways to explicitly allocate
 memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but
 not on
 arbitrary reference types.

 string is a reference type. We hear no complaints about strings being
 stored in ROM and the type of literals being immutable.

 string is not an immutable type. It is immutable(char)[] and char is a
 built-in value type.

 static assert(isMutable!string);

 It fits my definition of a valid enum reference type (immutable or
 implicitly castable to immutable). The point is that the data referenced
 is stored in ROM and therefore a) immutable and b) fully defined at
 compile-time.

 Indeed. But fact is, the data that is qualified with immutable is not
 of reference type therefore it behaves nicely. And you don't get that
 in the general case.

 In the general case, there is always a library function for
 construction. In other words, what the compiler currently does for array
 literals can be done in a library. But a library cannot create ROM
 space. The compiler-based features (and CTFE in general) should be
 helping us create things at compile time, not at run time. We already
 have the tools to construct arrays at runtime.

 I am looking for something like this:

 template writeln(T...)(T){
 alias writelnImpl!(writelnInferConst!T) writeln;
 }

 (it is even trivial to parse, unlike normal function definitions!)

 What does writelnInferConst!T do? I'm afraid I'm not getting what you
 are saying.

 I was thinking writeln should do this:

 void writeln(T...)(const T args) {...}

As you pointed out, this cannot print types that have a non-const 
toString method (caching the result could be a perfectly valid reason 
for that.)

writelnInferConst finds out which parameters can be treated as const and 
still be printed so that the correct version of writeln may be called.

For example:

class Foo{ // can be printed if const
     string toString()const{return "Foo";}
}

class Bar{ // cannot be printed if const
     string cache;
     string toString(){return cache!is null?cache:(cache="Bar");}
}

template hasConstToString(T){
     enum hasConstToString = is(typeof((const T t){return t.toString();}));
}

template writelnInferConstImpl(T...){
     static if(!T.length) alias T X;
     else static if(hasConstToString!(T[0])){
             alias T[0] _;
             alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X;
     }else
         alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X;
}
template writelnInferConst(T...){alias writelnInferConstImpl!T.X 
writelnInferConst;} // (bug 6966)


static 
assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar)));


The real thing would also do stuff like

actual argument immutable(Foo[])[] => formal argument const(Foo[][]).

In order to get rid of bloat created by pointless instantiations of 
writelnImpl.



 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

 It has a complexity of at least 2^numparams and probably all kinds of
 odd implications for the compiler internals that would lead to a buggy
 implementation.

 I'm not a compiler writer, but I don't see how this is the case.

There were/are quite a few error gagging related bugs. I guess this 
would be similar.

 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation
 for IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

 The issue with all this is, IFTI doesn't work that way:

 void foo(T: short)(T t) {}

 void main()
 {
 foo(1);
 }

 testifti.d(5): Error: template testifti.foo(T : short) does not match
 any function template declaration
 testifti.d(5): Error: template testifti.foo(T : short) cannot deduce
 template function from argument types !()(int)

 IFTI decides the types of literals before trying to find a proper
 template to instantiate. Any possibility to intercept the decision of
 literal type or of instantiation would be useful. I think that it's
 better suited to the constraints, because there is more power there. But
 I'm not sure. If you can find a more straightforward way, I'm all for it.

 In any case, I need to update the bug report, because the general case
 is if foo has an overload. For instance:

 foo(short s);
 foo(wstring w);

 foo2 should be able to call both with 1 and "hello" without issue.

 My driving use case to create the enhancement was creating a wrapper
 type that intercepted function calls. I planned to use opDispatch, but
 it didn't quite work with literals.

Ok, I see the problem. My proposed IFTI template mechanism would save 
the day.

It would look like this (static foreach would have to be replaced by a 
recursive mixin template because Walter encountered implementation 
difficulties).


template OverloadsOf(alias symbol){ // should this be in std.traits?
     alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol), 
__traits(identifier,symbol))) OverloadsOf;
}

auto wrapper(alias foo)(ParameterTypeTuple!foo args){
     return foo(args);
}
template opDispatch(string op,T...)(T){
     static foreach(foo; OverloadsOf!(mixin(op))){
         alias wrapper!foo opDispatch;
     }
     static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing 
with a template function
         auto opDispatch(T args){
             return foo(args);
         }
     }
}




 I think the bloat is a wash. Every time I use an array literal, there is
 a complete instantiation of what would be in an array template inline in
 the calling function.

 With the template function you have that too because the calling
 function builds the arguments. It is just that you also get the
 template bloat and an additional function call. Unless it is inlined,
 of course.

 You are right, I hadn't thought of that. But what about this: most array
 literals are actually literals (made up of CTFE-decided values). Making
 them ROM-stored would *eliminate* bloat as compared to the current
 implementation.

Yes, and the compiler should do that. That works fine with the current 
semantics of array literals.

 Also, given how template-centric D and phobos are, I think at some point
 we need to examine how to minimize template bloat in general, by
 coalescing identical code into one function, or not emitting functions
 that are always inlined, not to mention avoiding storing templates only
 used at compile-time in the code (e.g. isInputRange).

I agree.

 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

 There is:

 void main(){
 static immutable x = [1,2,3];
 }

 Seems rather odd you should have to jump through these hoops to get the
 compiler to use ROM space. But I concede that I did not know of this
 trick. It does not sway my opinion that CTFE should produce ROM-stored
 references, and library function should be used for runtime-constructed
 references.

CTFE should produce ROM-stored data iff it is used during run time, I 
agree on that. However if the enum is typed as mutable, it should create 
a copy of the ROM-stored data.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Thu, 17 Nov 2011 12:31:58 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:

 What does writelnInferConst!T do? I'm afraid I'm not getting what you
 are saying.

 I was thinking writeln should do this:

 void writeln(T...)(const T args) {...}

 As you pointed out, this cannot print types that have a non-const  
 toString method (caching the result could be a perfectly valid reason  
 for that.)

Caching string representation IMO is not a significant use case.  Not only  
that, but toString should be deprecated anyways in favor of a stream-based  
system.

 writelnInferConst finds out which parameters can be treated as const and  
 still be printed so that the correct version of writeln may be called.

 For example:

 class Foo{ // can be printed if const
      string toString()const{return "Foo";}
 }

 class Bar{ // cannot be printed if const
      string cache;
      string toString(){return cache!is null?cache:(cache="Bar");}
 }

 template hasConstToString(T){
      enum hasConstToString = is(typeof((const T t){return  
 t.toString();}));
 }

 template writelnInferConstImpl(T...){
      static if(!T.length) alias T X;
      else static if(hasConstToString!(T[0])){
              alias T[0] _;
              alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X;
      }else
          alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X;
 }
 template writelnInferConst(T...){alias writelnInferConstImpl!T.X  
 writelnInferConst;} // (bug 6966)


 static  
 assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar)));

If your goal is to reduce template bloat, I think this is not the solution.

But also note that this still does not guarantee const.  I don't really  
see the point of doing all these templates if you aren't going to  
guarantee writeln doesn't modify the data.

 The issue with all this is, IFTI doesn't work that way:

 void foo(T: short)(T t) {}

 void main()
 {
 foo(1);
 }

 testifti.d(5): Error: template testifti.foo(T : short) does not match
 any function template declaration
 testifti.d(5): Error: template testifti.foo(T : short) cannot deduce
 template function from argument types !()(int)

 IFTI decides the types of literals before trying to find a proper
 template to instantiate. Any possibility to intercept the decision of
 literal type or of instantiation would be useful. I think that it's
 better suited to the constraints, because there is more power there. But
 I'm not sure. If you can find a more straightforward way, I'm all for  
 it.

 In any case, I need to update the bug report, because the general case
 is if foo has an overload. For instance:

 foo(short s);
 foo(wstring w);

 foo2 should be able to call both with 1 and "hello" without issue.

 My driving use case to create the enhancement was creating a wrapper
 type that intercepted function calls. I planned to use opDispatch, but
 it didn't quite work with literals.

 Ok, I see the problem. My proposed IFTI template mechanism would save  
 the day.

 It would look like this (static foreach would have to be replaced by a  
 recursive mixin template because Walter encountered implementation  
 difficulties).


 template OverloadsOf(alias symbol){ // should this be in std.traits?
      alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol),  
 __traits(identifier,symbol))) OverloadsOf;
 }

 auto wrapper(alias foo)(ParameterTypeTuple!foo args){
      return foo(args);
 }
 template opDispatch(string op,T...)(T){
      static foreach(foo; OverloadsOf!(mixin(op))){
          alias wrapper!foo opDispatch;
      }
      static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing  
 with a template function
          auto opDispatch(T args){
              return foo(args);
          }
      }
 }

Pardon my saying so, but this looks horrendous.  Not to mention that I  
don't think it would work.  IFTI instantiates templates, it does not look  
inside instantiated templates for overloads.

BTW, your proposed IFTI template mechanism, do you have it stated  
anywhere?  Maybe it fixes the problem I mentioned.

 Seems rather odd you should have to jump through these hoops to get the
 compiler to use ROM space. But I concede that I did not know of this
 trick. It does not sway my opinion that CTFE should produce ROM-stored
 references, and library function should be used for runtime-constructed
 references.

 CTFE should produce ROM-stored data iff it is used during run time, I  
 agree on that. However if the enum is typed as mutable, it should create  
 a copy of the ROM-stored data.

Well, this is closer than I thought we were to an agreement.  I would  
agree with this, as long as it could be implicitly cast to immutable or  
const.

i.e.:

enum foo = [1, 2, 3];

immutable(int)[] a = foo; // no allocation
const(int)[] b = foo; // no allocation

-Steve

Timon Gehr <timon.gehr gmx.ch> writes:

On 11/17/2011 07:23 PM, Steven Schveighoffer wrote:
 On Thu, 17 Nov 2011 12:31:58 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:

 What does writelnInferConst!T do? I'm afraid I'm not getting what you
 are saying.

 I was thinking writeln should do this:

 void writeln(T...)(const T args) {...}

 As you pointed out, this cannot print types that have a non-const
 toString method (caching the result could be a perfectly valid reason
 for that.)

 Caching string representation IMO is not a significant use case. Not
 only that, but toString should be deprecated anyways in favor of a
 stream-based system.

 writelnInferConst finds out which parameters can be treated as const
 and still be printed so that the correct version of writeln may be
 called.

 For example:

 class Foo{ // can be printed if const
 string toString()const{return "Foo";}
 }

 class Bar{ // cannot be printed if const
 string cache;
 string toString(){return cache!is null?cache:(cache="Bar");}
 }

 template hasConstToString(T){
 enum hasConstToString = is(typeof((const T t){return t.toString();}));
 }

 template writelnInferConstImpl(T...){
 static if(!T.length) alias T X;
 else static if(hasConstToString!(T[0])){
 alias T[0] _;
 alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X;
 }else
 alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X;
 }
 template writelnInferConst(T...){alias writelnInferConstImpl!T.X
 writelnInferConst;} // (bug 6966)


 static
 assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar)));

 If your goal is to reduce template bloat, I think this is not the solution.

 But also note that this still does not guarantee const. I don't really
 see the point of doing all these templates if you aren't going to
 guarantee writeln doesn't modify the data.

 The issue with all this is, IFTI doesn't work that way:

 void foo(T: short)(T t) {}

 void main()
 {
 foo(1);
 }

 testifti.d(5): Error: template testifti.foo(T : short) does not match
 any function template declaration
 testifti.d(5): Error: template testifti.foo(T : short) cannot deduce
 template function from argument types !()(int)

 IFTI decides the types of literals before trying to find a proper
 template to instantiate. Any possibility to intercept the decision of
 literal type or of instantiation would be useful. I think that it's
 better suited to the constraints, because there is more power there. But
 I'm not sure. If you can find a more straightforward way, I'm all for
 it.

 In any case, I need to update the bug report, because the general case
 is if foo has an overload. For instance:

 foo(short s);
 foo(wstring w);

 foo2 should be able to call both with 1 and "hello" without issue.

 My driving use case to create the enhancement was creating a wrapper
 type that intercepted function calls. I planned to use opDispatch, but
 it didn't quite work with literals.

 Ok, I see the problem. My proposed IFTI template mechanism would save
 the day.

 It would look like this (static foreach would have to be replaced by a
 recursive mixin template because Walter encountered implementation
 difficulties).


 template OverloadsOf(alias symbol){ // should this be in std.traits?
 alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol),
 __traits(identifier,symbol))) OverloadsOf;
 }

 auto wrapper(alias foo)(ParameterTypeTuple!foo args){
 return foo(args);
 }
 template opDispatch(string op,T...)(T){
 static foreach(foo; OverloadsOf!(mixin(op))){
 alias wrapper!foo opDispatch;
 }
 static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing with
 a template function
 auto opDispatch(T args){
 return foo(args);
 }
 }
 }

 Pardon my saying so, but this looks horrendous. Not to mention that I
 don't think it would work.

Oh, it would certainly work.

 IFTI instantiates templates, it does not look
 inside instantiated templates for overloads.

This works, does this solve the confusion?:

void foo(int){writeln("foo!");}
void bar(double){writeln("bar!");}

template merge(){
     alias foo qux;
     alias bar qux;
}
alias merge!().qux qux;

void main(){
     qux(1);   // calls foo
     qux(1.0); // calls bar
}



 BTW, your proposed IFTI template mechanism, do you have it stated
 anywhere? Maybe it fixes the problem I mentioned.

Not yet, I will file a bugzilla enhancement request and post a link here.

What it does is quite simple:

1. Apply normal IFTI instantiation rules to the IFTI template, as if it 
was a function template.
2. Instantiate the IFTI template with the deduced arguments.
3. The result of the instantiation must be callable with the original 
arguments. Call it.

This allows the function that is called to have a different (albeit 
compatible) signature from what IFTI would give you.


 Seems rather odd you should have to jump through these hoops to get the
 compiler to use ROM space. But I concede that I did not know of this
 trick. It does not sway my opinion that CTFE should produce ROM-stored
 references, and library function should be used for runtime-constructed
 references.

 CTFE should produce ROM-stored data iff it is used during run time, I
 agree on that. However if the enum is typed as mutable, it should
 create a copy of the ROM-stored data.

 Well, this is closer than I thought we were to an agreement. I would
 agree with this, as long as it could be implicitly cast to immutable or
 const.

 i.e.:

 enum foo = [1, 2, 3];

 immutable(int)[] a = foo; // no allocation
 const(int)[] b = foo; // no allocation

 -Steve

Yes, that is exactly how I would imagine it to work.

=?utf-8?Q?Simen_Kj=C3=A6r=C3=A5s?= <simen.kjaras gmail.com> writes:

On Wed, 16 Nov 2011 21:00:16 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

 As I've said, there are already ways to explicitly allocate memory. A
 suggested replacement for this is:

 int[] a = array(1, 2, 3);

 And you could always do:

 int[] a = [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a = "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not on  
 arbitrary reference types.

 string is a reference type.  We hear no complaints about strings being  
 stored in ROM and the type of literals being immutable.

immutable(int[][]) a = [[1]];
int[][] b = a.dup; // Fails.

The problem is with more than a single layer of indirection.

"Steven Schveighoffer" <schveiguy yahoo.com> writes:

On Wed, 16 Nov 2011 16:47:58 -0500, Simen Kj=C3=A6r=C3=A5s <simen.kjaras=
 gmail.com>  =

wrote:

 On Wed, 16 Nov 2011 21:00:16 +0100, Steven Schveighoffer  =

 <schveiguy yahoo.com> wrote:

 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>  =


 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> =




 =

 wrote:

 Note that this is an explicit allocation:

 int[] a =3D [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know t=





he
 involved types.

 As I've said, there are already ways to explicitly allocate memory.=




 A
 suggested replacement for this is:

 int[] a =3D array(1, 2, 3);

 And you could always do:

 int[] a =3D [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a =3D "hello".dup;

 I don't see why we couldn't do the same for all array literals.

 Because 'immutable' behaves nicely on built-in value types, but not =



on  =

 arbitrary reference types.

 string is a reference type.  We hear no complaints about strings bein=


g  =

 stored in ROM and the type of literals being immutable.

 immutable(int[][]) a =3D [[1]];
 int[][] b =3D a.dup; // Fails.

 The problem is with more than a single layer of indirection.

Solved via library:

auto b =3D a.deepdup;

-Steve

so <so so.so> writes:

On Mon, 14 Nov 2011 15:13:20 +0200, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 There is definitely some debatable practice here for wherever enum is  
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*.  This is  
 counter-intuitive, one uses enum to define things using the compiler,  
 not during runtime.  It's used to invoke CTFE, to avoid heap  
 allocation.  It's not a glorified #define macro.

Thanks Steve, this is exactly what i was trying to say.

 The deep issue here is not that enum is used as a manifest constant, but  
 rather the fact that enum can map to a *function call* rather than the  
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should be  
 acceptable as well.

 My view is that enum should only be acceptable on data that is  
 immutable, or implicitly cast to immutable, and should *never* map to an  
 expression that calls a function during runtime.

I agree, enum in its current shape implies just that.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/