www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Request: Evaluate constant ~ and [] before instantiating templates

reply Don Clugston <dac nospam.com.au> writes:
Right now, string literals can be concatenated at compile time.
This is the essential features used in my 'static if' example in 
D.learn; it allows you to write compile-time itoa(), for example.

We're so close to having a metaprogramming language of comparable power 
to Lisp. We can already write functions that return arbitrary lists, 
provided that each entry in the list is a char.

But as of DMD 0.137, there are a few limitations:

(1). It's only possible to concatenate strings, not arrays of other 
constant types.

eg:
const int a = [1,2,3] ~ [4,5,6];

should be exactly equivalent to:

const int a = [1,2,3,4,5,6];

but unfortunately it's evaluated after template instantiation, not 
before. If this order was changed, it would be possible (for example) to 
create lookup tables at compile time. The fact that it already works for 
strings suggests that most of the machinery is already in place.

(2).  [] is also evaluated after template instantiation.

const char [] str = "abcdefghi"[2..4];
should be equivalent to
const char [] str = "bcd";

If both restrictions (1) and (2) were removed, we'd be able to do almost 
  any functional programming at compile time.


However, some significant syntactic sugar is also possible. Most useful 
would be a relaxation of the rules for template value parameters.
C++ doesn't allow floating point numbers, though some vendors have 
implemented it. AFAIK, the reason C++ doesn't allow it is because of
lack of standards in floating point representation, so that there are 
problems with round-off error. Since D specifies
IEEE floating point, includes the hex float format, and most 
importantly, has 'static if', I don't think those arguments apply.

Casts are evaluated before template instantiation, so it is possible to 
pass a real as a template parameter, you just need to do a really ugly 
cast. (Or two casts, and pass two parameters, because 80 bit reals won't 
fit into a long).

So,
request (3) : allow reals as template value parameters.

Ideally, an compile-time constant could be used as a value parameter, 
including string literals and arrays. But I think that would be much 
more difficult, and would be a distraction. (1) and (2) are the 
important things. Either of them would open up a lot of possibilities, 
and if D had both, the brag value would be immense :-).

And maybe they would not be so difficult to implement?
Nov 02 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Don Clugston" <dac nospam.com.au> wrote in message
news:dka37t$2c01$1 digitaldaemon.com...

 (1). It's only possible to concatenate strings, not arrays of other
 constant types.

 eg:
 const int a = [1,2,3] ~ [4,5,6];

 should be exactly equivalent to:

 const int a = [1,2,3,4,5,6];

 but unfortunately it's evaluated after template instantiation, not
 before. If this order was changed, it would be possible (for example) to
 create lookup tables at compile time. The fact that it already works for
 strings suggests that most of the machinery is already in place.


The reason it does not currently work is because there is no syntax for
array literals as a PrimaryExpression. This is fixable, but not at the
moment.


 (2).  [] is also evaluated after template instantiation.

 const char [] str = "abcdefghi"[2..4];
 should be equivalent to
 const char [] str = "bcd";

 If both restrictions (1) and (2) were removed, we'd be able to do almost
   any functional programming at compile time.


This will be fixed in the upcoming update.



 However, some significant syntactic sugar is also possible. Most useful
 would be a relaxation of the rules for template value parameters.
 C++ doesn't allow floating point numbers, though some vendors have
 implemented it. AFAIK, the reason C++ doesn't allow it is because of
 lack of standards in floating point representation, so that there are
 problems with round-off error. Since D specifies
 IEEE floating point, includes the hex float format, and most
 importantly, has 'static if', I don't think those arguments apply.

 Casts are evaluated before template instantiation, so it is possible to
 pass a real as a template parameter, you just need to do a really ugly
 cast. (Or two casts, and pass two parameters, because 80 bit reals won't
 fit into a long).

 So,
 request (3) : allow reals as template value parameters.


The reason it doesn't at the moment is it makes for really wretched template
instance names :-(
Nov 05 2005
next sibling parent Sai <Sai_member pathlink.com> writes:
The reason it does not currently work is because there is no syntax for
array literals as a PrimaryExpression. This is fixable, but not at the
moment.


I thought that is easy to implement once the syntax is figured out. Any idea why
it is not implemented yet ? whats keeping it from implementing ? are we waiting
for something ? 

Just curious.
Sai
Nov 06 2005
prev sibling next sibling parent Georg Wrede <georg.wrede nospam.org> writes:
Walter Bright wrote:

 "Don Clugston" <dac nospam.com.au> wrote in message 
 news:dka37t$2c01$1 digitaldaemon.com...
 

 (1). It's only possible to concatenate strings, not arrays of other
  constant types.
 
 eg: const int a = [1,2,3] ~ [4,5,6];
 
 should be exactly equivalent to:
 
 const int a = [1,2,3,4,5,6];
 
 but unfortunately it's evaluated after template instantiation, not 
 before. If this order was changed, it would be possible (for
 example) to create lookup tables at compile time. The fact that it
 already works for strings suggests that most of the machinery is
 already in place.


 
 The reason it does not currently work is because there is no syntax
 for array literals as a PrimaryExpression. This is fixable, but not
 at the moment.
 

 (2).  [] is also evaluated after template instantiation.
 
 const char [] str = "abcdefghi"[2..4]; should be equivalent to 
 const char [] str = "bcd";
 
 If both restrictions (1) and (2) were removed, we'd be able to do
 almost any functional programming at compile time.


 
 This will be fixed in the upcoming update.
 

 However, some significant syntactic sugar is also possible. Most
 useful would be a relaxation of the rules for template value
 parameters. C++ doesn't allow floating point numbers, though some
 vendors have implemented it. AFAIK, the reason C++ doesn't allow it
 is because of lack of standards in floating point representation,
 so that there are problems with round-off error. Since D specifies 
 IEEE floating point, includes the hex float format, and most 
 importantly, has 'static if', I don't think those arguments apply.
 
 Casts are evaluated before template instantiation, so it is
 possible to pass a real as a template parameter, you just need to
 do a really ugly cast. (Or two casts, and pass two parameters,
 because 80 bit reals won't fit into a long).
 
 So, request (3) : allow reals as template value parameters.


 
 The reason it doesn't at the moment is it makes for really wretched
 template instance names :-(


I don't know about the others, but I definitely would want to see 1,2 
and 3 implemented before DMD 1.0!! Even if it takes a a few months extra!

D's been under development for many years, and we've come a long way. 
Possibly longer than any of us could imagine two years ago.

I think getting the templates right (especially when we now have Don 
with us showing the way), is simply paramount. The situation is a bit 
like when Stepanov was this far with the STL, and the C++ standards 
committee reopened the already declared feature freeze!
Nov 06 2005
prev sibling next sibling parent Don Clugston <dac nospam.com.au> writes:
Walter Bright wrote:

 "Don Clugston" <dac nospam.com.au> wrote in message
 news:dka37t$2c01$1 digitaldaemon.com...
 

(1). It's only possible to concatenate strings, not arrays of other
constant types.

eg:
const int a = [1,2,3] ~ [4,5,6];

should be exactly equivalent to:

const int a = [1,2,3,4,5,6];

but unfortunately it's evaluated after template instantiation, not
before. If this order was changed, it would be possible (for example) to
create lookup tables at compile time. The fact that it already works for
strings suggests that most of the machinery is already in place.
 


 The reason it does not currently work is because there is no syntax for
 array literals as a PrimaryExpression. This is fixable, but not at the
 moment.


Ah. That does sound more difficult. Well, we go a long way with chars 
(and especially with wchars and dchars).
But could we do constant folding of char[] ~ char ?

(Right now, the only thing that works is char[] ~ char[].
The workaround I'm using is to convert a char to a string via: 
x"000102030405..."[n..n+1]. Basically a hack, and unfeasible for wide 
chars).


(2).  [] is also evaluated after template instantiation.

const char [] str = "abcdefghi"[2..4];
should be equivalent to
const char [] str = "bcd";

If both restrictions (1) and (2) were removed, we'd be able to do almost
  any functional programming at compile time.


 
 
 This will be fixed in the upcoming update.


Thanks! I've already had quite a bit of fun with that :-)


However, some significant syntactic sugar is also possible. Most useful
would be a relaxation of the rules for template value parameters.





request (3) : allow reals as template value parameters.


 
 The reason it doesn't at the moment is it makes for really wretched template
 instance names :-(


As I've posted elsewhere, it turns out we can do the general case by 
template aliases, although it's not as nice as a proper value parameter. 
But I don't really understand why a 64 bit IEEE double is so much worse 
than a 64 bit long? Surely you can just do a kind of reinterpret_cast, 
just whacking the bits in?
Nov 10 2005
prev sibling parent reply Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
In article <dkhtlf$10k6$1 digitaldaemon.com>, Walter Bright says...

The reason it does not currently work is because there is no syntax for
array literals as a PrimaryExpression. This is fixable, but not at the
moment.


Analogous to the proposed future syntax for stack vs heap allocated classes,
wouldn't a natural syntax for array-literals be:

ArrayType ( ArgumentList )

and

new ArrayType ( ArgumentList )

for stack and heap based array literas respectively.

i.e:

int[](1,2,4)
new double[](4,2,1.5)
new MyClass[](new MyClass(1), new MyClass(2))

Allowing compatibility with user defined collection classes:

Set!(int)(1,2,4)
new Set!(double)(4,2,1.5)

Maybe
const char[]('h','e','l','l','o',0)
could be the same as
"hello"

i.e. placed in the .rodata-section.

/Oskar
Nov 10 2005
parent reply Don Clugston <dac nospam.com.au> writes:
The reason it does not currently work is because there is no syntax for
array literals as a PrimaryExpression. This is fixable, but not at the
moment.


 
 Analogous to the proposed future syntax for stack vs heap allocated classes,
 wouldn't a natural syntax for array-literals be:
 
 ArrayType ( ArgumentList )
 
 and
 
 new ArrayType ( ArgumentList )
 
 for stack and heap based array literas respectively.
 
 i.e:
 
 int[](1,2,4)
 new double[](4,2,1.5)
 new MyClass[](new MyClass(1), new MyClass(2))


But already, in the code below, a and b and even c are valid, but d 
won't compile.
--------------------------------
int [] a = [1,2,3,4]; // ok - static

int main()
{
     const int [] b = [1,2,3,4]; ok - const
     int [] c = b;          // this is ok
     int [] d = [1,2,3,4]; // this doesn't compile
     return 0;
}
------------------
(Statics work as well).
For array literals, we want 'd' to compile (and be identical to 'c'). It 
seems clear to me that it should use the same syntax as for a const or 
static array.

The specific problem I'm interested in is

const int [] a = [1, 2, 3, 4];
const int [] b = [5, 6, 7, 8];

const int [] c = a ~ b; // doesn't work, unfortunately
int [] d = a ~ b;       // doesn't work

int main()
{
    int f[] = a ~ b;     // But this works!
    return 0;
}


 Allowing compatibility with user defined collection classes:
 
 Set!(int)(1,2,4)
 new Set!(double)(4,2,1.5)


It ought to be possible for the UDT to support

  Set!(int) a = [1, 2, 4];

or at least
   Set!(int)( [1, 2, 4] );
?

-Don

 
 Maybe
 const char[]('h','e','l','l','o',0)
 could be the same as
 "hello"
 
 i.e. placed in the .rodata-section.
 
 /Oskar
Nov 10 2005
parent reply Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
In article <dkvh10$1bc9$1 digitaldaemon.com>, Don Clugston says...

The reason it does not currently work is because there is no syntax for
array literals as a PrimaryExpression. This is fixable, but not at the
moment.


 
 Analogous to the proposed future syntax for stack vs heap allocated classes,
 wouldn't a natural syntax for array-literals be:
 
 ArrayType ( ArgumentList )
 
 and
 
 new ArrayType ( ArgumentList )
 
 for stack and heap based array literas respectively.
 
 i.e:
 
 int[](1,2,4)
 new double[](4,2,1.5)
 new MyClass[](new MyClass(1), new MyClass(2))


But already, in the code below, a and b and even c are valid, but d 
won't compile.
--------------------------------
int [] a = [1,2,3,4]; // ok - static

int main()
{
     const int [] b = [1,2,3,4]; ok - const
     int [] c = b;          // this is ok
     int [] d = [1,2,3,4]; // this doesn't compile
     return 0;
}
------------------
(Statics work as well).
For array literals, we want 'd' to compile (and be identical to 'c'). It 
seems clear to me that it should use the same syntax as for a const or 
static array.


Yes, I see no reason not to make d valid. But as I have understood, the future
goal is to make array literals a PrimaryExpression. This needs a syntax that
allows construction of anonymous arrays. There are of course plenty of
options... 

Dynamically typed languages have the luxury of allowing things similar to:

for i in [2,3,5,7,11,13]:
some_function(i,[1,4,6,8])


I guess similar cases are where anonymous array literals in D would be most
appreciated.

Java, IIRC, uses:

new int[4]{1,2,3,4} 

This does unfortunately give parsing ambiguities when
translated into D:s usage of [...,...,...] for array literals.

I merely suggested a syntax that would allow compatibility with UDTs.
Although, I'm finding it potentially dangerous to combine a simple syntax
for stack-based allocation with types that have reference semantics 
(classes and arrays). 


The specific problem I'm interested in is

const int [] a = [1, 2, 3, 4];
const int [] b = [5, 6, 7, 8];

const int [] c = a ~ b; // doesn't work, unfortunately
int [] d = a ~ b;       // doesn't work

int main()
{
    int f[] = a ~ b;     // But this works!
    return 0;
}


Yes, I agree with you that it would be most useful if this was implemented
and I see nothing preventing a ~ b to be constantly folded.
My post was about array literals as primary expressions, which is not the
original topic. Sorry if I hijacked the thread.


 Allowing compatibility with user defined collection classes:
 
 Set!(int)(1,2,4)
 new Set!(double)(4,2,1.5)


It ought to be possible for the UDT to support

  Set!(int) a = [1, 2, 4];

or at least
   Set!(int)( [1, 2, 4] );
?


What is the type of [1,2,4]?. Is it int[] or (const) int[3] or ...?
How would you specify for instance a ushort[]? What would get passed to the
constructor of Set!(int)?.

/Oskar
Nov 10 2005
parent reply Don Clugston <dac nospam.com.au> writes:
But already, in the code below, a and b and even c are valid, but d 
won't compile.
--------------------------------
int [] a = [1,2,3,4]; // ok - static

int main()
{
    const int [] b = [1,2,3,4]; ok - const
    int [] c = b;          // this is ok
    int [] d = [1,2,3,4]; // this doesn't compile
    return 0;
}
------------------
(Statics work as well).
For array literals, we want 'd' to compile (and be identical to 'c'). It 
seems clear to me that it should use the same syntax as for a const or 
static array.


 
 Yes, I see no reason not to make d valid. But as I have understood, the future
 goal is to make array literals a PrimaryExpression. This needs a syntax that
 allows construction of anonymous arrays. There are of course plenty of
 options... 
 
 Dynamically typed languages have the luxury of allowing things similar to:
 
 for i in [2,3,5,7,11,13]:
 some_function(i,[1,4,6,8])
 
 
 I guess similar cases are where anonymous array literals in D would be most
 appreciated.
 
 Java, IIRC, uses:
 
 new int[4]{1,2,3,4} 
 
 This does unfortunately give parsing ambiguities when
 translated into D:s usage of [...,...,...] for array literals.
 
 I merely suggested a syntax that would allow compatibility with UDTs.
 Although, I'm finding it potentially dangerous to combine a simple syntax
 for stack-based allocation with types that have reference semantics 
 (classes and arrays). 
 
 

The specific problem I'm interested in is

const int [] a = [1, 2, 3, 4];
const int [] b = [5, 6, 7, 8];

const int [] c = a ~ b; // doesn't work, unfortunately
int [] d = a ~ b;       // doesn't work

int main()
{
   int f[] = a ~ b;     // But this works!
   return 0;
}


 
 
 Yes, I agree with you that it would be most useful if this was implemented
 and I see nothing preventing a ~ b to be constantly folded.
 My post was about array literals as primary expressions, which is not the
 original topic. Sorry if I hijacked the thread.


Not at all, you've helped me understand what "array literals as primary 
expressions" means. But now I don't understand why Walter says they're 
necessary for the constant folding!


Allowing compatibility with user defined collection classes:

Set!(int)(1,2,4)
new Set!(double)(4,2,1.5)


It ought to be possible for the UDT to support

 Set!(int) a = [1, 2, 4];

or at least
  Set!(int)( [1, 2, 4] );
?


 
 
 What is the type of [1,2,4]?. Is it int[] or (const) int[3] or ...?
 How would you specify for instance a ushort[]? What would get passed to the
 constructor of Set!(int)?.


You're right, it might be too difficult. I was thinking that there could 
be a requirement that only one array constructor is permissible.
The compiler manages to work it out in this example, so all kinds of 
implicit casting must be going on:
----------------------
template whoknows(X)
{
    const X f[] = [1, 2, 3, 4];
}

int main()
{
    mixin whoknows!(int);
    int [] w = f;
    assert(w[1]==2);
    {
	   mixin whoknows!(double);
	   double [] q = f;
	   assert(q[3]==4.0);
    }
    return 0;
}
Nov 10 2005
next sibling parent reply Ivan Senji <ivan.senji_REMOVE_ _THIS__gmail.com> writes:
Don Clugston wrote:

 You're right, it might be too difficult. I was thinking that there could 
 be a requirement that only one array constructor is permissible.
 The compiler manages to work it out in this example, so all kinds of 
 implicit casting must be going on:


But it wouldn't be difficult if array litterals had a type.
Now you can write a template that does this:
int[] a = array!(int)[1,2,3,4,5];
Object[] b = array!(Object)[new Object, new Object];

and i wouldn't mind having to write:

int[] x = new int[][1,2,3,4,5];
or
int[] x = int[][1,2,3,4,5]; or something like this
instead of:
int[] x = [1,2,3,4,5];

Or am i missing something?
Nov 10 2005
parent Oskar Linde <oskar.linde gmail.com> writes:
In article <dkvqj2$1lft$1 digitaldaemon.com>, Ivan Senji says...

Don Clugston wrote:

 You're right, it might be too difficult. I was thinking that there could 
 be a requirement that only one array constructor is permissible.
 The compiler manages to work it out in this example, so all kinds of 
 implicit casting must be going on:


But it wouldn't be difficult if array litterals had a type.
Now you can write a template that does this:
int[] a = array!(int)[1,2,3,4,5];
Object[] b = array!(Object)[new Object, new Object];

and i wouldn't mind having to write:

int[] x = new int[][1,2,3,4,5];
or
int[] x = int[][1,2,3,4,5]; or something like this
instead of:
int[] x = [1,2,3,4,5];


Neither would I.


Or am i missing something?


It is unfortunately ambiguous:

new int[1][1];

Is this a 1 element array initialized with a 1, or an uninitialized 1x1 array?


literals.

/Oskar
Nov 10 2005
prev sibling parent reply Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
In article <dkvq4q$1l21$1 digitaldaemon.com>, Don Clugston says...


Allowing compatibility with user defined collection classes:

Set!(int)(1,2,4)
new Set!(double)(4,2,1.5)


It ought to be possible for the UDT to support

 Set!(int) a = [1, 2, 4];

or at least
  Set!(int)( [1, 2, 4] );
?


 
 
 What is the type of [1,2,4]?. Is it int[] or (const) int[3] or ...?
 How would you specify for instance a ushort[]? What would get passed to the
 constructor of Set!(int)?.


You're right, it might be too difficult. I was thinking that there could 
be a requirement that only one array constructor is permissible.


One would want a syntax that felt consistent and worked with
built in arrays, UDTs, built in associative arrays and combinations. 
Ideally, the syntax should also be similar for initilization and anonymous
literals.

[1,2,3] and ["jan":31, "feb":28] looks nice as syntax for arrays and aa,
but both need to be tagged with a type to work as expressions. Or,
they could be made implicitly castable to compatible types, which could
work when only one applicable constructor is available.


The compiler manages to work it out in this example, so all kinds of 
implicit casting must be going on:
----------------------
template whoknows(X)
{
    const X f[] = [1, 2, 3, 4];
}


Well... As far as I understand it, [1, 2, 3, 4] is parsed as an Initializer 
and not an Expression, and the entire declaration statement is not evaluated
until the template is instantiated and X known.

/Oskar
Nov 10 2005
parent Don Clugston <dac nospam.com.au> writes:
Oskar Linde wrote:

 In article <dkvq4q$1l21$1 digitaldaemon.com>, Don Clugston says...
 
 

Allowing compatibility with user defined collection classes:

Set!(int)(1,2,4)
new Set!(double)(4,2,1.5)


It ought to be possible for the UDT to support

Set!(int) a = [1, 2, 4];

or at least
 Set!(int)( [1, 2, 4] );
?



What is the type of [1,2,4]?. Is it int[] or (const) int[3] or ...?
How would you specify for instance a ushort[]? What would get passed to the
constructor of Set!(int)?.


You're right, it might be too difficult. I was thinking that there could 
be a requirement that only one array constructor is permissible.


 
 
 One would want a syntax that felt consistent and worked with
 built in arrays, UDTs, built in associative arrays and combinations. 
 Ideally, the syntax should also be similar for initilization and anonymous
 literals.
 
 [1,2,3] and ["jan":31, "feb":28] looks nice as syntax for arrays and aa,
 but both need to be tagged with a type to work as expressions. Or,
 they could be made implicitly castable to compatible types, which could
 work when only one applicable constructor is available.
 
 

The compiler manages to work it out in this example, so all kinds of 
implicit casting must be going on:
----------------------
template whoknows(X)
{
   const X f[] = [1, 2, 3, 4];
}


 
 
 Well... As far as I understand it, [1, 2, 3, 4] is parsed as an Initializer 
 and not an Expression, and the entire declaration statement is not evaluated
 until the template is instantiated and X known.


Ah, OK. Looking at Declaration.html, I see that each member of the array 
initialiser is an expression but the
initialiser itself is not an expression. But char [] initialisers are
expressions. No expressions involving array literals exist, but there 
ARE expressions involving const arrays. So it ought to be possible to 
support
const int [] c = a ~ b;
even without array literals as primary expressions. In fact, by my 
reading of the spec, this ought to work already.

The following already works:

template square(int c)
{
     const int square = c * c;
}

template whoknows(X)
{
    const X f[] = [square!(1), square!(2), square!(3), square!(4)];
}

Even when assigned to an short [], which is interesting (it checks each 
constant individually to see if it can fit into a short, even if it's 
declared as a long).
Nov 11 2005