www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Thoughts on possible tuple semantics

reply "Dicebot" <public dicebot.lv> writes:
Inspired by recent syntax thread I decided to write down by 
vision of how native language tuples should behave and how it may 
integrate into existing language state. There are lot of tricky 
corner cases and I'd like to see some more opinions on topic 
before trying DIP with full deprecation path design.

I didn't care about syntax or exact tuple features such as 
unpacking. Main questions for me were "what this thing actually 
is? and "how it can be expressed in existing language terms?"

----- Core -----

Define two distinct built-in "tuples":

     - Template argument sequence: may contain anything that is 
legal template
         argument.
     - Value tuple or simply "tuple": may contain any values, 
including run-time
     values. Value storage location is not defined.

Template argument sequence that contains only types is called 
"type sequence"
and is considered a type on its own. Type of value tuple is a 
type sequence.

Mixed template argument sequences are considered types too, but 
special one. Those
types can be aliased or passed to templates but can't be 
instantiated.

Each of these two entities has its own literal type. This is 
required to avoid
ambiguity between storing symbol as a type and taking its value 
on run-time. Tuple
always does the latter.

In further examples I will use imaginary syntax:
     ctseq(int, string, 42) : template argument sequence
     tuple("one", 2, three) : value tuple


 










// connections to existing template syntax

void foo(T...)(T args)
{
     static assert(is(T == ctseq(int, string)));
     static assert(is(typeof(args) == T));
     assert(args == tuple(1, "2"));
     int a = 1;
     string b = "2";
     assert(args == tuple(a, b));
     static assert(typeof(tuple(a, b)) == ctseq(int, string));
}

foo(1, "2");

// type semantics of type sequence

ctseq(int, int) twoVars;
twoVars[0] = 42;
twovars[1] = 43;
assert(twoVars == tuple(42, 43));

ctseq(int, 42) twoVars; // compile-time error, can't instantiate 
mixed template argument sequence
assert(twoVars != ctseq(42, 42)); // NOT the same, breaking change

// compile-time vs run-time vs type semantics

auto a1 = tuple(42, 42); // ok
auto b1 = ctseq(42, 42); // error

enum a2_1 = tuple(42, 42); // ok
int a, b;
enum a2_2 = tuple(a, b); // error
enum b2 = ctseq(42, 42); // error, breaking change

alias a3 = tuple(42, 42); // error
alias b3 = ctseq(42, 42); // ok

<<<<<

----- (auto) expansion / packing / unpacking -----

As Andrei has stated clearly that he does not like auto-expansion 
and considers
it a major mistake, I was trying to imagine how that idea can be 
incorporated
into idiomatic D code.


 










// existing syntax

// args can't be single entity and use normal parameter passing 
ABI at the same
// time.

void foo(T...)(T args) // following normal ABI implies unpacking
{
}

foo(1, 2, 3); // automatic packing

<<<<<

Breaking something like this does not seem reasonable. But I 
think salvation is
the "..." part. It may be explicitly defined to "implicit 
unpacking" and can be
used with palin tuple code like this:


 










void boo(T)(T args)
{
     foo(args.expand);
}

boo(tuple(1, 2, 3));

<<<<<

One thing to consisder is .tupleof - should it result in actual 
tuple or maintain
current behavior? Former is probably more reasonable but it is 
even more break

struct S { int a, b, c; }

foo(S.init.tupleof.expand); // huh

.expand should be probably defined as a simple syntax rewrite:
     - tuple(a, b)" to "a, b" for literals
     - "tupleVar" to "tupleVar[0], tupleVar[1]" for variables

That also implies that packing / unpacking syntax, whatever it 
can be,
is completely unrelated to expansion - former can't be expressed 
as a simple
syntax rewrite, latter can't have special semantics tied to it 
without creating
even more meta-type to represent it.

----- ABI -----

Once built-in value tuples get recognized as a distinct entity, 
there is no
reason to now allow using them for return values or as 
un-expanded parameters.

All is needed is to define that tuple(a, b, c) has same ABI for 
return values
and parameters as a struct Tuple{ typeof(a) a; typeof(b) b; 
typeof(c) c; } - I have been
told that there are some issues with that approach but with no 
clear explanation.

Mangling question remains open.


----- std.typetuple.TypeTuple / std.typecons.Tuple -----

No need to keep them other than for backwards compatibility ;)
Aug 21 2013
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
First, thank you so much ! The previous discussion was going 
nowhere as too much focused on syntax.

On Wednesday, 21 August 2013 at 15:22:59 UTC, Dicebot wrote:

 Define two distinct built-in "tuples":

     - Template argument sequence: may contain anything that is 
 legal template
         argument.
     - Value tuple or simply "tuple": may contain any values, 
 including run-time
     values. Value storage location is not defined.

 Template argument sequence that contains only types is called 
 "type sequence"
 and is considered a type on its own. Type of value tuple is a 
 type sequence.


Good let's drop tuple for sequence, that is much, much better. 
However, as sequence works now, it won't fit for the type of a 
tuple, as it auto expand.


 Each of these two entities has its own literal type. This is 
 required to avoid
 ambiguity between storing symbol as a type and taking its value 
 on run-time. Tuple
 always does the latter.

 In further examples I will use imaginary syntax:
     ctseq(int, string, 42) : template argument sequence
     tuple("one", 2, three) : value tuple


I have some syntax suggestion, but let's keep that for later. 
Semantic first.


 ----- (auto) expansion / packing / unpacking -----

 As Andrei has stated clearly that he does not like 
 auto-expansion and considers
 it a major mistake, I was trying to imagine how that idea can 
 be incorporated
 into idiomatic D code.


That is the hard point, clearly. This has advantages, but 
difficult to make everything fit together.


 










 // existing syntax

 // args can't be single entity and use normal parameter passing 
 ABI at the same
 // time.

 void foo(T...)(T args) // following normal ABI implies unpacking
 {
 }

 foo(1, 2, 3); // automatic packing


Which mean that I can call foo with a tuple directly ? What about 
:
auto a = tuple(1, 2);
foo(a, 3);

I guess this is the pack/unpack issue again.


 ----- std.typetuple.TypeTuple / std.typecons.Tuple -----

 No need to keep them other than for backwards compatibility ;)


Nobody understood them anyway.
Aug 21 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 21 August 2013 at 15:55:01 UTC, deadalnix wrote:

 Good let's drop tuple for sequence, that is much, much better. 
 However, as sequence works now, it won't fit for the type of a 
 tuple, as it auto expand.


My attitude is as follows "either we unify them even at cost of 
major breakage or is better to not touch it at all and just focus 
on some usability features, abandoning the hope to remove 
confusion". Creating new entities while keeping old ones as-is 
will make situation even worse :(


 As Andrei has stated clearly that he does not like 
 auto-expansion and considers
 it a major mistake, I was trying to imagine how that idea can 
 be incorporated
 into idiomatic D code.


 That is the hard point, clearly. This has advantages, but 
 difficult to make everything fit together.


Yes and I actually favor current auto-expansion despite all the 
issues - it is just easier to define what expansion is in terms 
of special tuple type semantics then some sort of action.

But Andrei seems to be pretty convinced about this and I have 
accepted this as a given starting point.


 // existing syntax

 // args can't be single entity and use normal parameter 
 passing ABI at the same
 // time.

 void foo(T...)(T args) // following normal ABI implies 
 unpacking
 {
 }

 foo(1, 2, 3); // automatic packing


 Which mean that I can call foo with a tuple directly ? What 
 about :
 auto a = tuple(1, 2);
 foo(a, 3);

 I guess this is the pack/unpack issue again.


Hm, the wording feels wrong now. No, here is the idea:
auto a = tuple(1, 2);
foo(a, 3); // rejected
foo(a.expand, 3); // works

"T..." here simply says, "work with T as it is a tuple but it is 
a normal function argument list in fact".

so no packing actually happens here: "foo(1, 2, 3)", my mistake
Aug 21 2013
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 21 August 2013 at 16:19:43 UTC, Dicebot wrote:

 Hm, the wording feels wrong now. No, here is the idea:
 auto a = tuple(1, 2);
 foo(a, 3); // rejected
 foo(a.expand, 3); // works

 "T..." here simply says, "work with T as it is a tuple but it 
 is a normal function argument list in fact".

 so no packing actually happens here: "foo(1, 2, 3)", my mistake


My concern here is to keep the foo(T...)(T args) equivalent to 
template foo(T...) { foo(T args) }

We also need to be more precise about what we mean when saying 
packing/unpacking.
Aug 21 2013
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 21 August 2013 at 16:51:18 UTC, deadalnix wrote:

 On Wednesday, 21 August 2013 at 16:19:43 UTC, Dicebot wrote:

 Hm, the wording feels wrong now. No, here is the idea:
 auto a = tuple(1, 2);
 foo(a, 3); // rejected
 foo(a.expand, 3); // works

 "T..." here simply says, "work with T as it is a tuple but it 
 is a normal function argument list in fact".

 so no packing actually happens here: "foo(1, 2, 3)", my mistake


 My concern here is to keep the foo(T...)(T args) equivalent to 
 template foo(T...) { foo(T args) }

 We also need to be more precise about what we mean when saying 
 packing/unpacking.


As far as I see it, equivalence will persist. However, this will 
change:

template foo(T...)
{
     void foo()
     {
         T values; // error, use "ctseq(T) values" instead, T... 
can't act as a single entity
     }
}
Aug 21 2013
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 21 August 2013 at 15:55:01 UTC, deadalnix wrote:

 First, thank you so much ! The previous discussion was going 
 nowhere as too much focused on syntax.


Also what do you thing about core idea behind proposal - defining
strict type system relation that typeof(tuple(int.init)) ==
ctseq(int)?
Aug 21 2013
parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 21 August 2013 at 16:35:04 UTC, Dicebot wrote:

 On Wednesday, 21 August 2013 at 15:55:01 UTC, deadalnix wrote:

 First, thank you so much ! The previous discussion was going 
 nowhere as too much focused on syntax.


 Also what do you thing about core idea behind proposal - 
 defining
 strict type system relation that typeof(tuple(int.init)) ==
 ctseq(int)?


ctseq(int) should be the type of the unpacked tuple, to keep 
thing flowing with foo(T...)(T args);
Aug 21 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 21 August 2013 at 15:22:59 UTC, Dicebot wrote:

 Inspired by recent syntax thread I decided to write down by 
 vision of how native language tuples should behave and how it 
 may integrate into existing language state. There are lot of 
 tricky corner cases and I'd like to see some more opinions on 
 topic before trying DIP with full deprecation path design.

 I didn't care about syntax or exact tuple features such as 
 unpacking. Main questions for me were "what this thing actually 
 is? and "how it can be expressed in existing language terms?"

 ----- Core -----

 Define two distinct built-in "tuples":

     - Template argument sequence: may contain anything that is 
 legal template
         argument.
     - Value tuple or simply "tuple": may contain any values, 
 including run-time
     values. Value storage location is not defined.

 Template argument sequence that contains only types is called 
 "type sequence"
 and is considered a type on its own. Type of value tuple is a 
 type sequence.

 Mixed template argument sequences are considered types too, but 
 special one. Those
 types can be aliased or passed to templates but can't be 
 instantiated.

 Each of these two entities has its own literal type. This is 
 required to avoid
 ambiguity between storing symbol as a type and taking its value 
 on run-time. Tuple
 always does the latter.

 In further examples I will use imaginary syntax:
     ctseq(int, string, 42) : template argument sequence
     tuple("one", 2, three) : value tuple


 










 // connections to existing template syntax

 void foo(T...)(T args)
 {
     static assert(is(T == ctseq(int, string)));
     static assert(is(typeof(args) == T));
     assert(args == tuple(1, "2"));
     int a = 1;
     string b = "2";
     assert(args == tuple(a, b));
     static assert(typeof(tuple(a, b)) == ctseq(int, string));
 }

 foo(1, "2");

 // type semantics of type sequence

 ctseq(int, int) twoVars;
 twoVars[0] = 42;
 twovars[1] = 43;
 assert(twoVars == tuple(42, 43));

 ctseq(int, 42) twoVars; // compile-time error, can't 
 instantiate mixed template argument sequence
 assert(twoVars != ctseq(42, 42)); // NOT the same, breaking 
 change

 // compile-time vs run-time vs type semantics

 auto a1 = tuple(42, 42); // ok
 auto b1 = ctseq(42, 42); // error

 enum a2_1 = tuple(42, 42); // ok
 int a, b;
 enum a2_2 = tuple(a, b); // error
 enum b2 = ctseq(42, 42); // error, breaking change

 alias a3 = tuple(42, 42); // error
 alias b3 = ctseq(42, 42); // ok

 <<<<<

 ----- (auto) expansion / packing / unpacking -----

 As Andrei has stated clearly that he does not like 
 auto-expansion and considers
 it a major mistake, I was trying to imagine how that idea can 
 be incorporated
 into idiomatic D code.


 










 // existing syntax

 // args can't be single entity and use normal parameter passing 
 ABI at the same
 // time.

 void foo(T...)(T args) // following normal ABI implies unpacking
 {
 }

 foo(1, 2, 3); // automatic packing

 <<<<<

 Breaking something like this does not seem reasonable. But I 
 think salvation is
 the "..." part. It may be explicitly defined to "implicit 
 unpacking" and can be
 used with palin tuple code like this:


 










 void boo(T)(T args)
 {
     foo(args.expand);
 }

 boo(tuple(1, 2, 3));

 <<<<<

 One thing to consisder is .tupleof - should it result in actual 
 tuple or maintain
 current behavior? Former is probably more reasonable but it is 
 even more break

 struct S { int a, b, c; }

 foo(S.init.tupleof.expand); // huh

 .expand should be probably defined as a simple syntax rewrite:
     - tuple(a, b)" to "a, b" for literals
     - "tupleVar" to "tupleVar[0], tupleVar[1]" for variables

 That also implies that packing / unpacking syntax, whatever it 
 can be,
 is completely unrelated to expansion - former can't be 
 expressed as a simple
 syntax rewrite, latter can't have special semantics tied to it 
 without creating
 even more meta-type to represent it.

 ----- ABI -----

 Once built-in value tuples get recognized as a distinct entity, 
 there is no
 reason to now allow using them for return values or as 
 un-expanded parameters.

 All is needed is to define that tuple(a, b, c) has same ABI for 
 return values
 and parameters as a struct Tuple{ typeof(a) a; typeof(b) b; 
 typeof(c) c; } - I have been
 told that there are some issues with that approach but with no 
 clear explanation.

 Mangling question remains open.


 ----- std.typetuple.TypeTuple / std.typecons.Tuple -----

 No need to keep them other than for backwards compatibility ;)


I like it :)

The question of ABI is interesing. I can think of a few options, 
working relative to the System V x86_64 ABI (read C on linux x64) 
as it's what i'm familiar with:

treat tuples as structs:

advantages: simple to implement, easy to interact with other ABI 
compliant code.

disadvantages: when returning tuple > 8 bytes requires using up 
an extra register on the *calling* side as a struct return is 
done via a pointer in EDI (i.e. 1st argument) to caller-allocated 
stack memory. This introduces an extra indirection. It's not the 
fastest option. Same for when passing them.


treat tuples as seperate arguments (when possible):

This would mean defining a new ABI on the returning side. If we 
used something like the System V *calling* ABI, then we'd get:

advantages: no indirection, arguments are ready in registers for 
callee, results are ready in registers for the caller to access 
quickly. Fast. Manu suggests that the advantages are greater on 
non-x86

disadvantages: not compatible with other ABI compliant code. 
Increased register pressure for both caller and callee in some 
circumstances. Could requires some extra movs from stack to 
registers on caller side, dependent on where the tuple is 
previously stored/later needed.



A consideration for all of this: I predict we would quickly start 
seeing a lot of code that takes a tuple and returns a modified 
version.
Using the struct option:
if larger than 8 bytes, pass pointer to old tuple-struct and new 
one. Return address of new one in RAX. Indirection but not too 
bad.

Using the seperate option:
pass tuple members in seperate registers (when small enough). If 
they're already in registers and the callee doesn't have to move 
them out, this couldn't be any faster.* Could (v. rarely these 
days) result in arguments overflowing on to stack.


*movs between registers are essentially free on modern x86/64
Aug 21 2013
prev sibling next sibling parent reply "Meta" <jared771 gmail.com> writes:
On Wednesday, 21 August 2013 at 15:22:59 UTC, Dicebot wrote:

     - Template argument sequence: may contain anything that is 
 legal template argument.


One thing I'd like to know. If we're at the point where we're 
discussing possible breaking changes to the language anyway, why 
is it so important that these tuples behave exactly like template 
argument lists? How important is it to be able to include both 
types AND values within the same tuple outside of variadic 
templates? I don't know if I've ever seen a use case for this.


     - Value tuple or simply "tuple": may contain any values, 
 including run-time
     values. Value storage location is not defined.


Are you implying that the address of a tuple cannot be taken?


 Template argument sequence that contains only types is called 
 "type sequence"
 and is considered a type on its own. Type of value tuple is a 
 type sequence.


I like this. Just to be clear, can you assign a type sequence to 
a variable or only alias it, like with regular types?


 Mixed template argument sequences are considered types too, but 
 special one. Those
 types can be aliased or passed to templates but can't be 
 instantiated.


What is the type of (42, string, dchar, false)? Is it (int, 
string, dchar, bool) (i.e., type sequence), or is it its own 
distinct type? Also, what operations can be done on a mixed 
tuple? Does this count as instantiation?:

void foo(T...)(T t) //Error?
{
     //...
}


 Each of these two entities has its own literal type. This is 
 required to avoid
 ambiguity between storing symbol as a type and taking its value 
 on run-time. Tuple
 always does the latter.


You're losing me. I sort of agree as to why, but I don't the idea 
of having two different tuples syntaces. That's part of why the 
situation is as it is in the first place. Yes, there needs to be 
a good way to make clear which type of tuple you're using, but I 
don't think two ways of making a tuple is a good thing.


 void foo(T...)(T args)


Looks like my earlier question was answered. Why is this not 
instantiation?


 {
     static assert(is(T == ctseq(int, string)));
     static assert(is(typeof(args) == T));
     assert(args == tuple(1, "2"));
     int a = 1;
     string b = "2";
     assert(args == tuple(a, b));
     static assert(typeof(tuple(a, b)) == ctseq(int, string));
 }


Something else. How does tuple(1, "a") relate to ctuple(1, "a"), 
if at all? How does ctuple(int, string) relate to ctuple(1, "a")? 
Is there ever a case where typeof(tuple(value, value)) == 
ctuple(value, value), or ctuple(type, type) == ctuple(value, 
value)?


 // type semantics of type sequence

 ctseq(int, int) twoVars;
 twoVars[0] = 42;
 twovars[1] = 43;
 assert(twoVars == tuple(42, 43));

 ctseq(int, 42) twoVars; // compile-time error, can't 
 instantiate mixed template argument sequence
 assert(twoVars != ctseq(42, 42)); // NOT the same, breaking 
 change


Okay, one question answered.


 // compile-time vs run-time vs type semantics

 auto a1 = tuple(42, 42); // ok
 auto b1 = ctseq(42, 42); // error


Good, but second example will probably cause confusion.


 enum a2_1 = tuple(42, 42); // ok
 int a, b;
 enum a2_2 = tuple(a, b); // error


Makes perfect sense.


 enum b2 = ctseq(42, 42); // error, breaking change


Will probably cause confusion.


 alias a3 = tuple(42, 42); // error
 alias b3 = ctseq(42, 42); // ok


Great.


 One thing to consisder is .tupleof - should it result in actual 
 tuple or maintain
 current behavior? Former is probably more reasonable but it is 
 even more break


I think it should be consistent with how your hypothetical tuple 
semantics work in the other situations you described. That would 
be much less surprising than having only .tupleof auto-expand.


 struct S { int a, b, c; }

 foo(S.init.tupleof.expand); // huh


I don't see the "huh" here. Seems perfectly reasonable to me. Any 
code relying on this behaviour would break if members were added 
to S anyway (and code shouldn't rely on this, IMO).


 .expand should be probably defined as a simple syntax rewrite:
     - tuple(a, b)" to "a, b" for literals
     - "tupleVar" to "tupleVar[0], tupleVar[1]" for variables

 That also implies that packing / unpacking syntax, whatever it 
 can be,
 is completely unrelated to expansion - former can't be 
 expressed as a simple
 syntax rewrite, latter can't have special semantics tied to it 
 without creating
 even more meta-type to represent it.


I don't know what you mean here.


 ----- std.typetuple.TypeTuple / std.typecons.Tuple -----

 No need to keep them other than for backwards compatibility ;)


Wouldn't both break with the changes you described?
Aug 21 2013
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 21 August 2013 at 16:53:12 UTC, Meta wrote:

 ...


Thanks for detailed response! :)


 On Wednesday, 21 August 2013 at 15:22:59 UTC, Dicebot wrote:

    - Template argument sequence: may contain anything that is 
 legal template argument.


 One thing I'd like to know. If we're at the point where we're 
 discussing possible breaking changes to the language anyway, 
 why is it so important that these tuples behave exactly like 
 template argument lists? How important is it to be able to 
 include both types AND values within the same tuple outside of 
 variadic templates? I don't know if I've ever seen a use case 
 for this.


There is a single huge use case - all templates :) If f we don't 
provide built-in way to express parameters accepted by templates, 
TypeTuple will appear again, because some way to keep and 
manipulate those parameters is a cornerstone for generic 
programming. Just count how many times template variadic 
parameters get sliced (and then aliased or passed to other 
template) in Phobos.


    - Value tuple or simply "tuple": may contain any values, 
 including run-time
    values. Value storage location is not defined.


 Are you implying that the address of a tuple cannot be taken?


Yes.


 Mixed template argument sequences are considered types too, 
 but special one. Those
 types can be aliased or passed to templates but can't be 
 instantiated.


 What is the type of (42, string, dchar, false)? Is it (int, 
 string, dchar, bool) (i.e., type sequence), or is it its own 
 distinct type?


Own distinct type.


 Also, what operations can be done on a mixed tuple? Does this 
 count as instantiation?:

 void foo(T...)(T t) //Error?
 {
     //...
 }


Yes, it is an error if T is not pure type sequence (it already 
is).


 You're losing me. I sort of agree as to why, but I don't the 
 idea of having two different tuples syntaces. That's part of 
 why the situation is as it is in the first place.


It has played minor role in current situation. Similar naming and 
vague meaning of TypeTuple has played major roles. Those have 
different literals because theu have _completely_ different usage 
semantics. Once those are defined and clear it is no more 
confusing than static array vs dynamic array.


 void foo(T...)(T args)


 Looks like my earlier question was answered. Why is this not 
 instantiation?


It is :)


 Something else. How does tuple(1, "a") relate to ctuple(1, 
 "a"), if at all?


tuple(1, "a") == tuple(ctseq(1, "a").expand) // only relation I 
can think of


 How does ctuple(int, string) relate to ctuple(1, "a")?


Completely unrelated.


 Is there ever a case where typeof(tuple(value, value)) == 
 ctuple(value, value), or ctuple(type, type) == ctuple(value, 
 value)?


No, never.

  Great.


 .expand should be probably defined as a simple syntax rewrite:
    - tuple(a, b)" to "a, b" for literals
    - "tupleVar" to "tupleVar[0], tupleVar[1]" for variables

 That also implies that packing / unpacking syntax, whatever it 
 can be,
 is completely unrelated to expansion - former can't be 
 expressed as a simple
 syntax rewrite, latter can't have special semantics tied to it 
 without creating
 even more meta-type to represent it.


 I don't know what you mean here.


I have changed my mind while was typing this answer :) Those may 
be more related than I have initially expected but I can't 
express it right now.


 ----- std.typetuple.TypeTuple / std.typecons.Tuple -----

 No need to keep them other than for backwards compatibility ;)


 Wouldn't both break with the changes you described?


Those can be re-implemented to keep old behavior during 
deprecation period but with all the breakage I don't know if 
makes any sense/
Aug 21 2013
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/21/13 8:22 AM, Dicebot wrote:

 Inspired by recent syntax thread I decided to write down by vision of
 how native language tuples should behave and how it may integrate into
 existing language state.


Awesome!


 Define two distinct built-in "tuples":


[snip]

 In further examples I will use imaginary syntax:
      ctseq(int, string, 42) : template argument sequence
      tuple("one", 2, three) : value tuple











 // connections to existing template syntax

 void foo(T...)(T args)
 {
      static assert(is(T == ctseq(int, string)));


There's going to be significant difficulty with this. This would be much 
easier:

       static assert(is(ctseq(T) == ctseq(int, string)));

Right now "T" as above has no type of its own, and ascribing a type to 
it would be a major change to which I think it's impossible to retrofit 
existing code without some breakages. If "T" is a type then "args" has a 
type and then we're back to asking ourselves whether writeln(args) 
passes one argument or more to writeln.

I do agree that we could take a different route if we designed the type 
system today, but we aren't. Current design is a legacy/liability.


      static assert(is(typeof(args) == T));
      assert(args == tuple(1, "2"));


So it seems the relationship between ctseq and tuple is that 
typeof(tuple(x, y. z)) is ctseq(typeof(x), typeof(y), typeof(z)). This 
leaves out of the discussion things like ctseq(1, "2") which contain 
compile-time values, not types. In that regard I'd prefer the two being 
entirely distinct.



Andrei



 Aug 21 2013



  parent  "Dicebot" <public dicebot.lv>  writes:


On Wednesday, 21 August 2013 at 19:18:52 UTC, Andrei Alexandrescu 
wrote:

 void foo(T...)(T args)
 {
     static assert(is(T == ctseq(int, string)));


 There's going to be significant difficulty with this. This 
 would be much easier:

       static assert(is(ctseq(T) == ctseq(int, string)));


It is not easier, it is returning to current issue I was trying 
to address - question "what is T and how it is related to 
existing type system".


 Right now "T" as above has no type of its own


Then why it can be aliased?


 If "T" is a type then "args" has a type and then we're back to 
 asking ourselves whether writeln(args) passes one argument or 
 more to writeln.


If we assume they auto-expansion is out, it is straightforward:
writeln(args) passes one argument, writeln(args.expand) - 
multiple.


 I do agree that we could take a different route if we designed 
 the type system today, but we aren't. Current design is a 
 legacy/liability.


Then, as I have already said, we can hardly do anything better 
than renaming TypeTuple and abandoning any hope to remove 
confusion. Well, maybe one can add some syntax sugar to make it 
more powerful, but built-ins will remain same weirdos.

And this is a time to step down for a moment and ask "is 
cumulative cost of documentation efforts and learning issues 
during all D lifespan worse than one hard by finite deprecation 
process?".

Current design is so hacky that we can't really do anything about 
it without breakage.


     static assert(is(typeof(args) == T));
     assert(args == tuple(1, "2"));


 So it seems the relationship between ctseq and tuple is that 
 typeof(tuple(x, y. z)) is ctseq(typeof(x), typeof(y), 
 typeof(z)). This leaves out of the discussion things like 
 ctseq(1, "2") which contain compile-time values, not types. In 
 that regard I'd prefer the two being entirely distinct.


They can't be distinct as both are accepted in one template 
argument list and this mandates that both must be able to be 
aliased. That makes them types. In my proposal ctseq(1, "2") also 
becomes a type, one that contains only meta-data and can't be 
instantiated.



 Aug 21 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


So, I took some time to think about this. Let me propose 
something close, but different.

I'll try to define sequences more precisely and define some tools 
that allow to implement tuples in a nice way on top of it.

A sequence is an ordered set. You can have types, alias and 
values sequences. Values sequence can be runtime or compile time.

You can ask for sequences as parameter template using ... and 
using the regular template parameter syntax.

template(T) => template(T...)
template(alias T) => template(alias T...)
template(T U, T) => template(T U, T...)
template(T alias U) => template(T alias U..., T...)

Obviously, we want template(T...) to be a special for a 
transition period. T... being a subset of alias T..., we can 
start with a warning when the use it outside the subset.

It is a breaking change, but reduce the confusion by reusing the 
template parameter mechanism. Considering that most people here 
are very confused by type tuples, I highly doubt a lot of code is 
outside abusing this feature.

A type sequence can be used to define a value sequence :
auto foo(T...)(T args) { ... }

args is a value sequence. It isn't packed as a struct or 
anything, simply several values (here several function 
parameters) hidden behing one identifier. As a result, args do 
not have address or type.

typeof(args) returns T, which is a type sequence (not a type !)
typeid(args) is invalid.

Each member of args however have an address, a type and 
everything a function parameter have.

A sequence provide an access to its member via [index] and length 
to indicate its length. index need to be a compile time value, 
and length is a compile time value.

It is possible to use a sequence to define any ordered set of 
stuff (value sequence can be used for function call, type 
sequence to create multiple fields in a struct, etc . . .).

Right, most of what is described here is close from what we 
already have. Types sequence in a struct can be used to create 
tuples and IFTI can make it very handy.

Sequences can be built using the coma operator. int, float is a 
type sequence. args[0], args[1] is a value sequence. They can be 
returned from functions :

int, int foo() {
     return 3, 4;
}

int, int a = foo();

However, due to syntax conflict, int, int a = 3, 4; is not 
possible (it the value on the right is an assignment, it conflict 
with multiple declarations). () can be used to disambiguate : 
int, int a = (3, 4);

The missing piece is an auto dispatch function. I propose here a 
simple rewrite rule, as this is simple to implement and can be 
really effective.

auto (a, b) = foo();

is rewritten as

auto tmp = foo(); // Here tmp is a sequence of declaration of 
value. No need to create lvalues (especially is a complex copy is 
involved).
assert(tmp.length == 2); // Should be removed anyway for 
sequences.
auto a = tmp[0];
auto b = tmp[1];

This allow to make any user type unpackable. Or even arrays, 
slices, randomAccessRanges, etc . . .



 Aug 22 2013



 next sibling parent reply "Dicebot" <public dicebot.lv>  writes:


On Thursday, 22 August 2013 at 11:19:55 UTC, deadalnix wrote:

 A sequence is an ordered set. You can have types, alias and 
 values sequences. Values sequence can be runtime or compile 
 time.



 typeof(args) returns T, which is a type sequence (not a type !)
 typeid(args) is invalid.


1) You propose typeof to result "not a type" entity. Well, if it 
can be result of typeof, can be used to declare variables and can 
be aliased - what makes it different from a type?

2) is(typeof(tuple(42, 42)) == typeof(ctseq(42, 42))) == ? 
(assuming typeof(args) == T and you don't make distinction 
between runtime and compile-time value sequences.

Also you don't seem to cover mixed sequences which are essential 
to D templates.



 Aug 22 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Thursday, 22 August 2013 at 12:50:06 UTC, Dicebot wrote:

 On Thursday, 22 August 2013 at 11:19:55 UTC, deadalnix wrote:

 A sequence is an ordered set. You can have types, alias and 
 values sequences. Values sequence can be runtime or compile 
 time.



 typeof(args) returns T, which is a type sequence (not a type !)
 typeid(args) is invalid.


 1) You propose typeof to result "not a type" entity. Well, if 
 it can be result of typeof, can be used to declare variables 
 and can be aliased - what makes it different from a type?


It is a sequence of types. It can be used to declare a sequence 
of variables.


 2) is(typeof(tuple(42, 42)) == typeof(ctseq(42, 42))) == ? 
 (assuming typeof(args) == T and you don't make distinction 
 between runtime and compile-time value sequences.


In my proposal, tuple are a library construct. The proposal 
introduce the necessary tooling to implement them nicely.

is(typeof(42, 42) == (int, int));
is(typeof(tuple(42, 42)) == <a library defined struct>);

It could be defined as follow :

struct Tuple(T...) {
     T expand;
     alias expand this;
}


 Also you don't seem to cover mixed sequences which are 
 essential to D templates.


Theses are alias sequences.



 Aug 22 2013



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Thursday, 22 August 2013 at 13:01:51 UTC, deadalnix wrote:

 It is a sequence of types. It can be used to declare a sequence 
 of variables.


That implies defining "sequence of X" as special entity on 
official spec and update 'alias', 'typeof' and template docs to 
reference it as a special case.


 In my proposal, tuple are a library construct. The proposal 
 introduce the necessary tooling to implement them nicely.

 is(typeof(42, 42) == (int, int));
 is(typeof(tuple(42, 42)) == <a library defined struct>);

 It could be defined as follow :

 struct Tuple(T...) {
     T expand;
     alias expand this;
 }


It is exactly what we have right now. So you think having two 
different types of expression/value tuples is fine?


 Also you don't seem to cover mixed sequences which are 
 essential to D templates.


 Theses are alias sequences.


As I have already said, it is no less confusing to classify as 
alias sequence something that is not limited to aliases.



 Aug 22 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Thursday, 22 August 2013 at 14:03:13 UTC, Dicebot wrote:

 It is exactly what we have right now. So you think having two 
 different types of expression/value tuples is fine?


One is a tuple, the other is a sequence. They are different beast 
and have different behavior.

The fact that it is similar to what we have now is on purpose, so 
we don't need to break a lot of code. Plus, with the proposed 
addition, it allow for nice library tuples.


 Also you don't seem to cover mixed sequences which are 
 essential to D templates.


 Theses are alias sequences.


 As I have already said, it is no less confusing to classify as 
 alias sequence something that is not limited to aliases.


It is what alias parameter for templates are. Simply keeping the 
terminology here.



 Aug 22 2013



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Thursday, 22 August 2013 at 14:22:50 UTC, deadalnix wrote:

 One is a tuple, the other is a sequence. They are different 
 beast and have different behavior.

 The fact that it is similar to what we have now is on purpose, 
 so we don't need to break a lot of code. Plus, with the 
 proposed addition, it allow for nice library tuples.


I am confused. You proposed tuple implementation is essentially 
the very same current std.typecons.Tuple is. So I can see only 
two differences between value sequence and tuple:

1) latter has address and ABI
2) former auto-expands

Anything else?


 Theses are alias sequences.


 As I have already said, it is no less confusing to classify as 
 alias sequence something that is not limited to aliases.


 It is what alias parameter for templates are. Simply keeping 
 the terminology here.


Wait, what? Alias parameters for templates have nothing to do 
with template argument lists (other than being one possible type 
of that list element). In other words, "T..." can contain more 
stuff than "alias T..." (imaginary syntax).

And, considering the fact that we already have two different 
alias semantics (for template alias parameters and normal 
aliases), it is a dangerous terminology to chose anyway.



 Aug 22 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Thursday, 22 August 2013 at 14:29:50 UTC, Dicebot wrote:

 Wait, what? Alias parameters for templates have nothing to do 
 with template argument lists (other than being one possible 
 type of that list element). In other words, "T..." can contain 
 more stuff than "alias T..." (imaginary syntax).


"other than being one possible type of that list element"

Indeed it is the only possible meaning and it the right one.


 And, considering the fact that we already have two different 
 alias semantics (for template alias parameters and normal 
 aliases), it is a dangerous terminology to chose anyway.


Yes, the term have 2 meanings. I simply reuse one of theses 
meaning without adding a new one.



 Aug 22 2013



prev sibling  parent reply "Regan Heath" <regan netmail.co.nz>  writes:


On Thu, 22 Aug 2013 15:03:12 +0100, Dicebot <public dicebot.lv> wrote:

 Also you don't seem to cover mixed sequences which are essential to D  
 templates.


 Theses are alias sequences.


 As I have already said, it is no less confusing to classify as alias  
 sequence something that is not limited to aliases.


I was thinking of it the other way round, as in .. It is not a sequence of  
aliases, but an alias for a sequence of .. expressions(?).  Perhaps  
calling it an "aliased sequence" makes that clearer?

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/



 Aug 22 2013



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Thursday, 22 August 2013 at 14:27:01 UTC, Regan Heath wrote:

 On Thu, 22 Aug 2013 15:03:12 +0100, Dicebot <public dicebot.lv> 
 wrote:

 Also you don't seem to cover mixed sequences which are 
 essential to D templates.


 Theses are alias sequences.


 As I have already said, it is no less confusing to classify as 
 alias sequence something that is not limited to aliases.


 I was thinking of it the other way round, as in .. It is not a 
 sequence of aliases, but an alias for a sequence of .. 
 expressions(?).  Perhaps calling it an "aliased sequence" makes 
 that clearer?

 R


Well, it falls into the same issue "named after what you can do 
with it" vs "named after what it is". The very meaning of word 
"alias" suggests that calling something "alias for X" is just 
adding naming indirection on top of X ;)



 Aug 22 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Thursday, 22 August 2013 at 14:32:27 UTC, Dicebot wrote:

 Well, it falls into the same issue "named after what you can do 
 with it" vs "named after what it is". The very meaning of word 
 "alias" suggests that calling something "alias for X" is just 
 adding naming indirection on top of X ;)


That is what an alias parameter is, and I do think that regular 
alias and alias parameter should converge.



 Aug 22 2013



prev sibling  parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx>  writes:


On Thu, Aug 22, 2013 at 01:19:53PM +0200, deadalnix wrote:

 So, I took some time to think about this. Let me propose something
 close, but different.
 
 I'll try to define sequences more precisely and define some tools
 that allow to implement tuples in a nice way on top of it.


[...]

Originally, I typed up a long response to what you posted, but suddenly,
it dawned on me that your idea, quoted below, is actually something much
more general and applicable than tuples alone. So I decided to dedicate
my reply to it:


 The missing piece is an auto dispatch function. I propose here a
 simple rewrite rule, as this is simple to implement and can be
 really effective.
 
 auto (a, b) = foo();
 
 is rewritten as
 
 auto tmp = foo(); // Here tmp is a sequence of declaration of value.
 No need to create lvalues (especially is a complex copy is
 involved).
 assert(tmp.length == 2); // Should be removed anyway for sequences.
 auto a = tmp[0];
 auto b = tmp[1];
 
 This allow to make any user type unpackable. Or even arrays, slices,
 randomAccessRanges, etc . . .


Now *this* is something new, and worth talking about.

Suppose we forget about the whole tuple fiasco, and forget that there's
such a thing as a tuple (or TypeTuple or whatever else there is that's
confusing everybody).  Just with this syntax alone, we can solve all
kinds of problems:

- If f is a function that returns some kind of array, we can have
  automatic unpacking of arrays:

	int[] func() { return [1,2,3]; }

	void main() {
		auto (x, y, z) = func();
		// Equivalent to:
		// auto tmp = func();
		// x = tmp[0];
		// y = tmp[1];
		// z = tmp[2];
	}

- We can automatically unpack regex matches:

	import std.regex;
	auto (x, y, z) = inputString.match(`(\d+)\s+(\w+)\s+(\S+)`).captures;
	// x = string matched by (\d+)
	// y = string matched by (\w+)
	// z = string matched by (\S+)

- Like you said, any indexable range can be supported by this syntax. In
  fact, I'd argue that you should be able to do this even with just an
  input range:

	auto (x, y, z) = makeInputRange();

  should be translated into:

	auto tmp = makeInputRange();
	assert(!tmp.empty);
	auto x = tmp.front;
	tmp.popFront();
	assert(!tmp.empty);
	auto y = tmp.front;
	tmp.popFront();
	assert(!tmp.empty);
	auto z = tmp.front;

  Since ranges are a major selling feature of D, I'd argue that
  in-language support should be completely appropriate, and even
  desirable. (In fact, foreach already understands what a range is, so
  why not extend it here as well.)


You said that the missing piece was an auto dispatch function. Well, I
think if we add yet another piece to it, this could become a killer
feature in D, even regardless of what happens with the whole tuples
issue:

The above is all nice and good, except that you can't pass a
tuple/array/range return from a function into a poly-adic function's
arguments. That is to say:

	int[] func1() { return [1,2,3]; }
	void func2(int x, int y, int z) { ... }

	// Currently this line doesn't compile:
	func2(func1());

I used int[] for illustration purposes only; it can also be, say, an
input range of ints:

	T func1() { ... }
	assert(isInputRange!T && is(ElementType!T == int));

	void func2(int x, int y, int z) { ... }

	func2(func1());
	// ^^^ this will be rewritten into:
	// auto tmp = func1();
	// auto arg1 = tmp.front;
	// tmp.popFront();
	// auto arg2 = tmp.front;
	// tmp.popFront();
	// auto arg3 = tmp.front;
	// func2(arg1, arg2, arg3);

Of course, if T is an array, then it will be rewritten into
func2(tmp[0], tmp[1], tmp[2]); same goes if T is a Tuple, etc.. Anything
indexable with array notation should undergo this rewriting. So you
could write:

	Tuple!(int,string,bool) func1() {
		return tuple(1, "a", true);
	}

	void func2(int x, string y, bool z) { ... }

	func2(func1());
	// ^^^^ this gets rewritten into:
	// auto tmp = func1();
	// func2(tmp[0], tmp[1], tmp[2]);

My point is that this auto-dispatch / auto-repack is not limited to
tuples alone. It can be made to work in a nice way to anything that has
array indexing notation or a range interface.

This would solve the problem of multiple return values, for example. You
could have a div() function that returns a quotient and remainder:

	auto div(int x, int y) {
		...
		return [q, r];
		// Or, (q, r), or a 2-element input range
	}

	auto (x, y) = div(13, 7);


T

-- 
Computers aren't intelligent; they only think they are.



 Aug 22 2013



 next sibling parent  "deadalnix" <deadalnix gmail.com>  writes:


On Thursday, 22 August 2013 at 19:48:36 UTC, H. S. Teoh wrote:

 Now *this* is something new, and worth talking about.

 Suppose we forget about the whole tuple fiasco, and forget that 
 there's
 such a thing as a tuple (or TypeTuple or whatever else there is 
 that's
 confusing everybody).  Just with this syntax alone, we can 
 solve all
 kinds of problems:

 - If f is a function that returns some kind of array, we can 
 have
   automatic unpacking of arrays:

 	int[] func() { return [1,2,3]; }

 	void main() {
 		auto (x, y, z) = func();
 		// Equivalent to:
 		// auto tmp = func();
 		// x = tmp[0];
 		// y = tmp[1];
 		// z = tmp[2];
 	}

 - We can automatically unpack regex matches:

 	import std.regex;
 	auto (x, y, z) = 
 inputString.match(`(\d+)\s+(\w+)\s+(\S+)`).captures;
 	// x = string matched by (\d+)
 	// y = string matched by (\w+)
 	// z = string matched by (\S+)

 - Like you said, any indexable range can be supported by this 
 syntax. In
   fact, I'd argue that you should be able to do this even with 
 just an
   input range:

 	auto (x, y, z) = makeInputRange();

   should be translated into:

 	auto tmp = makeInputRange();
 	assert(!tmp.empty);
 	auto x = tmp.front;
 	tmp.popFront();
 	assert(!tmp.empty);
 	auto y = tmp.front;
 	tmp.popFront();
 	assert(!tmp.empty);
 	auto z = tmp.front;

   Since ranges are a major selling feature of D, I'd argue that
   in-language support should be completely appropriate, and even
   desirable. (In fact, foreach already understands what a range 
 is, so
   why not extend it here as well.)


Yes that is the intended effect.


 You said that the missing piece was an auto dispatch function. 
 Well, I
 think if we add yet another piece to it, this could become a 
 killer
 feature in D, even regardless of what happens with the whole 
 tuples
 issue:

 The above is all nice and good, except that you can't pass a
 tuple/array/range return from a function into a poly-adic 
 function's
 arguments. That is to say:

 	int[] func1() { return [1,2,3]; }
 	void func2(int x, int y, int z) { ... }

 	// Currently this line doesn't compile:
 	func2(func1());

 I used int[] for illustration purposes only; it can also be, 
 say, an
 input range of ints:

 	T func1() { ... }
 	assert(isInputRange!T && is(ElementType!T == int));

 	void func2(int x, int y, int z) { ... }

 	func2(func1());
 	// ^^^ this will be rewritten into:
 	// auto tmp = func1();
 	// auto arg1 = tmp.front;
 	// tmp.popFront();
 	// auto arg2 = tmp.front;
 	// tmp.popFront();
 	// auto arg3 = tmp.front;
 	// func2(arg1, arg2, arg3);

 Of course, if T is an array, then it will be rewritten into
 func2(tmp[0], tmp[1], tmp[2]); same goes if T is a Tuple, etc.. 
 Anything
 indexable with array notation should undergo this rewriting. So 
 you
 could write:

 	Tuple!(int,string,bool) func1() {
 		return tuple(1, "a", true);
 	}

 	void func2(int x, string y, bool z) { ... }

 	func2(func1());
 	// ^^^^ this gets rewritten into:
 	// auto tmp = func1();
 	// func2(tmp[0], tmp[1], tmp[2]);

 My point is that this auto-dispatch / auto-repack is not 
 limited to
 tuples alone. It can be made to work in a nice way to anything 
 that has
 array indexing notation or a range interface.

 This would solve the problem of multiple return values, for 
 example. You
 could have a div() function that returns a quotient and 
 remainder:

 	auto div(int x, int y) {
 		...
 		return [q, r];
 		// Or, (q, r), or a 2-element input range
 	}

 	auto (x, y) = div(13, 7);


 T


I'd like to see that being allowed explicitly, not implicitly as 
you propose.

foo(int, int, int) must be different than foo(int[3]) . If not, 
the TypeTuple unpacking mess is taken to yet another level.

The good news is that this is implementable with the proposal :

int, int, int dispatch(int[3] args) {
     return args[0], args[1], args[2];
}

foo(dispatch(arr));

To sum up, the only major change my proposal got it the ability 
to return values sequences. That plus some syntaxic sugar provide 
everything we need.

I'm thinking about it for month now and it only poped recently. 
I'm now convinced that this is the way forward :
  - If ABI allow multiple return values, we can take advantage of 
it.
  - We can have clean tuples.
  - We give more expressiveness to user defined types.



 Aug 23 2013



prev sibling  parent  "bearophile" <bearophileHUGS lycos.com>  writes:


H. S. Teoh:


 any indexable range can be supported by this syntax. In
   fact, I'd argue that you should be able to do this even with 
 just an
   input range:

 	auto (x, y, z) = makeInputRange();

   should be translated into:

 	auto tmp = makeInputRange();
 	assert(!tmp.empty);
 	auto x = tmp.front;
 	tmp.popFront();
 	assert(!tmp.empty);
 	auto y = tmp.front;
 	tmp.popFront();
 	assert(!tmp.empty);
 	auto z = tmp.front;

   Since ranges are a major selling feature of D, I'd argue that
   in-language support should be completely appropriate, and even
   desirable.


Yes, this is an "obvious" nice feature to support, I didn't list 
it because I wasn't bold enough :-)

This is a good situation to show how other languages do, this is 
Python2:


 lazy = (x * x for x in xrange(1, 4))
 lazy






<generator object <genexpr> at 0x02230238>

 a, b, c = lazy
 a






1

 b






4

 c






9

Similar code is possible in Haskell and Perl6.

Bye,
bearophile



 Aug 23 2013