www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Asking for the opinion of DMD developers

reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
Hello everyone,

I'm the author of DIP 1023 [1], [2].

I'll try to keep this short to save you time by only mentioning 
the important
parts (and those are a lot anyway).
Feel free to ask me to elaborate on anything that it's not clear.

So, first of all, D can't cope with alias templates used as 
function
parameters (in template functions). For example:

struct TypeTemplate(T) {}
alias AliasTemplate(T) = TypeTemplate!T;
void functionTemplate(T)(AliasTemplate!T arg) {}

void main()
{
     AliasTemplate!int inst;
     functionTemplate(inst); /* "cannot deduce function from 
argument types !()(TypeTemplate!int)" */
}

DIP 1023 was introduced because the reasoning was that this is 
not simply
a bug and the feature was in fact, under-specified.
That is, a bug is when we have the form "When I do X, Z should 
happen" but
it doesn't. "Under-specified" means that "When I do X, it's not 
even clear
that Z should happen in the first place".
So, this is where the compiler writer requests a specification 
addition.

My position is that while I believe that Alias Templates are in 
general
under-specified in D, adding any more specification won't help
DMD writers _in this specific issue_.

Let me elaborate: Let's say that we do have a good enough 
specification,
like C++'s.

With the help of Nicholas Wilson who asked on reddit, I was 
redirected
to the C++ standard [3] in the sections 17.6.7 Alias Templates
and 17.5 Type Equivalence.

I think having such a specification won't help anyone here.
To be even more specific, as far as I can understand,
the issue at hand is handled in the C++ standard with 3 sentences 
(point 2,
section 17.6.7):

"When a template-id refers to the specialization of an alias 
template, it is equivalent to the associated type obtained by 
substitution of its template-arguments for the 
template-parameters in the type-id of the alias
template".

And this something that we already know, yet can't solve the 
issue.

What's your opinion on this ?

Btw, please consider that this is not my area of expertise. :)

Best regards,
Stefanos Baziotis

[1] https://github.com/dlang/DIPs/pull/176
[2] 
https://forum.dlang.org/post/dnyqxmgdazczwmmvayjx forum.dlang.org
[3] 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4713.pdf
Oct 13 2019
next sibling parent reply Elie Morisse <syniurge gmail.com> writes:
On Sunday, 13 October 2019 at 10:51:52 UTC, Stefanos Baziotis 
wrote:

 Hello everyone,

 I'm the author of DIP 1023 [1], [2].

 I'll try to keep this short to save you time by only mentioning 
 the important
 parts (and those are a lot anyway).
 Feel free to ask me to elaborate on anything that it's not 
 clear.

 So, first of all, D can't cope with alias templates used as 
 function
 parameters (in template functions). For example:

 struct TypeTemplate(T) {}
 alias AliasTemplate(T) = TypeTemplate!T;
 void functionTemplate(T)(AliasTemplate!T arg) {}

 void main()
 {
     AliasTemplate!int inst;
     functionTemplate(inst); /* "cannot deduce function from 
 argument types !()(TypeTemplate!int)" */
 }

 DIP 1023 was introduced because the reasoning was that this is 
 not simply
 a bug and the feature was in fact, under-specified.
 That is, a bug is when we have the form "When I do X, Z should 
 happen" but
 it doesn't. "Under-specified" means that "When I do X, it's not 
 even clear
 that Z should happen in the first place".
 So, this is where the compiler writer requests a specification 
 addition.

 My position is that while I believe that Alias Templates are in 
 general
 under-specified in D, adding any more specification won't help
 DMD writers _in this specific issue_.

 Let me elaborate: Let's say that we do have a good enough 
 specification,
 like C++'s.

 With the help of Nicholas Wilson who asked on reddit, I was 
 redirected
 to the C++ standard [3] in the sections 17.6.7 Alias Templates
 and 17.5 Type Equivalence.

 I think having such a specification won't help anyone here.
 To be even more specific, as far as I can understand,
 the issue at hand is handled in the C++ standard with 3 
 sentences (point 2,
 section 17.6.7):

 "When a template-id refers to the specialization of an alias 
 template, it is equivalent to the associated type obtained by 
 substitution of its template-arguments for the 
 template-parameters in the type-id of the alias
 template".

 And this something that we already know, yet can't solve the 
 issue.

 What's your opinion on this ?

 Btw, please consider that this is not my area of expertise. :)

 Best regards,
 Stefanos Baziotis

 [1] https://github.com/dlang/DIPs/pull/176
 [2] 
 https://forum.dlang.org/post/dnyqxmgdazczwmmvayjx forum.dlang.org
 [3] 
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4713.pdf


This is because DMD's current implementation of template argument 
deduction isn't advanced enough to cover this case, but it could 
be improved to cover it.

In your example, in "functionTemplate(inst);" inst has its type 
resolved to TypeTemplate!T and DMD's argument deduction tries 
matching the identifier "TypeTemplate" to the identifier 
"AliasTemplate", which fails. The argument deduction function in 
DMD doesn't know/check that AliasTemplate!T would actually 
resolve to TypeTemplate!T.

Regarding C++, C++ template argument deduction is more 
"sophisticated" but also has its limits, maybe to keep the 
complexity of the implementation at an acceptable level.
An example of C++ template function that doesn't work with the 
rules of C++ template argument deduction is 
std::forward<T>(std::remove_reference_t<T>& t) (see 
https://en.cppreference.com/w/cpp/utility/forward), C++ compilers 
cannot deduce T by matching the argument type to 
std::remove_reference<T>. Thus std::forward can only be used with 
an explicit template argument.

The C++ equivalent of your example does work, but alias templates 
are different in C++ and D. In C++ they only alias types, while 
in D there is no concept of alias template, your AliasTemplate 
may have one or more overloads that instantiate a variable or 
some other kind of symbol completely unrelated to aliases.
So C++ has it easier and the paragraph you quoted from the spec 
makes argument deduction possible by first substituting the 
function parameter by a "dummy instantiation of 
AliasTemplate<T>". A similar solution wouldn't be easy to 
implement in DMD for the aforementioned reasons, but it may be 
do-able.

IMHO if not done yet you should submit a bugzilla issue with your 
example and perhaps dive into DMD's code and try figuring out a 
solution.
Oct 13 2019
next sibling parent Elie Morisse <syniurge gmail.com> writes:
On Sunday, 13 October 2019 at 15:06:11 UTC, Elie Morisse wrote:

 In C++ they only alias types, while in D there is no concept of 
 alias template,


By that I meant that:

alias SomeAlias(T) = ...;

is syntactic sugar for:

template SomeAlias(T) { alias SomeAlias = ...; }


And unlike in C++ SomeAlias might have overloads that instantiate 
something completely different :

template SomeAlias(T : char) { enum SomeAlias = 987; }
template SomeAlias(T : int) { void SomeAlias() {} }
Oct 13 2019
prev sibling parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Sunday, 13 October 2019 at 15:06:11 UTC, Elie Morisse wrote:

 This is because DMD's current implementation of template 
 argument deduction isn't advanced enough to cover this case, 
 but it could be improved to cover it.


Yes, the problem though is that DMD developers could not cover it,
so the DIP was introduced to help.


 Regarding C++, C++ template argument deduction is more 
 "sophisticated" but also has its limits, maybe to keep the 
 complexity of the implementation at an acceptable level.
 An example of C++ template function that doesn't work with the 
 rules of C++ template argument deduction is 
 std::forward<T>(std::remove_reference_t<T>& t) (see 
 https://en.cppreference.com/w/cpp/utility/forward), C++ 
 compilers cannot deduce T by matching the argument type to 
 std::remove_reference<T>. Thus std::forward can only be used 
 with an explicit template argument.


I didn't know that, interesting.


 The C++ equivalent of your example does work, but alias 
 templates are different in C++ and D. In C++ they only alias 
 types, while in D there is no concept of alias template, your 
 AliasTemplate may have one or more overloads that instantiate a 
 variable or some other kind of symbol completely unrelated to 
 aliases.
 So C++ has it easier and the paragraph you quoted from the spec 
 makes argument deduction possible by first substituting the 
 function parameter by a "dummy instantiation of 
 AliasTemplate<T>". A similar solution wouldn't be easy to 
 implement in DMD for the aforementioned reasons, but it may be 
 do-able.


I have some little experience with DMD, but not enough to
answer this fully. From a draft implementation I did, it seems
that for the simple case, it should be doable.
However, what you mentioned with the overloads etc. is 
interesting,
I'll refer to it again below.


 IMHO if not done yet you should submit a bugzilla issue with 
 your example and perhaps dive into DMD's code and try figuring 
 out a solution.


You can check the DIP I referenced above [1]. There are links to 
a draft
implementation and a lot of bugzilla reports.


 And unlike in C++ SomeAlias might have overloads that 
 instantiate something completely different :
 
 template SomeAlias(T : char) { enum SomeAlias = 987; }
 template SomeAlias(T : int) { void SomeAlias() {} }


This is interesting in that a specification for alias templates 
as C++'s
may not be enough and an addition may actually help. TBH, I don't 
think though.
Because DMD currently can't handle even the simplest case.
I left the implementation and started the DIP when I stumbled 
upon problems
that didn't have to do with aliases but with template 
instantiation.
This is something that definitely should be able to be solved 
with the
current specification, yet it doesn't.


 A possible solution like C++'s, but taking into account the 
 multiplicity of TemplateDeclaration overloads would be:


Some of the things you mentioned are done but I don't remember 
wholly.
However, I get the idea.
Currently, I don't have time to work on the implementation though.
Consider that this was supposed to have ended by the end of May.
Unfortunately, it's unclear how much time will I have to do only 
the DIP work.
So basically, I can focus only on the specification side of 
things and
leave the implementation reasoning (i.e. whether the DIP will 
help with that,
how it can be done etc.) to the DMD developers.

Thanks for your input!
- Stefanos

[1] https://github.com/dlang/DIPs/pull/176
Oct 13 2019
prev sibling parent reply Elie Morisse <syniurge gmail.com> writes:
On Sunday, 13 October 2019 at 10:51:52 UTC, Stefanos Baziotis 
wrote:

 [1] https://github.com/dlang/DIPs/pull/176
 [2] 
 https://forum.dlang.org/post/dnyqxmgdazczwmmvayjx forum.dlang.org


I read your DIP and the review thread.


 if (arg is templateInstance && arg.TD is 
 aliasTemplateDeclaration) {


Checking that a TemplateDeclaration is an "alias template 
declaration" is easy : TemplateDeclaration.onemember && 
TemplateDeclaration.onemember.isAliasDeclaration()

However since they may be more than one TemplateDeclaration 
overload, your templateInstance cannot exist yet and you don't 
know which TemplateDeclaration overload should be probed..

A possible solution like C++'s, but taking into account the 
multiplicity of TemplateDeclaration overloads would be:

  - If the first round of arg deduction fails, try transforming 
the TypeFunction by substituting each function parameter that 
failed matching with a dummy instance from each 
TemplateDeclaration overload that contains an alias (one 
substitution attempt per overload) and redo the arg deduction. 
For each function parameter, no more than one TemplateDeclaration 
overload must match or else it's ambiguous.
  - If every combination of "one substitution per param" fails, 
keep substituting the remaining non-matching already substituted 
function parameters until there's no deeper alias template (i.e 
no more alias Temp(T) = SomeOtherAliasTemp(T)).

Downside : there may be A LOT of possible combinations.
Oct 13 2019
parent Elie Morisse <syniurge gmail.com> writes:
On Sunday, 13 October 2019 at 16:28:00 UTC, Elie Morisse wrote:

 Downside : there may be A LOT of possible combinations.


Actually function template argument deduction is done parameter 
by parameter (left to right), so the number of possible 
combinations shouldn't be too high.
Oct 13 2019