www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Brainstorming: Explicit instantiation of function template with `auto

reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
Function templates with `auto ref` parameters can’t be explicitly 
instantiated, but must be IFTI’d to infer `ref`-ness of `auto 
ref` parameters.

When doing meta-programming, that is annoying. Some template 
might have verified that some alias refers to a function template 
and that it can be called with the kinds of arguments the 
template wants to use. Great job, D, for making this rather easy! 
However, for some reason, we need a function pointer (or 
delegate) to the respective instance. That is always possible, 
except if the alias is to a function template with `auto ref` 
parameters.

We should find a solution to this. It does not have to be pretty, 
there should be just *some* way to do it.

The ideal test case is a function template with a sequence 
parameter and `auto ref` on non-template-argument-type parameters:
```d
void example(Ts...)(auto ref int arg, auto ref Ts args) { }

auto fp = &example/*YOUR IDEA*/;
```

One way I thought could work is just passing `ref T`: 
`example!(ref long)`, but that doesn’t work for the `int` 
parameter.

While I think the following looks okay, it occupies the syntax 
`![]` which we might use for something more useful:

```d
auto fp1 = &example![ref](); // void function(ref int)
auto fp2 = &example![auto](); // void function(int)

auto fp1 = &example![ref, auto, ref](char, wchar); // void 
function(ref int, char, ref wchar)
auto fp2 = &example![auto, ref, auto](char, wchar); // void 
function(int, ref char, wchar)
```

While required for explicit function template instantiation, even 
IFTI can profit from it: Not every lvalue argument should be 
passed by reference.
```d
int x;
example(x); // Full IFTI: `ref`, Ts empty
example!()(x); // Partial IFTI: infers `ref`, Ts empty, same as 
above
example![ref]()(x); // No IFTI: `ref` explicit, Ts empty, same as 
above
example![auto]()(x); // No IFTI: `auto` means not `ref`. Pass `x` 
by value/copy

```
Feb 20
next sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Thursday, 20 February 2025 at 23:24:10 UTC, Quirin Schroll 
wrote:

 Function templates with `auto ref` parameters can’t be 
 explicitly instantiated, but must be IFTI’d to infer `ref`-ness 
 of `auto ref` parameters.

 [...]


unittest? Im not entirely sure its impossible
Feb 20
prev sibling next sibling parent Basile B. <b2.temp gmx.com> writes:
On Thursday, 20 February 2025 at 23:24:10 UTC, Quirin Schroll 
wrote:

 [...]
 One way I thought could work is just passing ref T: 
 example!(ref long), but that doesn’t work for the int parameter.


maybe a way to go further with this idea is not to forget that we 
know the name of the `int` parameter, so something like

```d
void example(Ts...)(auto ref int arg, auto ref Ts args) { }

// void function(ref int arg);
auto fp1 = &example!(ref int arg);

// void function(ref int, char, ref wchar);
auto fp2 = &example!(ref int arg, char, ref wchar);
```

looks possible even if that only works if the parameter name is 
specified.

But otherwise the other syntax, I mean the one you've explained 
more, would work I think. That looks a bit like if the 
instantiation is for an array literal but it can be distinguished 
with two lookups (unless I'm loosing my D, an expression cannot 
start with `auto` or `ref` ?), or otherwise three (once reached a 
comma or a closing squared bracket).
Feb 21
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 20 February 2025 at 23:24:10 UTC, Quirin Schroll 
wrote:

 Function templates with `auto ref` parameters can’t be 
 explicitly instantiated, but must be IFTI’d to infer `ref`-ness 
 of `auto ref` parameters.

 When doing meta-programming, that is annoying. Some template 
 might have verified that some alias refers to a function 
 template and that it can be called with the kinds of arguments 
 the template wants to use. Great job, D, for making this rather 
 easy! However, for some reason, we need a function pointer (or 
 delegate) to the respective instance. That is always possible, 
 except if the alias is to a function template with `auto ref` 
 parameters.


It seems to me like this is a general problem with IFTI and/or 
overload resolution, not just `auto ref`. For example, suppose we 
use separate overloads instead of `auto ref`:

     template fun(T)
     {
         void fun(    T x,     T y) {}
         void fun(ref T x,     T y) {}
         void fun(    T x, ref T y) {}
         void fun(ref T x, ref T y) {}
     }

In this example, `fun` is callable using IFTI with any 
combination of lvalue and rvalue arguments, but obtaining a 
pointer to the specific overload of `fun` that's called for a 
given argument list is, let's say, non-trivial.

So, given the above, here's my proposed solution for this entire 
class of problems:

     int lvalue;
     enum int rvalue = 123;

     auto fp = &__traits(resolve, fun(lvalue, rvalue));
     // fp == &fun!int.fun(ref int, int)

`__traits(resolve)` takes a function call expression as its 
argument, and returns an alias to the function symbol that would 
be called by that expression, with both IFTI and overload 
resolution taken into account. If either IFTI or overload 
resolution fail, `__traits(compiles)` also fails--errors are not 
gagged.
Feb 21
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
Just yet another in the endless stream of cases why ref should be part of
the type and not a 'storage class'! Literally everything ref touches gets
more complex than it should.
*=F0=9F=8E=89*

On Fri, 21 Feb 2025 at 09:26, Quirin Schroll via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Function templates with `auto ref` parameters can=E2=80=99t be explicitly
 instantiated, but must be IFTI=E2=80=99d to infer `ref`-ness of `auto
 ref` parameters.

 When doing meta-programming, that is annoying. Some template
 might have verified that some alias refers to a function template
 and that it can be called with the kinds of arguments the
 template wants to use. Great job, D, for making this rather easy!
 However, for some reason, we need a function pointer (or
 delegate) to the respective instance. That is always possible,
 except if the alias is to a function template with `auto ref`
 parameters.

 We should find a solution to this. It does not have to be pretty,
 there should be just *some* way to do it.

 The ideal test case is a function template with a sequence
 parameter and `auto ref` on non-template-argument-type parameters:
 ```d
 void example(Ts...)(auto ref int arg, auto ref Ts args) { }

 auto fp =3D &example/*YOUR IDEA*/;
 ```

 One way I thought could work is just passing `ref T`:
 `example!(ref long)`, but that doesn=E2=80=99t work for the `int`
 parameter.

 While I think the following looks okay, it occupies the syntax
 `![]` which we might use for something more useful:

 ```d
 auto fp1 =3D &example![ref](); // void function(ref int)
 auto fp2 =3D &example![auto](); // void function(int)

 auto fp1 =3D &example![ref, auto, ref](char, wchar); // void
 function(ref int, char, ref wchar)
 auto fp2 =3D &example![auto, ref, auto](char, wchar); // void
 function(int, ref char, wchar)
 ```

 While required for explicit function template instantiation, even
 IFTI can profit from it: Not every lvalue argument should be
 passed by reference.
 ```d
 int x;
 example(x); // Full IFTI: `ref`, Ts empty
 example!()(x); // Partial IFTI: infers `ref`, Ts empty, same as
 above
 example![ref]()(x); // No IFTI: `ref` explicit, Ts empty, same as
 above
 example![auto]()(x); // No IFTI: `auto` means not `ref`. Pass `x`
 by value/copy

 ```
Feb 23
next sibling parent Nick Treleaven <nick geany.org> writes:
On Monday, 24 February 2025 at 00:24:03 UTC, Manu wrote:

 Just yet another in the endless stream of cases why ref should 
 be part of
 the type and not a 'storage class'!


Modern languages avoid doing that, just like D. Herb Sutter:


 C++ references were invented to be used as function 
 parameter/return types, and that’s what they’re still primarily 
 useful for. Since C++11, that includes the range-for loop which 
 conceptually works like a function call (see Q&A).



 Sometimes, a reference can also be useful as a local variable, 
 though in modern C++ a pointer or structured binding is usually 
 better (see Q&A).



 That’s it. All other uses of references should be avoided.


https://herbsutter.com/2020/02/23/references-simply/

In particular, the Q & A section:


 For example, if you’re writing a class template, just assume 
 (or document) that it can’t be instantiated with reference 
 types.


Which concludes:


 the dual nature of references is always the problem.



    If the design embraces the pointer-ness of references (one 
 level of indirection), then one set of use cases works and 
 people with alias-like use cases get surprised.



    If the design embraces the alias-ness of references (no 
 indirection), then the other set of use cases works and people 
 with pointer-like use cases get surprised.



    If the design mixes them, then a variety of people get 
 surprised in creative ways.
Feb 24
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 24 February 2025 at 00:24:03 UTC, Manu wrote:

 Just yet another in the endless stream of cases why ref should 
 be part of
 the type and not a 'storage class'! Literally everything ref 
 touches gets
 more complex than it should.
 *🎉*


+1000

The consistent, principled way to bring ref into the type system 
would be to assign all lvalue expressions a type of ref(T), and 
all rvalue expressions a type of T, with an implicit ref(T) -> T 
conversion for copyable T.

Unfortunately this is not really feasible for D, since it would 
be an enormous breaking change. But anyone designing a new 
language should consider this approach.
Feb 24
parent reply M. M. <matus email.cz> writes:
On Monday, 24 February 2025 at 13:50:07 UTC, Paul Backus wrote:

 On Monday, 24 February 2025 at 00:24:03 UTC, Manu wrote:

 Just yet another in the endless stream of cases why ref should 
 be part of
 the type and not a 'storage class'! Literally everything ref 
 touches gets
 more complex than it should.
 *🎉*


 +1000

 The consistent, principled way to bring ref into the type 
 system would be to assign all lvalue expressions a type of 
 ref(T), and all rvalue expressions a type of T, with an 
 implicit ref(T) -> T conversion for copyable T.

 Unfortunately this is not really feasible for D, since it would 
 be an enormous breaking change. But anyone designing a new 
 language should consider this approach.


not even with the editions coming? . . .
Feb 24
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 24 February 2025 at 16:30:59 UTC, M. M. wrote:

 On Monday, 24 February 2025 at 13:50:07 UTC, Paul Backus wrote:

 On Monday, 24 February 2025 at 00:24:03 UTC, Manu wrote:

 Just yet another in the endless stream of cases why ref 
 should be part of
 the type and not a 'storage class'! Literally everything ref 
 touches gets
 more complex than it should.
 *🎉*


 +1000

 The consistent, principled way to bring ref into the type 
 system would be to assign all lvalue expressions a type of 
 ref(T), and all rvalue expressions a type of T, with an 
 implicit ref(T) -> T conversion for copyable T.

 Unfortunately this is not really feasible for D, since it 
 would be an enormous breaking change. But anyone designing a 
 new language should consider this approach.


 not even with the editions coming? . . .


Nope. Even with editions, some changes are simply too disruptive.
Feb 24
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Thursday, 20 February 2025 at 23:24:10 UTC, Quirin Schroll 
wrote:

 Function templates with `auto ref` parameters can’t be 
 explicitly instantiated, but must be IFTI’d to infer `ref`-ness 
 of `auto ref` parameters.


I thought you could at least do it if I was willing to jump 
through the hoops to completely specify the function type I was 
trying to instantiate:

```D
void example(Ts...)(auto ref int arg, auto ref Ts args)
{	import std.stdio;
	writeln(is(typeof(&arg))? "lvalue " : "rvalue ", arg);
}

void main()
{	void function(int) fPtr1 = &example!();
	void function(ref int) fPtr2 = &example!();
	
	fPtr1(10);
	fPtr2(*new int(20));
}
```

. But the compiler plain out stated
```
app.d(1): Error: cannot explicitly instantiate template function 
with `auto ref` parameter
```
, so this problem is even worse than I thought. I agree some sort 
of solution is needed, though I don't have a good idea what it 
should be like.
Feb 24