www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Tuples, CTFE, and Sliding Template Arguments

reply Walter Bright <newshound2 digitalmars.com> writes:
Given the interest in CTFE of istrings, I have been thinking about making a 
general use case out of it instead of one specific to istrings.
---------------------

Sliding Template Arguments

Consider the following template:

```
void pluto(string s)()
{
     pragma(msg, s);
}

void test()
{
     pluto!"hello"();
}
```

This compiles because `s` is a compile time argument, and `pragma(msg, s)`
expects s to be a compile time value.

```
void pluto()(string s)
{
     pragma(msg, s);
}

void test()
{
     pluto("hello");
}
```

This fails to compile because `s` is a runtime argument that is not accessible
at compile time, even if it is inlined. (Inlining doesn't help here because 
inlining is done after CTFE and semantic analysis.)

Those examples illustrate the difference between a compile time and a runtime 
argument.

For background, to generate a tuple of elements:

```
alias AliasSeq(T...) = T;
```
and a function argument list that accepts a tuple:
```
void func(Args...)(Args args)
{
}
```
But notice that `args` are runtime arguments. It turns out there is no way
to use tuples to split an argument tuple into compile time and runtime tuples:

```
void pluto(Args...)(Args args)
{
     exec!(args[0])(args[1 .. args.length]);
}
```
This is the problem that DIP1036e ran into. It's clever solution is to have the 
compiler (because it cannot be done with metaprogramming) take the first 
argument, and use it as a compile time argument to a templated dummy value. 
Then, the type of that argument is a template with the value encoded into the 
type, which can be programmatically extracted and then processed at compile
time.

The awkwardness of this is it only happens with istrings, it is not a general 
purpose capability, plus the insertion of dummy arguments into the argument
list 
just so their types can be extracted.

Hence, the genesis of this proposal which describes a language capability to 
create a compile time parameter from a tuple of runtime expressions.

For lack of a better term, I'll call it "Sliding Template Arguments".

Consider a template function:

```
void pluto(string s, Args...)(Args args)
{
     pragma(msg, s);
}

void exec()
{
     pluto!"hello"(1,2,3);
}
```
which works now. But this fails to compile:
```
pluto("hello",1,2,3);
```
because there is no argument for `s`.

So, instead of issuing a compilation error, the compiler can "slide" the 
arguments to the left, so the first argument is moved into the compile time 
parameter list. Then, the call will compile.

The rule would be something like:

1. the function is a template taking a variadic runtime argument list
2. the compile time parameters are a sequence of N value parameters, followed by
the variadic type parameter.
3. the value parameters do not have default values
4. no compile time arguments are provided in the template invocation
5. the leftmost N runtime arguments are matched against the compile time
parameters,
and removed from the runtime argument list
6. if they match, then the template instantiation is rewritten to reflect this
7. compilation then proceeds normally

Template Sliding thus becomes a general use facility. One interesting 
consequence of this is it opens up a whole new class of functions that can now 
do CTFE computations on the leftmost arguments.
Jan 12
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/12/24 23:35, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking about 
 making a general use case out of it instead of one specific to istrings.
 ---------------------
 
 Sliding Template Arguments
 
 Consider the following template:
 
 ```
 void pluto(string s)()
 {
      pragma(msg, s);
 }
 
 void test()
 {
      pluto!"hello"();
 }
 ```
 
 This compiles because `s` is a compile time argument, and `pragma(msg, s)`
 expects s to be a compile time value.
 
 ```
 void pluto()(string s)
 {
      pragma(msg, s);
 }
 
 void test()
 {
      pluto("hello");
 }
 ```
 
 This fails to compile because `s` is a runtime argument that is not 
 accessible
 at compile time, even if it is inlined. (Inlining doesn't help here 
 because inlining is done after CTFE and semantic analysis.)
 
 Those examples illustrate the difference between a compile time and a 
 runtime argument.
 
 For background, to generate a tuple of elements:
 
 ```
 alias AliasSeq(T...) = T;
 ```
 and a function argument list that accepts a tuple:
 ```
 void func(Args...)(Args args)
 {
 }
 ```
 But notice that `args` are runtime arguments. It turns out there is no way
 to use tuples to split an argument tuple into compile time and runtime 
 tuples:
 
 ```
 void pluto(Args...)(Args args)
 {
      exec!(args[0])(args[1 .. args.length]);
 }
 ```
 This is the problem that DIP1036e ran into. It's clever solution is to 
 have the compiler (because it cannot be done with metaprogramming) take 
 the first argument, and use it as a compile time argument to a templated 
 dummy value. Then, the type of that argument is a template with the 
 value encoded into the type, which can be programmatically extracted and 
 then processed at compile time.
 
 The awkwardness of this is it only happens with istrings, it is not a 
 general purpose capability, plus the insertion of dummy arguments into 
 the argument list just so their types can be extracted.
 
 Hence, the genesis of this proposal which describes a language 
 capability to create a compile time parameter from a tuple of runtime 
 expressions.
 
 For lack of a better term, I'll call it "Sliding Template Arguments".
 
 Consider a template function:
 
 ```
 void pluto(string s, Args...)(Args args)
 {
      pragma(msg, s);
 }
 
 void exec()
 {
      pluto!"hello"(1,2,3);
 }
 ```
 which works now. But this fails to compile:
 ```
 pluto("hello",1,2,3);
 ```
 because there is no argument for `s`.
 
 So, instead of issuing a compilation error, the compiler can "slide" the 
 arguments to the left, so the first argument is moved into the compile 
 time parameter list. Then, the call will compile.
 
 The rule would be something like:
 
 1. the function is a template taking a variadic runtime argument list
 2. the compile time parameters are a sequence of N value parameters, 
 followed by
 the variadic type parameter.
 3. the value parameters do not have default values
 4. no compile time arguments are provided in the template invocation
 5. the leftmost N runtime arguments are matched against the compile time 
 parameters,
 and removed from the runtime argument list
 6. if they match, then the template instantiation is rewritten to 
 reflect this
 7. compilation then proceeds normally
 
 Template Sliding thus becomes a general use facility. One interesting 
 consequence of this is it opens up a whole new class of functions that 
 can now do CTFE computations on the leftmost arguments.


Generally I think this is a good idea, but the proposed syntax is a bit 
too specialized, arbitrarily restricted and the behavior may be 
unexpected and should be opt-in instead.

Maybe we can do it like this instead:

```d
void pluto(string s, Args...)(enum string x = s, Args args){

}
```

I.e., you can use `enum` in a function argument list and you have to 
default-initialize them. This means that this parameter always needs to 
have exactly that value.

Then, the arguments matched to an `enum` parameter are evaluated at 
compile time and matched against the initializer.
Jan 12
parent Nickolay Bukreyev <buknik95 ya.ru> writes:
On Friday, 12 January 2024 at 23:04:58 UTC, Timon Gehr wrote:

 I.e., you can use `enum` in a function argument list and you 
 have to default-initialize them. This means that this parameter 
 always needs to have exactly that value.

 Then, the arguments matched to an `enum` parameter are 
 evaluated at compile time and matched against the initializer.


Looks promising, but I have a question. Will this work?

```d
void pluto(){ }

void pluto(string s, Args...)(enum string x = s, Args args){
     pragma(msg, s);
     pluto(args); // <-
}

pluto("abc", "def", "ghi"); // pluto!("abc", string, string)
```

I’m afraid it can’t. When we call `pluto(args)`, `args` are not 
compile-time-known anymore. How can you overcome this?

---

I still think that Walter’s idea of relaxing requirements on 
alias parameters is the simplest yet general enough and is not 
limited to interpolations.

```d
struct Interpolation {
     immutable(string)[ ] parts;
}

void fooImpl(Interpolation interp, Args...)(Args args) { ... }

template foo(Interpolation interp, args...) {
     alias foo = fooImpl!interp(args);
}

void main() {
     string name = "Steve";
     int fourty = 40;
     foo!(i"Hello, $name, I see you are $(fourty + 2) years old.");
     // Desugars to:
     foo!(Interpolation(["Hello, ", "name", ", I see you are ", 
"fourty + 2", " years old."]), name, fourty + 2);
     // `foo!(...)` expands to:
     fooImpl!(Interpolation(["Hello, ", "name", ", I see you are 
", "fourty + 2", " years old."]))(name, fourty + 2);
}
```

This almost works now, except 1) you cannot pass `fourty + 2` to 
a template parameter, 2) you cannot do `alias foo = 
fooImpl!interp(args)`.

To say it another way, it should be allowed to instantiate a 
template with something unmangleable, but in that case the 
template must not declare anything that requires mangling 
(functions, variables, structs, unions, classes; uncertain about 
enums).

This approach has an advantage over several other proposals: if 
something is being accessed at compile time, you must indicate it 
clearly by putting an exclamation mark at the call site.
Jan 12
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
I started working on an alternative mechanism to templates for 1036e 
yesterday. Which I alluded to.

Being able to pass a UDA at a call site and be able to get access to it 
via ``__traits(getAttributes, parameter)`` within the (templated) function.

``i"$ident$(expr)$(ident:format)${1:format}"``

becomes

```d
 IExpression("ident")
ident,
 IExpression("expr")
expr,
 IExpression("ident")
 IFormat("format")
ident,
 IFormat("format")
 IPosition(1)
IPosition.init
```

This would be a general purpose language feature.

```d
string username, password;

getopt(
      description("My program")
      description("Second line")
     commandsInfo,
      description("My programs help info")
      flag("help")  flag("h") helpInfo,
      description("The username to connect with")
      flag("username")  flag("u") username,
      description("The password to connect with")
      flag("password")  flag("p") password
);
```

I have already mocked up the getopt, the only additional template usage 
is for formattedRead. This is a general purpose feature, that string 
interpolation could tie into.

https://gist.github.com/rikkimax/812de12e600a070fe267f3bdc1bb3928
Jan 12
parent reply Nickolay Bukreyev <buknik95 ya.ru> writes:
On Friday, 12 January 2024 at 23:10:08 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 I started working on an alternative mechanism to templates for 
 1036e yesterday. Which I alluded to.

 Being able to pass a UDA at a call site and be able to get 
 access to it via ``__traits(getAttributes, parameter)`` within 
 the (templated) function.

 ``i"$ident$(expr)$(ident:format)${1:format}"``

 becomes

 ```d
  IExpression("ident")
 ident,
  IExpression("expr")
 expr,
  IExpression("ident")
  IFormat("format")
 ident,
  IFormat("format")
  IPosition(1)
 IPosition.init
 ```

 This would be a general purpose language feature.


Your idea looks attracting to me, but to be honest, I haven’t 
fully grasped it. Could you show an example implementation of 
`execi` & `generateSql` please? Particularly interested in how 
you would pass and process literal text chunks.
Jan 13
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 14/01/2024 8:56 PM, Nickolay Bukreyev wrote:

 Your idea looks attracting to me, but to be honest, I haven’t fully 
 grasped it. Could you show an example implementation of `execi` & 
 `generateSql` please? Particularly interested in how you would pass and 
 process literal text chunks.


I'm working off of:

https://forum.dlang.org/post/cbwqnanabefqjvxrzszx forum.dlang.org

I have not thought this use case through, but the basic premise should 
be sound.

For instance suffix strings, positional arguments are not handled here 
and if something wasn't an interpolated string.

```d
db.execi(i"SELECT * FROM items WHERE id = $desiredId");
```

Becomes:

```d
db.execi( IPrefix("SELECT * FROM items WHERE id = ") desiredId);
```

```d
auto execi(FormatString fmt, Args...)(Sqlite db, Args args) {
	enum query = () {
		string ret;

		static foreach(i; 0 .. Args.length) {
			foreach(prefix; __traits(getAttributes, args[i], IPrefix)) {
				ret ~= prefix.value;
			}
			ret ~= "?";
			ret ~= i.text;
		}

		return ret;
	}();

	auto statement = Statement(db, query);
	static foreach (i, arg; args)
		statement.bind(i + 1, arg);
	return statement.execute();
}
```
Jan 14
next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 14/01/2024 9:23 PM, Richard (Rikki) Andrew Cattermole wrote:

 auto execi(FormatString fmt, Args...)(Sqlite db, Args args) { enum query 
 = () { string ret; static foreach(i; 0 .. Args.length) { foreach(prefix; 
 __traits(getAttributes, args[i], IPrefix)) { ret ~= prefix.value; } ret 
 ~= "?"; ret ~= i.text; } return ret; }(); auto statement = Statement(db, 
 query); static foreach (i, arg; args) statement.bind(i + 1, arg); return 
 statement.execute(); }


Ugh oops, left in the FormatString template parameter.

Should be:

```d
auto execi(Args...)(Sqlite db, Args args) {
     enum query = () {
         string ret;

         static foreach(i; 0 .. Args.length) {
             foreach(prefix; __traits(getAttributes, args[i], IPrefix)) {
                 ret ~= prefix.value;
             }
             ret ~= "?";
             ret ~= i.text;
         }

         return ret;
     }();

     auto statement = Statement(db, query);
     static foreach (i, arg; args)
         statement.bind(i + 1, arg);
     return statement.execute();
}
```
Jan 14
prev sibling parent reply Nickolay Bukreyev <buknik95 ya.ru> writes:
On Sunday, 14 January 2024 at 08:23:47 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 For instance suffix strings, positional arguments are not 
 handled here and if something wasn't an interpolated string.


Yes, suffix strings. There’s simply nowhere to put them in. We 
cannot even attach them to the last argument as ` ISuffix`… 
because there can be no arguments at all! (Not to mention this 
would break symmetry—which I’m not fond of.) Until there is a 
satisfactory solution, I cannot compare inline attributes with 
other interpolation approaches.
Jan 14
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 15/01/2024 6:12 AM, Nickolay Bukreyev wrote:

 On Sunday, 14 January 2024 at 08:23:47 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 For instance suffix strings, positional arguments are not handled here 
 and if something wasn't an interpolated string.


 
 Yes, suffix strings. There’s simply nowhere to put them in. We cannot 
 even attach them to the last argument as ` ISuffix`… because there can 
 be no arguments at all! (Not to mention this would break symmetry—which 
 I’m not fond of.) Until there is a satisfactory solution, I cannot 
 compare inline attributes with other interpolation approaches.


There is a solution.

Use a positional argument instead, set to -1.

Quite easy really :)
Jan 14
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.
 
 ---------------------

 Sliding Template Arguments


This is a workable solution. But still not with format strings 
(which are specific to writef).

If I were to use this to implement interpolation tuples, I'd 
prefer the first parameter to be of type `struct Interpolation { 
immutable string[] parts; }`, and therefore the compiler would do:

```d
void foo(Interpolation interp, Args...)(Args args) {...}

void main()
{
     string name = "Steve";
     int age = 42;
     foo(i"Hello, $name, I see you are $age years old.");
     // equivalent to:
     foo!(Interpolation(["Hello, ", "name", ", I see you are ", 
"age", " years old."]))(name, age);
}
```

There is still value, I think, in passing the parameters in the 
order they appear. And this doesn't allow for e.g. functions that 
take multiple istrings. But maybe that's fine?

Another interesting development from this, is that you can take 
the string literal data at runtime as well (when you prefer, or 
are ok with, runtime processing of the string literal data):

```d
void writeln(Args...)(Interpolation interp, Args args)
{
     assert(interp.parts.length == args.length * 2 + 1);
     write(interp.parts[0]); // always a leading string;
     static foreach(i; 0 .. args.length)
         write(args[i], interp.parts[(i+1)*2]);
     writeln();
}
```

I don't view this as simpler than DIP1036e or DIP1027 -- a simple 
transformation is a simple transformation. Certainly a hybrid 
DIP1027 with a format string passed at compile time is still not 
viable, due to reasons already stated.

But it will be slightly less bloat. And it has benefits 
elsewhere, as you say.

If this mechanism is what gets it over the finish line, I can 
compromise with this.

Is this something that can be implemented and played with?

This does seem like it has the potential to break code:

```d
void foo(int x, Args...)(Args args) {

}
void foo(Args...)(Args args) {
}

foo(1, 2, 3); // which is called? Today, the second is
```

So I would heed Timon's advice, maybe we do need an explicit 
opt-in.

-Steve
Jan 12
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 00:15:56 UTC, Steven 
Schveighoffer wrote:


 This does seem like it has the potential to break code:

 ```d
 void foo(int x, Args...)(Args args) {

 }
 void foo(Args...)(Args args) {
 }

 foo(1, 2, 3); // which is called? Today, the second is
 ```


A more realistic example:

```d
writefln("blah %d", 1)
```

Since `writefln` (and `format`) have string template format 
parameter versions.

Even with an opt-in, there will still be a decision to make (or 
throw an ambiguity error) for matching overloads.

-Steve
Jan 12
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 4:39 PM, Steven Schveighoffer wrote:

 ```d
 writefln("blah %d", 1)
 ```
 
 Since `writefln` (and `format`) have string template format parameter versions.
 
 Even with an opt-in, there will still be a decision to make (or throw an 
 ambiguity error) for matching overloads.


That's why the template shifting one would be a lower priority match.
Jan 12
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 4:15 PM, Steven Schveighoffer wrote:

 I don't view this as simpler than DIP1036e or DIP1027 -- a simple
transformation 
 is a simple transformation.


Adding extra hidden templates isn't that simple. If a user is not using a
canned 
version, he'd have to be pretty familiar with D to be able to write his own 
handler. 1027 is simpler in that if the generated tuple is examined, it matches 
just what one would have written using a format. Nothing much to learn there.



 Certainly a hybrid DIP1027 with a format string 
 passed at compile time is still not viable, due to reasons already stated.


The other reasons:

1. preventing calls to functions passing an ordinary string as opposed to an 
istring tuple
2. preventing nested istrings

have already been addressed. The compile time thing was the only one left.


 This does seem like it has the potential to break code:


The shifted one would be a more costly match, and so the legacy others would be 
favored first.
Jan 12
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/13/24 03:16, Walter Bright wrote:

 
 The other reasons:
 
 1. preventing calls to functions passing an ordinary string as opposed 
 to an istring tuple
 2. preventing nested istrings
 
 have already been addressed. The compile time thing was the only one left.


There is more, e.g.:

- You cannot interpolate an expression sequence, it will cause the 
format string to get out of synch with the arguments.

- DIP1027's format strings allow manual overrides that may then be 
incompatible with the library functions.

- DIP1027's format strings require the user to manually escape e.g. '%s' 
so that the library does not mistake it for a format specifier, even 
though the user does not otherwise interact with format specifiers at all.
Jan 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 6:56 PM, Timon Gehr wrote:

 On 1/13/24 03:16, Walter Bright wrote:

 The other reasons:

 1. preventing calls to functions passing an ordinary string as opposed to an 
 istring tuple
 2. preventing nested istrings

 have already been addressed. The compile time thing was the only one left.


 
 There is more, e.g.:
 
 - You cannot interpolate an expression sequence, it will cause the format
string 
 to get out of synch with the arguments.


I don't know what "interpolate an expression sequence" means. As for things 
getting out of sync, execi() with CTFE can reject a mismatch between format 
specifiers and arguments.

If you mean nested istrings, that can be simply rejected by not allowing a
tuple 
argument for an argument. Or, execi() can detect arguments with the `Format` 
type and handle the nested istrings.



 - DIP1027's format strings allow manual overrides that may then be
incompatible 
 with the library functions.


execi() with CTFE can reject them.



 - DIP1027's format strings require the user to manually escape e.g. '%s' so
that 
 the library does not mistake it for a format specifier, even though the user 
 does not otherwise interact with format specifiers at all.


execi() with CTFE can automatically escape them.
Jan 12
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/13/24 04:36, Walter Bright wrote:

 
 I don't know what "interpolate an expression sequence" means. As for 
 things getting out of sync, execi() with CTFE can reject a mismatch 
 between format specifiers and arguments.


Oh, not at all.

```d
import std.stdio;
alias Seq(T...)=T;
void main(){
     writefln(i"$(Seq!(1,2)) %s");
}
```
Jan 12
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 03:59:03 UTC, Timon Gehr wrote:

 On 1/13/24 04:36, Walter Bright wrote:

 
 I don't know what "interpolate an expression sequence" means. 
 As for things getting out of sync, execi() with CTFE can 
 reject a mismatch between format specifiers and arguments.


 Oh, not at all.

 ```d
 import std.stdio;
 alias Seq(T...)=T;
 void main(){
     writefln(i"$(Seq!(1,2)) %s");
 }
 ```


Yes, and there is more:

```d
writefln(i"is it here? ${}(1) Or here? %s");
```

Bottom line is that if we make `%s` special, then all functions 
must deal with the consequences. There is not a format specifier 
you can come up with that is not easily reproduced in the string 
literal directly -- you have to escape it and *know* that you 
must escape it. The easier path is just not to deal with format 
specifiers at all -- tell the library exactly where the pieces 
are.

And by the way, your example brings up another point not recently 
brought up where 1036e handles and DIP1027 does not: tuples as 
interpolated expressions. Because each expression is enclosed by 
`InterpolatedLiteral` pieces, you can tell which ones were 
actually tuples.

-Steve
Jan 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 8:13 PM, Steven Schveighoffer wrote:

 On Saturday, 13 January 2024 at 03:59:03 UTC, Timon Gehr wrote:

 On 1/13/24 04:36, Walter Bright wrote:

 I don't know what "interpolate an expression sequence" means. As for things 
 getting out of sync, execi() with CTFE can reject a mismatch between format 
 specifiers and arguments.


 Oh, not at all.

 ```d
 import std.stdio;
 alias Seq(T...)=T;
 void main(){
     writefln(i"$(Seq!(1,2)) %s");
 }
 ```




1027 can write a format string for a tuple as: "%s%s %%s" because the number of 
elements in the tuple is known at compile time.



 Yes, and there is more:
 
 ```d
 writefln(i"is it here? ${}(1) Or here? %s");
 ```


An empty format ${} would be a compile time error. The %s would be rewritten as
%%s.



 Bottom line is that if we make `%s` special, then all functions must deal with 
 the consequences. There is not a format specifier you can come up with that is 
 not easily reproduced in the string literal directly -- you have to escape it 
 and *know* that you must escape it. The easier path is just not to deal with 
 format specifiers at all -- tell the library exactly where the pieces are.


Escaping % is not hard to do. It's ordinary.



 And by the way, your example brings up another point not recently brought up 
 where 1036e handles and DIP1027 does not: tuples as interpolated expressions. 
 Because each expression is enclosed by `InterpolatedLiteral` pieces, you can 
 tell which ones were actually tuples.


And with 1027 the format string will be of type `FormatString` (not `string`), 
and you can tell which ones were actually tuples. So I take that back, nested 
formats are easily supported.

These are all straightforward solutions.
Jan 12
next sibling parent reply Adam Wilson <flyboynw gmail.com> writes:
On Saturday, 13 January 2024 at 06:27:51 UTC, Walter Bright wrote:

 Escaping % is not hard to do. It's ordinary.


I don't see people arguing that escaping is *difficult* to do. 
It's not. What *is* difficult is remembering to do it perfectly, 
every time, and accidentally building a silent injection attack 
when you (inevitably) fail. Especially since the attack vector is 
not detectable to linting tools. All systems with a special 
format-specifier are unsafe for use with SQL. Period.

Think of it this way: You have the option to install a fail-safe 
critical system in your jet, and everybody is telling you do it, 
but you're saying "Nah fam, it'll be fine. The odds are so low 
that a human will screw up this one manual step and this design 
will burn 0.1% less fuel and costs 10% less." *cough*MCAS*cough*.

Actually, MCAS is a pretty fair analogy here. The system mostly 
works as designed, except for the one button that if you don't 
push it when things go bad, brings down two airframes. That's how 
big a deal SQL injections are.

Don't be 2010's Boeing, be 1970s Boeing. Please build an 
indestructible 757.

Note that Java considered and rejected your premise in their 
version of this feature, with their reasoning laid out in the 
[spec-document](https://openjdk.org/jeps/430).
Jan 12
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Saturday, 13 January 2024 at 07:03:14 UTC, Adam Wilson wrote:

 Note that Java considered and rejected your premise in their 
 version of this feature, with their reasoning laid out in the 
 [spec-document](https://openjdk.org/jeps/430).


Looks like it's already a feature: 
https://www.baeldung.com/java-21-string-templates

Thanks, I wasn't aware of this.
Jan 13
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 11:03 PM, Adam Wilson wrote:

 On Saturday, 13 January 2024 at 06:27:51 UTC, Walter Bright wrote:

 Escaping % is not hard to do. It's ordinary.


 
 I don't see people arguing that escaping is *difficult* to do. It's not. What 
 *is* difficult is remembering to do it perfectly, every time, and accidentally 
 building a silent injection attack when you (inevitably) fail. Especially
since 
 the attack vector is not detectable to linting tools. All systems with a
special 
 format-specifier are unsafe for use with SQL. Period.


1027 can do that automatically, so it will work every time. I've written code 
that parsed a string and escaped the naughty bits many times.


 Note that Java considered and rejected your premise in their version of this 
 feature, with their reasoning laid out in the 
 [spec-document](https://openjdk.org/jeps/430).


It's a long document. I'm not sure what you see as my premise and what the Java 
doc is specific about.

As for what my premise is, CTFE code can be written to validate strings. This
is 
not difficult to do. It's put in the execi() function. It happens for every 
string. Once the code is written, it will work on every user-supplied istring.

DIP1036e doesn't have any magic bean to validate strings. It also has to code
up 
a validator. Coding up the validator is a task for both proposals, and they'd
do 
the same thing.

Furthermore, if a string type was passed to execi() as the first argument, it 
would not compile, as 1027 would type it as a `FormatString`, not a `string`.
So 
execi() would only work on the output of the string interpolator, not any
random 
string.
Jan 15
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/15/24 09:03, Walter Bright wrote:

 
 Furthermore, if a string type was passed to execi() as the first 
 argument, it would not compile, as 1027 would type it as a 
 `FormatString`, not a `string`. So execi() would only work on the output 
 of the string interpolator, not any random string.


Do you believe that if the documentation has the following examples:

```d
writefln("%s", x);
writefln(i"$x");
```

Then it is not inevitable that at some point, someone will write an 
implementation of `execi` with the signature `void execi(T...)(string, T)` ?
Jan 15
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/13/24 07:27, Walter Bright wrote:

 On 1/12/2024 8:13 PM, Steven Schveighoffer wrote:

 On Saturday, 13 January 2024 at 03:59:03 UTC, Timon Gehr wrote:

 On 1/13/24 04:36, Walter Bright wrote:

 I don't know what "interpolate an expression sequence" means. As for 
 things getting out of sync, execi() with CTFE can reject a mismatch 
 between format specifiers and arguments.


 Oh, not at all.

 ```d
 import std.stdio;
 alias Seq(T...)=T;
 void main(){
     writefln(i"$(Seq!(1,2)) %s");
 }
 ```




 
 1027 can write a format string for a tuple as: "%s%s %%s" because the 
 number of elements in the tuple is known at compile time.
 ...


I am testing all of my DIP1027 snippets against your implementation, and 
this is not what it does (it prints "1 2\n", in accordance with the 
DIP1027 specification). And if it were, it still suffers from the 
drawback that the library cannot detect that the user passed arguments 
in this fashion.


 

 Yes, and there is more:

 ```d
 writefln(i"is it here? ${}(1) Or here? %s");
 ```


 
 An empty format ${} would be a compile time error. The %s would be 
 rewritten as %%s.
 ...


It's not rewritten like that with DIP1027.


 

 Bottom line is that if we make `%s` special, then all functions must 
 deal with the consequences. There is not a format specifier you can 
 come up with that is not easily reproduced in the string literal 
 directly -- you have to escape it and *know* that you must escape it. 
 The easier path is just not to deal with format specifiers at all -- 
 tell the library exactly where the pieces are.


 
 Escaping % is not hard to do. It's ordinary.
 ...


It's completely unnecessary to require this of a user who wants to use 
an istring. Why doesn't DIP1027 escape '%' automatically? (Not that that 
would solve all issues with the format string.)


 

 And by the way, your example brings up another point not recently 
 brought up where 1036e handles and DIP1027 does not: tuples as 
 interpolated expressions. Because each expression is enclosed by 
 `InterpolatedLiteral` pieces, you can tell which ones were actually 
 tuples.


 
 And with 1027 the format string will be of type `FormatString` (not 
 `string`), and you can tell which ones were actually tuples.


How can you tell which ones were actually tuples?


 So I take that back, nested formats are easily supported.
 ...


You are moving the goalposts. I am glad you agree that DIP1027 is 
insufficient.


 These are all straightforward solutions.
 


DIP1036e is even more straightforward than the collection of fixes that 
would be required to make something that is closer to DIP1027 viable.
Jan 13
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/13/2024 1:24 AM, Timon Gehr wrote:

 It's not rewritten like that with DIP1027.


I know. Based on our discussions, several improvements need to be made to 
DIP1027. I'm basing my argument on what DIP1027 would be with those 
improvements. For example, changing the type of the format string from `string` 
to `Format`, doing proper escaping of %, handling tuple arguments, handling 
nested interpolations, etc. The draft implementation of it also lacks things 
like proper parsing of `(expression)`, which is straightforward to fix and not 
any fundamental flaw in the design.

I also am not criticizing things in DIP1036e that should be improved, as that
is 
irrelevant to the fundamental issues discussed here. The current 
implementation/spec seems to be missing things like escaping stray ? that could 
appear in the string literals.

I.e. we should both be discussing what the two proposals *could* be, rather
than 
what they *are* with their feet of clay.
Jan 14
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/15/24 08:50, Walter Bright wrote:

 On 1/13/2024 1:24 AM, Timon Gehr wrote:

 It's not rewritten like that with DIP1027.


 
 I know. Based on our discussions, several improvements need to be made 
 to DIP1027. I'm basing my argument on what DIP1027 would be with those 
 improvements.


But you have not really been giving the same consideration to DIP1036e. 
Why should we accept your moving of the goalposts on DIP1027 while you 
argue as if DIP1036e drawbacks cannot be overcome?

What is the fundamental difference between DIP1027 and DIP1036e? Your 
insistence on a format string? That `printf` can be called with an 
istring? What are the design constraints here?


 For example, changing the type of the format string from 
 `string` to `Format`, doing proper escaping of %, handling tuple 
 arguments, handling nested interpolations, etc.


Ok. But how do you handle tuple arguments and nested interpolations? It 
does not seem easy, especially if the format string parts should all be 
accessible at compile time.


 The draft implementation 
 of it also lacks things like proper parsing of `(expression)`, which is 
 straightforward to fix and not any fundamental flaw in the design.
 ...


Well, the DIP1036e implementation is viable as-is.


 I also am not criticizing things in DIP1036e that should be improved, as 
 that is irrelevant to the fundamental issues discussed here.


So you are criticizing things in DIP1036e with the intention that they 
should not be improved?


 The current 
 implementation/spec seems to be missing things like escaping stray ? 
 that could appear in the string literals.
 ...


No, the implementation of the language feature is complete. It's just 
that the proof-of-concept usage example did not do some sorts of 
validation. And anyway, this only serves to further highlight the 
error-prone nature of approaches based on special characters in a string.


 I.e. we should both be discussing what the two proposals *could* be, 
 rather than what they *are* with their feet of clay.


We can do that, but then I do not really see why there still has to be a 
debate framed around DIP1027 vs DIP1036e in the first place. Clearly 
both of them *could* be the other one. The possibilities are endless!
Jan 15
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/13/24 07:27, Walter Bright wrote:

 Yes, and there is more:

 ```d
 writefln(i"is it here? ${}(1) Or here? %s");
 ```


 
 An empty format ${} would be a compile time error. The %s would be 
 rewritten as %%s.


./dmd -run test.d
is it here?  Or here? 1
Jan 13
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 02:16:06 UTC, Walter Bright wrote:

 On 1/12/2024 4:15 PM, Steven Schveighoffer wrote:

 I don't view this as simpler than DIP1036e or DIP1027 -- a 
 simple transformation is a simple transformation.


 Adding extra hidden templates isn't that simple. If a user is 
 not using a canned version, he'd have to be pretty familiar 
 with D to be able to write his own handler.


Yes, that is intentional. You should not be able to call 
functions with new syntax because the parameters happen to match. 
We have a type system for a reason.


 1027 is simpler in that if the generated tuple is examined, it 
 matches just what one would have written using a format. 
 Nothing much to learn there.


In other words: "it matches just what one wouldn't have written, 
unless one is calling `writef`".


 Certainly a hybrid DIP1027 with a format string passed at 
 compile time is still not viable, due to reasons already 
 stated.


 The other reasons:

 1. preventing calls to functions passing an ordinary string as 
 opposed to an istring tuple


I don't see how this proposal fixes that. I'm assuming a function 
like `void foo(string s, int x)` will match `foo(i"something: 
$(1)")`


 2. preventing nested istrings


Why do we want to prevent nested istrings? That's not a goal.


 have already been addressed. The compile time thing was the 
 only one left.


A compile time format string still needs parsing. Why would we 
want to throw away all the work the compiler already did?

If you want to call `writef`, you can construct a format string 
easily at compile time. Or just... call `writef` the normal way.

Compile-time string parsing is way more costly than compile-time 
string concatenation.


 This does seem like it has the potential to break code:


 The shifted one would be a more costly match, and so the legacy 
 others would be favored first.


Ok. This does mean, for *intentional* overloading of a function 
to accept a compile-time first parameter, you will have to rename 
the function.

Possibly, if you have an opt-in syntax like Timon mentioned, then 
you can make the sliding template parameter more preferable.

-Steve
Jan 12
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 8:35 PM, Steven Schveighoffer wrote:

 On Saturday, 13 January 2024 at 02:16:06 UTC, Walter Bright wrote:

 On 1/12/2024 4:15 PM, Steven Schveighoffer wrote:

 I don't view this as simpler than DIP1036e or DIP1027 -- a simple 
 transformation is a simple transformation.


 Adding extra hidden templates isn't that simple. If a user is not using a 
 canned version, he'd have to be pretty familiar with D to be able to write his 
 own handler.


 
 Yes, that is intentional.


So you agree it is not simpler :-)


 You should not be able to call functions with new 
 syntax because the parameters happen to match. We have a type system for a
reason.


I proposed in the other topic to type the format string as Format (or 
FormatString), which resolves that issue, as a string is not implicitly 
convertible to a FormatString.



 1027 is simpler in that if the generated tuple is examined, it matches just 
 what one would have written using a format. Nothing much to learn there.


 
 In other words: "it matches just what one wouldn't have written, unless one is 
 calling `writef`".


Yes, it is meant for writef, not writeln.



 The other reasons:

 1. preventing calls to functions passing an ordinary string as opposed to an 
 istring tuple


 
 I don't see how this proposal fixes that. I'm assuming a function like `void 
 foo(string s, int x)` will match `foo(i"something: $(1)")`


Yes, we've seen that example. It's a bit contrived. I've sent a format string
to 
a function unexpectedly now and then. The result is the format string gets 
printed. I see it, I fix it. I can't see how it would be some disastrous 
problem. If it indeed a super problem, `Format` can be made to be a type that
is 
not implicitly convertible to a string, but can have a string extracted from it 
with CTFE.

What it does fix is your other concern about sending a string to a function 
(like execi()) that expects a Format as its first argument.



 2. preventing nested istrings


 
 Why do we want to prevent nested istrings? That's not a goal.


I mentioned in another reply to you a simple solution.



 have already been addressed. The compile time thing was the only one left.


 
 A compile time format string still needs parsing. Why would we want to throw 
 away all the work the compiler already did?


For the same reason writefln() exists in std.stdio, and people use it instead
of 
writeln(). Also, the SQL example generates a format string.



 If you want to call `writef`, you can construct a format string easily at 
 compile time. Or just... call `writef` the normal way.


??


 Compile-time string parsing is way more costly than compile-time string 
 concatenation.


I suspect you routinely use CTFE for far, far more complex tasks. This is a 
rounding error.



 Ok. This does mean, for *intentional* overloading of a function to accept a 
 compile-time first parameter, you will have to rename the function.


You can overload it with existing functions, or give it a new name. Your
choice, 
I don't see problem.
Jan 12
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 06:46:54 UTC, Walter Bright wrote:

 On 1/12/2024 8:35 PM, Steven Schveighoffer wrote:

 On Saturday, 13 January 2024 at 02:16:06 UTC, Walter Bright 
 wrote:

 On 1/12/2024 4:15 PM, Steven Schveighoffer wrote:

 I don't view this as simpler than DIP1036e or DIP1027 -- a 
 simple transformation is a simple transformation.


 Adding extra hidden templates isn't that simple. If a user is 
 not using a canned version, he'd have to be pretty familiar 
 with D to be able to write his own handler.


 
 Yes, that is intentional.


 So you agree it is not simpler :-)


No it is just as simple. I agree that the user should have to 
understand the feature before hooking it. I meant it is 
intentional that you can't "accidentally" hook istring calls 
without understanding what you are doing.

And I don't understand this line of argument to begin with. You 
have to be pretty familiar with D to hook anything:

* operator overloads
* foreach
* toString
* put
* DIP1027
* DIP1036e
* this thing you are proposing

And I'm sure there's more.


 You should not be able to call functions with new syntax 
 because the parameters happen to match. We have a type system 
 for a reason.


 I proposed in the other topic to type the format string as 
 Format (or FormatString), which resolves that issue, as a 
 string is not implicitly convertible to a FormatString.


This doesn't help, as an enum implicitly converts to its base 
type.


 1027 is simpler in that if the generated tuple is examined, 
 it matches just what one would have written using a format. 
 Nothing much to learn there.


 
 In other words: "it matches just what one wouldn't have 
 written, unless one is calling `writef`".


 Yes, it is meant for writef, not writeln.


In none of the proposals I have written or supported, has it been 
meant for `writef`. I don't understand the desire to hook 
`writef` and `format`. The feature to make 1036e hook `writeln` 
is just an easy added thing (just add a toString and it works), 
but is not fundamentally necessary. We could just as easily 
change writeln to handle whatever template types we create.

Hooking `writef` involves adding semantic requirements on the 
library author that are specialized for `writef`, for what 
benefit, I can't really say. You can always create a `writef` 
overload that handles these things, but I don't see the point of 
it. String interpolation isn't aimed at formatting, though it can 
be used for it (as demonstrated).


 The other reasons:

 1. preventing calls to functions passing an ordinary string 
 as opposed to an istring tuple


 
 I don't see how this proposal fixes that. I'm assuming a 
 function like `void foo(string s, int x)` will match 
 `foo(i"something: $(1)")`


 Yes, we've seen that example. It's a bit contrived. I've sent a 
 format string to a function unexpectedly now and then. The 
 result is the format string gets printed. I see it, I fix it.


Me too. But shouldn't we prefer compiler errors? Shouldn't we use 
the type system for what it is intended?

I've literally left bugs like this in code for years without 
noticing until the actual thing (an exception) was printed, and 
then it was hours to figure out what was happening.


 I can't see how it would be some disastrous problem. If it 
 indeed a super problem, `Format` can be made to be a type that 
 is not implicitly convertible to a string, but can have a 
 string extracted from it with CTFE.


This would be a step up, but still doesn't fix the format 
specifier problem.


 What it does fix is your other concern about sending a string 
 to a function (like execi()) that expects a Format as its first 
 argument.


Right, but this doesn't fix the format specifier problem. You 
seem to be solving all the problems but that one.


 2. preventing nested istrings


 
 Why do we want to prevent nested istrings? That's not a goal.


 I mentioned in another reply to you a simple solution.


Without a trailer, this isn't solvable technically. But I'm not 
really concerned about nested istrings. I just meant that it 
isn't a requirement to disallow them somehow.


 have already been addressed. The compile time thing was the 
 only one left.


 
 A compile time format string still needs parsing. Why would we 
 want to throw away all the work the compiler already did?


 For the same reason writefln() exists in std.stdio, and people 
 use it instead of writeln(). Also, the SQL example generates a 
 format string.


The compiler is *required* to parse out the parameters. It has, 
sitting in it's memory, the list of literals. Why would it 
reconstruct a string, with an arbitrarily decided placeholder, 
that you then have to deal with at runtime or CTFE? You are 
adding unnecessary work for the user, for the benefit of hooking 
`writef` -- a function *we control and can change to do whatever 
we want*.

The SQL example *DOES NOT* generate a format string, I've told 
you this multiple times. It generates a string with placeholders. 
There is no formatting. In fact, the C function doesn't even 
accept the parameters, those happen later after you generate the 
prepared statement.

But also, SQL requires you do it this way. And the C libraries 
being used require construction of a string (because that's the 
API C has). An sql library such as mysql-native, which is fully 
written in D, would not require building a string (and I intend 
to do this if string interpolation ever happens).

Things other than SQL *do not require building a string*.


 If you want to call `writef`, you can construct a format 
 string easily at compile time. Or just... call `writef` the 
 normal way.


 ??


Yeah, I've never cared about hooking `writef`, it's fine as-is 
(well, it's fine if that's what you like). The fact that you have 
to put in `%s` everywhere, it's a klunky mechanism for "output 
this thing to a character stream".

Can't tell you how many times I've written a `toString` hook that 
calls `outputRange.formattedWrite("%s", thing);`. That `"%s"` is 
so ugly and useless. But this is all D gives me to use, so I use 
it.


 Compile-time string parsing is way more costly than 
 compile-time string concatenation.


 I suspect you routinely use CTFE for far, far more complex 
 tasks. This is a rounding error.


Wait, so generating an extra template is a bridge too far, but 
parsing a DSL at compile time is a rounding error?

In my testing, CTFE concatenation is twice as fast as parsing, 
and uses 1/3 less memory.

Not to mention that concatenation is easy. I can do it in one 
line (if I don't really care about performance). The same cannot 
be said for parsing.

So I'd say, the user must understand that he's receiving a 
template, but also does not have to learn how to properly parse a 
specialized unrelated DSL. Format strings are weird, confusing, 
klunky, and less efficient.


 Ok. This does mean, for *intentional* overloading of a 
 function to accept a compile-time first parameter, you will 
 have to rename the function.


 You can overload it with existing functions, or give it a new 
 name. Your choice, I don't see problem.


If the template-parameter version is less preferred, it will only 
be used with an explicit template parameter.

It's not a problem, it just is one more quirk that is surprising.

-Steve
Jan 13
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, January 13, 2024 10:12:14 AM MST Steven Schveighoffer via 
Digitalmars-d wrote:

 The compiler is *required* to parse out the parameters. It has,
 sitting in it's memory, the list of literals. Why would it
 reconstruct a string, with an arbitrarily decided placeholder,
 that you then have to deal with at runtime or CTFE? You are
 adding unnecessary work for the user, for the benefit of hooking
 `writef` -- a function *we control and can change to do whatever
 we want*.

 The SQL example *DOES NOT* generate a format string, I've told
 you this multiple times. It generates a string with placeholders.
 There is no formatting. In fact, the C function doesn't even
 accept the parameters, those happen later after you generate the
 prepared statement.

 But also, SQL requires you do it this way. And the C libraries
 being used require construction of a string (because that's the
 API C has). An sql library such as mysql-native, which is fully
 written in D, would not require building a string (and I intend
 to do this if string interpolation ever happens).

 Things other than SQL *do not require building a string*.


To be honest, this is the only part of the string interpolation proposals
that seems even vaguely desirable to me. I absolutely hate their syntax in
comparison to just calling format. IMHO, it's extremely hard to read strings
with variables put into the middle of them, whereas format allows you to see
the string's contents with minimal interference from what's going to be
inserted into it (and when the interference is greater, it's because you're
doing something fancier than %s, and you need to be doing something that
you're not gonig to be doing with string interpolation anwyay). Obviously,
that's a very subjective thing, but given that I hate the syntax, string
interpolation would need to provide an actual technical benefit for it to
have any value for me.

So, if the string interpolation produces information for the function being
called in a manner that lets it see all of the individual pieces and what
their types are such that it can do whatever is appropriate with that
information rather than necessarily dealing with a format string that it has
to parse, then there's value in that. I question that it's worth the
terrible syntax, but at least it is providing some technical benefit at that
point rather than just a different syntax to produce format strings. IMHO,
if you're just going to create a format string out of the deal, then I see
no value over calling format (though obviously, there's some disagreement on
that or string interpolation wouldn't have even been proposed in the first
place.

Of course, ultimately, I'm not sure that it matters much to me what the
exact specification for interpolated strings is if they make it into the
language, since I'd rather simply never have to deal with them, but if we're
going to get them, I would hope that they would at least provide a technical
benefit rather than just a different syntax that some folks think is an
improvement.

- Jonathan M Davis
Jan 13
parent reply Daniel N <no public.email> writes:
On Sunday, 14 January 2024 at 03:09:23 UTC, Jonathan M Davis 
wrote:

 To be honest, this is the only part of the string interpolation 
 proposals that seems even vaguely desirable to me. I absolutely 
 hate their syntax in comparison to just calling format. IMHO, 
 it's extremely hard to read strings with variables put into the 
 middle of them, whereas format allows you to see the string's 
 contents with minimal interference from what's going to be 
 inserted into it (and when the interference is greater, it's 
 because you're doing something fancier than %s, and you need to 
 be doing something that you're not gonig to be doing with 
 string interpolation anwyay). Obviously, that's a very 
 subjective thing, but given that I hate the syntax, string 
 interpolation would need to provide an actual technical benefit 
 for it to have any value for me.


I kinda agree, although I consider $variable to be more readable 
than format.

1) "$a $b $c" (DIP1027)
2) "%s %s %s".format(a, b, c)
3) "${a} ${b} ${c}" (DIP1036) Too much punctuation noise in the 
common case.

When you are forced to add {} for simple variables, the entire 
feature loses its point, even worse when you add expressions with 
function calls and what not.
1) "%s %s %s".format(a+1, b+1, c+1)
2) "${a+1} ${b+1} ${c+1}" (DIP1036)

Funnily enough, alias parameters can already handle runtime 
variables...
Jan 14
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Sunday, 14 January 2024 at 09:41:16 UTC, Daniel N wrote:

 When you are forced to add {} for simple variables, the entire 
 feature loses its point, even worse when you add expressions


Just let's use groovy syntax for 1036:
1. $myVar - for variable refs.
2. ${1 + myVar} - for expressions and vars.
3. \$myVar - for string "$myVar".

Then all lazyness will be satisfied.

Best regards,
Alexandru.
Jan 14
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
I'll summarize up front, and so one can skip the detailed reply below. It's 
obvious there's no meeting of the minds here. I'll just explain where I'm
coming 
from.

Features doing simple things should be simple. That's the case with 1027. Doing 
complicated things should be expected to be more complicated to use. With 1036, 
it starts out at the other end. It's complicated, with complexity that one has 
to write additional code to make it simple again (the "filter out the 
unnecessary templates" thing we talked about).

It's fine that transmorgrifying the string to adapt it to SQL has complexities 
to it. That's expected. If that's all string interpolation would be used for, 
that's ok.

But people want to use string interpolation for mixins, etc., all that have 
nothing to do with SQL. The transmorgrification should not be necessary for 
those cases, it should just work. But it doesn't, if the filter isn't applied 
there's a significant runtime and code space cost.

Simple things should be simple, complicated things should expect complexity.
But 
1036 has simple things being complicated. Doing SQL is about the same level of 
complexity for 1027 as 1036. For doing simple things, 1036 remains complicated, 
and 1027 gets simple.

That's the extent of my unease with it.

Does 1027 do *everything* 1036 does? No. There's the Format can be implicitly 
converted to a string thing. So there's a tradeoff. Do we trade off a minor 
thing for a fair amount of complexity that yes, the user will see? Every
feature 
in D is a compromise of one sort or another. I fall on one side of that, you
the 
other.

Another illustration of this idea. For algorithmic code, I invented opApply(). 
But after experience with it, I slowly grew to dislike it because I could never 
remember how it worked, and would have to reread the documentation on it. The 
implementation of it is also ugly, as it has to rewrite the code that surrounds 
it. Heaven help anyone who has to read the code gen output of that. I much 
prefer the later approach using lambdas. They're simple, easy to remember and 
use. There isn't much implementation for them; they grew naturally out of other 
features in the language.

The one thing that lambdas don't do that opApply does is recursion. Some klunky 
code has to be written to make a lambda do recursion. But that's ok, as there 
are relatively few needs for recursion, so the simple cases with lambdas are 
simple, and the complicated cases are more work. This is as it should be.

Nothing comes for free, everything is a compromise.

----------------------------

On 1/13/2024 9:12 AM, Steven Schveighoffer wrote:

 This doesn't help, as an enum implicitly converts to its base type.


It's a point for 1036.



 In none of the proposals I have written or supported, has it been meant for 
 `writef`. I don't understand the desire to hook `writef` and `format`.


A format string is needed to support formats like "%03d". I understand that is 
not needed for SQL, but support for formats is not so straightforward with 1036.



 Me too. But shouldn't we prefer compiler errors? Shouldn't we use the type 
 system for what it is intended?


We should prefer compiler errors, I agree. But everything has a cost to it.



 I've literally left bugs like this in code for years without noticing until
the 
 actual thing (an exception) was printed, and then it was hours to figure out 
 what was happening.


When I've had an exception printed, I'll grep the code base for the message to 
see where it came from. The message should also say what was wrong. If it's a 
generic exception, I'll use the debugger to find where it came from. I doubt a 
format string error would be hard to track down, as it's only one level below 
the cause of the error.



 This would be a step up, but still doesn't fix the format specifier problem.


I'm not seeing a format specifier problem. %s, ?1, what's the difference? What 
if the string literals generated by 1036 have a stray ?1 in them? If there's a 
check for that, I missed it.



 Without a trailer, this isn't solvable technically.


It can by counting arguments and the %s.



 I just meant that it isn't a requirement to disallow them somehow.


I did mention a way it can work.



 The SQL example *DOES NOT* generate a format string, I've told you this
multiple 
 times. It generates a string with placeholders. There is no formatting. In
fact, 
 the C function doesn't even accept the parameters, those happen later after
you 
 generate the prepared statement.


CTFE work is required to generate both "hello ?1 ?2" and "hello %s %s". I 
understand your point that the latter requires an unacceptable level of CTFE 
processing, though I don't agree with the "unacceptable" part. I don't see how 
1036 is going to deal with stray ? in the string literals without CTFE.



 Things other than SQL *do not require building a string*.


Building a format string enables formats other than %s. For example, %03d. Adam 
wrote a library function for that, 
https://github.com/adamdruppe/interpolation-examples/blob/master/lib/format.d 
which does a lot of CTFE. Noted in it is "the format string is built at compile 
time".
	>> I suspect you routinely use CTFE for far, far more complex tasks. This is a

 rounding error.


 Wait, so generating an extra template is a bridge too far, but parsing a DSL
at 
 compile time is a rounding error?


It's ((number of args) - 1)*2+3 templates. Looking at the object file, they are 
templates with pretty long names, along with their struct instantiations, which 
is not free in compiler time or memory consumption.



 Not to mention that concatenation is easy. I can do it in one line (if I don't 
 really care about performance). The same cannot be said for parsing.


As you mentioned before, the code only has to be written once!
Jan 14
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/15/24 02:27, Walter Bright wrote:

 I'll summarize up front, and so one can skip the detailed reply below. 
 It's obvious there's no meeting of the minds here. I'll just explain 
 where I'm coming from.
 ...


Well, it is sad that you feel this way.


 Features doing simple things should be simple. That's the case with 
 1027.


It's not the case. Format strings are not simpler, you even got it 
wrong. In fact, you got it wrong in the DIP1027 specification, in the 
DIP1027 implementation and now in the format string parser within the 
`execi` example.


 Doing complicated things should be expected to be more complicated 
 to use. With 1036, it starts out at the other end. It's complicated, 
 with complexity that one has to write additional code to make it simple 
 again (the "filter out the unnecessary templates" thing we talked about).


That's DIP1036e, not DIP1036. DIP1036 does not have templates separating 
the arguments, it only has a header at the start.


 
 It's fine that transmorgrifying the string to adapt it to SQL has complexities
to it. That's expected. If that's all string interpolation would be used for,
that's ok.
 ...


DIP1027 is not simpler.


 But people want to use string interpolation for mixins, etc., all that have
nothing to do with SQL. The transmorgrification should not be necessary for
those cases, it should just work.


It does not with DIP1027, you still need to call some function, same as 
with DIP1036e, and that function itself will be more complex and harder 
to get right. (Though that really hardly matters as the functions needed 
for simple use cases are in Phobos anyway.)


 But it doesn't, if the filter isn't applied there's a significant runtime and
code space cost.
 ...


DIP1036e can be slightly inefficient by default.
DIP1027 is unsafe by default. Companies went out of business due to a 
SQL injection attack. People's private information got leaked.

I really do not understand why instead of using DIP1036e as a sane 
starting point and addressing the issues with efficiency, you use 
DIP1027 as a starting point make it only slightly safer, relying on the 
library author to get things right in terms of safety, while nudging 
them into the direction of doing it unsafely. D has a long track record 
of encouraging safer patterns, and this should not be an exception.


 Simple things should be simple, complicated things should expect complexity.
But 1036 has simple things being complicated. Doing SQL is about the same level
of complexity for 1027 as 1036. For doing simple things, 1036 remains
complicated, and 1027 gets simple.
 ...


That is not true.


 That's the extent of my unease with it.
 ...


Well, then I think you should be able to see it my way, by recognizing 
that format strings are not actually simpler.


 Does 1027 do *everything* 1036 does? No. There's the Format can be implicitly
converted to a string thing.


I am amazed this is still the only drawback that comes to your mind.


 So there's a tradeoff. Do we trade off a minor thing for a fair amount of
complexity that yes, the user will see? Every feature in D is a compromise of
one sort or another. I fall on one side of that, you the other.
 ...


I think not having istrings at all is significantly better than having 
DIP1027.
Jan 14
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 1/15/24 04:41, Timon Gehr wrote:

 

 Doing complicated things should be expected to be more complicated to 
 use. With 1036, it starts out at the other end. It's complicated, with 
 complexity that one has to write additional code to make it simple 
 again (the "filter out the unnecessary templates" thing we talked about).


 
 That's DIP1036e, not DIP1036. DIP1036 does not have templates separating 
 the arguments, it only has a header at the start.


Seems I got that mixed up myself, with YAIDIP.

DIP1036 in fact does have the templates interspersed:
https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1036.md

But YAIDIP only has a header:
https://github.com/John-Colvin/YAIDIP
Jan 14
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

 Simple things should be simple, complicated things should 
 expect complexity.


Indeed.


 But 1036 has simple things being complicated.


Maybe, but it does is uniformly, consistently and extensibly.


 Doing SQL is about the same level of complexity for 1027 as 
 1036. For doing simple things, 1036 remains complicated, and 
 1027 gets simple.


1027 does it deceptively simply, wrong and unsafe.
Level of complexity is completely irrelevant here.
Jan 14
prev sibling next sibling parent zjh <fqbqrr 163.com> writes:
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

 I'll summarize up front, and so one can skip the detailed reply 
 below. It's obvious there's no meeting of the minds here. I'll 
 just explain where I'm coming from.





Don't argue anymore, there's `no result`. Use `two switches` and 
`experiment first`.
Jan 14
prev sibling next sibling parent Nickolay Bukreyev <buknik95 ya.ru> writes:
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

 I'll summarize up front, and so one can skip the detailed reply 
 below. It's obvious there's no meeting of the minds here. I'll 
 just explain where I'm coming from.


Thank you for your thorough explanation. Such things really help 
us to understand one another better.


 Simple things should be simple, complicated things should 
 expect complexity.


Agreed.

Could you take a look at [this 
post](https://forum.dlang.org/post/jgniissqzeybvmkdorlk forum.dlang.org)
please? I think it presented a valuable counter-argument to some of the points
you addressed.
Jan 14
prev sibling next sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 15/01/2024 2:27 PM, Walter Bright wrote:

 I'll summarize up front, and so one can skip the detailed reply below. 
 It's obvious there's no meeting of the minds here. I'll just explain 
 where I'm coming from.
 
 Features doing simple things should be simple. That's the case with 
 1027. Doing complicated things should be expected to be more complicated 
 to use. With 1036, it starts out at the other end. It's complicated, 
 with complexity that one has to write additional code to make it simple 
 again (the "filter out the unnecessary templates" thing we talked about).
 
 It's fine that transmorgrifying the string to adapt it to SQL has 
 complexities to it. That's expected. If that's all string interpolation 
 would be used for, that's ok.
 
 But people want to use string interpolation for mixins, etc., all that 
 have nothing to do with SQL. The transmorgrification should not be 
 necessary for those cases, it should just work. But it doesn't, if the 
 filter isn't applied there's a significant runtime and code space cost.
 
 Simple things should be simple, complicated things should expect 
 complexity. But 1036 has simple things being complicated. Doing SQL is 
 about the same level of complexity for 1027 as 1036. For doing simple 
 things, 1036 remains complicated, and 1027 gets simple.


As others have said, 1027 is not simple. In fact it is the sort of 
feature when used will be considered a program security risk that 
companies will forbid the use of. A little bit of bad syntax sugar is 
not worth losing money over and it will happen, the question is when not if.

It requires an extensive infrastructure to process, and even then you 
are unlikely to get it right. As a feature it is a net negative. Almost 
every person who has had an opinion on such a feature has expressed 
dislike and out right thinking it is problematic. It cannot go in.


 Another illustration of this idea. For algorithmic code, I invented 
 opApply(). But after experience with it, I slowly grew to dislike it 
 because I could never remember how it worked, and would have to reread 
 the documentation on it. The implementation of it is also ugly, as it 
 has to rewrite the code that surrounds it. Heaven help anyone who has to 
 read the code gen output of that. I much prefer the later approach using 
 lambdas. They're simple, easy to remember and use. There isn't much 
 implementation for them; they grew naturally out of other features in 
 the language.


I on the other hand quite like it.


 On 1/13/2024 9:12 AM, Steven Schveighoffer wrote:

 This doesn't help, as an enum implicitly converts to its base type.


 
 It's a point for 1036.
 
 

 In none of the proposals I have written or supported, has it been 
 meant for `writef`. I don't understand the desire to hook `writef` and 
 `format`.


 
 A format string is needed to support formats like "%03d". I understand 
 that is not needed for SQL, but support for formats is not so 
 straightforward with 1036.


It is also not straight forward for 1027 either.

People are moving away from percentage escape formats. They are variable 
length and therefore cannot be parsed generically.

Whereas f-string's which is what everyone is moving towards are 
incredibly easy to parse for their format. I have mine working with both 
formatting and date/time (which has its format based upon PHP's 
percentage escape yuck).

Overall I'm happy with it.


 Me too. But shouldn't we prefer compiler errors? Shouldn't we use the 
 type system for what it is intended?


 
 We should prefer compiler errors, I agree. But everything has a cost to it.


Yeah turn around time.

Sure compilation might be slower by a few millisecond, but hey lets use 
more of the developer time which is far more constly and could cost 
hours instead!


Realistically though, have you reviewed my proposal for call site UDA's?

In a very short space of time, while I was tired I was able to mock up 
getopt without any additional templates.

Later on whilst tired and not thinking straight I was asked to 
demonstrate how SQL with string interpolation could work. I wrote the 
few lines to do that succinctly in a matter of a couple of minutes.

I don't understand the problem here. We have a solution that appears to 
tick the design requirements you have expressed and you have not so far 
reviewed it. Please do.
Jan 14
prev sibling next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:


 I've literally left bugs like this in code for years without 
 noticing until the actual thing (an exception) was printed, 
 and then it was hours to figure out what was happening.


 When I've had an exception printed, I'll grep the code base for 
 the message to see where it came from. The message should also 
 say what was wrong. If it's a generic exception, I'll use the 
 debugger to find where it came from. I doubt a format string 
 error would be hard to track down, as it's only one level below 
 the cause of the error.


That's not an option when working with relational databases, as 
their primary use is having multiple applications / tools / batch 
/ maintenance people working on them concurrently.

Doing something wrong, _in production_, on a sql database, like 
updating / deleting / inserting something wrongly, for an 
erroneous sql or _erroneous_ binding of a D variable to a SQL 
parameters can't simply be resolved firing up a debugger or 
halting a company and restore a backup.

/P
Jan 15
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Monday, 15 January 2024 at 01:27:00 UTC, Walter Bright wrote:

 Features doing simple things should be simple. That's the case 
 with 1027. Doing complicated things should be expected to be 
 more complicated to use. With 1036, it starts out at the other 
 end. It's complicated, with complexity that one has to write 
 additional code to make it simple again (the "filter out the 
 unnecessary templates" thing we talked about).


[...]

 Simple things should be simple, complicated things should 
 expect complexity. But 1036 has simple things being 
 complicated. Doing SQL is about the same level of complexity 
 for 1027 as 1036. For doing simple things, 1036 remains 
 complicated, and 1027 gets simple.


This part of the post is an argument about *usage* complexity, so 
let's compare what 1027 and 1036 are like to use.

For DIP 1027:

* writef just works
* For anything else, you have to process a format string at 
runtime using string parsing.

For DIP 1036:

* writeln just works
* For anything else, you have to process an argument list at 
compile time using introspection.

In the "just works" case, both are equally easy to use.

What about the case where it doesn't "just work"? For DIP 1036, D 
has world-class introspection facilities built into the language 
and standard library. For DIP 1027, the facilities we have for 
string parsing are...regex, I guess? So I would say 1036 has an 
edge here.


 Another illustration of this idea. For algorithmic code, I
 invented opApply(). But after experience with it, I slowly grew
 to dislike it because I could never remember how it worked, and
 would have to reread the documentation on it. The
 implementation of it is also ugly, as it has to rewrite the
 code that surrounds it. Heaven help anyone who has to read the
 code gen output of that. I much prefer the later approach using
 lambdas. They're simple, easy to remember and use. There isn't
 much implementation for them; they grew naturally out of other
 features in the language.


This part is an argument about *implementation* complexity, so 
let's compare implementations.

DIP 1036's PR includes a lot more tests and documentation than 
DIP 1027's, so comparing the PRs in their entirety doesn't make 
sense. Instead, I'm going to focus on the specific modules in the 
DMD frontend where the "meat" of the implementation lives.

For DIP 1027 [1], we have

* +141 lines in dmd.expressionsem
* +11 lines in dmd.lexer
* +11 lines in dmd.tokens (discounting the unrelated comment)

...which totals to 163 lines.

For DIP 1036 [2], we have

* +78 lines in dmd.expressionsem
* +166 lines in dmd.lexer
* +33 lines in dmd.tokens

...which totals to 277 lines.

So, by this measure, DIP 1036 has a more complex implementation 
than DIP 1027. Although in absolute terms, neither is especially 
complex.

For comparison, if I apply the same methodology to DIP 1038 
( mustuse) [3], I count 260 lines (in the dsymbolsem, 
expressionsem, statementsem, and mustuse modules)--pretty close 
to DIP 1036. Perhaps I'm flattering myself, but I would consider 
 mustuse to be a feature with a fairly simple implementation.

[1] https://github.com/dlang/dmd/pull/15722/files
[2] https://github.com/dlang/dmd/pull/15715/files
[3] https://github.com/dlang/dmd/pull/13589/files
Jan 15
prev sibling parent reply Nickolay Bukreyev <buknik95 ya.ru> writes:
On Saturday, 13 January 2024 at 06:46:54 UTC, Walter Bright wrote:

 So you agree it is not simpler :-)


Let’s compare them side by side.

```d
// DIP1027 + Sliding Template Arguments

// core.interpolation:

enum FormatString: string;
// Or, alternatively:
// struct FormatString { string s; }

// User-library code:

string generateSql(FormatString fmt) {
     import std.conv: text;

     string sql;
     int number;
     for (size_t i = 0; i < fmt.length; ++i) {
         char c = fmt[i];
         if (c == '%' && i + 1 < fmt.length && fmt[i + 1] == 's') {
             sql ~= text('?', ++number);
             ++i;
         } else if (c == '%' && i + 1 < fmt.length && fmt[i + 1] 
== '%')
             ++i;
         else
             sql ~= c;
     }
     return sql;
}

auto execi(FormatString fmt, Args...)(Sqlite db, Args args) {
     enum query = generateSql(fmt); // CTFE.
     auto statement = Statement(db, query);
     static foreach (i, arg; args)
         statement.bind(i + 1, arg);
     return statement.execute();
}

// User code:

db.execi(i"SELECT * FROM items WHERE id = $desiredId");
// Desugars to:
db.execi(cast(FormatString)"SELECT * FROM items WHERE id = %s", 
desiredId);
```

Vs.

```d
// Steven's proposal + Sliding Template Arguments

// core.interpolation:

struct Interpolation {
     immutable(string)[ ] parts;
}

// User-library code:

string generateSql(Interpolation interp) {
     import std.conv: text;

     string sql;
     foreach (i, part; interp.parts)
         sql ~= i & 0x1 ? text('?', i / 2 + 1) : part;
     return sql;
}

// No changes here.
auto execi(Interpolation interp, Args...)(Sqlite db, Args args) {
     enum query = generateSql(interp); // CTFE.
     auto statement = Statement(db, query);
     static foreach (i, arg; args)
         statement.bind(i + 1, arg);
     return statement.execute();
}

// User code:

db.execi(i"SELECT * FROM items WHERE id = $desiredId");
// Desugars to:
db.execi(Interpolation(["SELECT * FROM items WHERE id = ", 
"desiredId", ""]), desiredId);
```

Which one is simpler?

(Note that neither of them will actually work due to the fact 
`FormatString`/`Interpolation` is not the first argument, but 
that’s not the point I’m arguing here.)
Jan 13
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 18:43:46 UTC, Nickolay Bukreyev 
wrote:

 On Saturday, 13 January 2024 at 06:46:54 UTC, Walter Bright 
 wrote:

 So you agree it is not simpler :-)


 Let’s compare them side by side.


Proposal 1:

 ```d
 // User-library code:

 string generateSql(FormatString fmt) {
     import std.conv: text;

     string sql;
     int number;
     for (size_t i = 0; i < fmt.length; ++i) {
         char c = fmt[i];
         if (c == '%' && i + 1 < fmt.length && fmt[i + 1] == 
 's') {
             sql ~= text('?', ++number);
             ++i;
         } else if (c == '%' && i + 1 < fmt.length && fmt[i + 1] 
 == '%')
             ++i;
         else
             sql ~= c;
     }
     return sql;
 }
 ```


Note: the above code has a bug in it (I found it when testing), 
it's even easy for the language author to get these things wrong!

Proposal 2:


 ```d
 // User-library code:

 string generateSql(Interpolation interp) {
     import std.conv: text;

     string sql;
     foreach (i, part; interp.parts)
         sql ~= i & 0x1 ? text('?', i / 2 + 1) : part;
     return sql;
 }
 ```

 Which one is simpler?


Thank you for showcasing!

I'll add also the code to generate a `writef` format string is 
even easier (if you really wanted to call writef):

```d
string writefFormatString(Interpolation interp)
{
     import std.range : stride;
     import std.algorithm : filter;
     import std.array: join;
     return interp.parts.stride(2).join("%s");
}
```

And of course, sql dialects that use a static placeholder could 
just do:

```d
string mysqlQueryString(Interpolation interp)
{
     import std.range : stride;
     import std.algorithm : filter;
     import std.array: join;
     return interp.parts.stride(2).join("?");
}
```

And in fact, we can provide this as a nice library function for 
users:

```d
string blueprintWithPlaceholder(Interpolation interp, string 
placeholder)
{
     import std.range : stride;
     import std.algorithm : filter;
     import std.array: join;
     return interp.parts.stride(2).join(placeholder);
}
```


 (Note that neither of them will actually work due to the fact 
 `FormatString`/`Interpolation` is not the first argument, but 
 that’s not the point I’m arguing here.)


Ugh, I hadn't thought of that. This is kind of a downer for this 
idea. It's... still saveable, but it's not as neat.

-Steve
Jan 13
prev sibling next sibling parent reply zjh <fqbqrr 163.com> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:


 Sliding Template Arguments

 Consider the following template:

 ```
 void pluto(string s)()
 {
     pragma(msg, s);
 }



```d
void pluto(Args...)(Args args)
{
     exec!(args[0])(args[1 .. args.length]);
}
```
Like this, I think as long as the left side is known at compile 
time, we can directly add this feature!

```d
void pluto(string s, Args...)(string x = s, Args args){

}
```
I think if there is an `overloaded version`, then choose the 
overloaded version. If not, you can simply slide it because there 
are usually `no side effects`, but there should be a method to 
determine whether the variable is known or unknown` at compile 
time`.

  I found out earlier that we can create an `attribute 
dictionary`. `Functions/variables/etc...` can all have an 
`attribute dictionary`.

Here, if 'x' has an attribute dictionary, it can extract 
information that if `'x'` is a compile time variable.

So, you can use this information for `compile time 
metaprogramming`!

Moreover, try to `deduct` the function's attribute dictionary as 
much as possible. This way, there is no need for each function's 
endless attribute soup.
Jan 12
parent zjh <fqbqrr 163.com> writes:
On Saturday, 13 January 2024 at 02:14:55 UTC, zjh wrote:
  `attribute

 dictionary`. `Functions/variables/etc...` can all have an 
 `attribute dictionary`.



For things like `named parameters`, as long as there is an 
`attribute dictionary`, it is very simple to implement named 
parameters because we can give each parameter a position. With 
named parameters, the value can be directly set through the 
mapping of `name and position`.
Jan 12
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.


What happens here?

```D
void pluto(string s = "default", Args...)(Args args)
{
}

void main () {
     pluto("foo");
     pluto("foo", "bar");
}
```
Jan 12
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/12/2024 7:05 PM, Andrej Mitrovic wrote:

 What happens here?
 
 ```D
 void pluto(string s = "default", Args...)(Args args)
 {
 }
 
 void main () {
      pluto("foo");
      pluto("foo", "bar");
 }
 ```


They will compile as:

```
pluto!"default"("foo");
pluto!"default"("foo", "bar");
```
i.e. as they would now. Existing conventional overloads would be the "better" 
match. Sliding templates would only be tried if the alternative was a failure
to 
match.
Jan 12
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 03:05:57 UTC, Andrej Mitrovic 
wrote:

 What happens here?

 ```D
 void pluto(string s = "default", Args...)(Args args)
 {
 }

 void main () {
     pluto("foo");
     pluto("foo", "bar");
 }
 ```


 From original post:


 3. the value parameters do not have default values


-Steve
Jan 12
prev sibling next sibling parent claptrap <clap trap.com> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.


Im sorry but this is ridiculous. In order to try and bring 1027 
up to a par with 1036 you are willing to add a special case to 
template args?
Jan 13
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 So, instead of issuing a compilation error, the compiler can 
 "slide" the arguments to the left, so the first argument is 
 moved into the compile time parameter list. Then, the call will 
 compile.


FeepingCreature proposed this instead, which seems to be more 
flexible and clearer:
https://forum.dlang.org/post/arzmecuotonnomsehrmk forum.dlang.org
Jan 13
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Saturday, 13 January 2024 at 13:14:28 UTC, Nick Treleaven 
wrote:

 On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 So, instead of issuing a compilation error, the compiler can 
 "slide" the arguments to the left, so the first argument is 
 moved into the compile time parameter list. Then, the call 
 will compile.


 FeepingCreature proposed this instead, which seems to be more 
 flexible and clearer:
 https://forum.dlang.org/post/arzmecuotonnomsehrmk forum.dlang.org



I like this one, the only thing to think about is what to do with 
existing calls like:

```d
format!"%s"("hello");
```

What goes where? Does "hello" get pushed to the args, or is it 
still the format string? Does it just not match that overload, 
and now you need separate overloads for when you explicitly 
instantiate with a parameter?

-Steve
Jan 13
prev sibling next sibling parent Daniel N <no public.email> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.
 ---------------------


I agree, good idea, but it can be even more generalized, please 
check Rikkis idea.
https://gist.github.com/rikkimax/812de12e600a070fe267f3bdc1bb3928

Or distilled to a few lines by me...

```d
alias I(T...) = T;

void pluto(int a)
{
     pragma(msg, __traits(getArgumentAttributes, a));
}

void main()
{
     pluto( "%s" I!1);
}
```
Jan 13
prev sibling next sibling parent Daniel N <no public.email> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.
 ---------------------
 ```
 But notice that `args` are runtime arguments. It turns out 
 there is no way
 to use tuples to split an argument tuple into compile time and 
 runtime tuples:

 ```
 void pluto(Args...)(Args args)
 {
     exec!(args[0])(args[1 .. args.length]);
 }
 ```


What if you split the type-tuple instead but store a value at [0].

```d

import std.stdio;

// Works today!
void pluto_v1(Args...)(string, Args[1] a1, Args[2] a2)
{
     pragma(msg, Args[0]); // Compile-time value!
}

// Almost compiles... add 'void' to explicitly slide instead of 
implicit magic.
void pluto_v2(Args...)(void, Args[1..$] args)
{
     pragma(msg, Args[0]); // Compile-time value!
     args.writeln;
}

void main()
{
     pluto_v1!("x", int, int)("x", 2, 3);
//  pluto_v2("x", 2, 3);
}
  ```
Jan 13
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, January 12, 2024 3:35:54 PM MST Walter Bright via Digitalmars-d 
wrote:

 Given the interest in CTFE of istrings, I have been thinking about making a
 general use case out of it instead of one specific to istrings.


Honestly, this proposal just feels to me like it's making function
overloading even more complicated for no obvious benefit. Of course, I don't
like the idea of interpolated strings in the first place, so that's likely
coloring my judgement on this sort of thing, but all I'm really seeing here
is extra complexity being added to something that's already prety complex.
And I'd really rather not have runtime arguments silently become
compile-time arguments base on how a function signature is written (which
could change when the code is refactored). I feel like function overloading
is already seriously pushing it with how complicated it is (particularly
when templates are involved). If anything, I wish that it were simpler, not
more complex.

- Jonathan M Davis
Jan 13
parent Nickolay Bukreyev <buknik95 ya.ru> writes:
On Sunday, 14 January 2024 at 02:49:24 UTC, Jonathan M Davis 
wrote:

 And I'd really rather not have runtime arguments silently 
 become compile-time arguments base on how a function signature 
 is written (which could change when the code is refactored).


Agreed.

Let me tell you a story. I once had to refactor a D codebase that 
exhibited monstrous memory usage during compile time. I quickly 
figured out the cause: it was instantiating tons of different 
templates with tons of different arguments, most notably value 
(i.e., non-type) arguments. Many of those template instantiations 
could be turned into CTFE computations, and some of those 
arguments could even become runtime arguments with little to no 
drawbacks.

So what I did was pretty straightforward: I went through the 
codebase looking for `!(...)` and considered whether I could move 
those template arguments to runtime (or CTFE, which is 
syntactically the same). If a snippet had no `!(...)`, I was 
confident it didn’t pass value arguments to templates and thus 
wasn’t related to the problem I was fighting.

If something like Sliding Template Arguments existed at that 
time, my task would be much harder. For a call `foo(arg)`, I 
would have to check whether `arg` was compile-time-known, and if 
so, look at the signature of `foo` to find out how it was 
receiving it. I would have to do that for _every_ argument of 
_every_ function call.

Now you probably understand why I’m not keen on the idea of 
magically turning what syntactically looks like a runtime 
argument into a template argument. I’d really prefer it [the 
other way 
around](https://forum.dlang.org/post/cfizhoqdmxrexupcwpmc forum.dlang.org): if
you need a separate instantiation per value (as opposed to type) of a
parameter, put an exclamation point. `!(...)` is what turns DMD from one of the
fastest compilers into the most memory-hungry one; I think it should remain
explicit.
Jan 13
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 
 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.
 

 Sliding Template Arguments


There actually is a [DIP 
coming](https://github.com/dlang/DIPs/pull/232) that accomplishes 
the same thing as proposed here. It's waiting for Mike to open 
the DIP pipeline again.
Jan 15
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 1/15/24 11:39, Dukc wrote:

 On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright wrote:

 Given the interest in CTFE of istrings, I have been thinking about 
 making a general use case out of it instead of one specific to istrings.


 Sliding Template Arguments


 
 There actually is a [DIP coming](https://github.com/dlang/DIPs/pull/232) 
 that accomplishes the same thing as proposed here. It's waiting for Mike 
 to open the DIP pipeline again.


It's essentially a more detailed version of Steven's proposal, with more 
use cases.
Jan 15
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Monday, 15 January 2024 at 14:55:22 UTC, Timon Gehr wrote:

 On 1/15/24 11:39, Dukc wrote:

 On Friday, 12 January 2024 at 22:35:54 UTC, Walter Bright 
 wrote:

 Given the interest in CTFE of istrings, I have been thinking 
 about making a general use case out of it instead of one 
 specific to istrings.


 Sliding Template Arguments


 
 There actually is a [DIP 
 coming](https://github.com/dlang/DIPs/pull/232) that 
 accomplishes the same thing as proposed here. It's waiting for 
 Mike to open the DIP pipeline again.


 It's essentially a more detailed version of Steven's proposal, 
 with more use cases.


Let’s be clear, [my 
proposal](https://forum.dlang.org/post/ojotybtiosjikmmcxemw forum.dlang.org) is
a poor description of this DIP, which I swear I didn’t know existed!

Obviously this one has more thought and elegance put into it, I 
would support it!

-Steve
Jan 15
parent Daniel N <no public.email> writes:
On Monday, 15 January 2024 at 15:42:54 UTC, Steven Schveighoffer 
wrote:

 Sliding Template Arguments


 
 There actually is a [DIP 
 coming](https://github.com/dlang/DIPs/pull/232) that 
 accomplishes the same thing as proposed here. It's waiting 
 for Mike to open the DIP pipeline again.


 It's essentially a more detailed version of Steven's proposal, 
 with more use cases.


 Let’s be clear, [my 
 proposal](https://forum.dlang.org/post/ojotybtiosjikmmcxemw forum.dlang.org)
is a poor description of this DIP, which I swear I didn’t know existed!

 Obviously this one has more thought and elegance put into it, I 
 would support it!

 -Steve


Wow, that is cool. maybe it would be possible to prioritize that 
DIP as it is an enabler for future DIPS. :)
Jan 15