www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The state of string interpolation

reply o <o o.o> writes:
I really wish that D had string interpolation, and I can see that 
a ton of other people also do. String interpolation has been 
brought up before, but it has been ignored/put down every time. 
Jonathan Marler even created a Pull Request 
(https://github.com/dlang/dmd/pull/7988) to add this feature, but 
he ended up closing it because it was getting nowhere. The excuse 
that keeps being given for not adding this feature is something 
along the lines of "There are ways to work around this, so it is 
unnecessary to add it". If this is the case, then why do people 
keep on requesting it again and again? Here are just some cases 
of other people asking for string interpolation:

- https://forum.dlang.org/thread/c2q7dt$67t$1 digitaldaemon.com
- 
https://forum.dlang.org/thread/qpuxtedsiowayrhgyell forum.dlang.org
- 
https://forum.dlang.org/thread/ncwpezwlgeajdrigegee forum.dlang.org
And there are 2 closed PRs for this:
- https://github.com/dlang/dmd/pull/6703
- https://github.com/dlang/dmd/pull/7988

In my opinion, having a sizable amount of people request a 
feature (about 100 out of 280 as of The State of D Survey 
[https://dlang.typeform.com/report/H1GTak/PY9NhHkcBFG0t6ig]) is a 
good enough to realize that maybe we should start thinking of 
adding it to the language.

I understand that this may involve a DIP process, which takes 
time. Therefore I am willing to offer up $100 to someone if they 
are willing to go through the DIP process for string 
interpolation.

And please don't mention Phobos' sorry excuse for string 
interpolation:
"a is: %s, b is: %s, sum is %s.".format(a, b, a + b)
Is no better than
"a is: "~a.to!string~"b is: "~b.to!string~" and the sum is: 
"~(a+b).to!string~"."

This is just so much more readable, and maintainable:
"a is: ${a}, b is: ${b}, and the sum is: ${a+b}."

And, it gets even more readable with syntax highlighting: 
https://i.imgur.com/Bz79vtV.png

So please, what I am asking is that we start to *seriously* 
consider adding string interpolation to D. Thanks for taking the 
time to read this!
Dec 05 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 06, 2018 at 12:10:56AM +0000, o via Digitalmars-d wrote:

 I really wish that D had string interpolation, and I can see that a
 ton of other people also do. String interpolation has been brought up
 before, but it has been ignored/put down every time. Jonathan Marler
 even created a Pull Request (https://github.com/dlang/dmd/pull/7988)
 to add this feature, but he ended up closing it because it was getting
 nowhere. The excuse that keeps being given for not adding this feature
 is something along the lines of "There are ways to work around this,
 so it is unnecessary to add it". If this is the case, then why do
 people keep on requesting it again and again?


[...]

Why can't this be added to to code.dlang.org as a library / dub package
/ whatever?

Heck, I'd even use it.  Getting anything into Phobos these days is an
uphill battle, and not the kind I have the time (or patience) to fight.

In fact, I'd even *implement* it, if someone else hasn't already done
it.


T

-- 
Talk is cheap. Whining is actually free. -- Lars Wirzenius
Dec 05 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 00:20:53 UTC, H. S. Teoh wrote:

 Why can't this be added to to code.dlang.org as a library / dub 
 package / whatever?


Library things cannot access the local scope of the string unless 
you mixin.

So



is doable as a library (and I know Nick Sabalusky's scriptlike 
has it, probably among others), but



is only doable as a language feature.
Dec 05 2018
next sibling parent o <o o.o> writes:
On Thursday, 6 December 2018 at 00:32:48 UTC, Adam D. Ruppe wrote:



 is only doable as a language feature.


Exactly. This is what we need. No messing around with templates 
or mixins.
Dec 05 2018
prev sibling next sibling parent reply Zoadian <no no.no> writes:
would it be possible to have some kind of eponymous mixin 
templates in the language?
I'd imagine it to look something like this:

````
import std.stdio;

mixin template interp(string S) {
     mixin interp = S; //some fancy code to replace ${_var_} with 
content of _var_
}

void main() {
     int a = 5;
     string b = interp!"hello ${a} world";
}
```
Dec 06 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 6 December 2018 at 11:03:23 UTC, Zoadian wrote:

 would it be possible to have some kind of eponymous mixin 
 templates in the language?
 I'd imagine it to look something like this:

 ````
 import std.stdio;

 mixin template interp(string S) {
     mixin interp = S; //some fancy code to replace ${_var_} 
 with content of _var_
 }

 void main() {
     int a = 5;
     string b = interp!"hello ${a} world";
 }
 ```


Not sure if its the same thing you're thinking of, but please see 
this already filed enhancement request:  
https://issues.dlang.org/show_bug.cgi?id=18586

Mike
Dec 07 2018
parent Zoadian <no no.no> writes:
Let me clarify my ideas:

The goal is to be able to implement string interpolation _with 
nice syntax_ as library code.
The code we are aiming for is something like this:
```
int a = 5;
int b = 42;
string b = interp!"a is $(a) and b is $(b)";
```

What is possible today?
````
mixin template interp(string S) {
	enum INTERP_RESULT = tuple("a is ", a, " and b is ", b); // 
replace with code that generates tuple at compile time, based on 
S and captured scope variables
}
void main() {
	int a = 5;
	int b = 42;
	mixin interp!"a is $(a) and b is $(b)";
	writeln(INTERP_RESULT);
}
```

So what do we need to support our goal?
a) mixin as expressions
````
mixin template interp(string S) {
	enum INTERP_RESULT = tuple("a is ", a, " and b is ", b); // 
replace with code that generates tuple at compile time, based on 
S and captured scope variables
}
void main() {
	int a = 5;
	int b = 42;
	writeln(mixin interp!"a is $(a) and b is $(b)".INTERP_RESULT);
}
```

b) a way to refer to a symbol inside that mixin automatically 
(Eponymous mixin templates)
    i'm using 'mixin interp =' as a way to say this is an 
Eponymous mixin template. But maybe it is enough to have 'alias 
interp =' or 'enum interp ='.
    like in normal templates: template ASDF(T) { enum ASDF = ...; }
````
mixin template interp(string S) {
	mixin interp = tuple("a is ", a, " and b is ", b); // replace 
with code that generates tuple at compile time, based on S and 
captured scope variables
}
void main() {
	int a = 5;
	int b = 42;
	writeln(mixin interp!"a is $(a) and b is $(b)");
}
```

c) (optional) a way to not require 'mixin' to be repeated at the 
call site.
````
mixin template interp(string S) {
	mixin interp = tuple("a is ", a, " and b is ", b); // replace 
with code that generates tuple at compile time, based on S and 
captured scope variables
}
void main() {
	int a = 5;
	int b = 42;
	writeln(interp!"a is $(a) and b is $(b)");
}
```

A problem that might exist is that you can define multiple things 
inside a mixin template.
but for it to be useable as an mixin expression only one does 
make sense, so my idea was to only allow Eponymous mixin 
templates to be used as mixin expressions.
Also i might be missing something totally obvious and this does 
not work at all. But i think we should try to explore ways to 
make implementation in library code possible _and_ pretty, before 
we implement string interpolation in the compiler.

So looking at this enhancement request 
(https://issues.dlang.org/show_bug.cgi?id=18586), i'd change this 
code:
```
mixin template fun(string body) {
     mixin("auto fun() { "~body~" }");
}
```
to this:
```
mixin template fun(string body) {
     mixin("mixin fun = (){ return "~body~"; }");
//or maybe: mixin("alias fun = (){ return "~body~"; }");
}
```
Dec 07 2018
prev sibling next sibling parent reply Martin Tschierschke <mt smartdolphin.de> writes:
On Thursday, 6 December 2018 at 00:32:48 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 00:20:53 UTC, H. S. Teoh wrote:

 Why can't this be added to to code.dlang.org as a library / 
 dub package / whatever?


 Library things cannot access the local scope of the string 
 unless you mixin.

 So



 is doable as a library (and I know Nick Sabalusky's scriptlike 
 has it, probably among others), but



 is only doable as a language feature.


I am referring to this post, because it is the first mentioning 
the mixin way in this thread:

           Why is mixin() not allowed in UFCS?


it may become way nicer, and with an alias i=interp; we would have

=>



or:
Dec 08 2018
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Saturday, 8 December 2018 at 19:17:26 UTC, Martin Tschierschke 
wrote:




I don't like one-letter methods/templates that can easily clash 
with local variables.

foreach(i; 0..10) writeln(i!"oops!".mixin);
Dec 08 2018
parent Martin Tschierschke <mt smartdolphin.de> writes:
On Saturday, 8 December 2018 at 19:47:42 UTC, Dennis wrote:

 On Saturday, 8 December 2018 at 19:17:26 UTC, Martin 
 Tschierschke wrote:




 I don't like one-letter methods/templates that can easily clash 
 with local variables.

 foreach(i; 0..10) writeln(i!"oops!".mixin);


You are right using only i would clash easily, but my main 
question was, why it is not allowed to use .mixin ?
Dec 08 2018
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 8 December 2018 at 19:17:26 UTC, Martin Tschierschke 
wrote:

 I am referring to this post, because it is the first mentioning 
 the mixin way in this thread:

           Why is mixin() not allowed in UFCS?


Because mixin isn't a function, it's a special type of expression 
(like `sizeof` is in C). UFCS only applies to functions.

Here's the relevant part of the language specification: 
https://dlang.org/spec/expression.html#mixin_expressions
Dec 08 2018
prev sibling parent Claude <no no.no> writes:
On Thursday, 6 December 2018 at 00:32:48 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 00:20:53 UTC, H. S. Teoh wrote:

 Why can't this be added to to code.dlang.org as a library / 
 dub package / whatever?


 Library things cannot access the local scope of the string 
 unless you mixin.

 So



 is doable as a library (and I know Nick Sabalusky's scriptlike 
 has it, probably among others), but



 is only doable as a language feature.


I have some syntax suggestion...

Interpolated string would fit well with string-mixins. I 
sometimes tend to use the following pattern:

         enum densityToBitmap(int x, int y, int z) =
             q{
                 if (length(%d, %d, %d) <= 0)
                     index |= (1 << cornerIds[%d][%d][%d]);
             }.format(x, y, z, x, y, z);

         mixin(densityToBitmap!(0, 0, 0));
         mixin(densityToBitmap!(0, 0, 1));
         // etc ...

So I use the q{ ... } syntax when the string contains D code to 
be "mixed-in" later. Using "format" is a bit awkward, so 
interpolated strings would help to make the code more readable, 
and we could still keep the braces-syntax.

For example, replacing the "q{ ... }.format(...)" by a "i{ ... 
}.text"...
Dec 10 2018
prev sibling next sibling parent reply Neia Neutuladh <neia ikeran.org> writes:
On Thu, 06 Dec 2018 00:10:56 +0000, o wrote:

 And please don't mention Phobos' sorry excuse for string interpolation:
 "a is: %s, b is: %s, sum is %s.".format(a, b, a + b)
 Is no better than "a is: "~a.to!string~"b is: "~b.to!string~" and the
 sum is: "~(a+b).to!string~"."


I mean, let's not be hyperbolic. Or contemptuous. format() is a lot more 
readable than a bunch of concatenation for a lot of things. It gives you a 
coherent view of the structure of the string as a whole. As a tradeoff, 
it's harder to see the relationship between the format specifiers and the 
parameters.

String interpolation gets rid of that tradeoff.

However, that would also force druntime to include formatting code that it 
currently lacks. It's very likely that this would have some 
inconsistencies with std.conv. Like, the most straightforward way to make 
this work is to have a runtime function taking variadic arguments:

    string _d_stringinterpc(...);
    wstring _d_stringinterpw(...);
    dstring _d_stringinterpd(...);

That would allow a pluggable approach, which allows for the maximum amount 
of compatibility.

So you write:

    auto bar = "hello world".lazyReplace('l', 'p');
    writeln("foo ${bar}");

This compiles down to:

    writeln(_d_stringinterpc("foo ", bar));

And it prints:

    foo scratch.LazyReplacementRange!(immutable(char), 
string).LazyReplacementRange

lazyReplace returns a LazyReplacementRange, which doesn't define 
toString(). Its corresponding TypeInfo_Struct doesn't have a xtoString 
function pointer set, so we can't convert it to a string by 
calling .toString() on it (even indirectly through reflection). We can't 
list its properties and fields at runtime, so we can't iterate through it 
like std.format would. We can't even assemble a toString() that looks like 
the default for structs in std.conv.

The only thing we have available is the name of the type.

In 2007, this would have worked.

----

Okay, it's bad to use that sort of runtime function. What about putting a 
template inside object.d?

That would require us to put std.conv and std.format into object.d. It 
wouldn't be even remotely pluggable.

This isn't an issue in most languages. Most languages tie their standard 
libraries to their compilers. D maintains a greater separation than 
normal, in part because of the Tango / Phobos separation.
Dec 05 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 02:14:12 UTC, Neia Neutuladh 
wrote:

 However, that would also force druntime to include formatting 
 code that it currently lacks.


Not necessarily. There is precedent in the language for a 
built-in feature actually lowering to a Phobos call: the ** 
operator is translated to std.math.pow.

The interpolate thing could very well be translated to a call to 
std.conv.text.
Dec 05 2018
parent Daniel Kozak <kozzi11 gmail.com> writes:
wow I did not know that.
P.S. ** should be ^^ right?

On Thu, Dec 6, 2018 at 3:55 AM Adam D. Ruppe via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 6 December 2018 at 02:14:12 UTC, Neia Neutuladh
 wrote:

 However, that would also force druntime to include formatting
 code that it currently lacks.


 Not necessarily. There is precedent in the language for a
 built-in feature actually lowering to a Phobos call: the **
 operator is translated to std.math.pow.

 The interpolate thing could very well be translated to a call to
 std.conv.text.
Dec 06 2018
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 6 December 2018 at 02:14:12 UTC, Neia Neutuladh 
wrote:

 However, that would also force druntime to include formatting 
 code that it currently lacks.


Jonathan Marler's implementation [1] has a really nice approach 
of lowering interpolated strings to tuples. You can pass it to 
variadic functions like `writeln` or `text` and bring your own 
formatting / memory allocation schemes, giving better performance 
and flexibility.

[1] https://github.com/dlang/dmd/pull/7988
Dec 06 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 09:01:03 UTC, Dennis wrote:

 Jonathan Marler's implementation [1] has a really nice approach 
 of lowering interpolated strings to tuples.


Yes, yes, yes, I like that a lot. I think that's the way to do it 
if D is ever to get these.

I propose that we all either

1) drop the idea

or

2) unite around that proposal and try to push it through.
Dec 06 2018
next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Thursday, 6 December 2018 at 14:02:09 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 09:01:03 UTC, Dennis wrote:

 Jonathan Marler's implementation [1] has a really nice 
 approach of lowering interpolated strings to tuples.


 Yes, yes, yes, I like that a lot. I think that's the way to do 
 it if D is ever to get these.

 I propose that we all either

 1) drop the idea

 or

 2) unite around that proposal and try to push it through.


I don't mind spearheading the charge, but I am not going to do 
this all by myself as I got the other DIP in mind(Most 
noticeably, making properties great again, as I have to pick up 
the torch that the other dev left off.)
Let arrange a meet up and let us do this thing.
Dec 06 2018
prev sibling parent Martin Tschierschke <mt smartdolphin.de> writes:
On Thursday, 6 December 2018 at 14:02:09 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 09:01:03 UTC, Dennis wrote:

 Jonathan Marler's implementation [1] has a really nice 
 approach of lowering interpolated strings to tuples.


 Yes, yes, yes, I like that a lot. I think that's the way to do 
 it if D is ever to get these.

 I propose that we all either

 1) drop the idea

 or

 2) unite around that proposal and try to push it through.


O.k. I take 2) how is the best way to "unite around that 
proposal?"
Dec 06 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 4:01 AM, Dennis wrote:

 On Thursday, 6 December 2018 at 02:14:12 UTC, Neia Neutuladh wrote:

 However, that would also force druntime to include formatting code 
 that it currently lacks.


 
 Jonathan Marler's implementation [1] has a really nice approach of 
 lowering interpolated strings to tuples. You can pass it to variadic 
 functions like `writeln` or `text` and bring your own formatting / 
 memory allocation schemes, giving better performance and flexibility.
 
 [1] https://github.com/dlang/dmd/pull/7988


I supported the concept[1] back when I read it from Dmitry Olshansky[2].

If I had known about the debate on the PR I would have jumped in. But I 
don't have much to add. All that really dictates my views is the 
experience I've had with other languages. However, none of those other 
languages have the same concept that I know of -- they basically just 
create a string with toString-ing the interpolations.

With the concept of lowering to a tuple, I'd love to use such a thing 
for database queries.

For instance:

db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = ?", aval, 
bval, cval, dval, id);

vs.

db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = $dval 
WHERE id = $id");

The mixin/library solution is much less approachable. What's awesome 
about the language solution is that it just works without much extra 
understanding and verbosity, and need for using mixins. mixins are cool, 
but are best tucked away behind templates or generative functions. They 
shouldn't seen much in user code. Saying we should use a library 
solution for this is like saying we can replace foreach usage with a 
library foreach that lowers to some for-loop and explicit delegates (or 
maybe a mixin?). Yes, it could be done. No, it shouldn't be done. This 
is one of those types of syntax sugar that should not be ignored.

+1000 from me, I'd love to see the PR merged, or the DIP created, 
whatever needs to happen.

-Steve

[1] https://forum.dlang.org/post/odb9hk$2jqm$1 digitalmars.com
[2] https://forum.dlang.org/post/ocut06$261n$1 digitalmars.com
Dec 06 2018
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 6 December 2018 at 16:19:12 UTC, Steven 
Schveighoffer wrote:

 [snip]

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = 
 ?", aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = 
 $dval WHERE id = $id");


Very pretty.
Dec 06 2018
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 6 December 2018 at 16:19:12 UTC, Steven 
Schveighoffer wrote:

 On 12/6/18 4:01 AM, Dennis wrote:

 [...]


 I supported the concept[1] back when I read it from Dmitry 
 Olshansky[2].

 If I had known about the debate on the PR I would have jumped 
 in. But I don't have much to add. All that really dictates my 
 views is the experience I've had with other languages. However, 
 none of those other languages have the same concept that I know 
 of -- they basically just create a string with toString-ing the 
 interpolations.

 With the concept of lowering to a tuple, I'd love to use such a 
 thing for database queries.

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = 
 ?", aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = 
 $dval WHERE id = $id");

 The mixin/library solution is much less approachable. What's 
 awesome about the language solution is that it just works 
 without much extra understanding and verbosity, and need for 
 using mixins. mixins are cool, but are best tucked away behind 
 templates or generative functions. They shouldn't seen much in 
 user code. Saying we should use a library solution for this is 
 like saying we can replace foreach usage with a library foreach 
 that lowers to some for-loop and explicit delegates (or maybe a 
 mixin?). Yes, it could be done. No, it shouldn't be done. This 
 is one of those types of syntax sugar that should not be 
 ignored.

 +1000 from me, I'd love to see the PR merged, or the DIP 
 created, whatever needs to happen.

 -Steve

 [1] https://forum.dlang.org/post/odb9hk$2jqm$1 digitalmars.com
 [2] https://forum.dlang.org/post/ocut06$261n$1 digitalmars.com


Does I understand your sql example right, although it looks like 
it is prone for sql injection attacks, it isn't because you 
evaluate the tuples and not use the string as whole?

I really like this feature. Everytime I have to concatenate 
strings and variables I really really miss this feature.

I would propose Mike starts a donation campaign for this DIP. 
Then op and others can donate in an official way.

Kind regards
Andre
Dec 06 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 17:47:58 UTC, Andre Pany wrote:

 Does I understand your sql example right, although it looks 
 like it is prone for sql injection attacks, it isn't because 
 you evaluate the tuples and not use the string as whole?


Yeah, since it is tuples the function itself gets to manage how 
they are used, including doing some escaping, etc.

I would take it one step further and put the other stuff in a 
wrapped type from the compiler, so the function receiving it can 
static if and tell what it is, so

i"foo $(foo)"
would be

tuple("foo ", FromInterpolation("foo", foo))


so you can identify when something was passed vs being literally 
in the string. And it included the name as a string so we can do 
some other crazy stuff too.

i"foo $(a + b)"

FromInterpolation("a + b", a+b)

so you can then print

a + b = 4


i think that would be kinda cool - the stuff inside is passed as 
a code string. So really generally speaking it would be


tuple("string literal", FromInterpolation(code_as_string, 
mixin(code)), " more string literal"); // and so on


I think that would be seriously cool and quite useful. You can 
then see From Interpolation as a type in there and know to call 
sql escape or replace with ? and move the arg or whatever - the 
function can use it all with CT reflection.
Dec 06 2018
next sibling parent reply Neia Neutuladh <neia ikeran.org> writes:
On Thu, 06 Dec 2018 18:06:51 +0000, Adam D. Ruppe wrote:

 I would take it one step further and put the other stuff in a wrapped
 type from the compiler, so the function receiving it can static if and
 tell what it is, so
 
 i"foo $(foo)"
 would be
 
 tuple("foo ", FromInterpolation("foo", foo))


I was about to suggest wrapping the non-parameters in a Literal{} struct, 
but FromInterpolation makes more sense.

I was thinking about protecting against errors produced when you have to 
use an even/odd rule to figure out what's part of the literal and what's 
part of the interpolation:

    auto c = ");drop table foo;--";
    // whoops, forgot a comma
    db.exec("SELECT * FROM foo WHERE id IN ($a,$b$c)");
      ->
    db.prepare("SELECT * FROM foo WHERE id IN(?, ?);drop table foo;--?")
      .inject(a, b, ")");

With FromInterpolation, you'd be able to reliably come up with the correct 
SQL: "SELECT * FROM foo WHERE id IN (?, ??)". Which is invalid and would 
be rejected.
Dec 06 2018
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 1:28 PM, Neia Neutuladh wrote:

 On Thu, 06 Dec 2018 18:06:51 +0000, Adam D. Ruppe wrote:

 I would take it one step further and put the other stuff in a wrapped
 type from the compiler, so the function receiving it can static if and
 tell what it is, so

 i"foo $(foo)"
 would be

 tuple("foo ", FromInterpolation("foo", foo))


 
 I was about to suggest wrapping the non-parameters in a Literal{} struct,
 but FromInterpolation makes more sense.
 
 I was thinking about protecting against errors produced when you have to
 use an even/odd rule to figure out what's part of the literal and what's
 part of the interpolation:


Hm... this is a good point. Two symbols back to back would be confusing 
if one was a string.

One possibility is to pass an empty string to separate them, but this is 
hackish.

Or you could check the TBA trait that says which parameters were 
interpolations and which ones were literals.


 With FromInterpolation, you'd be able to reliably come up with the correct
 SQL: "SELECT * FROM foo WHERE id IN (?, ??)". Which is invalid and would
 be rejected.


Right, but it also requires that the callee deal with the 
FromInterpolation type, no? Whereas just passing the tuple doesn't 
require current code changes.

I suppose FromInterpolation could alias itself to the actual value. That 
way, it lowers to the original proposal.

One thing I'd say is that FromInterpolation should take the string 
interpolation as a comiple-time item.

In other words, I'd say it should be FromInterpolation!"foo"(foo). But I 
still like the idea of getting the alias from the original source, you 
can do so much more that way (thinking of UDAs).

Actually, considering the generative capabilities of D, you could have a 
function that accepts the DB query as a compile-time interpolated 
string, and simply generate the query string and call to the underlying 
library.

So many possibilities...

-Steve
Dec 06 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 18:41:34 UTC, Steven 
Schveighoffer wrote:

 Right, but it also requires that the callee deal with the 
 FromInterpolation type, no? Whereas just passing the tuple 
 doesn't require current code changes.

 I suppose FromInterpolation could alias itself to the actual 
 value. That way, it lowers to the original proposal.


Yes, with alias this it would work in most cases.

struct FromInterpolation(T) {
    string originalCode;
    T value;
    alias value this;
}

void main() {
   import std.stdio;
   string name = "adam";
   writeln("hi ", FromInterpolation!string("name", mixin("name")));
}


We can prove that works today by just writing it manually and 
running it. It also works for other types:

void foo(string s, string b);
foo("hi ", FromInterpolation!string("name", mixin("name")));


So I dare say in the majority of cases where you might use this, 
an alias this solution covers it. And if you DO want to 
specialize on the special data, it is there for you to use. (do a 
deconstructing is() expression to check if it is 
FromInterpolation and extract the type, then go crazy with it).


Passing the actual variable as a CT arg is doable.. but only if 
there IS an actual variable.

i"hi $(name)"; // could pass T!("hi ", name)
i"hi $(a+b)"; // error: cannot read variables at compile time


Now, maybe we don't want to support $(a+b)... but I think people 
would miss it. How many times in PHP or Ruby or whatever do we 
say:



I think if we don't support that, people will see it as a 
crippled interpolation. So in my view, we get the most 
bang-for-buck by making it a runtime argument rather than a 
compile time alias.

(BTW it is interesting passing an int to a function with an 
interpolated string. but since it doesn't automatically convert 
and just does a tuple.. you can totally do that.)
Dec 06 2018
next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 6 December 2018 at 19:52:27 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 18:41:34 UTC, Steven 
 Schveighoffer wrote:

 [...]


 Yes, with alias this it would work in most cases.

 [...]


Just to clarify, I was already assuming this is what 
FromInterplation would do.
Dec 06 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 2:52 PM, Adam D. Ruppe wrote:


 Passing the actual variable as a CT arg is doable.. but only if there IS 
 an actual variable.
 
 i"hi $(name)"; // could pass T!("hi ", name)
 i"hi $(a+b)"; // error: cannot read variables at compile time


Yeah, I know. I was hoping we could bend the rules a bit in this case. 
Since an interpolation would be a different kind of alias (with special 
traits attached), it could have its own allowances that normally don't 
happen. Make it like a lazy alias ;)


 Now, maybe we don't want to support $(a+b)... but I think people would 
 miss it. How many times in PHP or Ruby or whatever do we say:


We have to support it. Of course, it would be supported for the runtime 
mechanism (where you lose the compile-time introspection). But it would 
be cool to have the compile-time traits allowed as well.


 (BTW it is interesting passing an int to a function with an interpolated 
 string. but since it doesn't automatically convert and just does a 
 tuple.. you can totally do that.)


This is why I think the idea is different and vastly superior to most 
other language string interpolation mechanisms.

-Steve
Dec 06 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 3:08 PM, Steven Schveighoffer wrote:

 On 12/6/18 2:52 PM, Adam D. Ruppe wrote:
 

 Passing the actual variable as a CT arg is doable.. but only if there 
 IS an actual variable.

 i"hi $(name)"; // could pass T!("hi ", name)
 i"hi $(a+b)"; // error: cannot read variables at compile time


 
 Yeah, I know. I was hoping we could bend the rules a bit in this case. 
 Since an interpolation would be a different kind of alias (with special 
 traits attached), it could have its own allowances that normally don't 
 happen. Make it like a lazy alias ;)


This doesn't work:

foo!(a + b);

But this does:

foo!(() => a + b);

However, the type is different, and it's not implicitly called, as it's 
a lambda.

So writeln(Args) would show "hi <someFunctionSignature>".

We still need some compiler-magic here, but it's almost supported...

-Steve
Dec 06 2018
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
Marler's original proposal is simple, orthogonal, and elegant. It 
makes use of existing D features in natural ways, and is both 
easy to understand, and easy to use in simple cases without 
*needing* to understand it. I think adding additional machinery 
like FromInterpolation on top of it would be a mistake. If users 
want to opt in to that extra complexity, it can always be made 
available as a library.

On Thursday, 6 December 2018 at 18:28:09 UTC, Neia Neutuladh 
wrote:

 I was thinking about protecting against errors produced when 
 you have to use an even/odd rule to figure out what's part of 
 the literal and what's part of the interpolation:

     auto c = ");drop table foo;--";
     // whoops, forgot a comma
     db.exec("SELECT * FROM foo WHERE id IN ($a,$b$c)");
       ->
     db.prepare("SELECT * FROM foo WHERE id IN(?, ?);drop table 
 foo;--?")
       .inject(a, b, ")");

 With FromInterpolation, you'd be able to reliably come up with 
 the correct SQL: "SELECT * FROM foo WHERE id IN (?, ??)". Which 
 is invalid and would be rejected.


The actual solution here is to use the type system to distinguish 
between trusted and untrusted strings. E.g.,

     UnsafeString c = getUserInput(); // e.g., "); drop table 
foo;--"
     db.exec("SELECT * FROM foo WHERE id IN ($a,$b$c)");

...and `db.exec` knows to escape its arguments iff they're 
UnsafeStrings.
Dec 06 2018
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 2:12 PM, Paul Backus wrote:

 Marler's original proposal is simple, orthogonal, and elegant. It makes 
 use of existing D features in natural ways, and is both easy to 
 understand, and easy to use in simple cases without *needing* to 
 understand it. I think adding additional machinery like 
 FromInterpolation on top of it would be a mistake. If users want to opt 
 in to that extra complexity, it can always be made available as a library.


I agree with this. But the points brought up are good ones.


 
 On Thursday, 6 December 2018 at 18:28:09 UTC, Neia Neutuladh wrote:

 I was thinking about protecting against errors produced when you have 
 to use an even/odd rule to figure out what's part of the literal and 
 what's part of the interpolation:

     auto c = ");drop table foo;--";
     // whoops, forgot a comma
     db.exec("SELECT * FROM foo WHERE id IN ($a,$b$c)");
       ->
     db.prepare("SELECT * FROM foo WHERE id IN(?, ?);drop table foo;--?")
       .inject(a, b, ")");

 With FromInterpolation, you'd be able to reliably come up with the 
 correct SQL: "SELECT * FROM foo WHERE id IN (?, ??)". Which is invalid 
 and would be rejected.


 
 The actual solution here is to use the type system to distinguish 
 between trusted and untrusted strings. E.g.,
 
      UnsafeString c = getUserInput(); // e.g., "); drop table foo;--"
      db.exec("SELECT * FROM foo WHERE id IN ($a,$b$c)");
 
 ....and `db.exec` knows to escape its arguments iff they're UnsafeStrings.


The more I think about it, the better it is to use the original 
proposal, and just pass the parameters at compile time in order to make 
it work. The non-string portions will be aliases to the expressions or 
variables, making them easily distinguishable from the string portions.

So instead of my original db.exec(...), you'd do db.exec!(...), exactly 
the same as before, just it's passed at compile time vs. runtime. The 
runtime possibility is there too, but you wouldn't use it in a database 
setting.

This can be used to forward to whatever you want. If you want to change 
the parameters to FromInterpolation("foo", foo), it will be possible.

This is very D-ish, too, where we supply the compile-time mechanisms, 
and you come up with the cool way to deal with them.

-Steve
Dec 06 2018
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 2:27 PM, Steven Schveighoffer wrote:

 The more I think about it, the better it is to use the original 
 proposal, and just pass the parameters at compile time in order to make 
 it work. The non-string portions will be aliases to the expressions or 
 variables, making them easily distinguishable from the string portions.
 
 So instead of my original db.exec(...), you'd do db.exec!(...), exactly 
 the same as before, just it's passed at compile time vs. runtime. The 
 runtime possibility is there too, but you wouldn't use it in a database 
 setting.
 
 This can be used to forward to whatever you want. If you want to change 
 the parameters to FromInterpolation("foo", foo), it will be possible.


e.g. (need some support here, from new __traits):

foo(StringInterp...)()
{
    auto makeInterpolation(alias x)()
    {
       static if(isCompileTimeString!x) return x;
       else return FromInterpolation!(__traits(interpolationOf, x))(x);
    }
    return fooImpl(staticMap!(makeInterpolation, StringInterp));
}

-Steve
Dec 06 2018
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 06, 2018 at 02:27:05PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:

 On 12/6/18 2:12 PM, Paul Backus wrote:

 Marler's original proposal is simple, orthogonal, and elegant. It
 makes use of existing D features in natural ways, and is both easy
 to understand, and easy to use in simple cases without *needing* to
 understand it. I think adding additional machinery like
 FromInterpolation on top of it would be a mistake. If users want to
 opt in to that extra complexity, it can always be made available as
 a library.




[...]

 The more I think about it, the better it is to use the original
 proposal, and just pass the parameters at compile time in order to
 make it work. The non-string portions will be aliases to the
 expressions or variables, making them easily distinguishable from the
 string portions.
 
 So instead of my original db.exec(...), you'd do db.exec!(...),
 exactly the same as before, just it's passed at compile time vs.
 runtime. The runtime possibility is there too, but you wouldn't use it
 in a database setting.
 
 This can be used to forward to whatever you want. If you want to
 change the parameters to FromInterpolation("foo", foo), it will be
 possible.
 
 This is very D-ish, too, where we supply the compile-time mechanisms,
 and you come up with the cool way to deal with them.


[...]

Yes, this.  I support this.

Now, I'm also thinking about how to minimize the "surface area" of
language change in order to make this proposal more likely to be
accepted by W&A.  Because as it stands, the exact syntax of interpolated
strings is probably going to ignite a bikeshedding war over nitty-gritty
details that will drown the core proposal in less important issues.

While it's clear that *some* language support will be needed, since
otherwise we have no way of accessing the surrounding scope of the
interpolated string, I wonder how much of the implementation can be
pushed to library code (which would make this more attractive to W&A),
and what are the bare essentials required, in terms of language change,
to make a library implementation possible.

Essentially, what you want to be able to do is essentially for a
template to be able to reference symbols in the caller's scope, rather
than in the template's scope. We want to be able to do this in a clean
way that doesn't break encapsulation (too badly).  If we could somehow
pass some kind of reference or symbol that captures the caller's scope
to the template, then we can implement interpolated strings as library
code instead of adding yet another form of string literal to the
language.  For example, perhaps something like this:

	// Note: very crude tentative syntax, bikeshed over this later.
	template myTpl(string s, import context = __CALLER_CONTEXT)
		if (is(typeof(context.name) : string))
	{
		enum myTpl = s ~ context.name;
	}

	void main() {
		string name = "abc";
		enum z = myTpl!"blah";
		pragma(msg, z);	// prints "blahabc"
	}

Basically, `context` serves as a proxy object for looking up symbols in
the caller's context. It behaves like a module import in pulling in
symbols from the caller's context into the template, as if the template
had "imported" the caller's context (like you would import a module).

Armed with such a construct, the template can actually reject
instantation unless the caller's context contains a symbol called "name"
(see the sig constraint).  This allows a string interpolation template
to produce a helpful error message when instantiation fails, e.g., by
moving the sig constraint into a static if with a static assert that
says "symbol 'blah' in interpolated string not found in caller's
context" or something to that effect.

This solves the problem of the template referencing symbols outside its
own scope that may not be guaranteed to exist.

Passing the context as a template parameter also lets the user pass in a
different scope, e.g., a module, as parameter instead:

	template findSymbols(import context)
	{
		alias findSymbols = __traits(allSymbols, context);
	}
	alias stdStdioSymbols = findSymbols!(std.stdio);

A lot of other possibilities open up here as applications, not just
interpolated strings.  Though the primary motivation for this feature
would be interpolated strings.

Just throwing out some ideas to see if there's a way to do this without
making a big change to the language that will likely be rejected.


T

-- 
To err is human; to forgive is not our policy. -- Samuel Adler
Dec 06 2018
next sibling parent reply Meta <jared771 gmail.com> writes:
On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Essentially, what you want to be able to do is essentially for 
 a template to be able to reference symbols in the caller's 
 scope, rather than in the template's scope.


Am I missing something? D has this already in the form of mixin 
templates.
Dec 06 2018
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 6 December 2018 at 20:45:57 UTC, Meta wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Essentially, what you want to be able to do is essentially for 
 a template to be able to reference symbols in the caller's 
 scope, rather than in the template's scope.


 Am I missing something? D has this already in the form of mixin 
 templates.


Mixin templates can *add* symbols to the caller's scope, but they 
can't refer to *existing* symbols unless they're passed in as 
template parameters.
Dec 06 2018
next sibling parent Meta <jared771 gmail.com> writes:
On Thursday, 6 December 2018 at 20:49:08 UTC, Paul Backus wrote:

 On Thursday, 6 December 2018 at 20:45:57 UTC, Meta wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Essentially, what you want to be able to do is essentially 
 for a template to be able to reference symbols in the 
 caller's scope, rather than in the template's scope.


 Am I missing something? D has this already in the form of 
 mixin templates.


 Mixin templates can *add* symbols to the caller's scope, but 
 they can't refer to *existing* symbols unless they're passed in 
 as template parameters.


mixin template myTpl()
{
     pragma(msg, "name: ", typeof(name));
}

void main()
{
     auto name = "abc";
     mixin myTpl!(); //Prints "name: string"
}

But you do have to know the name of the symbol in advance. I 
assume that's what you mean by "they can't refer to *existing* 
symbols unless they're passed in as template parameters".
Dec 06 2018
prev sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Thu, 06 Dec 2018 20:49:08 +0000, Paul Backus wrote:

 Mixin templates can *add* symbols to the caller's scope, but they can't
 refer to *existing* symbols unless they're passed in as template
 parameters.


A mixin template is declared as `mixin template Foo() {}`. This form of 
template can refer to symbols in the instantiating scope. For instance:

mixin template Foo()
{
    auto b = a;
}

void main()
{
    int a = 12;
    mixin Foo;
    writeln(b);
}

All templates can be mixed in, but if it wasn't declared as `mixin 
template`, it only has access to symbols in the declaration scope.
Dec 06 2018
prev sibling parent reply Neia Neutuladh <neia ikeran.org> writes:
On Thu, 06 Dec 2018 20:45:57 +0000, Meta wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Essentially, what you want to be able to do is essentially for a
 template to be able to reference symbols in the caller's scope, rather
 than in the template's scope.


 
 Am I missing something? D has this already in the form of mixin
 templates.


From the spec:

"A TemplateMixin can occur in declaration lists of modules, classes, 
structs, unions, and as a statement."

Also, it still requires the `mixin` keyword at the place of instantiation. 

So you want to write:

    writeln(interp!"Hi, ${user.name}!");

But you currently have to write:

    mixin interp!"Hi, ${user.name}!";
    writeln(interpResult);
Dec 06 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 06, 2018 at 09:06:16PM +0000, Neia Neutuladh via Digitalmars-d
wrote:

 On Thu, 06 Dec 2018 20:45:57 +0000, Meta wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Essentially, what you want to be able to do is essentially for a
 template to be able to reference symbols in the caller's scope,
 rather than in the template's scope.


 
 Am I missing something? D has this already in the form of mixin
 templates.


 

From the spec:


 
 "A TemplateMixin can occur in declaration lists of modules, classes,
 structs, unions, and as a statement."
 
 Also, it still requires the `mixin` keyword at the place of instantiation. 
 
 So you want to write:
 
     writeln(interp!"Hi, ${user.name}!");
 
 But you currently have to write:
 
     mixin interp!"Hi, ${user.name}!";
     writeln(interpResult);


Ah, so mixin templates *can* access symbols from the containing scope.
That's nice... so actually, we can already implement string
interpolation in the library.  The only complaint is the ugly syntax
required `mixin blah!"abc ${def}"` instead of simply `blah!"abc ${def}"`
or `blah("abc ${def}")`.

I'm in favor of making the syntax less ugly, but I fear Andrei's
objection is going to be that (1) this is already possible without a
language change, and (2) the proposed language change would effectively
only add syntactic sugar, which is unlikely to be enough justification
for doing it.

The above syntax *could* possibly be made just slightly less ugly, if we
extended mixin templates to allow mixin *expressions*, e.g.:

	writeln(mixin interp!"Hi, ${user.name}!");

instead of needing to break it into two lines and accessing an implicit
symbol. Having to break it into two lines is really a serious blow to
writability / usability.  I'd like to get rid of the `mixin` as well,
but I have a feeling Andrei isn't going to like that.

Or perhaps, one could employ the alternative:

	mixin interpFun!(writeln, "Hi, ${user.name}");

where the interpFun template essentially implements the function call on
your behalf.  This is essentially a hack, though, and wouldn't be easily
extendible to the general case without further contortions.  Extending
mixin templates to allow expressions seems to be the least undesirable
option, barring actually adding string interpolation support to the
compiler / language spec.


T

-- 
Don't get stuck in a closet---wear yourself out.
Dec 06 2018
parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 6 December 2018 at 21:32:22 UTC, H. S. Teoh wrote:

 Ah, so mixin templates *can* access symbols from the containing 
 scope. That's nice... so actually, we can already implement 
 string interpolation in the library.  The only complaint is the 
 ugly syntax required `mixin blah!"abc ${def}"` instead of 
 simply `blah!"abc ${def}"` or `blah("abc ${def}")`.


Library versions have existed for a while now. See for example 
`interp` from the scriptlike package on DUB [1], and Marler's own 
library version [2] linked in the original Github discussion.

[1] 
https://github.com/Abscissa/scriptlike/blob/v0.10.2/README.md#string-interpolation
[2] https://github.com/dlang/phobos/pull/6339


 I'm in favor of making the syntax less ugly, but I fear 
 Andrei's objection is going to be that (1) this is already 
 possible without a language change, and (2) the proposed 
 language change would effectively only add syntactic sugar, 
 which is unlikely to be enough justification for doing it.


Unless D someday adds AST macros, new syntax is always going to 
require language changes, so I don't think the fact that this is 
"only" a syntax change should disqualify it from consideration.

Whether or not the difference between 
`writeln(mixin(interp!"..."))` and `writeln(i"...")` is 
significant enough to be worth it is, of course, a value 
judgement, but given that string interpolation has a proven 
track-record in other languages, it's a 100% backwards-compatible 
change, and the implementation work has already been done, I 
think it stands a decent chance of being accepted. The most 
direct comparison I can find is DIP 1009, which added syntactic 
sugar for contracts, and was accepted "without objection."
Dec 06 2018
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 3:00 PM, H. S. Teoh wrote:

 On Thu, Dec 06, 2018 at 02:27:05PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:

 On 12/6/18 2:12 PM, Paul Backus wrote:

 Marler's original proposal is simple, orthogonal, and elegant. It
 makes use of existing D features in natural ways, and is both easy
 to understand, and easy to use in simple cases without *needing* to
 understand it. I think adding additional machinery like
 FromInterpolation on top of it would be a mistake. If users want to
 opt in to that extra complexity, it can always be made available as
 a library.




 [...]

 The more I think about it, the better it is to use the original
 proposal, and just pass the parameters at compile time in order to
 make it work. The non-string portions will be aliases to the
 expressions or variables, making them easily distinguishable from the
 string portions.

 So instead of my original db.exec(...), you'd do db.exec!(...),
 exactly the same as before, just it's passed at compile time vs.
 runtime. The runtime possibility is there too, but you wouldn't use it
 in a database setting.

 This can be used to forward to whatever you want. If you want to
 change the parameters to FromInterpolation("foo", foo), it will be
 possible.

 This is very D-ish, too, where we supply the compile-time mechanisms,
 and you come up with the cool way to deal with them.


 [...]
 
 Yes, this.  I support this.
 
 Now, I'm also thinking about how to minimize the "surface area" of
 language change in order to make this proposal more likely to be
 accepted by W&A.  Because as it stands, the exact syntax of interpolated
 strings is probably going to ignite a bikeshedding war over nitty-gritty
 details that will drown the core proposal in less important issues.
 
 While it's clear that *some* language support will be needed, since
 otherwise we have no way of accessing the surrounding scope of the
 interpolated string, I wonder how much of the implementation can be
 pushed to library code (which would make this more attractive to W&A),
 and what are the bare essentials required, in terms of language change,
 to make a library implementation possible.


Why do we need support for accessing the surrounding scope? The 
passed-in variable aliases and expressions should be all that is needed.

[snip]


 Just throwing out some ideas to see if there's a way to do this without
 making a big change to the language that will likely be rejected.


I don't think we need to start with such a drastic addition. Let's start 
with lowering the interpolation string to the tuple, and see how far we 
can get from that.

-Steve
Dec 06 2018
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Now, I'm also thinking about how to minimize the "surface area" 
 of language change in order to make this proposal more likely 
 to be accepted by W&A.


IMO the one thing that will likely get this across the finish 
line is to generalize it:

1) What is the fundamental missing language feature that is 
preventing us from implementing interpolated strings in the 
library?

2) What other use cases besides string interpolation would be 
enabled by adding such a feature?

If a compelling language feature that has more uses beyond string 
interpolation can be engineered and/or discovered, I think it 
will have a much better chance of being accepted.

Mike
Dec 06 2018
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin wrote:

 1) What is the fundamental missing language feature that is 
 preventing us from implementing interpolated strings in the 
 library?


To access symbols from the caller's scope.


 2) What other use cases besides string interpolation would be 
 enabled by adding such a feature?


Any thing from scala's implicit playbook.
Dec 06 2018
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 6 December 2018 at 21:47:02 UTC, Sebastiaan Koppe 
wrote:

 On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin 
 wrote:

 1) What is the fundamental missing language feature that is 
 preventing us from implementing interpolated strings in the 
 library?


 To access symbols from the caller's scope.


 2) What other use cases besides string interpolation would be 
 enabled by adding such a feature?


 Any thing from scala's implicit playbook.


There's a reason why there's no way to access the callers scope 
without a mixin...because it breaks encapsulation.  Imagine this:

int x = 10;
foo();


What is the value of x after this?  If a function (or whatever 
foo is) could access the caller's scope, then it could be 
anything.  This adds a very large mental burden on the reader 
because now every function call could modify any local state at 
any point.  That's why you need this:

int x = 10;
mixin foo();

Now you've got a flag to indicate that you need to understand foo 
before you can make any assumptions about your local state.  
There's a balance between power and reasonable encapsulation.  If 
you make functions too powerful, then understanding code becomes 
a nightmare.

Interpolated strings are a special case because even though they 
can access the caller's scope, they do this explicitly:

int x = 10;
foo(i"$(x=20)");

You can clearly see that x is being set to 20, you don't need to 
know the implementation specifics of interpolated strings to see 
that.

A feature that allows functions and/or templates to access the 
caller's state implicitly would allow us to implement 
interpolated strings, however, this would come at the cost of 
adding a very large mental burden to understand any and all D 
code going forward.

On that note, you could make an argument that you should be able 
to access the caller's state in a "read-only" capacity, but that 
can get complicated.  Say you have this:

int x = 10;
int* y = &x;
foo();

What's the value of x after this?  What if we define foo as:

void foo(CallerScope s = __CALLER_SCOPE__)
{
     *(cast(int*)s.y) = 20;
}

We did have to cast `y` from const(int*) to int*, but we were 
able to change x nonetheless. You end up with the same problem 
that you have to understand every function you call at a much 
deeper level to know when it can modify your local state.
Dec 06 2018
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 6 December 2018 at 22:31:07 UTC, Jonathan Marler 
wrote:

 There's a reason why there's no way to access the callers scope 
 without a mixin...because it breaks encapsulation.  Imagine 
 this:

 int x = 10;
 foo();

 What is the value of x after this?  If a function (or whatever 
 foo is) could access the caller's scope, then it could be 
 anything.


I agree, it is thin ice. But Mike asked for a generalisation and 
this is it.

Also, scala's implicit parameters do also allow mutation. 
Although in scala stuff is generally immutable to begin with.


 Interpolated strings are a special case because even though 
 they can access the caller's scope, they do this explicitly:

 int x = 10;
 foo(i"$(x=20)");

 You can clearly see that x is being set to 20, you don't need 
 to know the implementation specifics of interpolated strings to 
 see that.


Good point.


 On that note, you could make an argument that you should be 
 able to access the caller's state in a "read-only" capacity,


I was about to say that.


 but that can get complicated.  Say you have this:

 int x = 10;
 int* y = &x;
 foo();

 What's the value of x after this?  What if we define foo as:

 void foo(CallerScope s = __CALLER_SCOPE__)
 {
     *(cast(int*)s.y) = 20;
 }

 We did have to cast `y` from const(int*) to int*, but we were 
 able to change x nonetheless. You end up with the same problem 
 that you have to understand every function you call at a much 
 deeper level to know when it can modify your local state.


Same problem indeed, but one with a lower probability.

Still thin ice, for sure. But I have to say a small part of me 
likes it (besides enabling string interpolation, it would also 
enable things like 'importing' user defined functions in a 
library without explicitly passing them via a template parameter.)
Dec 07 2018
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 06, 2018 at 09:20:54PM +0000, Mike Franklin via Digitalmars-d wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Now, I'm also thinking about how to minimize the "surface area" of
 language change in order to make this proposal more likely to be
 accepted by W&A.


 
 IMO the one thing that will likely get this across the finish line is
 to generalize it:
 
 1) What is the fundamental missing language feature that is preventing
 us from implementing interpolated strings in the library?
 
 2) What other use cases besides string interpolation would be enabled
 by adding such a feature?
 
 If a compelling language feature that has more uses beyond string
 interpolation can be engineered and/or discovered, I think it will
 have a much better chance of being accepted.


[...]

Thanks for stating what I had in mind in a much clearer way than I did.
:D

After thinking about it more, I realize we do have mixin templates that
are able to access symbols in the surrounding scope already. So, at
least in principle, we could already implement string interpolation in
the library, e.g.:

	int i, j;
	string s, t;
	mixin interpolate!"${s} owes ${i} favors and ${j} free tickets to ${t}";
	writeln(interpolateResult);

or something along similar lines.

The only real objection I can see is the ugliness of the syntax, since
currently mixin templates can only be used in statements, rather than
expressions, so to pass multiple interpolated strings to a function,
you'd need a bunch of ugly assignments and temporaries on multiple
lines, thus diminishing the usability of the feature.

If syntactic sugar is the only essential ingredient, then I'm not seeing
a high chance of this proposal being accepted.

But OTOH, making mixin templates usable as expressions, not just
statements, extends beyond mere syntactic niceness. It would allow a
wider application of mixin templates to be usable as smaller chunks of a
larger expression, that otherwise would require onerous amounts of
boilerplate to express.  It's also arguably a natural extension of mixin
templates as an existing language feature (i.e., we already have mixin
template statements, the logical next step is mixin template
expressions). So that could possibly be the angle we should push for.


T

-- 
"Hi." "'Lo."
Dec 06 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 4:20 PM, Mike Franklin wrote:

 On Thursday, 6 December 2018 at 20:00:02 UTC, H. S. Teoh wrote:

 Now, I'm also thinking about how to minimize the "surface area" of 
 language change in order to make this proposal more likely to be 
 accepted by W&A.


 
 IMO the one thing that will likely get this across the finish line is to 
 generalize it:
 
 1) What is the fundamental missing language feature that is preventing 
 us from implementing interpolated strings in the library?
 
 2) What other use cases besides string interpolation would be enabled by 
 adding such a feature?
 
 If a compelling language feature that has more uses beyond string 
 interpolation can be engineered and/or discovered, I think it will have 
 a much better chance of being accepted.


It's really not what's enabled (clearly, things can still be used 
without string interpolation, and we can use mixins to simulate close to 
what we would want).

It's how one writes and reads code. The efficiency and elegance is not 
to be ignored.

I view `foreach` as a fundamental feature of D. Yet, it's completely 
unnecessary, given the existence of `for`. AND given the power of D's 
mixins, we can completely replace the functionality with a Foreach 
library type or function. Despite all that, I'd rather have foreach as a 
language construct, because it's so damned clean.

It's the boiling down of the syntax to the fundamental expression of the 
developer. I want to write the parameters to a function or to a template 
without having to reach for the comma-quote-separator-quote-comma 
sequence. Not only is this noisy and cumbersome, but it breaks up the 
data and the presentation in a way that is difficult to piece together.

writeln(a, " ", b, " ", c, " ", d);

is messy and distracting. It's like a photograph with lots of extraneous 
items taking away from what you should be looking at.

writefln("%s %s %s %s", a, b, c, d);

Better, I can envision how the result comes out. But get that string to 
be long and complicated, or add 5 more parameters, and now it's hard to 
figure out what variable goes where. It's not coupled to the location it 
belongs. Now the photograph has cutouts for your family members, and the 
actual photos of your family scrunched up together on the side, out of 
the picture. You just have to imagine what it would be like if they were 
in there.

writeln("$a $b $c $d");

Simple, elegant, obvious. With syntax highlighting, the symbols are 
identified easily. Its like a photograph with your family in focus 
against a nice background. I literally would have to explain the usage 
of this feature to nobody who is familiar with C. Or even PHP for that 
matter!

So it's not about what it enables, it's about enabling your 
expressiveness, and bringing focus to what is important when writing code.

Now, on top of that, if we do go with the lowering to tuples, we get 
even more power. We have so many more options than just a simple 
concatenated string. The DB query possibility makes my mouth water. And 
I've written a lot of DB queries with D. They are hard to write, and 
hard to read. Every time I add a stupid column, and then I have to add a 
new placeholder, and put the new value in the right place in the 
trailing arguments, it's a long process.

So everything this enables is doable, right now, in a library. It 
wouldn't be nearly as pretty, and the alternatives today without mixins 
are ugly, unfocused, and distracting. I won't say that we absolutely 
have to have this feature in the language, but I would say it would be a 
shining star of a feature when compared with other languages.

-Steve
Dec 06 2018
next sibling parent reply o <o o.o> writes:
On Thursday, 6 December 2018 at 22:04:13 UTC, Steven 
Schveighoffer wrote:

 [...]
 Now, on top of that, if we do go with the lowering to tuples, 
 we get even more power. We have so many more options than just 
 a simple concatenated string. The DB query possibility makes my 
 mouth water. And I've written a lot of DB queries with D. They 
 are hard to write, and hard to read. Every time I add a stupid 
 column, and then I have to add a new placeholder, and put the 
 new value in the right place in the trailing arguments, it's a 
 long process.

 -Steve


While I agree that lowering to tuples opens awesome 
possibilities, it is rather different than string interpolation. 
"string interpolation" should lower to a string, so an 
interpolated string can be used identically to any other string. 
It just makes a lot more sense if an "interpolated string" 
actually is a string, not a tuple.

String interpolation (at least using the 'i' prefix) should 
produce a string. If 'tuple interpolation' really is beneficial, 
maybe it could be added as a different string prefix
i.e `assert(t"my name is $name" == tuple("my name is ", name))`
and `assert(i"my name is $name" == "my name is Jeff")`

Just think how much nicer this is (lowering to a string):
`foo(i"My name is $name");`

Compared to this (lowering to a tuple, requiring a conversion to 
a string):
```
import std.conv: text;
foo(text(i"My name is $name"));
```
Dec 06 2018
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:

 Just think how much nicer this is (lowering to a string):
 `foo(i"My name is $name");`

 Compared to this (lowering to a tuple, requiring a conversion 
 to a string):
 ```
 import std.conv: text;
 foo(text(i"My name is $name"));
 ```


With UFCS it looks a lot nicer already:
```
foo(i"My name is $name".text);
```
How often do you actually need to pass an interpolated string to 
a function in string form? That's not a rethorical question, it 
is important to consider. If 90% of string interpolation happens 
in writeln / logging then it's not worth forcing the use of 
phobos + GC + eager toString and concatenation just to make the 
remaining 10% of cases a bit easier.
Dec 06 2018
parent reply o <o o.o> writes:
On Thursday, 6 December 2018 at 22:52:28 UTC, Dennis wrote:

 On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:

 Just think how much nicer this is (lowering to a string):
 `foo(i"My name is $name");`

 Compared to this (lowering to a tuple, requiring a conversion 
 to a string):
 ```
 import std.conv: text;
 foo(text(i"My name is $name"));
 ```


 With UFCS it looks a lot nicer already:
 ```
 foo(i"My name is $name".text);
 ```
 How often do you actually need to pass an interpolated string 
 to a function in string form? That's not a rethorical question, 
 it is important to consider. If 90% of string interpolation 
 happens in writeln / logging then it's not worth forcing the 
 use of phobos + GC + eager toString and concatenation just to 
 make the remaining 10% of cases a bit easier.


"forcing the use of phobos" does not only apply to 
lower-to-string interpolation. Tuple and 'text()' functionality 
all comes from phobos in the first place.


 If 90% of string interpolation happens in writeln / logging 
 then it's not worth forcing the use of phobos + GC + eager 
 toString and concatenation just to make the remaining 10% of 
 cases a bit easier.


So lets say that tuple-interpolation covers 90% of the use cases. 
What about the other 10%? If we used 2 different string prefixes, 
ie `i"..."` for lower-to-string interpolation and `t"..."` for 
lower-to-tuple, then 100% of the use cases are covered and 
everyone is happy.
Dec 06 2018
parent Dennis <dkorpel gmail.com> writes:
On Thursday, 6 December 2018 at 23:10:48 UTC, o wrote:

 "forcing the use of phobos" does not only apply to 
 lower-to-string interpolation. Tuple and 'text()' functionality 
 all comes from phobos in the first place.


But `text` is something you choose to import and call yourself. 
You are free to implement your own text function for a BetterC / 
WebAssembly environment. And, as Adam clarified, it's not lowered 
to a Phobos Tuple but a 'compiler tuple'.


 If we used 2 different string prefixes, ie `i"..."` for 
 lower-to-string interpolation and `t"..."` for lower-to-tuple, 
 then 100% of the use cases are covered and everyone is happy.


I think we can be happy if we can make a convincing case that one 
of these is worth the extra complexity. Trying to get in two new 
string types with huge overlap is probably not worth it. It also 
raises the question 'when to use which' which will not be obvious 
to many users.
Dec 06 2018
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:

 Just think how much nicer this is (lowering to a string):
 `foo(i"My name is $name");`

 Compared to this (lowering to a tuple, requiring a conversion 
 to a string):
 ```
 import std.conv: text;
 foo(text(i"My name is $name"));
 ```


You can write `foo` in such a way that the user never has to 
manually insert a call to `text`:

void foo(string s)
{
     // do whatever
}

void foo(Args...)(Args args)
{
     import std.conv: text;
     foo(text(args));
}

In fact, you can even encapsulate this pattern in a mixin:

template interpArgs(alias fun)
{
     import std.format: format;

     enum interpArgs = q{
         auto %1$s(Args...)(Args args)
         {
             import std.conv: text;
             return %1$s(text(args));
         }
     }.format(__traits(identifier, fun));
}

mixin(interpArgs!foo);
Dec 06 2018
parent reply o <o o.o> writes:
On Thursday, 6 December 2018 at 22:53:26 UTC, Paul Backus wrote:

 On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:

 Just think how much nicer this is (lowering to a string):
 `foo(i"My name is $name");`

 Compared to this (lowering to a tuple, requiring a conversion 
 to a string):
 ```
 import std.conv: text;
 foo(text(i"My name is $name"));
 ```


 You can write `foo` in such a way that the user never has to 
 manually insert a call to `text`:

 void foo(string s)
 {
     // do whatever
 }

 void foo(Args...)(Args args)
 {
     import std.conv: text;
     foo(text(args));
 }

 In fact, you can even encapsulate this pattern in a mixin:

 template interpArgs(alias fun)
 {
     import std.format: format;

     enum interpArgs = q{
         auto %1$s(Args...)(Args args)
         {
             import std.conv: text;
             return %1$s(text(args));
         }
     }.format(__traits(identifier, fun));
 }

 mixin(interpArgs!foo);


I think that this just further proves my point. In order use an 
interpolated string as a real string, you ended up going back 
down this rabbit-hole of mixins/libraries/templetes. There would 
be no need for this if `foo(i"a is $a)` was the same as `foo("a 
is 12")`.
Dec 06 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 6:05 PM, o wrote:

 On Thursday, 6 December 2018 at 22:53:26 UTC, Paul Backus wrote:

 On Thursday, 6 December 2018 at 22:34:14 UTC, o wrote:

 Just think how much nicer this is (lowering to a string):
 `foo(i"My name is $name");`

 Compared to this (lowering to a tuple, requiring a conversion to a 
 string):
 ```
 import std.conv: text;
 foo(text(i"My name is $name"));
 ```


 You can write `foo` in such a way that the user never has to manually 
 insert a call to `text`:

 void foo(string s)
 {
     // do whatever
 }

 void foo(Args...)(Args args)
 {
     import std.conv: text;
     foo(text(args));
 }

 In fact, you can even encapsulate this pattern in a mixin:

 template interpArgs(alias fun)
 {
     import std.format: format;

     enum interpArgs = q{
         auto %1$s(Args...)(Args args)
         {
             import std.conv: text;
             return %1$s(text(args));
         }
     }.format(__traits(identifier, fun));
 }

 mixin(interpArgs!foo);


 
 I think that this just further proves my point. In order use an 
 interpolated string as a real string, you ended up going back down this 
 rabbit-hole of mixins/libraries/templetes. There would be no need for 
 this if `foo(i"a is $a)` was the same as `foo("a is 12")`.


But it's not. It's foo("a is " ~ a.to!string);

I'd rather the allocation(s) be explicit.

foo(i"a is $a".text)

looks good to me, and solves the problem.

-Steve
Dec 06 2018
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 6 December 2018 at 22:04:13 UTC, Steven 
Schveighoffer wrote:


 It's really not what's enabled (clearly, things can still be 
 used without string interpolation, and we can use mixins to 
 simulate close to what we would want).

 It's how one writes and reads code. The efficiency and elegance 
 is not to be ignored.


[..]

Of course, you are right, and that's what I was trying to say.  
"What language feature do we need, or limitation removed, that 
gets us what we want:  the ability to implement interpolated 
strings and potentially other features in the library without too 
much compromise on the efficiency, elegance, and expressiveness 
you would get if you implemented it in the language, and without 
turning D into a DSL".  That probably doesn't do it justice 
either, but I hope you get the idea and we're on more-or-less the 
same page.

Jacob mentioned AST macros (I knew that was coming, and even had 
a note in my first draft say "Sorry, Jacob, except for AST 
macros", but I deleted it :D ).  But Jacob is on to something.  
We already pass around symbols with `alias`.  Can that be 
extended somehow to pass around expressions too (i.e. expressions 
as rvalues)?  That would probably help address the issue of 
printing assert expressions as well 
(https://github.com/dlang/dmd/pull/8517)

Zodian mentioned eponymous mixin template at 
https://forum.dlang.org/post/tnebtoplmhtpabnlvydw forum.dlang.org

H. S. Teoh mentioned using mixins as expressions at 
https://forum.dlang.org/post/mailman.5553.1544132869.29801.digitalmars-d puremagic.com

As for myself, I don't see much value in forcing users to add 
`mixin` at the site of a mixin template instantiation, but 
there's a lot I don't see, so I may just be missing something.
```
import std.stdio;

mixin template Foo(T)
{
     T x = 5;
}

extern(C) void main()
{
     // Why do we need `mixin` here?
     // Why can't we just write `Foo!int;`?
     // It's already yelling at us with a ! operator.
     mixin Foo!int;
     writeln(x);
}
```

Simen Kjærås mentioned a few things when we were discussing ways 
to implement properties in the library at 
https://forum.dlang.org/post/jqcqsriizuewdqotbnop forum.dlang.org 
  There might be some feature to be added, or limitation removed, 
that will permit help both the string interpolation case and the 
properties case.

I know I'm just thinking out loud and not contributing much, but 
I can't help but feel that there's something common in all these 
ideas that just hasn't been identified yet.  And no, Jacob, not 
AST macros (Actually, that would probably do it, but I think 
we've already been there) :D

Mike
Dec 06 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 7 December 2018 at 00:43:43 UTC, Mike Franklin wrote:

 Can that be extended somehow to pass around expressions too 
 (i.e. expressions as rvalues)?


We kinda-sorta can, with `lazy void`. If we could get the code 
out of it as a string, that might be interesting. But I don't 
think it would hit the interp thing (whose library implementation 
is really just the mixin expression on the outside)


     // It's already yelling at us with a ! operator.


Keep in mind that mixin templates currently do not actually need 
arguments listed.
Dec 06 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 7 December 2018 at 01:19:35 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 00:43:43 UTC, Mike Franklin wrote:

 Can that be extended somehow to pass around expressions too 
 (i.e. expressions as rvalues)?


 We kinda-sorta can, with `lazy void`.


I'm not sure what you mean.  Please elaborate.

Steven gave this example[1] here:


 This doesn't work:
 
 foo!(a + b);
 
 But this does:
 
 foo!(() => a + b);


What I'm thinking is if `foo` is declared as `foo(alias a)`, then 
`a` could either accept either a symbol (currently possible) or 
an expression (not currently possible; this is what I meant by 
"expressions as rvalues").  Depending on how the body of `foo` 
utilized `a` it could more or less do the equivalent of `foo!(() 
=> a + b)`.

Mike

[1] - https://forum.dlang.org/post/puc2f3$27tr$1 digitalmars.com
Dec 06 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 7 December 2018 at 02:38:30 UTC, Mike Franklin wrote:

 We kinda-sorta can, with `lazy void`.


 I'm not sure what you mean.  Please elaborate.


It is the runtime version of what Steven gave.

void foo(lazy int something) {
    import std.stdio;
    writeln(something);
}

void main() {
   int a = 1, b = 2;
   foo(a + b);
}


And you can `lazy void` too, which takes anything which has an 
effect (it is an error if it has no effect), but discards its 
return value. example here: 
https://dlang.org/spec/function.html#lazy-params


The compiler simply wraps the expression you pass to the argument 
in a delegate, and automatically calls any reference to it (so 
the ref of something above actually does something())
Dec 06 2018
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 7 December 2018 at 03:00:54 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 02:38:30 UTC, Mike Franklin wrote:

 We kinda-sorta can, with `lazy void`.


 I'm not sure what you mean.  Please elaborate.


 It is the runtime version of what Steven gave.

 void foo(lazy int something) {
    import std.stdio;
    writeln(something);
 }

 void main() {
   int a = 1, b = 2;
   foo(a + b);
 }


 And you can `lazy void` too, which takes anything which has an 
 effect (it is an error if it has no effect), but discards its 
 return value. example here: 
 https://dlang.org/spec/function.html#lazy-params


It's stuff like this that makes D so cool; I wasn't aware of this 
feature.  Thanks, Adam, I totally get it now.
Dec 06 2018
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 12:43:43AM +0000, Mike Franklin via Digitalmars-d wrote:
[...]

 As for myself, I don't see much value in forcing users to add `mixin`
 at the site of a mixin template instantiation, but there's a lot I
 don't see, so I may just be missing something.
 ```
 import std.stdio;
 
 mixin template Foo(T)
 {
     T x = 5;
 }
 
 extern(C) void main()
 {
     // Why do we need `mixin` here?
     // Why can't we just write `Foo!int;`?
     // It's already yelling at us with a ! operator.
     mixin Foo!int;
     writeln(x);
 }
 ```


Yes it's already yelling at us with a ! operator, but the difference is
that Foo!Bar is generally read as a templated type of some sort in
typical D code.  It's surprising behaviour to see Foo!Bar that not only
has side-effects, but may access symbols in the current scope that
aren't explicitly passed to it.  The 'mixin' is the cue that whatever's
inside Foo may reach out and touch stuff in the surrounding code with
its grubby hands, as opposed to, say, instantiating a template type, or
calling a template function, both of which can only exert an effect
within their own scope and/or arguments explicitly handed to them.

While I understand the cumbersomeness of having to type `mixin`
everywhere string interpolation is used, part of me kinda wants to keep
the `mixin` in the syntax, if for nothing else than to warn anyone
reading the code that we're potentially touching stuff outside of what
template functions can usually touch.

Which leads to the absolute minimal language change that would let us
implement nice interpolated strings: mixin expressions. Writing:

	string name = ...;
	writeln(mixin interp!"My name is ${name}.");

is miles ahead of:

	string name = ...;
	mixin interp!"My name is ${name}.";
	writeln(interpResult);	// bad: `interpResult` is an implicit
				// symbol leaked out of interp's
				// implementation

Esp. if multiple interpolated arguments are being used:

	string name = ...;
	int age = ...;
	mixin interp!"My name is ${name}.";
	string s1 = interpResult;	// ouch
	mixin interp!"My age is ${age}.";
	string s2 = interpResult;	// double ouch
	writefln("%s %s", s1, s2);

Far too much boilerplate compared to:

	string name = ...;
	int age = ...;
	writefln("%s %s",
		 mixin interp!"My name is ${name}.",
		 mixin interp!"My age is ${age}.");

(I know, I know, bad example, since the whole point is not to use format
strings.  But it was the quickest example that came to mind.)

And yes, `mixin interp!"..."` is horribly verbose. But ostensibly we
could abbreviate it to something like `mixin i!"..."`.


On that note, though, a thought occurred to me: what if we implement
interpolated strings using something (admittedly a major hack) like:

	mixin(interpolate!q{
		writeln("My name is ${name} and age is ${age}");
		writeln("How are you, ${guestName}?");
	});

where `interpolate` parses the passed-in code, expands any interpolated
variables, and returns a code string that's mixed into the current
context?  It will allow completely convenient syntax, since you can put
multiple statements in the token string, even arbitrarily complex bits
of code if you want. All you have to do is to wrap the whole thing
inside the token string.

(I can already see y'all cringe at this. :-P  But for completeness' sake
in the prospective DIP, we should address why such an approach is not
desirable, since it's clearly *possible*, and does, ostensibly, solve
the problem at hand.)


T

-- 
Let's eat some disquits while we format the biskettes.
Dec 06 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 7 December 2018 at 01:47:13 UTC, H. S. Teoh wrote:


 And yes, `mixin interp!"..."` is horribly verbose. But 
 ostensibly we could abbreviate it to something like `mixin 
 i!"..."`.


Perhaps what's needed is a different mixin operator that is not 
verbose and significantly distinguishes it from the template 
instantiation operator.

mixin i!"...";  // instead of this...  (too verbose)
i!"...";        // or this... (looks too much like a template 
instantiation)


Mike
Dec 06 2018
next sibling parent Martin Tschierschke <mt smartdolphin.de> writes:
On Friday, 7 December 2018 at 07:53:14 UTC, Mike Franklin wrote:

 On Friday, 7 December 2018 at 01:47:13 UTC, H. S. Teoh wrote:


 And yes, `mixin interp!"..."` is horribly verbose. But 
 ostensibly we could abbreviate it to something like `mixin 
 i!"..."`.


 Perhaps what's needed is a different mixin operator that is not 
 verbose and significantly distinguishes it from the template 
 instantiation operator.

 mixin i!"...";  // instead of this...  (too verbose)
 i!"...";        // or this... (looks too much like a template 
 instantiation)


 Mike


When I tried - some time ago - to find a way to be able to just 
write

exho!"${name} you are app. ${age*365} days old";

instead of - when using scriptlike (DUB package)

import scriptlike;
...
writeln(mixin(interp!"${name} you are app. ${age*365} days old"));

(Sorry for the naming of the function was to distinguish it from 
echo.)

I defined an additional modul exho.d which
only defines an mixin sting:

module exho;

enum use_exho="auto mixinter(string x)(){return mixin(interp!x);}
auto exho(string x)(){return 
mixin(\"writeln(\"~interp!x~\")\");}";

Now in use it comes down to this:

import scriptlike;
import exho;
//Now you can write in  every scope you need it: mixin(use_exho);
// or mixinter! as shortcut for mixin(interp!...)
void main()
{
   auto name = userInput!string("Please enter your name");
   auto age = userInput!int("And your age");

   mixin(use_exho);
   exho!"${name} you are app. ${age*365} days old";
}

You still need "mixin" to get the definition in the right scope 
to define your shortcut in place.
Dec 07 2018
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 07:53:14 UTC, Mike Franklin wrote:

 On Friday, 7 December 2018 at 01:47:13 UTC, H. S. Teoh wrote:


 And yes, `mixin interp!"..."` is horribly verbose. But 
 ostensibly we could abbreviate it to something like `mixin 
 i!"..."`.


 Perhaps what's needed is a different mixin operator that is not 
 verbose and significantly distinguishes it from the template 
 instantiation operator.

 mixin i!"...";  // instead of this...  (too verbose)
 i!"...";        // or this... (looks too much like a template 
 instantiation)


 Mike


An interesting idea.



Keep in mind though, that interpolated strings implemented using 
this will have the disadvantages of mixin code, bad error 
messages, hard to debug and a hit to compile time performance. 
Plus you still need the extra import. Even if we could implement 
interoplated strings this way, it may still be worth it to add 
the small amount of code to the compiler for the extra benefits. 
But this idea is the first I heard that could work.
Dec 07 2018
next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 14:40:05 UTC, Jonathan Marler wrote:

 On Friday, 7 December 2018 at 07:53:14 UTC, Mike Franklin wrote:

 On Friday, 7 December 2018 at 01:47:13 UTC, H. S. Teoh wrote:


 And yes, `mixin interp!"..."` is horribly verbose. But 
 ostensibly we could abbreviate it to something like `mixin 
 i!"..."`.


 Perhaps what's needed is a different mixin operator that is 
 not verbose and significantly distinguishes it from the 
 template instantiation operator.

 mixin i!"...";  // instead of this...  (too verbose)
 i!"...";        // or this... (looks too much like a template 
 instantiation)


 Mike


 An interesting idea.



 Keep in mind though, that interpolated strings implemented 
 using this will have the disadvantages of mixin code, bad error 
 messages, hard to debug and a hit to compile time performance. 
 Plus you still need the extra import. Even if we could 
 implement interoplated strings this way, it may still be worth 
 it to add the small amount of code to the compiler for the 
 extra benefits. But this idea is the first I heard that could 
 work.


One more thing.  If we implement it as a string operator (i.e 
i"...") then we can use just the letter 'i' because it doesn't 
live in the identifier namespace.  If we made it a library, we 
should probably give it a longer name (i.e. interp) so we don't 
take away the identifier `i` from the app.

writeln(i"a is $(a)");

vs

import std.typecons : interp;


There are a handful of benefits to making it a part of the 
language and the importance of these benefits should be carefully 
considered.  It's the shortest syntax, has normal error messages, 
fast compile time performance, no need for extra imports or 
definitions, does not take away identifiers/symbols from the app 
and just looks nicer. If string interpolation is going to be a 
heavily used feature, showing up in code samples promoting the 
language and all over the standard library then it deserves the 
language support for these benefits.  If it was accepted it would 
show up all over the place in my code and we want to make a good 
impression.
Dec 07 2018
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 7 December 2018 at 14:40:05 UTC, Jonathan Marler wrote:

 Perhaps what's needed is a different mixin operator that is 
 not verbose and significantly distinguishes it from the 
 template instantiation operator.

 mixin i!"...";  // instead of this...  (too verbose)
 i!"...";        // or this... (looks too much like a template 
 instantiation)


 Mike


 An interesting idea.




Congratulations, you've invented Lisp reader macros. :)

https://lisper.in/reader-macros
Dec 07 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 7:43 PM, Mike Franklin wrote:

 On Thursday, 6 December 2018 at 22:04:13 UTC, Steven Schveighoffer wrote:
 

 It's really not what's enabled (clearly, things can still be used 
 without string interpolation, and we can use mixins to simulate close 
 to what we would want).

 It's how one writes and reads code. The efficiency and elegance is not 
 to be ignored.


 
 [..]
 
 Of course, you are right, and that's what I was trying to say. "What 
 language feature do we need, or limitation removed, that gets us what we 
 want:  the ability to implement interpolated strings and potentially 
 other features in the library without too much compromise on the 
 efficiency, elegance, and expressiveness you would get if you 
 implemented it in the language, and without turning D into a DSL".  That 
 probably doesn't do it justice either, but I hope you get the idea and 
 we're on more-or-less the same page.


I get the idea, but the answer is, nothing. We can implement it today 
(take a look at scriptlike).

As long as there is an insistence that 'if it's doable in the library, 
the elegance is not worth the effort,' any proposal will fail. We need 
to first convince those in charge think the feature should be added 
because of the *human* impact. That was the point of my longer answer.


 Jacob mentioned AST macros (I knew that was coming, and even had a note 
 in my first draft say "Sorry, Jacob, except for AST macros", but I 
 deleted it :D ).  But Jacob is on to something. We already pass around 
 symbols with `alias`.  Can that be extended somehow to pass around 
 expressions too (i.e. expressions as rvalues)?  That would probably help 
 address the issue of printing assert expressions as well 
 (https://github.com/dlang/dmd/pull/8517)


I think we can't exactly pass around expressions in the general case, 
because it's too fraught with ambiguity.

For example, if foo accepts an alias, then what is foo!(a + b)? Today, 
if a and b are compile-time known, then it's the sum of a and b. If a or 
b cannot be determined at compile time it's an error.

If foo!(a + b) all of a sudden compiled for runtime variables, but did 
something wildly different, we would probably have issues.

I think we would need a new type of alias. My hope was that we could let 
this remain without syntax, and just say it's only accessible if you use 
string interpolation. This makes the change simpler to implement, and 
narrowly focused.

However, if I had to give it a name (and syntax), I'd say it's a lazy alias.


 
 As for myself, I don't see much value in forcing users to add `mixin` at 
 the site of a mixin template instantiation, but there's a lot I don't 
 see, so I may just be missing something.
 ```
 import std.stdio;
 
 mixin template Foo(T)
 {
      T x = 5;
 }
 
 extern(C) void main()
 {
      // Why do we need `mixin` here?
      // Why can't we just write `Foo!int;`?
      // It's already yelling at us with a ! operator.
      mixin Foo!int;
      writeln(x);
 }
 ```


I think the reason is that you are giving the template free access to 
ALL your scope. My expectation of the string interpolation mechanism is 
that it only gives access to the variables or expressions passed to it. 
I think it's probably a good thing to require mixin for a flag to look 
for, even if it's ugly.

vibe.d does string interpolation in it's diet templates, but the rub is 
that you have to pass in an alias to the variables you want to be 
visible in the interpolation. So it requires a very un-DRY solution, and 
also doesn't work with arbitrary expressions.


 Simen Kjærås mentioned a few things when we were discussing ways to 
 implement properties in the library at 
 https://forum.dlang.org/post/jqcqsriizuewdqotbnop forum.dlang.org  There 
 might be some feature to be added, or limitation removed, that will 
 permit help both the string interpolation case and the properties case.
 
 I know I'm just thinking out loud and not contributing much, but I can't 
 help but feel that there's something common in all these ideas that just 
 hasn't been identified yet.  And no, Jacob, not AST macros (Actually, 
 that would probably do it, but I think we've already been there) :D


As I said, there's already possible libraries that allow what we want. 
It's the usage syntax that is the pain-point I would say.

-Steve
Dec 06 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 06, 2018 at 10:29:11PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 vibe.d does string interpolation in it's diet templates, but the rub
 is that you have to pass in an alias to the variables you want to be
 visible in the interpolation. So it requires a very un-DRY solution,
 and also doesn't work with arbitrary expressions.


I've discovered that since alias parameters accept module symbols, I can
pass my module to a Diet template, and then the template has free access
to everything in the module. :-D  E.g.,

	// mymodule.d
	int x;
	int y;
	...
	render!("mytemplate.dt", mymodule);

	// mytemplate.dt
	html
	    body


If your render!"" call is inside a class, you can also pass in `this`,
then you can access all class members.


[...]

 As I said, there's already possible libraries that allow what we want.
 It's the usage syntax that is the pain-point I would say.


[...]

True. But we'll have to think of a convincing argument of why building
the syntax into the language is a good thing -- good enough to pass
W&A's scrutiny.  That's the difficult part.

There's also the matter of exactly what syntax we're going to use for
interpolated strings. Since it's going to be baked into the language,
any changes / tweaks will be a lot harder to make. So we have to get it
(almost) perfect on first try (or at least, a perfect subset that can be
extended later without breaking anything that depends on it), which is
not easy to achieve. There will probably be bikeshedding over the finer
details of syntax.


T

-- 
Тише едешь, дальше будешь.
Dec 06 2018
prev sibling next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 7 December 2018 at 03:29:11 UTC, Steven Schveighoffer 
wrote:


 As long as there is an insistence that 'if it's doable in the 
 library, the elegance is not worth the effort,' any proposal 
 will fail. We need to first convince those in charge think the 
 feature should be added because of the *human* impact. That was 
 the point of my longer answer.


Understood.  In Jonathan's PR Andrei challenged the merit of a 
language string interpolation feature because a library solution 
already existed, and it wasn't being heavily utilized.  I think 
that overlooks the fact that perhaps the reason the library 
solution is not heavily utilized is because its syntax is not 
very elegant.  But I think he also simply wanted the potential of 
a library solution to be thoroughly investigated.  That is what 
is supposed to happen in the DIP process.

I still think there could be some general, broadly useful 
features to add or limitations to remove so we could build 
something like this without having to justify it to anyone.


 Jacob mentioned AST macros (I knew that was coming, and even 
 had a note in my first draft say "Sorry, Jacob, except for AST 
 macros", but I deleted it :D ).  But Jacob is on to something. 
 We already pass around symbols with `alias`.  Can that be 
 extended somehow to pass around expressions too (i.e. 
 expressions as rvalues)?  That would probably help address the 
 issue of printing assert expressions as well 
 (https://github.com/dlang/dmd/pull/8517)


 I think we can't exactly pass around expressions in the general 
 case, because it's too fraught with ambiguity.


[..]


 However, if I had to give it a name (and syntax), I'd say it's 
 a lazy alias.


Yep, that's one of the features I had in mind.  We already have 
it at runtime; would be nice to have feature parity at 
compile-time.  Maybe it wouldn't do anything to help the problem 
at hand though.  I'll have to spend some time on it.

Mike
Dec 06 2018
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-12-07 04:29, Steven Schveighoffer wrote:


 vibe.d does string interpolation in it's diet templates, but the rub is 
 that you have to pass in an alias to the variables you want to be 
 visible in the interpolation. So it requires a very un-DRY solution, and 
 also doesn't work with arbitrary expressions.


I haven't used the diet templates but I have used views, the 
corresponding system, in Ruby on Rails quite a lot. There it's not 
required to pass in the variables to the view/template. But in my 
experience it's best to create an explicit object that will contain all 
the data you need from the view. This usually leads to better code than 
passing in a bunch of different modules, which you usually end up adding 
view specific methods to. Following this module you would only need to 
pass in one object/struct value.

-- 
/Jacob Carlborg
Dec 08 2018
prev sibling next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 00:43:43 UTC, Mike Franklin wrote:

 I know I'm just thinking out loud and not contributing much, 
 but I can't help but feel that there's something common in all 
 these ideas that just hasn't been identified yet.  And no, 
 Jacob, not AST macros (Actually, that would probably do it, but 
 I think we've already been there) :D

 Mike


The truth is, we already have a feature that allows us to do 
this, its "mixin".  I empathize with the feeling that there must 
be something else out there that allows us to do interpolation as 
a library, but I think you'll find there just isn't.

As D exists today, with a library solution, we need 2 extra 
"composed" function calls to `mixin` and the "interpolating" 
function. Plus we need an extra import to include the 
interpolating function.

import std.typecons : interp;
writeln(mixin(interp("a is $(a)")));

Even if we could make mixin a little more friendly, you still 
have:

import std.typecons : interp;
writeln(interp("a is $(a)").mixin);

These 2 extra calls are the difference between string 
interpolation spreading like an infectious disease and being 
completely useless. String interpolation in this form is 
laughable when you compare it to existing solutions:

writeln("a is ", a);
writeln("a is %s", a);

For comparison, here's the simple language solution for it:

writeln(i"a is $(a)");

Looking for a feature that can enable "interpolation as a 
library" as well as other constructs is the right question. 
You're looking for a feature that's almost as powerful as mixin 
but also doesn't require the caller to be so verbose.  This is 
where the rubber meets the road.  Sadly, no such feature can or 
should exist.  If you allow functions to have this power you are 
going to make the language impossibly hard to understand moving 
forward.  Any function now has the ability to reach into their 
caller's scope meaning you now have to understand the source of 
every function to know how it affects any and all of your local 
data.  There's no more isolation, local variables are now almost 
as bad as global variables **shudder**.

Interpolation is an interesting case because it can reach into 
the caller's scope, but all the code to access locally scoped 
state is inside the string being interpolated at the call site.  
Here's the way I see it:

    Allowing a library to write functions that reach into their 
caller's scope breaks the assumed isolation between functions.  
This means that no data is safe, any function could mess with 
your data at any time. We don't want that, so this means if we 
want interpolation to be a library, then the caller MUST invoke 
`mixin`. But for string interpolation to be adopted, it needs to 
be less verbose than current solutions which is impossible if it 
needs to use mixin. Thus we arrive at an impass.  We want to 
implement interpolation as a library but it can't be done well 
enough unless we break function encapsulation.  Assuming all of 
this logic is correct, the only conclusion is that interpolation 
requires language support.

All that being said, if someone does find a feature that allows 
interpolation as a library without mixin and without breaking 
function isolation then great.  But no feature has been found yet 
and my confidence that no such feature can exist is growing over 
time. At some point we'll have to admit the unicorn doesn't exist 
so let's stick a horn on our horse and paint it...it's not as 
pretty as the unicorn, but at least its real :)
Dec 06 2018
parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Friday, 7 December 2018 at 05:38:13 UTC, Jonathan Marler wrote:

 All that being said, if someone does find a feature that allows 
 interpolation as a library without mixin and without breaking 
 function isolation then great.  But no feature has been found 
 yet and my confidence that no such feature can exist is growing 
 over time. At some point we'll have to admit the unicorn 
 doesn't exist so let's stick a horn on our horse and paint 
 it...it's not as pretty as the unicorn, but at least its real :)


I've long wished for something like the __CALLER_CONTEXT__ that's 
been suggested elsewhere in this thread, but as has been pointed 
out, it's a maintenance nightmare.

An idea struck me, though: what if __CALLER_CONTEXT__ needed to 
be explicitly passed, and i"" is simply syntactic sugar?

Specifically, i"value: {a+b}" is lowered to 
interpolateString!("value: {a+b}", __traits(getContext)). 
interpolateString is a template in druntime that does the heavy 
lifting.

This would require two new features:

1) __traits(getContext), which returns a scope that can be passed 
to templates, and has some way of invoking mixins in the context 
of that scope (for stuff like i"value: ${a+b}", where simple 
member lookup isn't enough), possibly as context.mixin("a+b").

2) A lowering from i"{a+b}" to interpolateString!("{a+b}", 
__traits(getContext)).

This way we get a shiny new feature to play with and see if there 
are other valuable uses (__traits(getContext)), and a neat way to 
implement string interpolation without destroying every 
expectation of encapsulation. Strictly speaking, 
interpolateString could still do stupid things, but I'd argue we 
should be able to trust code in druntime.

--
   Simen
Dec 07 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-12-07 01:43, Mike Franklin wrote:


 I know I'm just thinking out loud and not contributing much, but I can't 
 help but feel that there's something common in all these ideas that just 
 hasn't been identified yet.  And no, Jacob, not AST macros (Actually, 
 that would probably do it, but I think we've already been there) :D


It's quite annoying to see so many suggestions for new language features 
and existing highly specialized features that could be solved with AST 
macros instead of new language features.

-- 
/Jacob Carlborg
Dec 08 2018
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Saturday, 8 December 2018 at 12:23:28 UTC, Jacob Carlborg 
wrote:

 It's quite annoying to see so many suggestions for new language 
 features and existing highly specialized features that could be 
 solved with AST macros instead of new language features.


It quite clear from walter that it will be a snowball chance in 
hell that AST macros will be accepted in the language. The AST 
macros DIP need to have overwhelming arguments and evidence in 
order to penetrate the steel fortress that is armed with 
artillery, militarily grade assault-rifles and helicopters that 
is Walter. Which from looking at the current 50dip, it does not 
have.

Unless you are going to committed to a writing such a dip, I 
think it is safe to say AST macros will never going to be 
implemented.
Dec 08 2018
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Thursday, 6 December 2018 at 21:20:54 UTC, Mike Franklin wrote:


 IMO the one thing that will likely get this across the finish 
 line is to generalize it:

 1) What is the fundamental missing language feature that is 
 preventing us from implementing interpolated strings in the 
 library?


AST macros :). That would give access to the caller's scope and 
it can replace the call expression with whatever the macro 
returns.


 2) What other use cases besides string interpolation would be 
 enabled by adding such a feature?


A whole lot of different use cases.

https://wiki.dlang.org/DIP50

--
/Jacob Carlborg
Dec 06 2018
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 19:12:52 UTC, Paul Backus wrote:

 If users want to opt in to that extra complexity, it can always 
 be made available as a library.


Well, maybe, but libraries can't get the info if it isn't there.

We might be able to alias this the FromInterpolation thing so it 
just works in most places where you would use something else... 
but indeed, perhaps would be nice to do it in __traits.

(Or just abandon it and settle for the basic thing, I am OK with 
that too.)


 The actual solution here is to use the type system to 
 distinguish between trusted and untrusted strings. E.g.,


If you do that, then you really must forbid any plain string to 
force the programmer to declare it...
Dec 06 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 6 December 2018 at 19:39:17 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 19:12:52 UTC, Paul Backus wrote:

 If users want to opt in to that extra complexity, it can 
 always be made available as a library.


 Well, maybe, but libraries can't get the info if it isn't there.


I'm not convinced that libraries *should* have access to that 
info. Can you imagine having to debug a function that works when 
you call it as `fun(i"foo$(bar)")`, but fails when you call it as 
`fun("foo", bar)`? Sure, you can tell people not to write code 
like that, but if you're going to take that position, why go out 
of your way to make it possible in the first place?


 The actual solution here is to use the type system to 
 distinguish between trusted and untrusted strings. E.g.,


 If you do that, then you really must forbid any plain string to 
 force the programmer to declare it...


Sure, there are ways to make it even safer. My point is just that 
this isn't a string interpolation problem, it's a problem of 
conflating two different kinds of data just because they happen 
to have the same representation.
Dec 06 2018
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 20:40:02 UTC, Paul Backus wrote:

 I'm not convinced that libraries *should* have access to that 
 info.



Yeah. I can imagine some cool stuff I'd do to play with it... but 
it isn't really important.


Let's go back to the simple proposal, turning it into the tuple 
like Jonathan Marler's PR.
Dec 06 2018
prev sibling next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Thursday, 6 December 2018 at 18:06:51 UTC, Adam D. Ruppe wrote:

 I would take it one step further and put the other stuff in a 
 wrapped type from the compiler, so the function receiving it 
 can static if and tell what it is, so

 i"foo $(foo)"
 would be

 tuple("foo ", FromInterpolation("foo", foo))


 so you can identify when something was passed vs being 
 literally in the string. And it included the name as a string 
 so we can do some other crazy stuff too.

 i"foo $(a + b)"

 FromInterpolation("a + b", a+b)

 so you can then print

 a + b = 4

 [...]


Very neat idea.
Dec 06 2018
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 1:06 PM, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 17:47:58 UTC, Andre Pany wrote:

 Does I understand your sql example right, although it looks like it is 
 prone for sql injection attacks, it isn't because you evaluate the 
 tuples and not use the string as whole?


 
 Yeah, since it is tuples the function itself gets to manage how they are 
 used, including doing some escaping, etc.
 
 I would take it one step further and put the other stuff in a wrapped 
 type from the compiler, so the function receiving it can static if and 
 tell what it is, so
 
 i"foo $(foo)"
 would be
 
 tuple("foo ", FromInterpolation("foo", foo))
 
 
 so you can identify when something was passed vs being literally in the 
 string. And it included the name as a string so we can do some other 
 crazy stuff too.
 
 i"foo $(a + b)"
 
 FromInterpolation("a + b", a+b)
 
 so you can then print
 
 a + b = 4
 
 
 i think that would be kinda cool - the stuff inside is passed as a code 
 string. So really generally speaking it would be
 
 
 tuple("string literal", FromInterpolation(code_as_string, mixin(code)), 
 " more string literal"); // and so on
 
 
 I think that would be seriously cool and quite useful. You can then see 
  From Interpolation as a type in there and know to call sql escape or 
 replace with ? and move the arg or whatever - the function can use it 
 all with CT reflection.


It would be useful, but I don't like the mechanism. It requires you have 
a lot more machinery to deal with the paramters, where as now it Just 
Works with things like writeln.

It could be an extra piece of data that is accessible from the 
compile-time tuple:

foo(i"a + b = $(a+b));

void foo(Params...)(Params p) {
    static assert(__traits(getInterpolation, p[1]) == "a+b");
    static assert(__traits(getInterpolation, p) == AliasSeq!("a + b = ", 
"a+b"));
}

Maybe __traits(getInterpolation, ...) returns empty sequence for 
non-interpolated portions? Or maybe it's an error?

Just thought of a cool other possibility, if the aliases are accessible 
in the function:

 formatSpec("%x") int a;

writeln(i"a is $a"); -> prints hex version of `a`

The downside of this is that there is potentially a separate 
instantiation for when string interpolation is used, vs. when the 
parameters are passed normally. Maybe that means it requires you use 
compile-time arguments. A small price to pay I would think.

-Steve
Dec 06 2018
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 12:47 PM, Andre Pany wrote:

 On Thursday, 6 December 2018 at 16:19:12 UTC, Steven Schveighoffer wrote:

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = ?", 
 aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = $dval 
 WHERE id = $id");


 Does I understand your sql example right, although it looks like it is 
 prone for sql injection attacks, it isn't because you evaluate the 
 tuples and not use the string as whole?


Yes, that's exactly right, the whole thing gets lowered into:

db.exec("UPDATE Foo SET a = ", aval, ", b = ", bval, ", c = ", cval, ", 
d = ", dval", " WHERE id = ", id);

The one thing that this requires, in order to not allocate a string to 
pass to the SQL engine, is direct protocol access, such as we have in 
mysql-native.

There are other possibilities too. For instance, it's possible we could 
somehow pass the strings for compile time, so the real SQL string is 
generated internally. But I don't know how that would look. Perhaps, you 
just pass the query string at compile time, and deal with the tuple to 
generate the string.

-Steve
Dec 06 2018
prev sibling next sibling parent reply o <o o.o> writes:
On Thursday, 6 December 2018 at 16:19:12 UTC, Steven 
Schveighoffer wrote:

 [...]
 Most languages I know of create a string with toString-ing the 
 interpolations.

 [...]

 The mixin/library solution is much less approachable. What's 
 awesome about the language solution is that it just works 
 without much extra understanding and verbosity, and need for 
 using mixins. mixins are cool, but are best tucked away behind 
 templates or generative functions. They shouldn't seen much in 
 user code. Saying we should use a library solution for this is 
 like saying we can replace foreach usage with a library foreach 
 that lowers to some for-loop and explicit delegates (or maybe a 
 mixin?). Yes, it could be done. No, it shouldn't be done. This 
 is one of those types of syntax sugar that should not be 
 ignored.


Well said.

I agree that lowering to a tuple opens up some cool 
possibilities, but what about the times when it is desired to 
just use string interpolation as just a plain string?
For example, if `i"a is ${a}."` is equivalent to 'tuple("a is ", 
a, ".")', then if you want to use the interpolated string as a 
string, you would have to do something like `text(i"a is 
${a}.")`. If this is the case, how much better is this over a 
library implementation, which would look like this: `interp!"a is 
${a}."`?

I am definitely not saying that a library solution is a better 
idea. I personally think that this only makes sense as a language 
feature. But I just think it would be best if `i"a is ${a}"` is 
more or less equivalent to `"a is "~a.to!string~"."`.

As you said, string interpolation is just (very nice) syntactic 
sugar over concatenation, so I think we should make this sugar as 
sweet as possible.
Dec 06 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 5:03 PM, o wrote:

 I agree that lowering to a tuple opens up some cool possibilities, but 
 what about the times when it is desired to just use string interpolation 
 as just a plain string?
 For example, if `i"a is ${a}."` is equivalent to 'tuple("a is ", a, 
 ".")', then if you want to use the interpolated string as a string, you 
 would have to do something like `text(i"a is ${a}.")`. If this is the 
 case, how much better is this over a library implementation, which would 
 look like this: `interp!"a is ${a}."`?


`text(...)` is the way to go. Or rather `i"a is $a".text`;

And no, it wouldn't be interp!"a is $a.", as you HAVE to put mixin in 
there (this is how scriptlike works).

Consider if you are actually writing a string interpolation to do a 
mixin, it would be:

mixin(mixin(interp!"..."));


 
 I am definitely not saying that a library solution is a better idea. I 
 personally think that this only makes sense as a language feature. But I 
 just think it would be best if `i"a is ${a}"` is more or less equivalent 
 to `"a is "~a.to!string~"."`.


I get what you are saying. But this solution is inferior for a few reasons:

1. It allocates. Maybe many times (lots of to!string calls allocate).
2. It loses information.
3. It depends on a library feature (i.e. unusable in betterC).


 As you said, string interpolation is just (very nice) syntactic sugar 
 over concatenation, so I think we should make this sugar as sweet as 
 possible.


It's sugar, but I'd prefer it to be D-flavored sugar. Expose what the 
developer wrote at compile-time, and let the language gurus work their 
magic. Concatenation is accessible, easily, but so are so many other 
things that make this very useful.

D is capable of doing so much, because of the ability to introspect, and 
discover at compile-time. I'm still freaking amazed that vibe.d can take 
a line like:

render!("myview.dt", foo, bar);

and process a DSL which has access to foo and bar, by those names, and 
outputs a compiled D function. I want to allow enabling such things with 
string interpolation too.

-Steve
Dec 06 2018
parent reply o <o o.o> writes:
On Thursday, 6 December 2018 at 22:24:50 UTC, Steven 
Schveighoffer wrote:

 `text(...)` is the way to go. Or rather `i"a is $a".text`;

 -Steve


I just tried this out (https://run.dlang.io/is/RmRBZ8), but I 
don't know why it doesn't work.

This:
```
import std.stdio: writeln;
import std.conv: text;
import std.typecons: tuple;

void main() {
	string name = "Jeff";
	writeln(tuple("my name is ", name).text);
}
```
Prints "Tuple!(string, string)("my name is ", "Jeff")", not "My 
my name is Jeff".
Dec 06 2018
next sibling parent Dennis <dkorpel gmail.com> writes:
On Thursday, 6 December 2018 at 22:54:54 UTC, o wrote:

 Prints "Tuple!(string, string)("my name is ", "Jeff")", not "My 
 my name is Jeff".


You're converting the tuple object to a string and passing that 
string as a single argument to writeln. Try:
```
writeln(tuple("my name is ", name).expand);
```
Dec 06 2018
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 6 December 2018 at 22:54:54 UTC, o wrote:

 import std.typecons: tuple;


That's a library tuple, which is just syntax salt over a struct.

When we say tuple here, we are actually referring to a compiler 
tuple, which is more like AliasSeq from std.meta - a magic type 
that represents an argument list.
Dec 06 2018
next sibling parent o <o o.o> writes:
On Thursday, 6 December 2018 at 23:02:43 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 22:54:54 UTC, o wrote:

 import std.typecons: tuple;


 That's a library tuple, which is just syntax salt over a struct.

 When we say tuple here, we are actually referring to a compiler 
 tuple, which is more like AliasSeq from std.meta - a magic type 
 that represents an argument list.


Thanks for the clarification. I know realize the real potential 
of using tuples here instead of toString-ing :-).

+1 for lowering to tuples.
Dec 06 2018
prev sibling next sibling parent o <o o.o> writes:
On Thursday, 6 December 2018 at 23:02:43 UTC, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 22:54:54 UTC, o wrote:

 import std.typecons: tuple;


 That's a library tuple, which is just syntax salt over a struct.

 When we say tuple here, we are actually referring to a compiler 
 tuple, which is more like AliasSeq from std.meta - a magic type 
 that represents an argument list.


Thanks for the clarification. Now this makes a lot more sense.
+1 for lowering to tuples/sequences. Disregard my previous 
messages about lowering to strings :-).
Dec 06 2018
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/6/18 6:02 PM, Adam D. Ruppe wrote:

 On Thursday, 6 December 2018 at 22:54:54 UTC, o wrote:

 import std.typecons: tuple;


 
 That's a library tuple, which is just syntax salt over a struct.


Syntax salt. Love that.

-Steve
Dec 07 2018
prev sibling next sibling parent o <o o.o> writes:
On Thursday, 6 December 2018 at 16:19:12 UTC, Steven 
Schveighoffer wrote:

 On 12/6/18 4:01 AM, Dennis wrote:

 On Thursday, 6 December 2018 at 02:14:12 UTC, Neia Neutuladh 
 wrote:

 However, that would also force druntime to include formatting 
 code that it currently lacks.


 
 Jonathan Marler's implementation [1] has a really nice 
 approach of lowering interpolated strings to tuples. You can 
 pass it to variadic functions like `writeln` or `text` and 
 bring your own formatting / memory allocation schemes, giving 
 better performance and flexibility.
 
 [1] https://github.com/dlang/dmd/pull/7988


 I supported the concept[1] back when I read it from Dmitry 
 Olshansky[2].

 If I had known about the debate on the PR I would have jumped 
 in. But I don't have much to add. All that really dictates my 
 views is the experience I've had with other languages. However, 
 none of those other languages have the same concept that I know 
 of -- they basically just create a string with toString-ing the 
 interpolations.

 With the concept of lowering to a tuple, I'd love to use such a 
 thing for database queries.

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = 
 ?", aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = 
 $dval WHERE id = $id");

 The mixin/library solution is much less approachable. What's 
 awesome about the language solution is that it just works 
 without much extra understanding and verbosity, and need for 
 using mixins. mixins are cool, but are best tucked away behind 
 templates or generative functions. They shouldn't seen much in 
 user code. Saying we should use a library solution for this is 
 like saying we can replace foreach usage with a library foreach 
 that lowers to some for-loop and explicit delegates (or maybe a 
 mixin?). Yes, it could be done. No, it shouldn't be done. This 
 is one of those types of syntax sugar that should not be 
 ignored.

 +1000 from me, I'd love to see the PR merged, or the DIP 
 created, whatever needs to happen.

 -Steve

 [1] https://forum.dlang.org/post/odb9hk$2jqm$1 digitalmars.com
 [2] https://forum.dlang.org/post/ocut06$261n$1 digitalmars.com


Can you add some examples/explanations about the database 
querying to the DIP?
Dec 09 2018
prev sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Thursday, 6 December 2018 at 16:19:12 UTC, Steven 
Schveighoffer wrote:

 With the concept of lowering to a tuple, I'd love to use such a 
 thing for database queries.

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = 
 ?", aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = 
 $dval WHERE id = $id");


I really like that, but I'd add a caveat: this syntax makes it 
harder for db.exec to know which arguments you give it are 
trusted compile-time inputs, and which arguments are unsafe 
runtime inputs.

I'd argue for a second syntax to be added as well, fi"formatted 
string". So:

db.exec(fi"UPDATE Foo SET a = $aval, b = $bval, c = $cval, WHERE 
id = $id");

would lower to:

db.exec("UPDATE Foo SET a = %s, b = %s, c = %s, WHERE id = %s", 
aval, bval, cval, id);

That way, db.exec would be able to interpret its first argument 
as trusted input and everything else as unsafe.

That syntax might be redundant if a builtin FromInterpolation 
struct is created. The thing is, there have been arguments 
against using a builtin struct, and a db.exec method built for 
interpolation would be incompatible with regular use, so this:

db.exec("UPDATE Foo SET a = ",aval,", b = ",bval,", c = ",cval,", 
WHERE id = ",id);

would not work, which would have to be documented, etc.

On the other hand, a fi"some string with $var" syntax could 
leverage any already existing db.exec method with a "C format" 
first parameter (including actual C functions?), without forcing 
the library implementation to differentiate between normal 
arguments and FromInterpolation arguments.
Dec 10 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/10/18 8:48 AM, Olivier FAURE wrote:

 On Thursday, 6 December 2018 at 16:19:12 UTC, Steven Schveighoffer wrote:

 With the concept of lowering to a tuple, I'd love to use such a thing 
 for database queries.

 For instance:

 db.exec("UPDATE Foo SET a = ?, b = ?, c = ?, d = ? WHERE id = ?", 
 aval, bval, cval, dval, id);

 vs.

 db.exec(i"UPDATE Foo SET a = $aval, b = $bval, c = $cval, d = $dval 
 WHERE id = $id");


 
 I really like that, but I'd add a caveat: this syntax makes it harder 
 for db.exec to know which arguments you give it are trusted compile-time 
 inputs, and which arguments are unsafe runtime inputs.


Yes, if you read down into this thread, Neia brought up the same issue.

The answer is -- pass them at compile time. I.e.:

db.exec!(i"UPDATE ...");

Now you have access to what are string literals and what are parameters. 
You can even build the query string at compile time if using a library 
that requires it all as one string.

-Steve
Dec 10 2018
prev sibling next sibling parent reply Arun Chandrasekaran <aruncxy gmail.com> writes:
On Thursday, 6 December 2018 at 00:10:56 UTC, o wrote:

 I really wish that D had string interpolation, and I can see 
 that a ton of other people also do. String interpolation has 
 been brought up before, but it has been ignored/put down every 
 time. Jonathan Marler even created a Pull Request 
 (https://github.com/dlang/dmd/pull/7988) to add this feature, 
 but he ended up closing it because it was getting nowhere. The 
 excuse that keeps being given for not adding this feature is 
 something along the lines of "There are ways to work around 
 this, so it is unnecessary to add it". If this is the case, 
 then why do people keep on requesting it again and again? Here 
 are just some cases of other people asking for string 
 interpolation:

 [...]


https://github.com/dlang/dmd/pull/7988#issuecomment-434100869 was 
saddening to see.

On one hand people talk about D community not having enough man 
power + wanting to have a corporate back-up, etc. On the other 
hand... :(
Dec 06 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 7 December 2018 at 02:47:01 UTC, Arun Chandrasekaran 
wrote:

 On one hand people talk about D community not having enough man 
 power + wanting to have a corporate back-up, etc. On the other 
 hand... :(


Yeah, D has plenty of workers. I rarely even bother trying to 
work with D's management though. My motivation was killed years 
ago.

I recently tried again though after the new PR manager actually 
pinged me! I fixed the PR.... and now it is sitting in silence 
again. Wolf, wolf, wolf, the boy cried.
Dec 06 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 03:02:02AM +0000, Adam D. Ruppe via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 02:47:01 UTC, Arun Chandrasekaran wrote:

 On one hand people talk about D community not having enough man
 power + wanting to have a corporate back-up, etc. On the other
 hand... :(


 
 Yeah, D has plenty of workers. I rarely even bother trying to work
 with D's management though. My motivation was killed years ago.
 
 I recently tried again though after the new PR manager actually pinged
 me! I fixed the PR.... and now it is sitting in silence again. Wolf,
 wolf, wolf, the boy cried.


The thing is, the manpower we lack is more in the area of reviewing and
merging PRs, not so much in coding up those PRs in the first place.  The
scope of D is huge, and covers many different areas of expertise;
reviewing PRs quite often requires expertise in a certain area in order
to do it justice, but there are only a scant few who can do the job,
usually just one person, who is also responsible for many other areas,
besides non-D-related responsibilities such as a full-time job.
Speaking as a Phobos committer, I simply don't have the expertise to
have the confidence to review many PRs that involve areas I'm not
familiar with, and my time is already too limited even in areas I *am*
confident to review.  As a result, I refrain from reviewing for fear of
making the wrong decision(s) -- or even if I do review the parts of the
PR I'm confident enough to review, I would refrain from merging unless
somebody else more competent reviews it.

Also, there's a lack of clear communication of expectations from W&A.
Some years ago, when I and a few others were actively merging PRs, at
one point Andrei stepped in and expressed unhappiness at the kind of
changes that were being merged. He had good reasons for it, and in
retrospect he did the right thing.  But OTOH he also did not give clear
enough directions as what exactly *is* desired / undesired, so, lacking
enough information to proceed, passivity set in, from the fear of
getting vetoed again.  Their general reticence on many smaller issues,
while understandable given how much is how their plate, nonetheless
forms a very bad combination with wanting everything to be perfect --
it's not always clear what their idea of "perfect" is, and when nobody
knows and W&A don't speak up, things just grind to a standstill.

The slow rate of adding new committers also does not help, given our
already lacking manpower and the general busy-ness of existing
committers. There are many more eager contributors submitting PRs than
there are committers who are trusted enough to review and merge them, so
we're bottlenecking in a bad way.

There's also too much of the perfect becoming the enemy of the good,
which seems to be a common malady in D circles, and that doesn't help
things either.

I think at some point, W&A are just going to have to loosen their rein
on things, and appoint and trust more (many more) delegates to make
decisions.  Keeping everything bottlenecking on just the two of them is
not a viable long-term strategy.  While they have acknowledged in the
past that they lack experience in management and are trying their best
with what they have, this situation has been going on for years, and the
improvements have been slow.  It's showing signs that this mode of
management isn't working very well for an open source project, and it
may be time to seriously reconsider how things are run, lest the
technical excellence of D gets dragged under by ineffective management.


T

-- 
"A one-question geek test. If you get the joke, you're a geek: Seen on a
California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs -
Natural Intelligence, Inc.
Dec 06 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 6 December 2018 at 00:10:56 UTC, o wrote:

 I really wish that D had string interpolation, and I can see 
 that a ton of other people also do. String interpolation has 
 been brought up before, but it has been ignored/put down every 
 time. Jonathan Marler even created a Pull Request 
 (https://github.com/dlang/dmd/pull/7988) to add this feature, 
 but he ended up closing it because it was getting nowhere. The 
 excuse that keeps being given for not adding this feature is 
 something along the lines of "There are ways to work around 
 this, so it is unnecessary to add it". If this is the case, 
 then why do people keep on requesting it again and again? Here 
 are just some cases of other people asking for string 
 interpolation:

 - https://forum.dlang.org/thread/c2q7dt$67t$1 digitaldaemon.com
 - 
 https://forum.dlang.org/thread/qpuxtedsiowayrhgyell forum.dlang.org
 - 
 https://forum.dlang.org/thread/ncwpezwlgeajdrigegee forum.dlang.org
 And there are 2 closed PRs for this:
 - https://github.com/dlang/dmd/pull/6703
 - https://github.com/dlang/dmd/pull/7988

 In my opinion, having a sizable amount of people request a 
 feature (about 100 out of 280 as of The State of D Survey 
 [https://dlang.typeform.com/report/H1GTak/PY9NhHkcBFG0t6ig]) is 
 a good enough to realize that maybe we should start thinking of 
 adding it to the language.

 I understand that this may involve a DIP process, which takes 
 time. Therefore I am willing to offer up $100 to someone if 
 they are willing to go through the DIP process for string 
 interpolation.

 And please don't mention Phobos' sorry excuse for string 
 interpolation:
 "a is: %s, b is: %s, sum is %s.".format(a, b, a + b)
 Is no better than
 "a is: "~a.to!string~"b is: "~b.to!string~" and the sum is: 
 "~(a+b).to!string~"."


It's better, the latter is much worse. Longer, harder, to read, 
uglier. AFAIC `.to!string` is an anti-pattern.


 This is just so much more readable, and maintainable:
 "a is: ${a}, b is: ${b}, and the sum is: ${a+b}."


So is this:

text("a is", a, ", b is ", b, " and the sum is: ", a + b);

Which works today.

The only use I'd have for string interpolation is when generating 
code (as a string) that spans multiple lines. Before this thread 
I wasn't aware that scriptlike had string interpolation. Now that 
I know it does, I don't know what I'd need "proper" (i.e. 
language assisted) SI.

Every language change is a cost, and therefore should justify its 
inclusion. I personally don't think that it is in this case just 
to make a niche use case slightly easier, and this coming from 
someone from that niche! From now on it's:

const code = mixin(interp!q{
     // stuff that I used to write with many %s and it was hard to 
match them up
});
Dec 07 2018
next sibling parent Dennis <dkorpel gmail.com> writes:
On Friday, 7 December 2018 at 17:11:35 UTC, Atila Neves wrote:

 So is this:

 text("a is", a, ", b is ", b, " and the sum is: ", a + b);

 Which works today.


Except you made a mistake: there is no whitespace between "a is" 
and a, so it will be glued together. Kind of hard to notice 
without string interpolation. ;)
Dec 07 2018
prev sibling next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 17:11:35 UTC, Atila Neves wrote:

 The only use I'd have for string interpolation is when 
 generating code (as a string) that spans multiple lines. Before 
 this thread I wasn't aware that scriptlike had string 
 interpolation. Now that I know it does, I don't know what I'd 
 need "proper" (i.e. language assisted) SI.

 Every language change is a cost, and therefore should justify 
 its inclusion. I personally don't think that it is in this case 
 just to make a niche use case slightly easier, and this coming 
 from someone from that niche! From now on it's:

 const code = mixin(interp!q{
     // stuff that I used to write with many %s and it was hard 
 to match them up
 });


Your reasoning is sound but we differ in opinion here.  In my 
opinion the cost is low.  The implementation is very isolated.  
It's a simple change to the lexer and a new function call in 
parse stage that returns a tuple, that's it. You also say the 
usefullness is "niche" but I would disagree.  If it's implemented 
in the language then it could potentially replace the majority of 
calls to writeln/writefln.  It's more efficient than writefln and 
easier to read/maintain than writeln.  You yourself made a 
mistake in your own example because writeln is so clumsy to use.  
That's how I would use it anyway, I can't speak for everyone else 
but I think many people see this benefit as well.
Dec 07 2018
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 05:11:35PM +0000, Atila Neves via Digitalmars-d wrote:
[...]

 Every language change is a cost, and therefore should justify its
 inclusion.  I personally don't think that it is in this case just to
 make a niche use case slightly easier, and this coming from someone
 from that niche! From now on it's:
 
 const code = mixin(interp!q{
     // stuff that I used to write with many %s and it was hard to match them
 up
 });


One thing I like about format() that string interpolation doesn't
(easily) give, is the ability to specify additional formatting
parameters, e.g., %6.2f, or %04X. You'd have to basically "uglify"
string interpolation in much the same ways in order to get equivalent
functionality, e.g., (hypothetical syntax) "Price is ${6.2:price}.".

The one thing missing from format(), as you point out above, is the lack
of named arguments for long format strings.  This could be fixed by
extending format() with named placeholders, so that you can do something
like:

	// Hypothetical syntax
	format(q"CODETEMPLATE
		void %{funcName}s(%{funcParams}(%s, %)) {
			%{boilerPlate}s
			for (i; 0 .. %{numIter}d)
			{
				%{loopBody}s
			}
		}
	CODETEMPLATE", NamedArgs([
		"funcName", "myFunc",
		"funcParams", ["int x", "float y", "string z"],
		"boilerPlate", generateBoilerplate(),
		"loopBody", generateFuncBody(),
		"numIter", 255
	]));

Basically, instead of sequential arguments (or numerically-indexed

associative array from which it can look up named arguments. This gets
rid of the "too many %s I can't figure out which argument is which"
problem, plus it makes your format string more readable and
maintainable. It allows the same argument to be referenced multiple
times, and arguments don't have to appear in the order they appear in
the format string (important for i18n uses).

Note that NamedArgs can be dispensed with if the language supported
named arguments (like in Python, `myfunc(width = 123, height = 234)`).


T

-- 
I am Ohm of Borg. Resistance is voltage over current.
Dec 07 2018
next sibling parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 12/7/18 11:26 AM, H. S. Teoh wrote:

 One thing I like about format() that string interpolation doesn't
 (easily) give, is the ability to specify additional formatting
 parameters, e.g., %6.2f, or %04X. You'd have to basically "uglify"
 string interpolation in much the same ways in order to get equivalent
 functionality, e.g., (hypothetical syntax) "Price is ${6.2:price}.".


This seems like it would fit with the existing language:

"Price is ${6.2.price}."

SomeFormattingRange price(double d) {..}
Dec 07 2018
prev sibling next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 18:26:51 UTC, H. S. Teoh wrote:

 On Fri, Dec 07, 2018 at 05:11:35PM +0000, Atila Neves via 
 Digitalmars-d wrote: [...]

 Every language change is a cost, and therefore should justify 
 its inclusion.  I personally don't think that it is in this 
 case just to make a niche use case slightly easier, and this 
 coming from someone from that niche! From now on it's:
 
 const code = mixin(interp!q{
     // stuff that I used to write with many %s and it was hard 
 to match them
 up
 });


 One thing I like about format() that string interpolation 
 doesn't (easily) give, is the ability to specify additional 
 formatting parameters, e.g., %6.2f, or %04X. You'd have to 
 basically "uglify" string interpolation in much the same ways 
 in order to get equivalent functionality, e.g., (hypothetical 
 syntax) "Price is ${6.2:price}.".

 The one thing missing from format(), as you point out above, is 
 the lack of named arguments for long format strings.  This 
 could be fixed by extending format() with named placeholders, 
 so that you can do something like:

 	// Hypothetical syntax
 	format(q"CODETEMPLATE
 		void %{funcName}s(%{funcParams}(%s, %)) {
 			%{boilerPlate}s
 			for (i; 0 .. %{numIter}d)
 			{
 				%{loopBody}s
 			}
 		}
 	CODETEMPLATE", NamedArgs([
 		"funcName", "myFunc",
 		"funcParams", ["int x", "float y", "string z"],
 		"boilerPlate", generateBoilerplate(),
 		"loopBody", generateFuncBody(),
 		"numIter", 255
 	]));

 Basically, instead of sequential arguments (or 

 amounts to a polymorphic associative array from which it can 
 look up named arguments. This gets rid of the "too many %s I 
 can't figure out which argument is which" problem, plus it 
 makes your format string more readable and maintainable. It 
 allows the same argument to be referenced multiple times, and 
 arguments don't have to appear in the order they appear in the 
 format string (important for i18n uses).

 Note that NamedArgs can be dispensed with if the language 
 supported
 named arguments (like in Python, `myfunc(width = 123, height = 
 234)`).


 T


Got that one covered:

import std.stdio;

auto formatHex(T)(T value)
{
     static struct Formatter
     {
         T value;
         void toString(scope void delegate(const(char)[]) sink)
         {
             import std.format : formattedWrite;
             writef("%x", value);
         }
     }
     return Formatter(value);
}

void main(string[] args)
{
     int a = 42;
     writeln("a is ", a);
     writeln("a is ", a.formatHex);
     /*
     writeln(i"a is $(a)");
     writeln(i"a is $(a.formatHex)");
     */
}
Dec 07 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 1:26 PM, H. S. Teoh wrote:

 On Fri, Dec 07, 2018 at 05:11:35PM +0000, Atila Neves via Digitalmars-d wrote:
 [...]

 Every language change is a cost, and therefore should justify its
 inclusion.  I personally don't think that it is in this case just to
 make a niche use case slightly easier, and this coming from someone
 from that niche! From now on it's:

 const code = mixin(interp!q{
      // stuff that I used to write with many %s and it was hard to match them
 up
 });


 
 One thing I like about format() that string interpolation doesn't
 (easily) give, is the ability to specify additional formatting
 parameters, e.g., %6.2f, or %04X. You'd have to basically "uglify"
 string interpolation in much the same ways in order to get equivalent
 functionality, e.g., (hypothetical syntax) "Price is ${6.2:price}.".


You'd need a valid D expression there with the given proposal (to lower 
to a tuple). Something like ${price.formatted!("%6.2s")}.

Or as I had suggested elsewhere, you could use UDAs:

 formatted!("%6.2s") double price;

It's not exactly easy, but it's doable. Plus you could alias it:

alias moneyf = formatted!("%6.2s");
${price.moneyf}

Which makes things actually quite pleasant.


 
 The one thing missing from format(), as you point out above, is the lack
 of named arguments for long format strings.  This could be fixed by
 extending format() with named placeholders, so that you can do something
 like:
 
 	// Hypothetical syntax
 	format(q"CODETEMPLATE
 		void %{funcName}s(%{funcParams}(%s, %)) {
 			%{boilerPlate}s
 			for (i; 0 .. %{numIter}d)
 			{
 				%{loopBody}s
 			}
 		}
 	CODETEMPLATE", NamedArgs([
 		"funcName", "myFunc",
 		"funcParams", ["int x", "float y", "string z"],
 		"boilerPlate", generateBoilerplate(),
 		"loopBody", generateFuncBody(),
 		"numIter", 255
 	]));
 
 Basically, instead of sequential arguments (or numerically-indexed

 associative array from which it can look up named arguments. This gets
 rid of the "too many %s I can't figure out which argument is which"
 problem, plus it makes your format string more readable and
 maintainable. It allows the same argument to be referenced multiple
 times, and arguments don't have to appear in the order they appear in
 the format string (important for i18n uses).


The benefit I see from named parameters is using them more than once. 
That snippet above is very unreadable IMO.

But just for fun, let's compare to string interpolation strawman:

	text(iq"CODETEMPLATE
		void ${funcName}(${funcParams.formatted!"%-(%s, %)"}) {
			${generateBoilerplate}
			for (i; 0 .. ${numIter})
			{
				${generateFuncBody}
			}
		}
	CODETEMPLATE");

Oh, and it requires zero library support. It just works, as if you 
called it with the parameters in the right place.

With:

alias argf = formatted!("%-(%s, %)";

then the function declaration becomes:
		void ${funcName}(${funcParams.argf}) {

-Steve
Dec 07 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 01:46:05PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 	text(iq"CODETEMPLATE
 		void ${funcName}(${funcParams.formatted!"%-(%s, %)"}) {
 			${generateBoilerplate}
 			for (i; 0 .. ${numIter})
 			{
 				${generateFuncBody}
 			}
 		}
 	CODETEMPLATE");
 
 Oh, and it requires zero library support. It just works, as if you
 called it with the parameters in the right place.


The irony is that formatted!"%-%(s, %)" is essentially std.format
wrapped in nice clothes, the very thing which we're trying to replace
with string interpolation, and the very example of my point, which is
that (standard) string interpolation syntax is limited in what it can
express before you have to resort to inline code. If we didn't have
std.format, you'd have to come up with various helper functions for
formatting stuff, which is essentially an escape hatch from the
simplistic syntax of string interpolation.  Without the std.format DSL,
the only thing we have left is "string with embedded code snippets".

Of course, we could extend the string interpolation syntax with
shorthands for common formatting operations, like array formatting,
width/precision specifiers, etc.. But then you end up reinventing the
std.format DSL (perhaps with a different syntax, which may or may not be
nicer).



 With:
 
 alias argf = formatted!("%-(%s, %)";
 
 then the function declaration becomes:
 		void ${funcName}(${funcParams.argf}) {


[...]

Again, the irony is that we're depending on std.format for the
formatted!... template.  So why go through the trouble of implementing
string interpolation instead of just using std.format to begin with?

Now, mind you, I've nothing against string interpolation, in the
language or as a library.  All I'm saying is that having a DSL does make
certain things easier to express, though the tradeoff is that you have
to learn (possibly obscure-looking) new syntax.  Dispensing with a
complex DSL leaves you with convenient basic formatting, but when you
need something more than basic formatting, you have to drop down to
code.

//

What would be potentially awesome, is if we take the best of both worlds
and combine them into a unified DSL that is both powerful (e.g. with
concise formatting capabilities like std.format) and readable (by
allowing in-place variable names). Then you could write something like
this:

	int idx = 20;
	string varname = "myMatrix";
	float[][] data = [
		[ PI, 1, 0 ],
		[ -1, -PI, 0 ],
		[ 0, 0, -1 ]
	];

			   "%{data}(\t[ %(%5.2f %) ]%|\n%)");

and it would output:


	[  3.14  1.00  0.00 ]
	[ -1.00 -3.14  0.00 ]
	[  0.00  0.00 -1.00 ]

This would be another application for passing function context as a
template parameter.  Once we have that in the language, the
implementation of formatNG could potentially look something like this:

	template formatNG(string fmt, context = __traits(getContext))
	{
		...
		enum varName = ...;
		result ~= formatItem(mixin("context." ~ varName));
		...
	}

where the default template argument fetches the caller's context by
default.


T

-- 
Do not reason with the unreasonable; you lose by definition.
Dec 07 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 2:52 PM, H. S. Teoh wrote:

 On Fri, Dec 07, 2018 at 01:46:05PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
 [...]

 	text(iq"CODETEMPLATE
 		void ${funcName}(${funcParams.formatted!"%-(%s, %)"}) {
 			${generateBoilerplate}
 			for (i; 0 .. ${numIter})
 			{
 				${generateFuncBody}
 			}
 		}
 	CODETEMPLATE");

 Oh, and it requires zero library support. It just works, as if you
 called it with the parameters in the right place.


 
 The irony is that formatted!"%-%(s, %)" is essentially std.format
 wrapped in nice clothes, the very thing which we're trying to replace
 with string interpolation, and the very example of my point, which is
 that (standard) string interpolation syntax is limited in what it can
 express before you have to resort to inline code.


My vision of formatted would be simply a pairing of a format specifier 
with a value. It would be up to the function to interpret that (i.e. 
text, or inlineformat or whatever you like).

But the point is you are coupling the format with the value in the same 
place. Which is the idea behind the "named parameters" that you have.

I don't want the language to do any sort of formatting at all. That's a 
library feature. We just need the interpolation feature to give us a 
nicer way to call it.

-Steve
Dec 07 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 03:13:09PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 My vision of formatted would be simply a pairing of a format specifier
 with a value. It would be up to the function to interpret that (i.e.
 text, or inlineformat or whatever you like).
 
 But the point is you are coupling the format with the value in the
 same place. Which is the idea behind the "named parameters" that you
 have.
 
 I don't want the language to do any sort of formatting at all. That's
 a library feature. We just need the interpolation feature to give us a
 nicer way to call it.


[...]

I see.  So that means what we really want is for the language to
translate interpolated strings into tuples that can then be processed by
library code. In other words, it's not really an interpolated string
anymore, it's a kind of "tuple literal" with convenient syntax. It has
general applicability, and can be used for all sorts of interesting
things beyond just interpolated strings.  Like your database query
example.  In fact, it can even form the basis for implementing my named
parameters idea.  Once we have it in tuple form, it can be used for all
sorts of things.  (Which makes me wonder if we can even make it a
replacement for std.meta.AliasSeq ...) It would be an enabler of cool
new D idioms, rather than mere syntactic sugar for a "niche" use case.

It also has limited surface area for troublesome / unwanted
interactions.  Implementing the scope-as-template-argument idea I
proposed would be a lot more dangerous, because it opens up the
possibility of innocent-looking function calls to access and modify
local variables that are not explicitly passed to it, thus breaking
function encapsulation in a pretty evil way.  Standardizing the syntax
in the language, where any accessed local symbols are clearly obvious in
the source representation, eliminates this possibility.

I think we need to sell it to W&A from this angle. :-D  This is a lot
more convincing than "PHP-style interpolated strings would be nice, can
we haz it plz, kthxbye".


T

-- 
This is not a sentence.
Dec 07 2018
next sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 20:59:51 UTC, H. S. Teoh wrote:

 [...]


Yes to everything you just said :)
Dec 07 2018
prev sibling next sibling parent o <o o.o> writes:
On Friday, 7 December 2018 at 20:59:51 UTC, H. S. Teoh wrote:

 I see.  So that means what we really want is for the language 
 to translate interpolated strings into tuples that can then be 
 processed by library code. In other words, it's not really an 
 interpolated string anymore, it's a kind of "tuple literal" 
 with convenient syntax. It has general applicability, and can 
 be used for all sorts of interesting things beyond just 
 interpolated strings.  Like your database query example.  In 
 fact, it can even form the basis for implementing my named 
 parameters idea.  Once we have it in tuple form, it can be used 
 for all sorts of things.  (Which makes me wonder if we can even 
 make it a replacement for std.meta.AliasSeq ...) It would be an 
 enabler of cool new D idioms, rather than mere syntactic sugar 
 for a "niche" use case.

 T


This is phrased **perfectly**. Would you mind adding it to the 
DIP 
(https://github.com/jash11/DIPs/blob/master/Drafts/1NNN-JSH.md)? 
It would be a great addition. Thanks so much!
Dec 07 2018
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 08/12/2018 9:59 AM, H. S. Teoh wrote:

 I think we need to sell it to W&A from this angle.:-D   This is a lot
 more convincing than "PHP-style interpolated strings would be nice, can
 we haz it plz, kthxbye".


Just a warning, what I'm seeing right now is compiler hooks into the 
language (I've wanted these before). It is very unlikely from what I 
remember what W&A have said that it would be accepted.

Whatever semantics+syntax that is came up with, you may need to be a bit 
careful so that it is clear what is going on to the compiler and user.
Dec 07 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 3:59 PM, H. S. Teoh wrote:

 On Fri, Dec 07, 2018 at 03:13:09PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
 [...]

 My vision of formatted would be simply a pairing of a format specifier
 with a value. It would be up to the function to interpret that (i.e.
 text, or inlineformat or whatever you like).

 But the point is you are coupling the format with the value in the
 same place. Which is the idea behind the "named parameters" that you
 have.

 I don't want the language to do any sort of formatting at all. That's
 a library feature. We just need the interpolation feature to give us a
 nicer way to call it.


 [...]
 
 I see.  So that means what we really want is for the language to
 translate interpolated strings into tuples that can then be processed by
 library code. In other words, it's not really an interpolated string
 anymore, it's a kind of "tuple literal" with convenient syntax. It has
 general applicability, and can be used for all sorts of interesting
 things beyond just interpolated strings.  Like your database query
 example.  In fact, it can even form the basis for implementing my named
 parameters idea.  Once we have it in tuple form, it can be used for all
 sorts of things.  (Which makes me wonder if we can even make it a
 replacement for std.meta.AliasSeq ...) It would be an enabler of cool
 new D idioms, rather than mere syntactic sugar for a "niche" use case.


I think we are all finally on the same page (which I didn't realize we 
weren't, sorry about that!)

The selling of the idea is the hard part, Jonathan Marler has already 
implemented it, and the spec is super-simple as you say.

I think part of the confusion here is that we are calling it string 
interpolation. It's not the same as what most languages use for string 
interpolation. It's really something more like "string formatted alias 
sequence" or "string interpolation tuple". That should be reflected in 
the actual DIP.

What's central to the utility is that it's based on string literals 
(runtime strings will NOT work here), and therefore, has the potential 
to be used in mixins, and in compile-time alias sequences. Sequences of 
expressions split by strings are so common in D, it just makes complete 
sense to me.

-Steve
Dec 07 2018
next sibling parent reply o <o o.o> writes:
On Friday, 7 December 2018 at 22:12:18 UTC, Steven Schveighoffer 
wrote:

 I think we are all finally on the same page (which I didn't 
 realize we weren't, sorry about that!)

 The selling of the idea is the hard part, Jonathan Marler has 
 already implemented it, and the spec is super-simple as you say.

 I think part of the confusion here is that we are calling it 
 string interpolation. It's not the same as what most languages 
 use for string interpolation. It's really something more like 
 "string formatted alias sequence" or "string interpolation 
 tuple". That should be reflected in the actual DIP.

 What's central to the utility is that it's based on string 
 literals (runtime strings will NOT work here), and therefore, 
 has the potential to be used in mixins, and in compile-time 
 alias sequences. Sequences of expressions split by strings are 
 so common in D, it just makes complete sense to me.

 -Steve


I sent you a github invite to the contributors group for the DIP. 
If you are okay with it, could you clarify what you said in the 
DIP? Thanks!
Dec 07 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 5:24 PM, o wrote:

 On Friday, 7 December 2018 at 22:12:18 UTC, Steven Schveighoffer wrote:

 I think we are all finally on the same page (which I didn't realize we 
 weren't, sorry about that!)

 The selling of the idea is the hard part, Jonathan Marler has already 
 implemented it, and the spec is super-simple as you say.

 I think part of the confusion here is that we are calling it string 
 interpolation. It's not the same as what most languages use for string 
 interpolation. It's really something more like "string formatted alias 
 sequence" or "string interpolation tuple". That should be reflected in 
 the actual DIP.

 What's central to the utility is that it's based on string literals 
 (runtime strings will NOT work here), and therefore, has the potential 
 to be used in mixins, and in compile-time alias sequences. Sequences 
 of expressions split by strings are so common in D, it just makes 
 complete sense to me.


 
 I sent you a github invite to the contributors group for the DIP. If you 
 are okay with it, could you clarify what you said in the DIP? Thanks!


Thanks, I got the invite, will respond there.

-Steve
Dec 07 2018
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Friday, 7 December 2018 at 22:12:18 UTC, Steven Schveighoffer 
wrote:

 On 12/7/18 3:59 PM, H. S. Teoh wrote:

 [...]


 I think we are all finally on the same page (which I didn't 
 realize we weren't, sorry about that!)

 [...]


I recall the same confusion back when I was implementing it.  The 
term "intupleated strings" came to mind...maybe a little to 
cheeky though :)
Dec 07 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 5:26 PM, Jonathan Marler wrote:

 On Friday, 7 December 2018 at 22:12:18 UTC, Steven Schveighoffer wrote:

 On 12/7/18 3:59 PM, H. S. Teoh wrote:

 [...]


 I think we are all finally on the same page (which I didn't realize we 
 weren't, sorry about that!)

 [...]


 
 I recall the same confusion back when I was implementing it.  The term 
 "intupleated strings" came to mind...maybe a little to cheeky though :)


Yes, I immediately read that as interpolated strings :) The human brain 
doesn't notice slightly different spellings 
(https://www.mrc-cbu.cam.ac.uk/people/matt.davis/cmabridge/)

I don't think it would have the desired effect, but it is clever.

-Steve
Dec 07 2018
prev sibling parent reply o <o o.o> writes:
On Friday, 7 December 2018 at 20:59:51 UTC, H. S. Teoh wrote:

 I think we need to sell it to W&A from this angle. :-D  This is 
 a lot more convincing than "PHP-style interpolated strings 
 would be nice, can we haz it plz, kthxbye".


 T


Also, see Andrei's comment: 
https://github.com/dlang/dmd/pull/7988#issuecomment-375760720
Dec 07 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment: 
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


He seems to also have the misunderstanding that was common 
earlier in this thread that tuple == std.typecons.

That is false.
Dec 07 2018
parent reply o <o o.o> writes:
On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment: 
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common 
 earlier in this thread that tuple == std.typecons.


This is probably going to confuse more people, so we should refer 
to them as "sequences" in the DIP. Also, I am calling it "String 
Sequence Literals" instead of "String Interpolation" because it 
is more broad - more than just interpolation can be done with 
this.
Dec 07 2018
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 8 December 2018 at 00:58:43 UTC, o wrote:

 On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment: 
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common 
 earlier in this thread that tuple == std.typecons.


 This is probably going to confuse more people, so we should 
 refer to them as "sequences" in the DIP. Also, I am calling it 
 "String Sequence Literals" instead of "String Interpolation" 
 because it is more broad - more than just interpolation can be 
 done with this.


For reference, the official documentation on dlang.org refers to 
them as "compile-time sequences" [1] or "AliasSeqs" [2].

Also, as long as we're bikeshedding, my vote for the title would 
be something like "String Syntax for Compile-Time Sequences", so 
that there's no potential for confusion between "String (Sequence 
Literals)" and "(String Sequence) Literals".

[1] https://dlang.org/articles/ctarguments.html
[2] https://dlang.org/spec/template.html#variadic-templates
Dec 07 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, December 7, 2018 5:58:43 PM MST o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment:
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common
 earlier in this thread that tuple == std.typecons.


 This is probably going to confuse more people, so we should refer
 to them as "sequences" in the DIP. Also, I am calling it "String
 Sequence Literals" instead of "String Interpolation" because it
 is more broad - more than just interpolation can be done with
 this.


Personally, I'd just use the term AliasSeq, because that's what the
associated template for them in Phobos is called. Just about anything else
is going to be confusing. It's really unfortunate that Walter originally
decided to call them tuples given that they aren't quite tuples, but there
really isn't a good term for them ultimately. As it is, AliasSeq was just
the best of a bad set of choices, but at least it's clear what you mean what
you use the term, whereas just about anything else breeds confusion.

- Jonathan M Davis
Dec 07 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 9:04 PM, Jonathan M Davis wrote:

 On Friday, December 7, 2018 5:58:43 PM MST o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment:
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common
 earlier in this thread that tuple == std.typecons.


 This is probably going to confuse more people, so we should refer
 to them as "sequences" in the DIP. Also, I am calling it "String
 Sequence Literals" instead of "String Interpolation" because it
 is more broad - more than just interpolation can be done with
 this.


 
 Personally, I'd just use the term AliasSeq, because that's what the
 associated template for them in Phobos is called. Just about anything else
 is going to be confusing. It's really unfortunate that Walter originally
 decided to call them tuples given that they aren't quite tuples, but there
 really isn't a good term for them ultimately. As it is, AliasSeq was just
 the best of a bad set of choices, but at least it's clear what you mean what
 you use the term, whereas just about anything else breeds confusion.


But it's not an AliasSeq. It's simply a list of items.

for example, I want this to work:

writeln(i"a + b = ${a+b}");

And this compiles:

writeln("a + b = ", a + b);

But this does not:

writeln(AliasSeq!("a + b = ", a + b));

-Steve
Dec 07 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 09:29:10PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 But it's not an AliasSeq. It's simply a list of items.
 
 for example, I want this to work:
 
 writeln(i"a + b = ${a+b}");
 
 And this compiles:
 
 writeln("a + b = ", a + b);
 
 But this does not:
 
 writeln(AliasSeq!("a + b = ", a + b));


[...]

Hmph.  I forgot about this limitation of compile-time tuples aka
AliasSeq: you cannot capture expressions this way. :-(

The best I could come up with was to replicate std.typecons.Tuple:

	import std.stdio;
	alias AliasSeq(T...) = T;
	struct Tuple(T...)
	{
		T t;
		alias t expand;
	}
	auto func(Args...)(Args args)
	{
		return Tuple!Args(args);
	}
	void main() {
		int a=1, b=2;

		// Desired effect
		writeln("Sum of ", a, " and ", b, " is ", a+b);

		// NG: doesn't compile:
		//writeln(AliasSeq!("Sum of ", a, " and ", b, " is ", a+b));

		// This works (but is ugly)
		writeln(func("Sum of ", a, " and ", b, " is ", a+b).expand);
	}


This certainly puts a damper in our proposal. :-(

How does Marler's implementation behave? Does it capture expressions? If
it does, we have good news (but we'll have to document this clearly,
since it would be unprecedented in the current language).  If not, we'll
have to think more carefully about how to implement this.


T

-- 
Doubtless it is a good thing to have an open mind, but a truly open mind should
be open at both ends, like the food-pipe, with the capacity for excretion as
well as absorption. -- Northrop Frye
Dec 07 2018
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 8 December 2018 at 02:49:46 UTC, H. S. Teoh wrote:

 Hmph.  I forgot about this limitation of compile-time tuples 
 aka AliasSeq: you cannot capture expressions this way. :-(


The problem with AliasSeq!() is the !() part - it is passing them 
as compile-time arguments, and thus it must be available at 
compile time *for the template it is being passed to*.

But the compiler itself has no such limitation. And as proof, 
consider std.traits.Parameters. You can set items in that to 
runtime values and pass them to functions... just not *templates* 
because the variables a+b need to be available.

Also, struct.tupleof works fine. We aren't capturing expressions 
per se, we are capturing the values of those evaluating those 
expressions. Passing it to a compile time thing would complain, 
but passing it to a runtime function would be fine, just like how

Template!(a+b)

only works if they can be CTFE'd, whereas

Function(a+b)

is totally fine
Dec 07 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 9:49 PM, H. S. Teoh wrote:

 On Fri, Dec 07, 2018 at 09:29:10PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
 [...]

 But it's not an AliasSeq. It's simply a list of items.

 for example, I want this to work:

 writeln(i"a + b = ${a+b}");

 And this compiles:

 writeln("a + b = ", a + b);

 But this does not:

 writeln(AliasSeq!("a + b = ", a + b));


 [...]
 
 Hmph.  I forgot about this limitation of compile-time tuples aka
 AliasSeq: you cannot capture expressions this way. :-(
 
 The best I could come up with was to replicate std.typecons.Tuple:
 
 	import std.stdio;
 	alias AliasSeq(T...) = T;
 	struct Tuple(T...)
 	{
 		T t;
 		alias t expand;
 	}
 	auto func(Args...)(Args args)
 	{
 		return Tuple!Args(args);
 	}


So this is completely different from an AliasSeq. An AliasSeq is a 
compile-time list, and ONLY allows symbols and values known at compile 
time. An expression like "a + b" cannot be aliased.

What your code is doing is making an AliasSeq of *types* and then 
assigning the given values to an instance of those types. That works, 
and is not dissimilar to what I expect the string list to do when used 
on a runtime function.

But we can't have this proposal without that support. So we have to say 
it's not exactly an AliasSeq.

I have a PR on the DIP right now, take a look, would appreciate input: 
https://github.com/jash11/DIPs/pull/3


 	void main() {
 		int a=1, b=2;
 
 		// Desired effect
 		writeln("Sum of ", a, " and ", b, " is ", a+b);
 
 		// NG: doesn't compile:
 		//writeln(AliasSeq!("Sum of ", a, " and ", b, " is ", a+b));
 
 		// This works (but is ugly)
 		writeln(func("Sum of ", a, " and ", b, " is ", a+b).expand);
 	}
 
 
 This certainly puts a damper in our proposal. :-(
 
 How does Marler's implementation behave? Does it capture expressions? If
 it does, we have good news (but we'll have to document this clearly,
 since it would be unprecedented in the current language).  If not, we'll
 have to think more carefully about how to implement this.


I'm yet unclear yet whether the implementation behaves that way, but if 
it doesn't we should just change it so it does. But he has said (a bit 
ambiguously) that it does do expressions (comment in that PR).

If we can't get arbitrary expressions to work at runtime, then this 
proposal is no good.

-Steve
Dec 07 2018
next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 8 December 2018 at 03:04:35 UTC, Steven 
Schveighoffer wrote:


 So this is completely different from an AliasSeq. An AliasSeq 
 is a compile-time list, and ONLY allows symbols and values 
 known at compile time. An expression like "a + b" cannot be 
 aliased.


That was the idea behind the `lazy alias` feature discussed 
earlier in this thread.  With that feature, I think expressions 
can be aliased.

`lazy void` already works for runtime arguments.  I think all 
that is required is `lazy alias` for compile-time arguments.

Mike
Dec 07 2018
prev sibling parent John Chapman <johnch_atms hotmail.com> writes:
On Saturday, 8 December 2018 at 03:04:35 UTC, Steven 
Schveighoffer wrote:

 I'm yet unclear yet whether the implementation behaves that 
 way, but if it doesn't we should just change it so it does. But 
 he has said (a bit ambiguously) that it does do expressions 
 (comment in that PR).

 If we can't get arbitrary expressions to work at runtime, then 
 this proposal is no good.

 -Steve


It definitely does do expressions. The implementation just calls 
"parseExpression" for stuff inside '$(…)'.

int fun(int n) { return n; }

string hal = "HAL";
assert(i"$(hal) $((() => 1_000)() * 8 + fun(1_000))".text == "HAL 
9000");
Dec 08 2018
prev sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Saturday, 8 December 2018 at 02:49:46 UTC, H. S. Teoh wrote:

 How does Marler's implementation behave? Does it capture 
 expressions? If it does, we have good news (but we'll have to 
 document this clearly, since it would be unprecedented in the 
 current language).  If not, we'll have to think more carefully 
 about how to implement this.


 T


Yes it captures expressions. I've gone ahead and reopened and 
rebased my PR so that people can try it out.  Once you build it, 
make sure to include the "-transition=interpolate" flag to enable 
string interpolation.  And take a look at the 
"test/runnable/istring.d" for examples.
Dec 07 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, December 7, 2018 7:29:10 PM MST Steven Schveighoffer via 
Digitalmars-d wrote:

 On 12/7/18 9:04 PM, Jonathan M Davis wrote:

 On Friday, December 7, 2018 5:58:43 PM MST o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment:
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common
 earlier in this thread that tuple == std.typecons.


 This is probably going to confuse more people, so we should refer
 to them as "sequences" in the DIP. Also, I am calling it "String
 Sequence Literals" instead of "String Interpolation" because it
 is more broad - more than just interpolation can be done with
 this.


 Personally, I'd just use the term AliasSeq, because that's what the
 associated template for them in Phobos is called. Just about anything
 else is going to be confusing. It's really unfortunate that Walter
 originally decided to call them tuples given that they aren't quite
 tuples, but there really isn't a good term for them ultimately. As it
 is, AliasSeq was just the best of a bad set of choices, but at least
 it's clear what you mean what you use the term, whereas just about
 anything else breeds confusion.


 But it's not an AliasSeq. It's simply a list of items.

 for example, I want this to work:

 writeln(i"a + b = ${a+b}");

 And this compiles:

 writeln("a + b = ", a + b);

 But this does not:

 writeln(AliasSeq!("a + b = ", a + b));


*sigh* So, we'd be creating yet another similar concept with this? That
seems really questionable. I can see why you'd want something like i"a + b =
${a+b}" to work, but we already have enough confusion around "tuples" as it
is without trying to add a third kind.

- Jonathan M Davis
Dec 07 2018
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 8 December 2018 at 03:11:10 UTC, Jonathan M Davis 
wrote:

 *sigh* So, we'd be creating yet another similar concept with 
 this?


No, this is the SAME concept. It is just that the library cannot 
express it fully due to a limitation of the interface.

What we're talking about here is the compiler's internal tuple, 
as seen in things like obj.tupleof (which is a public name btw)
Dec 07 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/7/18 10:11 PM, Jonathan M Davis wrote:

 On Friday, December 7, 2018 7:29:10 PM MST Steven Schveighoffer via
 Digitalmars-d wrote:

 On 12/7/18 9:04 PM, Jonathan M Davis wrote:

 On Friday, December 7, 2018 5:58:43 PM MST o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 23:56:48 UTC, Adam D. Ruppe wrote:

 On Friday, 7 December 2018 at 23:46:45 UTC, o wrote:

 Also, see Andrei's comment:
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


 He seems to also have the misunderstanding that was common
 earlier in this thread that tuple == std.typecons.


 This is probably going to confuse more people, so we should refer
 to them as "sequences" in the DIP. Also, I am calling it "String
 Sequence Literals" instead of "String Interpolation" because it
 is more broad - more than just interpolation can be done with
 this.


 Personally, I'd just use the term AliasSeq, because that's what the
 associated template for them in Phobos is called. Just about anything
 else is going to be confusing. It's really unfortunate that Walter
 originally decided to call them tuples given that they aren't quite
 tuples, but there really isn't a good term for them ultimately. As it
 is, AliasSeq was just the best of a bad set of choices, but at least
 it's clear what you mean what you use the term, whereas just about
 anything else breeds confusion.


 But it's not an AliasSeq. It's simply a list of items.

 for example, I want this to work:

 writeln(i"a + b = ${a+b}");

 And this compiles:

 writeln("a + b = ", a + b);

 But this does not:

 writeln(AliasSeq!("a + b = ", a + b));


 
 *sigh* So, we'd be creating yet another similar concept with this? That
 seems really questionable. I can see why you'd want something like i"a + b =
 ${a+b}" to work, but we already have enough confusion around "tuples" as it
 is without trying to add a third kind.


I don't believe it's much different from tuple.expand. But I don't know 
what the limitations of the existing tuple-like things are.

Really, I just wanted it to expand into "as if you typed all those 
things individually separated by commas".

-Steve
Dec 07 2018
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 8 December 2018 at 03:15:47 UTC, Steven 
Schveighoffer wrote:

 Really, I just wanted it to expand into "as if you typed all 
 those things individually separated by commas".


Sounds like a "Variadic Argument List of Strings" to me.

Mike
Dec 07 2018
parent Paul Backus <snarwin gmail.com> writes:
On Saturday, 8 December 2018 at 05:55:58 UTC, Mike Franklin wrote:

 On Saturday, 8 December 2018 at 03:15:47 UTC, Steven 
 Schveighoffer wrote:

 Really, I just wanted it to expand into "as if you typed all 
 those things individually separated by commas".


 Sounds like a "Variadic Argument List of Strings" to me.

 Mike


That's almost exactly what it is, except it's not just strings.

To anyone who's still confused by this, I cannot recommend 
strongly enough the "Compile-Time Sequences" article on 
dlang.org. [1] It explains very clearly what these "argument 
sequences" are and how they work, with copious examples.

[1] https://dlang.org/articles/ctarguments.html
Dec 07 2018
prev sibling parent reply o <o o.o> writes:
On Saturday, 8 December 2018 at 02:29:10 UTC, Steven 
Schveighoffer wrote:

 this compiles:

 writeln("a + b = ", a + b);

 But this does not:

 writeln(AliasSeq!("a + b = ", a + b));

 -Steve


As Paul Backus brought up in a PR, you can use 
`std.typecons.tuple`s instead, to use expressions. Example:

auto foo = tuple(i"a + b = $(a+b)");
Dec 08 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Dec 08, 2018 at 05:17:13PM +0000, o via Digitalmars-d wrote:

 On Saturday, 8 December 2018 at 02:29:10 UTC, Steven Schveighoffer wrote:

 this compiles:
 
 writeln("a + b = ", a + b);
 
 But this does not:
 
 writeln(AliasSeq!("a + b = ", a + b));
 
 -Steve


 
 As Paul Backus brought up in a PR, you can use `std.typecons.tuple`s
 instead, to use expressions. Example:
 
 auto foo = tuple(i"a + b = $(a+b)");


There is more than meets the eye here, that I think we should be aware
of, whether or not it affects the current prospective DIP.

Firstly, there's the compiler's internal representation that Walter
calls a "tuple" (N.B.: not to be confused with std.typecons.tuple) which
is basically a list of "stuff", where "stuff" can be any of: (1) a basic
type with a compile-time known value; (2) a type; (3) an alias to a
Symbol, where a Symbol can be anything that has an identifier associated
with it (basically, an entry in one of the symbol tables used by the
compiler to keep track of identifiers); (4) a function literal aka
lambda, which in my knowledge has some degree of special treatment,
though for the most part it behaves just like (3).

Secondly, there's std.meta.AliasSeq, which is defined as:

	alias AliasSeq(T...) = T;

The `T...` captures a template argument list, which is a "tuple" in
Walter's sense of the word, and gives it an identifier that can be
referred to in code (otherwise, there is no way to refer to the compiler
internal "tuple").  I had thought that AliasSeq ought to be able to
capture all instances of "tuple"s, but apparently I was wrong -- a
"tuple" apparently *can* contain an expression, but expressions cannot
be captured by AliasSeq because of grammatical / syntactical
restrictions placed on template argument lists.  Marler's PR apparently
*does* let interpolated strings capture expressions in a way that
AliasSeq cannot.

Thirdly, there's std.typecons.Tuple, of which std.typecons.tuple is
simply a thin wrapper mainly serving as syntactic sugar for constructing
instances of std.typecons.Tuple.  There's a very important difference
between std.typecons.Tuple and the compiler internal "tuple" in Walter's
sense of the word: std.typecons.Tuple is a *runtime* object that
contains a tuple of *values*.  In contrast, the compiler internal
"tuple" is (obviously) compile-time only.  It basically behaves like an
anonymous struct.  Note that because it is a runtime object, it CANNOT
be used with AST stage compile-time constructs like static if, static
foreach, template expansions, etc.. If you try to access its values in a
static if, you'll get an error. It *is* usable in CTFE, though, if it
can be initialized with compile-time known values.

What we need for this DIP is for interpolated strings to be lowered to
the first kind of tuple, the compiler-internal "tuple" that can capture
AST-accessible "stuff" like string literals and expressions.  I had
thought it was equivalent to AliasSeq, but apparently there's a
discrepancy, and AliasSeq cannot bind expressions which is a no-go. We
cannot use std.typecons.Tuple because it's generally frowned on for the
compiler to depend on Phobos (even though the ^^ operator "cheats" in
that respect -- but to be fair, that was a historical accident predating
the compiler/Phobos clean separation policy). Furthermore,
std.typecons.Tuple cannot be used in static if and other AST stage
constructs, so that's also a no-go.


T

-- 
I don't trust computers, I've spent too long programming to think that they can
get anything right. -- James Miller
Dec 09 2018
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 10 December 2018 at 05:43:48 UTC, H. S. Teoh wrote:

 On Sat, Dec 08, 2018 at 05:17:13PM +0000, o via Digitalmars-d 
 wrote:

 
 As Paul Backus brought up in a PR, you can use 
 `std.typecons.tuple`s instead, to use expressions. Example:
 
 auto foo = tuple(i"a + b = $(a+b)");


 There is more than meets the eye here, that I think we should 
 be aware of, whether or not it affects the current prospective 
 DIP.

 [...]

 What we need for this DIP is for interpolated strings to be 
 lowered to the first kind of tuple, the compiler-internal 
 "tuple" that can capture AST-accessible "stuff" like string 
 literals and expressions.  I had thought it was equivalent to 
 AliasSeq, but apparently there's a discrepancy, and AliasSeq 
 cannot bind expressions which is a no-go. We cannot use 
 std.typecons.Tuple because it's generally frowned on for the 
 compiler to depend on Phobos (even though the ^^ operator 
 "cheats" in that respect -- but to be fair, that was a 
 historical accident predating the compiler/Phobos clean 
 separation policy). Furthermore, std.typecons.Tuple cannot be 
 used in static if and other AST stage constructs, so that's 
 also a no-go.


 T


To be clear: the proposal is, and has always been (to my 
knowledge), that interpolated strings should lower to a 
compile-time sequence, and not a Phobos Tuple.

The point I was making, in the PR "o" mentioned, is that you can 
pass the compile-time sequence you get from the interpolated 
string to `std.typecons.tuple` in order to store the values of 
the expressions in a Phobos Tuple at runtime--as though you had 
typed:

     auto foo = tuple("a + b = ", a + b);

In other words, it's an example of usage, not part of the core 
proposal.
Dec 09 2018
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 11:46:45PM +0000, o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 20:59:51 UTC, H. S. Teoh wrote:

 I think we need to sell it to W&A from this angle. :-D  This is a
 lot more convincing than "PHP-style interpolated strings would be
 nice, can we haz it plz, kthxbye".
 
 
 T


 
 Also, see Andrei's comment:
 https://github.com/dlang/dmd/pull/7988#issuecomment-375760720


I think the DIP must take Andrei's comment into account. I.e., we should
create possible library solutions, and discuss the pros and cons, esp.
to enumerate the ways in which a language-backed implementation would be
superior.

A few specific points that come to mind:

- Mixins essentially grant the callee unrestricted access to the
  caller's scope.  This leads to the possibility of surprising behaviour
  (e.g., what looks like an innocent function call or interpolated
  literal causes local variables to change, that aren't directly passed
  to the function).  Mixins are also a last-resort feature that we
  generally don't prefer if less intrusive means are available to
  achieve the desired effect.

- Having the language segment the literal into a tuple of strings and
  aliases (as opposed to outright string interpolation) has
  applicability beyond mere string interpolations. Steven's database
  idea would be great to use to strengthen this point.  We could also
  think up some more advanced examples that embed, e.g., functions
  (rather than merely variables) that the callee can use to transform
  the input string segments in interesting ways that go far beyond what
  mere string interpolation does.  Code generation techniques come to
  mind (e.g., specify a function that generates a loop around a block of
  code specified in string form).

- Andrei's last point about needing to import std.typecons is false.
  Built-in tuples (aka AliasSeq) does not depend on
  std.typecons.AliasSeq; it's a compiler construct that AliasSeq merely
  gives a useful name for.  What we're really doing is turning an i"..."
  literal into a *template argument list*.

Andrei also mentioned nicer mixin syntaxes, which was also discussed in
this thread.  Our discussions should be rephrased and summarized in a
way that directly answers Andrei's points.


T

-- 
People tell me I'm stubborn, but I refuse to accept it!
Dec 07 2018
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 7 December 2018 at 18:46:05 UTC, Steven Schveighoffer 
wrote:

 But just for fun, let's compare to string interpolation 
 strawman:

 	text(iq"CODETEMPLATE
 		void ${funcName}(${funcParams.formatted!"%-(%s, %)"}) {
 			${generateBoilerplate}
 			for (i; 0 .. ${numIter})
 			{
 				${generateFuncBody}
 			}
 		}
 	CODETEMPLATE");

 Oh, and it requires zero library support. It just works, as if 
 you called it with the parameters in the right place.

 With:

 alias argf = formatted!("%-(%s, %)";

 then the function declaration becomes:
 		void ${funcName}(${funcParams.argf}) {

 -Steve


This is just plain awesome. I fail to see how anybody can be 
against this. The more so when you consider it only requires a 
small change to the frontend.

I especially like the formatted alias. It just keeps getting 
better.
Dec 07 2018
prev sibling parent reply o <o o.o> writes:
On Friday, 7 December 2018 at 17:11:35 UTC, Atila Neves wrote:

 Every language change is a cost, and therefore should justify 
 its inclusion. I personally don't think that it is in this case 
 just to make a niche use case slightly easier, and this coming 
 from someone from that niche!


Think about `foreach`, `foreach_reverse`, UFCS, WYSIWYG strings, 
and `switch`. These are all just so-called "niche cases". When 
you strip out all of the syntax "niches" from a language, you are 
more or less left with assembly. Hell, assembly itself is just 
sugar over pure machine code! So with this attitude, why not 
ditch D and program in machine language?

It's the same case for string interpolation. It *is* possible to 
live without it, but it is just damn nice when you have it.

---

As a side note, Many people have brought up some very good 
points, but the conversation looks like it is now starting to 
stall. I think we should create a DIP template at this point, 
just to have a place to throw out ideas, not committing to 
anything yet. Then, if we decide to actually go through with the 
DIP, we already have all the brainstorming done and in one place. 
Would someone else (more experienced) like to start the DIP, or 
should I?
Dec 07 2018
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Friday, 7 December 2018 at 19:52:41 UTC, o wrote:

Would someone else (more experienced) like to start the
 DIP, or should I?


If I have spare time I don't mind starting one. However if you 
really wanted this, you need start writing it yourself, and post 
it on the fourms for assistance and feedback. I am pretty sure 
other people here don't mind helping you writing the dip.

"Be the change that you wanted to see in the world".


-Alex
Dec 07 2018
parent o <o o.o> writes:
On Friday, 7 December 2018 at 20:12:17 UTC, 12345swordy wrote:

 If I have spare time I don't mind starting one. However if you 
 really wanted this, you need start writing it yourself, and 
 post it on the fourms for assistance and feedback. I am pretty 
 sure other people here don't mind helping you writing the dip.

 "Be the change that you wanted to see in the world".


 -Alex


I started a very rough draft of the DIP 
(https://github.com/jash11/DIPs/blob/master/Drafts/1NNN-JSH.md).

Everyone who is interested: please take a look and create a PR 
with any additions. I am completely new to the whole DIP process, 
so I would greatly appreciate all help and support with the whole 
process.

Jonathan Marler - If it is okay with you, could you write a 
little bit about your implementation in the "Description" section?
Dec 07 2018
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Dec 07, 2018 at 07:52:41PM +0000, o via Digitalmars-d wrote:

 On Friday, 7 December 2018 at 17:11:35 UTC, Atila Neves wrote:

 Every language change is a cost, and therefore should justify its
 inclusion. I personally don't think that it is in this case just to
 make a niche use case slightly easier, and this coming from someone
 from that niche!


 
 Think about `foreach`, `foreach_reverse`, UFCS, WYSIWYG strings, and
 `switch`. These are all just so-called "niche cases". When you strip
 out all of the syntax "niches" from a language, you are more or less
 left with assembly. Hell, assembly itself is just sugar over pure
 machine code! So with this attitude, why not ditch D and program in
 machine language?


I think the point isn't to throw out all "niche" uses, but to consider
the cost of a new feature *relative to the current state of the
language*.  It's clear that we want certain nice features and
abstractions that would allow us to not have to write assembly directly.
But what is less often recognized / acknowledged is that there's also
only a limited amount of "design space" available for you to put
features into, so to speak.  Furthermore, a good chunk of this "language
real estate" is already occupied by existing language features, so any
additional features needs to be carefully weighed against what's
currently there, in order not to "run out of design space" and start
incurring technical debt.

Well actually, to be more precise, *every* feature comes with a cost
(complexity, syntax, increase in learning curve, etc.), and the cost of
each additional feature compounds with the total cost of current
features (e.g., unexpected/undesired interactions with existing
features).  There's only a finite number of features you can add before
the cost begins to outweigh the benefits (too many features to learn /
remember, too many unexpected interactions between combinations of
features, too complex to implement in a straightforward way, resulting
in implementation bugs, etc.).  As the number of features grow, the cost
of adding even a seemingly-trivial feature also grows.  It doesn't mean
we should never add new features, but it needs to offer substantial
benefits that outweigh the cost of adding (and maintaining!) it.

It would be very nice if we could just throw everything and the kitchen
sink into the language, but it would end up being a monster of a
language (like C++ (j/k :-P)), unwieldy and overly complex.


[...]

 As a side note, Many people have brought up some very good points, but
 the conversation looks like it is now starting to stall. I think we
 should create a DIP template at this point, just to have a place to
 throw out ideas, not committing to anything yet. Then, if we decide to
 actually go through with the DIP, we already have all the
 brainstorming done and in one place. Would someone else (more
 experienced) like to start the DIP, or should I?


If you have the motivation right now, I think you should just go ahead
and draft it up.  The rest of us can help polish it, if need be.  The
worst scenario is if the person most motivated to do it doesn't do it,
then the rest, who are not as motivated, never get around to it. Then
nothing happens and we end up back in square one.


T

-- 
"How are you doing?" "Doing what?"
Dec 07 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 7 December 2018 at 19:52:41 UTC, o wrote:

 On Friday, 7 December 2018 at 17:11:35 UTC, Atila Neves wrote:

 Every language change is a cost, and therefore should justify 
 its inclusion. I personally don't think that it is in this 
 case just to make a niche use case slightly easier, and this 
 coming from someone from that niche!


 Think about `foreach`, `foreach_reverse`, UFCS, WYSIWYG 
 strings, and `switch`. These are all just so-called "niche 
 cases". When you strip out all of the syntax "niches" from a 
 language, you are more or less left with assembly. Hell, 
 assembly itself is just sugar over pure machine code! So with 
 this attitude, why not ditch D and program in machine language?

 It's the same case for string interpolation. It *is* possible 
 to live without it, but it is just damn nice when you have it.


Strawman.

My point isn't that syntax sugar isn't needed, is that in this 
case the problem is easily solved with a library (and has been), 
and doesn't apply to anything except for multiline code 
generation, which is a niche.
Dec 10 2018
next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 10 December 2018 at 10:55:05 UTC, Atila Neves wrote:


 My point isn't that syntax sugar isn't needed, is that in this 
 case the problem is easily solved with a library (and has 
 been), and doesn't apply to anything except for multiline code 
 generation, which is a niche.


It not quite a niche if you are dealing with sql scripts as part 
of your job description.

-Alex
Dec 10 2018
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 10 December 2018 at 10:55:05 UTC, Atila Neves wrote:

 Strawman.

 My point isn't that syntax sugar isn't needed, is that in this 
 case the problem is easily solved with a library (and has 
 been), and doesn't apply to anything except for multiline code 
 generation, which is a niche.


In languages that support it, string interpolation is used for 
many things other than multiline code generation.
Dec 10 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 10 December 2018 at 16:55:38 UTC, Paul Backus wrote:

 On Monday, 10 December 2018 at 10:55:05 UTC, Atila Neves wrote:

 Strawman.

 My point isn't that syntax sugar isn't needed, is that in this 
 case the problem is easily solved with a library (and has 
 been), and doesn't apply to anything except for multiline code 
 generation, which is a niche.


 In languages that support it, string interpolation is used for 
 many things other than multiline code generation.


I meant compared to std.conv.text. If it fits on one line then 
`text` is more than fine.
Dec 10 2018
next sibling parent Anonymouse <asdf asdf.net> writes:
On Monday, 10 December 2018 at 20:57:52 UTC, Atila Neves wrote:

 I meant compared to std.conv.text. If it fits on one line then 
 `text` is more than fine.


My use case is terminal colours in format patterns, which quickly 
become mental gymnastics and extremely bug prone/easy to get 
wrong. They're largely oneliners, and interpolation would be a 
huge help.

// string logtint, warningtint;
// Exception e;

logger.warningf("IRC Parse Exception: %s%s %s(at 
%1$s%4$s%3$s:%1$s%5$d%3$s)",
     logtint, e.msg, warningtint, e.file, e.line);

(Apologies if I misread your meaning)
Dec 10 2018
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 10 December 2018 at 20:57:52 UTC, Atila Neves wrote:

 On Monday, 10 December 2018 at 16:55:38 UTC, Paul Backus wrote:

 On Monday, 10 December 2018 at 10:55:05 UTC, Atila Neves wrote:

 Strawman.

 My point isn't that syntax sugar isn't needed, is that in 
 this case the problem is easily solved with a library (and 
 has been), and doesn't apply to anything except for multiline 
 code generation, which is a niche.


 In languages that support it, string interpolation is used for 
 many things other than multiline code generation.


 I meant compared to std.conv.text. If it fits on one line then 
 `text` is more than fine.


Compare:

     const expected = ...;
     const result = foo(x, y, z);

     // without interpolation
     assert(result == expected,
         text("`foo(", x, ", ", y, ", ", z, ")`: expected `", 
expected, "` but got `", result, "`"));

     // with library interpolation
     assert(result == expected, mixin(interp!
         "`foo($(x), $(y), $(z))`: expected `$(expected)` but got 
`$(result)`"));

     // with built-in interpolation syntax
     assert(result == expected,
         i"`foo($(x), $(y), $(z))`: expected `$(expected)` but got 
`$(result)`".text);

Even for a single-line string, the interpolated version has clear 
advantages.

I'll grant that the difference between interpolation as a library 
and as a language feature is a lot smaller than the difference 
between interpolation and plain-old `text`. So maybe getting 
`interp` into Phobos is a better line of attack than trying to 
sell W&A on new syntax.
Dec 10 2018
parent reply Sjoerd Nijboer <dlang sjoerdnijboer.com> writes:
On Monday, 10 December 2018 at 23:07:32 UTC, Paul Backus wrote:

 So maybe getting `interp` into Phobos is a better line of 
 attack than trying to sell W&A on new syntax.


I might be thinking something really silly, but since adding such 
a thing as string interpolation is pretty much (wanted) syntactic 
sugar, isn't it just possible to lower it? And if the answer to 
this is yes, why can't we have some much wanted syntactic library 
solutions as language features lowered in a compiler pass?
We can just maintain a separate set of lowerable language 
constructs into a combination of existing library functions 
instead of the true lowering of for loops and while loops and 
everything the like.
(we could lower a specific character before a string literal into 
a literal![STRING]; )
Isn't that more acceptable? Keeping syntactic sugar lowering 
separate from fundamental lowering? As far as I can see this 
would "promote" addition of syntactic sugar as a language feature 
because it can be maintained whitout a big impact on the language.
If certain syntactic sugar lowerings are deemed wanted, tested 
and found stable but can benefit from a performance increase by 
better compiler support using some other forms than just 
lowering, they might eventually be altered whitout impact to the 
user.
Also, these syntactic sugar language features might be 
disabled/enabled by a compiler flag or dub setting.
All with all, I don't see how this couldn't realistically be 
included into the language like a lot of syntactic sugar. I 
understand the argument against it, but the potential benefits I 
can see are just too big to ignore.
Dec 11 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 11 December 2018 at 13:49:17 UTC, Sjoerd Nijboer 
wrote:

 I might be thinking something really silly, but since adding 
 such a thing as string interpolation is pretty much (wanted) 
 syntactic sugar, isn't it just possible to lower it? And if the 
 answer to this is yes, why can't we have some much wanted 
 syntactic library solutions as language features lowered in a 
 compiler pass?
 We can just maintain a separate set of lowerable language 
 constructs into a combination of existing library functions 
 instead of the true lowering of for loops and while loops and 
 everything the like.
 (we could lower a specific character before a string literal 
 into a literal![STRING]; )


It sounds like what you're asking for is a way to define rules 
for "lowering" one kind of syntax (e.g., string literals) into 
another (e.g., calls to library functions or templates).

This feature exists in other programming languages. The technical 
name for it is "AST macros." It's a very powerful feature, 
because it allows the programmer to essentially build their own 
custom programming languages on top of the "base" one provided by 
the compiler. For the same reason, it can also make code very 
difficult to read and understand. If any piece of syntax can be 
"lowered" into anything else, the only way to understand the code 
is to know what all the lowering rules are, and that can get 
difficult when every programmer is free to add to the pile.

Because of those potential difficulties, Walter has rejected 
every previous proposal to add AST macros to D, and there's no 
evidence to suggest that he's changed his mind since the last 
one. So, while it's true that AST macros would solve the issue 
with string interpolation, they're not a realistic option if we 
want our proposal to be accepted.
Dec 11 2018
parent reply Sjoerd Nijboer <dlang sjoerdnijboer.com> writes:
On Tuesday, 11 December 2018 at 16:48:56 UTC, Paul Backus wrote:

 It sounds like what you're asking for is a way to define rules 
 for "lowering" one kind of syntax (e.g., string literals) into 
 another (e.g., calls to library functions or templates).


Maybe I explained it a littl weird, but I would really like D to 
not have AST macros that are accessible for the programmer. Like 
you explained, it makes the language into whatever the one who 
created the library wanted to be. Which probably isn't a good 
thing for a language like D.

I think that there might be realistically 5 or 8 or so syntactic 
sugar features people generally want in this language, and I 
think the compiler could provide that. This could be done using 
AST's, but should not be accessible to the programmer.
At most, I think this would add some compiler swithces like `dmd 
--syntaxextention=interpolation,null-conditional,etc...`
That way they would be defined by the D community, W&A wouldn't 
have to "deal" with it, the people who want their sugar have it, 
and the people who never ever want to touch it don't have to.

The biggest benefit I think of doing it this way is 
standardisation. Because if it were to be done with a library 
implementation, there will probably be a few libraries which will 
implement it with various syntax and in various stages of 
working. Furtheron, people would have to learn a library instead 
of a language feature, which on itsself isn't such big of a deal 
except that there are probably multiple libraries doing thesame 
thing, and libraries are harder to find documentation for. Its 
always annoying in a professional project to have to add external 
dependencies for small features. It becomes a liability and makes 
D less attractive.
Dec 11 2018
parent Paul Backus <snarwin gmail.com> writes:
On Tuesday, 11 December 2018 at 17:12:46 UTC, Sjoerd Nijboer 
wrote:

 The biggest benefit I think of doing it this way is 
 standardisation. Because if it were to be done with a library 
 implementation, there will probably be a few libraries which 
 will implement it with various syntax and in various stages of 
 working. Furtheron, people would have to learn a library 
 instead of a language feature, which on itsself isn't such big 
 of a deal except that there are probably multiple libraries 
 doing thesame thing, and libraries are harder to find 
 documentation for. Its always annoying in a professional 
 project to have to add external dependencies for small 
 features. It becomes a liability and makes D less attractive.


You can get the same benefit by putting the library version in 
the standard library.

I do agree that this is important, and I think more people would 
use `interp` if it came with Phobos rather than in a dub package.
Dec 11 2018
prev sibling next sibling parent Erik van Velzen <erik evanv.nl> writes:
On Thursday, 6 December 2018 at 00:10:56 UTC, o wrote:

 I really wish that D had string interpolation


I program in languages which have string interpolation but hardly 
use it.

Probably there is a better way of doing things if you're using it 
a lot.

For example for HTML there is hyperscript.
Dec 07 2018
prev sibling parent reply sighoya <sighoya gmail.com> writes:
What's the state nearly year later?
Oct 16 2019
parent Dennis <dkorpel gmail.com> writes:
On Wednesday, 16 October 2019 at 20:51:32 UTC, sighoya wrote:

 What's the state nearly year later?


There's a DIP proposing that i"" strings are lowered to 
printf-like format strings and arguments:
https://github.com/dlang/DIPs/pull/165

There's a competing proposal (not yet in DIP form) that allows 
more flexibility:
https://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_13.html

There's already some Draft review discussion, but once it hits 
community review I expect the best design will be fought out.
Oct 16 2019