www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - "string interpolation"

reply Amex <Amex gmail.com> writes:
I and many others write a code that uses string mixins simply to 
define a symbol based on others:

mixin(T~" "~id~" = "~value~");");


This is very ugly and it is impossible to debug properly.

Why not allivate this issue? Is is so common that there can be a 
short hand syntax that the compiler can decode naturally.

e.g.,

#T #id = #value;


Or whatever syntax one wants to device.

¡T¡¡id¡ = ¡value¡;

↕T ↕id = ↕value;


Whatever...

The compiler then types the symbol and tries to resolve it. If it 
is a string in it inserts it directly as a symbol.

Basically whatever the simplification it is effectively lowered 
in to a string mixin...

but because it is expected to be used in a statement or 
expression it can be parsed. Only the marked symbols are resolved 
and everything else must be a valid expression. Essentially the 
compiler resolves the symbols then parses the string as if it 
were typed in as direct code. Since the symbol substitution is 
well defined it can easily backtrack to pinpoint the error in the 
"mixin".

if id = "fdf";

↕"int" ↕id = ↕"3";

is the same as

int fdf = 3;


Of course, if T is a type then

↕T ↕id = ↕value;

is the same as

T ↕id = ↕value;
Jun 08 2019
next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 6/8/19 5:28 PM, Amex wrote:

 I and many others write a code that uses string mixins simply to define 
 a symbol based on others:
 
 mixin(T~" "~id~" = "~value~");");
 
 
 This is very ugly and it is impossible to debug properly.
 
 Why not allivate this issue? Is is so common that there can be a short 
 hand syntax that the compiler can decode naturally.
 


There is work being done on the idea of interpolated strings:

https://github.com/dlang/dmd/pull/7988
Jun 08 2019
parent reply Amex <Amex gmail.com> writes:
On Saturday, 8 June 2019 at 22:23:34 UTC, Nick Sabalausky 
(Abscissa) wrote:

 On 6/8/19 5:28 PM, Amex wrote:

 I and many others write a code that uses string mixins simply 
 to define a symbol based on others:
 
 mixin(T~" "~id~" = "~value~");");
 
 
 This is very ugly and it is impossible to debug properly.
 
 Why not allivate this issue? Is is so common that there can be 
 a short hand syntax that the compiler can decode naturally.
 


 There is work being done on the idea of interpolated strings:

 https://github.com/dlang/dmd/pull/7988


This is not what I'm talking about. Similar but different. I'm 
specifically talking about usage to replace string mixins.

I'm not really talking about string interpolation at all. It's 
symbol substitution.

I've already given examples.
Jun 08 2019
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Sunday, 9 June 2019 at 05:24:29 UTC, Amex wrote:

 On Saturday, 8 June 2019 at 22:23:34 UTC, Nick Sabalausky 
 (Abscissa) wrote:

 On 6/8/19 5:28 PM, Amex wrote:

 I and many others write a code that uses string mixins simply 
 to define a symbol based on others:
 
 mixin(T~" "~id~" = "~value~");");
 
 
 This is very ugly and it is impossible to debug properly.
 
 Why not allivate this issue? Is is so common that there can 
 be a short hand syntax that the compiler can decode naturally.
 


 There is work being done on the idea of interpolated strings:

 https://github.com/dlang/dmd/pull/7988


 This is not what I'm talking about. Similar but different. I'm 
 specifically talking about usage to replace string mixins.

 I'm not really talking about string interpolation at all. It's 
 symbol substitution.

 I've already given examples.


So it sounds like you want `#X` to lower to `mixin(X)`? So in 
your example

#T #id = #value;

would be

mixin(T) mixin(id) = mixin(value);


been proposed before, and I think the idea has merit.  However, 
as the language exists today there's no way to support the level 
of mixin you are suggesting.  Mixin only works at higher levels 
like expressions and statements, so you could do things like:

mixin("const a = 1 + 2 + 3");

and even things like:

const a = 1 + mixin("2 + 3");

but not things like this:

mixin("const a") = 1 + 2 + 3;

Or things like

const mixin("a") = 1 + 2 + 3;

Mixin can only appear at certain places in the grammar because 
the compiler needs to be able to analyze the code surrounding it 
before evaluating what's inside the mixin. This way you can use 
your own code at compile-time to generate what's inside the mixin 
without depending on what the mixin does to your code.

In any case, with Adam Ruppe's full string interpolation 
proposal, you're example can be done with:

mixin("$T $id = $value;");
Jun 09 2019
parent reply Amex <Amex gmail.com> writes:
On Sunday, 9 June 2019 at 07:05:26 UTC, Jonathan Marler wrote:

 On Sunday, 9 June 2019 at 05:24:29 UTC, Amex wrote:

 On Saturday, 8 June 2019 at 22:23:34 UTC, Nick Sabalausky 
 (Abscissa) wrote:

 On 6/8/19 5:28 PM, Amex wrote:

 I and many others write a code that uses string mixins 
 simply to define a symbol based on others:
 
 mixin(T~" "~id~" = "~value~");");
 
 
 This is very ugly and it is impossible to debug properly.
 
 Why not allivate this issue? Is is so common that there can 
 be a short hand syntax that the compiler can decode 
 naturally.
 


 There is work being done on the idea of interpolated strings:

 https://github.com/dlang/dmd/pull/7988


 This is not what I'm talking about. Similar but different. I'm 
 specifically talking about usage to replace string mixins.

 I'm not really talking about string interpolation at all. It's 
 symbol substitution.

 I've already given examples.


 So it sounds like you want `#X` to lower to `mixin(X)`? So in 
 your example

 #T #id = #value;

 would be

 mixin(T) mixin(id) = mixin(value);


 has been proposed before, and I think the idea has merit.  
 However, as the language exists today there's no way to support 
 the level of mixin you are suggesting.  Mixin only works at 
 higher levels like expressions and statements, so you could do 
 things like:

 mixin("const a = 1 + 2 + 3");

 and even things like:

 const a = 1 + mixin("2 + 3");

 but not things like this:

 mixin("const a") = 1 + 2 + 3;

 Or things like

 const mixin("a") = 1 + 2 + 3;

 Mixin can only appear at certain places in the grammar because 
 the compiler needs to be able to analyze the code surrounding 
 it before evaluating what's inside the mixin. This way you can 
 use your own code at compile-time to generate what's inside the 
 mixin without depending on what the mixin does to your code.

 In any case, with Adam Ruppe's full string interpolation 
 proposal, you're example can be done with:

 mixin("$T $id = $value;");



There is nothing that would stop the language from doing what I'm 
suggesting...


You agree that every statement S could be rewritten to 
mixin("S"); and then evaluated by the compiler?

If so, then it can do what I suggest.


or whatever).

2. If the statement has such a symbol then the statement is 
wrapped in the mixin but all the symbols are expanded first.


.e.g,


$T $id = $value;

$ is found in front of a symbol.

simply expand symbols

mixin(T~" "~id~" = "~value~";");

with the appropriate to!string's when needed.

It is simply a rewrite rule that simplifies meta code and makes 
certain string mixins more readable by making them look like 
standard code. It makes it easier to debug. It is only for symbol 
substitution so there is always a 1 to 1 relation. Any errors in 
the resolved string can be mapped back in to the original 
statement.


It's purpose is to get rid of explicit mixin entirely for certain 
types of very common string mixin statements.



$T $id = $value;

can be lifted to mixin without any compiler knowledge. It is a 
direct map. The only issues may be in determining how to convert 
the symbols to strings... but in most cases to!string will work 
fine.
Jun 10 2019
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 10 June 2019 at 08:53:56 UTC, Amex wrote:

 It's purpose is to get rid of explicit mixin entirely for 
 certain types of very common string mixin statements.


Show me a real world example you've written where you would use 
this. I'm currently convinced it is a harmful antipattern...
Jun 10 2019
prev sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 10 June 2019 at 08:53:56 UTC, Amex wrote:

 On Sunday, 9 June 2019 at 07:05:26 UTC, Jonathan Marler wrote:

 On Sunday, 9 June 2019 at 05:24:29 UTC, Amex wrote:

 On Saturday, 8 June 2019 at 22:23:34 UTC, Nick Sabalausky 
 (Abscissa) wrote:

 On 6/8/19 5:28 PM, Amex wrote:

 I and many others write a code that uses string mixins 
 simply to define a symbol based on others:
 
 mixin(T~" "~id~" = "~value~");");
 
 
 This is very ugly and it is impossible to debug properly.
 
 Why not allivate this issue? Is is so common that there can 
 be a short hand syntax that the compiler can decode 
 naturally.
 


 There is work being done on the idea of interpolated strings:

 https://github.com/dlang/dmd/pull/7988


 This is not what I'm talking about. Similar but different. 
 I'm specifically talking about usage to replace string mixins.

 I'm not really talking about string interpolation at all. 
 It's symbol substitution.

 I've already given examples.


 So it sounds like you want `#X` to lower to `mixin(X)`? So in 
 your example

 #T #id = #value;

 would be

 mixin(T) mixin(id) = mixin(value);


 has been proposed before, and I think the idea has merit.  
 However, as the language exists today there's no way to 
 support the level of mixin you are suggesting.  Mixin only 
 works at higher levels like expressions and statements, so you 
 could do things like:

 mixin("const a = 1 + 2 + 3");

 and even things like:

 const a = 1 + mixin("2 + 3");

 but not things like this:

 mixin("const a") = 1 + 2 + 3;

 Or things like

 const mixin("a") = 1 + 2 + 3;

 Mixin can only appear at certain places in the grammar because 
 the compiler needs to be able to analyze the code surrounding 
 it before evaluating what's inside the mixin. This way you can 
 use your own code at compile-time to generate what's inside 
 the mixin without depending on what the mixin does to your 
 code.

 In any case, with Adam Ruppe's full string interpolation 
 proposal, you're example can be done with:

 mixin("$T $id = $value;");



 There is nothing that would stop the language from doing what 
 I'm suggesting...


 You agree that every statement S could be rewritten to 
 mixin("S"); and then evaluated by the compiler?


Yes I believe that's true.


 If so, then it can do what I suggest.

 1. Scans statement for any symbol starting with special 


 2. If the statement has such a symbol then the statement is 
 wrapped in the mixin but all the symbols are expanded first.


 .e.g,


 $T $id = $value;

 $ is found in front of a symbol.

 simply expand symbols

 mixin(T~" "~id~" = "~value~";");

 with the appropriate to!string's when needed.


Ok I see how this is possible now.


 It is simply a rewrite rule that simplifies meta code and makes 
 certain string mixins more readable by making them look like 
 standard code. It makes it easier to debug. It is only for 
 symbol substitution so there is always a 1 to 1 relation. Any 
 errors in the resolved string can be mapped back in to the 
 original statement.


It seems like a pretty specific set of syntax/semantics to add to 
the language. I can't think of alot of cases that would benefit 
from it. If this is a very common pattern it could be justified. 
Can you point to a few places in some real code that could 
benefit from this pattern?


 It's purpose is to get rid of explicit mixin entirely for 
 certain types of very common string mixin statements.


Using mixin code shouldn't happen too often. Its powerfull but 
also comes with issues. Usually they are hidden away inside 
templates/functions in a library invisible to the user. Using a 
keyword like mixin makes them easy to find, and since they 
shouldn't happen too often, the extra characters don't create 
much of an issue.

However, your suggestion isn't as powerful or as dangerous as a 
full blown mixin, which makes them less concerning to have to 

if it turns out to be useful enough then it could justify a 
language change. But I'm failing to think of useful cases for it. 
Can you share the cases you have thought this could be useful 
for? Pointing to existing code would be the best way to show them.
Jun 10 2019
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 8 June 2019 at 21:28:13 UTC, Amex wrote:

 I and many others write a code that uses string mixins simply 
 to define a symbol based on others:

 mixin(T~" "~id~" = "~value~");");


 This is very ugly and it is impossible to debug properly.


Specifically for the case for mixins they accept comma separated 
stuff. Stuff that is not a string will be converted through it 
`.stringof` property (e.g. types).


mixin(T, " ", id, " = ", value, ");");
Jun 08 2019
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 9 June 2019 at 00:53:23 UTC, Nicholas Wilson wrote:

 Specifically for the case for mixins they accept comma 
 separated stuff.


all that does is change ~ for , ...


 Stuff that is not a string will be converted through it 
 `.stringof` property (e.g. types).


And this is bad! .stringof is the wrong thing to do in the vast 
majority of mixin cases, especially now that we have static 
foreach.


 mixin(T, " ", id, " = ", value, ");");


This will actually generate wrong code in there coincidentally 
happens to be another thing in scope with the same name as T, and 
if not, it is liable to cause a compile error.

The correct way to write these things is

mixin("T " ~ id ~ " = value;");


In well-written mixin code, concatenation is infrequent 
(operators (think opBinary) and declaration names are the only 
ones I wouldn't question. there are a few exceptions, but you 
probably aren't one of them) and stringof is extremely rare. 
Thorughout all my code generation libraries, I have *zero* uses 
of .stringof inside mixin. In my code generation tests and 
samples, I have one use of it... and it predates `static 
foreach`. Let me show you this code.

The idea here was to do a pass-by-value lambda. Usage:

     auto foo(int x)  nogc
     {
         auto f = lambda!(x, q{ (int y) => x + y });
         return f;
     }

Note the  nogc part, which is why we aren't just using the 
built-in one and a string instead. You list the arguments you 
capture, then the lambda as a string.


Old version of the code:

---
     template lambda(Args...) {
     	import std.conv;
	import std.range;
	import std.string;

     	string evil() {
		// build the functor
		import std.meta;
		string code = "static struct anon {";
		foreach(i; aliasSeqOf!(iota(0, Args.length-1)))
			code ~= "typeof(Args[" ~ to!string(i) ~ "]) " ~ 
Args[i].stringof ~ ";";

		string func = Args[$-1];
		auto idx = func.indexOf("=>");
		if(idx == -1)
			throw new Exception("No => in lambda"); // or we could use one 
of the other styles

		auto args = func[0 .. idx];
		auto bod  = func[idx + 2 .. $];

		code ~= "auto opCall(T...)" ~ args ~ "{ return " ~ bod ~ "; }";

		code ~= "this(T...)(T t) {
			this.tupleof = t;
		};";

		code ~= "}";
		return code;
	}
     	mixin(evil());

         anon lambda() {
                 anon a;
                 // copy the values in
                 a.tupleof = Args[0 .. $-1];
                 return a;
         }
     }
---


It worked, but you can tell by my name of evil(), I didn't even 
like this code back when I wrote it the first time.

But I use a lot of concats and some stringof there. Let's try to 
rewrite this to eliminate that stuff with D's new features.

---
     template lambda(Args...) {

	static struct anon {
		static foreach(i; 0 .. Args.length - 1)
			mixin("typeof(Args[i]) " ~ __traits(identifier, Args[i]) ~ 
";");

		mixin("auto opCall(T...)"~Args[$-1][0 .. 
Args[$-1].indexOf("=>")]~" {
			return " ~ Args[$-1][Args[$-1].indexOf("=>") + 2 .. $] ~ ";
		}");
	}

	anon lambda() {
		anon a;
		// copy the values in
		a.tupleof = Args[0 .. $-1];
		return a;
	}
     }
---

No more stringof! No more conversion of values to string, meaning 
no more need for two of those imports! Yay. And compile time cut 
down by 1/4.

But, that is still a lot of concatenation, and it isn't a 
declaration name or an operator, so this code is still suspect to 
me. Can we do even better?

Of course! Let's replace that mixin generated opCall with 
something simpler:

        auto opCall(T...)(T t) {
            return mixin(Args[$-1])(t);
        }

Heeey :) it still works, and it is a LOT simpler. Zero imports 
needed now!

And you get better error messages and more compatibility with 
other D syntaxes. Let's take the type out.

With old code:

lll.d-mixin-53(53): Error: undefined identifier y
lll.d(15): Error: template lll.foo.lambda!(x, " (y) => x + y 
").anon.opCall cannot deduce function from argument types 
!()(int), candidates are:


But you defined y! So that error message doesn't help.

With new code:

lll.d(58): Error: function literal __lambda3(y) is not callable 
using argument types (int)
lll.d(58):        cannot pass argument _param_0 of type int to 
parameter y
lll.d(15): Error: template instance `lll.foo.lambda!(x, " (y) => 
x + y ").anon.opCall!int` error instantiating


Oh, the literal types aren't matching up. This message helps.


Let's change the lambda to (int y) { return x + y; }. You get a 
range error or a static assert error about the syntax, depending 
on how robust the implementation is.

But with the new code? No error message, it actually works. 
Instead of trying to slice up D code, I'm just compiling it as a 
complete chunk. And it still satisfies the  nogc constraint.


Mixins without stringof and minimizing concatenation is:

1) possible!

2) easier to write

3) easier to read

4) has fewer bugs


Instead of changing concat/stringof syntax, we should be 
educating people on the techniques to get rid of it. It is rarely 
justified.
Jun 08 2019