www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Where are we with thoughts on string interpolation and mixins?

reply aliak <something something.com> writes:
Hello,

I do a lot of work with mixins. But, it's usually very hard to 
read, and therefore reason about, and therefor review, and 
therefor maintain. So I'm wondering what people's thoughts are on 
how we can improve the experience of using mixins.

I'm thinking two things:
1) String interpolation
2) mixin syntax (this is not as big a deal as 1 IMO)

I was triggered by this in rust: 
https://github.com/bodil/typed-html - not completely relevant but 
it made me think: "oh, that's really nice and readable" and then 
I thought of D mixins and felt sad.

I'm referring to (this is plucked form random projects on github) 
code like this:

template MPQ_F_GET(char[] type, char[] name, char[] name2 = name) 
{
     const char[] MPQ_F_GET = type ~ " " ~ name2 ~ "() { " ~
             type ~ " ret; " ~
             "file_" ~ name ~ "(am, fileno, &ret); " ~
             "return ret;" ~
         "}";
}

Which, frankly, I had no idea what it was doing till I 
transformed it to this:

template MPQ_F_GET(char[] type, char[] name, char[] name2 = name) 
{
     const char[] MPQ_F_GET = q{
         $type $name2() {
             $type ret;
             file_$name(am, fileno, &ret);
             return ret;
         }
     };
}

Then I realized, ok, it's creating a function. Before that, I 
didn't see the "()" because it was hidden in noise. I was 
confused about "ret" because I assumed the mixin was generated 
from the template parameters. And I had no idea "file_" ~ name ~ 
"(am, fileno, &ret); " ~ was a function call O_o. Here's another 
one:

private char[] bindCode( char[] symbol ) {
    return symbol ~ " = cast( typeof( " ~ symbol ~
       " ) )m_sLibrary.getSymbol( `" ~ symbol ~ "` );";
}

As opposed to:

private char[] bindCode( char[] symbol ) {
    return q{
        $symbol = 
cast(typeof($symbol)m_sLibrary.getSymbol($symbol);
    };
}

And here's a one liner:

mixin("self."~option.VarName ~ " = " ~ "assumedValue.to!bool;");

mixin("self.${option.VarName} = ${assumedValue.to!bool};");

I've found that you waste A LOT of time just trying to figure out 
where ~ and " or ` go when you're writing a mixin. And don't even 
get me started on this pattern: `"` ~var ~ `";`; The number of 
times I've had a bug there and been thinking ... "why does this 
not compile" ... "ooooh a missing semicolon"...

This:

`"`~x`"~`~"` ~var ~ `";`;

As opposed to this:

`"$x" ~= "$var"`;

The difference to someone coming from interpolated strings land 
is quite significant. For people used to mixins, it's maybe less 
so? I'm not sure. I feel the pain everytime I try and mixin some 
code.

And I feel like this is almost like asking people to move to a 
ranged for loop because the C for(;;) is just inferior in 
readability, maintainability, debuggability, and reviewability 
... which is basically time, money, sanity, developer 
satisfaction, etc, etc. But at the same time, the C for-loop 
works "just fine".

I tried to rewrite this one as well [0], because I think that 
could look/read much better with interpolated string, but I 
couldn't understand it because there was just too much noise and 
my head hurt and I gave up and I threw my mac out the window and 
then I dissolved in to the virtual ether and wrote this post in 
my now new binary, and starved, form.

If it makes a difference at all as well, I've gotten this look at 
work 🤨when I say, yeah, there's no string interpolation in this 
language. And also "how old is this language?"

Ok, so now for the mixin syntax:

mixin(function("blah"));

What're thoughts on something like template mixin syntax so that 
we can get rid of those extra parens?:

mixin function("blah");

I couldn't really think of anything other than that or to allow 
for a trailing mixin decleration, i.e.: function("blah").mixin;

Thoughts?

Cheers,
- Ali

PS: I know that there're dub packages, but they a) they add yet 
another layer of indent, yet another set of parens or a bang, b) 
more uses of mixin (i.e. - mixin(mixin s!`"$x" ~= "$var"`"); - 
erm, I'm not even sure if that'd work, and c) take up a symbol 
which adds the friction of having to alias it away to something 
both nice, and unoccupied in your that particular file. Which 
also, btw, adds a maintainability nightmare and a code review 
nightmare - "wait what, interpolation is 'i' here but 'interp' in 
this file? And 's' in this other file??? - no thanks". Plus, 
dependencies for quick scripts are just overkill, and then you 
just go and use python instead.

PPS: I also believe that string interpolation was one of the top 
requested things in the survey this year? I may be misremembering 
though.

[0]: 
https://github.com/atilaneves/unencumbered/blob/7ecc9a811268edd6170bd294ac846d4da72acc8a/source/cucumber/reflection.d#L86
Nov 20 2018
next sibling parent reply Zoadian <no no.no> writes:
On Tuesday, 20 November 2018 at 09:19:41 UTC, aliak wrote:

 Hello,

 I do a lot of work with mixins. But, it's usually very hard to 
 read, and therefore reason about, and therefor review, and 
 therefor maintain. So I'm wondering what people's thoughts are 
 on how we can improve the experience of using mixins.

 [...]


While I do agree that string interp would be nice, I think it 
could be done in library code. Let me show you how I currently do 
something like it:

```
private enum LIQUID_ASGRAM_DEFINE_API(string PRE, T, TC, TI) = `
	struct %PRE%_s;
	alias %PRE% = %PRE%_s*;
	%PRE% %PRE%_create(uint nfft);
	void %PRE%_destroy(%PRE% q);
	void %PRE%_reset(%PRE% q);
	void %PRE%_set_scale(%PRE% q, float ref_lvl, float div);
	void %PRE%_set_display(%PRE% q, const(char)* ascii);
	void %PRE%_push(%PRE% q, %TI% x);
	void %PRE%_write(%PRE% q, %TI%* x, uint n);
	void %PRE%_execute(%PRE% q, char*  ascii, float * peakval, float 
* peakfreq);
	void %PRE%_print(%PRE% q);
`.replace("%PRE%", PRE).replace("%T%", 
T.stringof).replace("%TC%", TC.stringof).replace("%TI%", 
TI.stringof);

mixin(LIQUID_ASGRAM_DEFINE_API!("asgramcf", float, 
liquid_float_complex, liquid_float_complex));

mixin(LIQUID_ASGRAM_DEFINE_API!("asgramf", float, 
liquid_float_complex, float));
```
Nov 20 2018
next sibling parent reply aliak <something something.com> writes:
On Tuesday, 20 November 2018 at 10:09:40 UTC, Zoadian wrote:

 On Tuesday, 20 November 2018 at 09:19:41 UTC, aliak wrote:

 Hello,

 I do a lot of work with mixins. But, it's usually very hard to 
 read, and therefore reason about, and therefor review, and 
 therefor maintain. So I'm wondering what people's thoughts are 
 on how we can improve the experience of using mixins.

 [...]


 While I do agree that string interp would be nice, I think it 
 could be done in library code. Let me show you how I currently 
 do something like it:

 ```
 private enum LIQUID_ASGRAM_DEFINE_API(string PRE, T, TC, TI) = `
 	struct %PRE%_s;
 	alias %PRE% = %PRE%_s*;
 	%PRE% %PRE%_create(uint nfft);
 	void %PRE%_destroy(%PRE% q);
 	void %PRE%_reset(%PRE% q);
 	void %PRE%_set_scale(%PRE% q, float ref_lvl, float div);
 	void %PRE%_set_display(%PRE% q, const(char)* ascii);
 	void %PRE%_push(%PRE% q, %TI% x);
 	void %PRE%_write(%PRE% q, %TI%* x, uint n);
 	void %PRE%_execute(%PRE% q, char*  ascii, float * peakval, 
 float * peakfreq);
 	void %PRE%_print(%PRE% q);
 `.replace("%PRE%", PRE).replace("%T%", 
 T.stringof).replace("%TC%", TC.stringof).replace("%TI%", 
 TI.stringof);

 mixin(LIQUID_ASGRAM_DEFINE_API!("asgramcf", float, 
 liquid_float_complex, liquid_float_complex));

 mixin(LIQUID_ASGRAM_DEFINE_API!("asgramf", float, 
 liquid_float_complex, float));
 ```


You almost prove some of the same points I'm trying to make :p

"replace("%T%", T.stringof).replace("%TC%", TC.stringof)"

Are not needed as %T%  and %TC% are not in that string.

And probably all the places you call LIQUID_ASGRAM_DEFINE_API 
have extra arguments that negatively impact any kind of code 
reviews. And this would happen constantly. Code evolves, so 
people will change LIQUID_ASGRAM_DEFINE_API and review it, and 
debug it.

But on a side note, I do think that in some cases a .format() 
would be a lot more readable than string interop, especially 
where you are reusing one variable a lot of times. Like in your 
example:

private enum LIQUID_ASGRAM_DEFINE_API(string PRE, T, TC, TI) = `
	struct %1_s;
	alias %1 = %1_s*;
	%1 %1_create(uint nfft);
	void %1_destroy(%1 q);
	void %1_reset(%1 q);
	void %1_set_scale(%1 q, float ref_lvl, float div);
	void %1_set_display(%1 q, const(char)* ascii);
	void %1_push(%1 q, %2 x);
	void %1_write(%1 q, %2* x, uint n);
	void %1_execute(%1 q, char*  ascii, float * peakval, float * 
peakfreq);
	void %1_print(%1 q);
`.format(PRE, TI.stringof);

Though, as the number of placeholders increase, so does your 
mental burden.

Cheer,
- Ali
Nov 20 2018
parent reply Zoadian <no no.no> writes:
 You almost prove some of the same points I'm trying to make :p

 "replace("%T%", T.stringof).replace("%TC%", TC.stringof)"

 Are not needed as %T%  and %TC% are not in that string.

 And probably all the places you call LIQUID_ASGRAM_DEFINE_API 
 have extra arguments that negatively impact any kind of code 
 reviews. And this would happen constantly. Code evolves, so 
 people will change LIQUID_ASGRAM_DEFINE_API and review it, and 
 debug it.

 But on a side note, I do think that in some cases a .format() 
 would be a lot more readable than string interop, especially 
 where you are reusing one variable a lot of times. Like in your 
 example:


Well, it's a direct translation of a C header 
(https://github.com/jgaeddert/liquid-dsp/blob/master/include/liquid.h). That's
why I kept it that way.
```
#define LIQUID_ASGRAM_DEFINE_API(ASGRAM,T,TC,TI)
````


The point I was trying to make is: I think we can build something 
like this (https://run.dlang.io/is/NjHUah):

```
import std.stdio;
import std.array;
import std.traits;

string sinterp(string code, alias P1)() {
     string s = code;
     s = s.replace("$"~__traits(identifier, P1), P1);
     return s;
}

void main() {
     enum txt = "auto test";
     pragma(msg, sinterp!(`$txt = 3;`, txt));
     mixin(sinterp!(`$txt = 3;`, txt));
}
```
Nov 20 2018
parent aliak <something something.com> writes:
On Tuesday, 20 November 2018 at 12:47:34 UTC, Zoadian wrote:

 The point I was trying to make is: I think we can build 
 something like this (https://run.dlang.io/is/NjHUah):


Ok I may have slightly missed that point :o, sorry.

But yeah, I actually mentioned that in the post as well -> 
https://code.dlang.org/packages/stri

You mean that kinda thing?


 ```
 import std.stdio;
 import std.array;
 import std.traits;

 string sinterp(string code, alias P1)() {
     string s = code;
     s = s.replace("$"~__traits(identifier, P1), P1);
     return s;
 }

 void main() {
     enum txt = "auto test";
     pragma(msg, sinterp!(`$txt = 3;`, txt));
     mixin(sinterp!(`$txt = 3;`, txt));
 }
 ```


mixin(sinterp!(`$txt = 3;`));

Vs

mixin("$txt = 3;");

I mean ... Add context, add more strings, and more complication 
and the level of noise just ++ to point of it becoming 
unnecessarily stressful to review code no?
Nov 20 2018
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 20 November 2018 at 10:09:40 UTC, Zoadian wrote:

 replace("%T%", T.stringof).replace("%TC%", 
 TC.stringof).replace("%TI%", TI.stringof);


Your case is an exception to the general rule, but I wanna point 
out to other people reading that stringof in mixins is actually 
*usually* a mistake and you are *usually* better off just using 
the TC etc names directly in the mixin string. stringof breaks 
with different scopes and even some symbol names/strings in a new 
parsing context.

The big exception is function names... and tbh I kinda prefer to 
give it an internal name and only mix in a public alias for it. 
But that depends on the situation (and oh i wish we could change 
the name more easily with an external trick or something, would 
make static foreach more useful too!)

more info i wrote up a few years ago:

https://stackoverflow.com/questions/32615733/struct-composition-with-mixin-and-templates/32621854#32621854
Nov 20 2018
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 20 November 2018 at 09:19:41 UTC, aliak wrote:

 Hello,

 I do a lot of work with mixins. But, it's usually very hard to 
 read, and therefore reason about, and therefor review, and 
 therefor maintain. So I'm wondering what people's thoughts are 
 on how we can improve the experience of using mixins.

 I'm thinking two things:
 1) String interpolation
 2) mixin syntax (this is not as big a deal as 1 IMO)

 I was triggered by this in rust: 
 https://github.com/bodil/typed-html - not completely relevant 
 but it made me think: "oh, that's really nice and readable" and 
 then I thought of D mixins and felt sad.

 I'm referring to (this is plucked form random projects on 
 github) code like this:

 template MPQ_F_GET(char[] type, char[] name, char[] name2 = 
 name) {
     const char[] MPQ_F_GET = type ~ " " ~ name2 ~ "() { " ~
             type ~ " ret; " ~
             "file_" ~ name ~ "(am, fileno, &ret); " ~
             "return ret;" ~
         "}";
 }

 Which, frankly, I had no idea what it was doing till I 
 transformed it to this:

 template MPQ_F_GET(char[] type, char[] name, char[] name2 = 
 name) {
     const char[] MPQ_F_GET = q{
         $type $name2() {
             $type ret;
             file_$name(am, fileno, &ret);
             return ret;
         }
     };
 }

 Then I realized, ok, it's creating a function. Before that, I 
 didn't see the "()" because it was hidden in noise. I was 
 confused about "ret" because I assumed the mixin was generated 
 from the template parameters. And I had no idea "file_" ~ name 
 ~ "(am, fileno, &ret); " ~ was a function call O_o. Here's 
 another one:

 private char[] bindCode( char[] symbol ) {
    return symbol ~ " = cast( typeof( " ~ symbol ~
       " ) )m_sLibrary.getSymbol( `" ~ symbol ~ "` );";
 }

 As opposed to:

 private char[] bindCode( char[] symbol ) {
    return q{
        $symbol = 
 cast(typeof($symbol)m_sLibrary.getSymbol($symbol);
    };
 }

 And here's a one liner:

 mixin("self."~option.VarName ~ " = " ~ "assumedValue.to!bool;");

 mixin("self.${option.VarName} = ${assumedValue.to!bool};");

 I've found that you waste A LOT of time just trying to figure 
 out where ~ and " or ` go when you're writing a mixin. And 
 don't even get me started on this pattern: `"` ~var ~ `";`; The 
 number of times I've had a bug there and been thinking ... "why 
 does this not compile" ... "ooooh a missing semicolon"...

 This:

 `"`~x`"~`~"` ~var ~ `";`;

 As opposed to this:

 `"$x" ~= "$var"`;

 The difference to someone coming from interpolated strings land 
 is quite significant. For people used to mixins, it's maybe 
 less so? I'm not sure. I feel the pain everytime I try and 
 mixin some code.

 And I feel like this is almost like asking people to move to a 
 ranged for loop because the C for(;;) is just inferior in 
 readability, maintainability, debuggability, and reviewability 
 ... which is basically time, money, sanity, developer 
 satisfaction, etc, etc. But at the same time, the C for-loop 
 works "just fine".

 I tried to rewrite this one as well [0], because I think that 
 could look/read much better with interpolated string, but I 
 couldn't understand it because there was just too much noise 
 and my head hurt and I gave up and I threw my mac out the 
 window and then I dissolved in to the virtual ether and wrote 
 this post in my now new binary, and starved, form.

 If it makes a difference at all as well, I've gotten this look 
 at work 🤨when I say, yeah, there's no string interpolation in 
 this language. And also "how old is this language?"

 Ok, so now for the mixin syntax:

 mixin(function("blah"));

 What're thoughts on something like template mixin syntax so 
 that we can get rid of those extra parens?:

 mixin function("blah");

 I couldn't really think of anything other than that or to allow 
 for a trailing mixin decleration, i.e.: function("blah").mixin;

 Thoughts?

 Cheers,
 - Ali

 PS: I know that there're dub packages, but they a) they add yet 
 another layer of indent, yet another set of parens or a bang, 
 b) more uses of mixin (i.e. - mixin(mixin s!`"$x" ~= "$var"`"); 
 - erm, I'm not even sure if that'd work, and c) take up a 
 symbol which adds the friction of having to alias it away to 
 something both nice, and unoccupied in your that particular 
 file. Which also, btw, adds a maintainability nightmare and a 
 code review nightmare - "wait what, interpolation is 'i' here 
 but 'interp' in this file? And 's' in this other file??? - no 
 thanks". Plus, dependencies for quick scripts are just 
 overkill, and then you just go and use python instead.

 PPS: I also believe that string interpolation was one of the 
 top requested things in the survey this year? I may be 
 misremembering though.

 [0]: 
 https://github.com/atilaneves/unencumbered/blob/7ecc9a811268edd6170bd294ac846d4da72acc8a/source/cucumber/reflection.d#L86


Perhaps not too well known, mixin now supports multiple arguments 
of any type like pragma(msg) does and stringises each arg. So 
returning an AliasSeq of the fragments should be passable 
straight to mixin.
Nov 20 2018