www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - proposal: mixin functions to auto-mixin at call site

reply Timothee Cour <thelastmammoth gmail.com> writes:
I'd like to be able to declare a function with a special  mixin property
that will auto-mixin at call site:

 mixin string foo(){return some_string;}
void main(){
 foo; //behaves as mixin(foo()); if  mixin weren't provided
}

This is purely syntax sugar, as it could be done with a mixin at call site,
but is very useful in a number of scenarios:

Example 1:
I wrote a string parsing function 'embed' that allows to embed variables in
current scope in a string:
void main(){
int x1=11;
double x2=2.3;
assert(mixin("variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed) ==
"variables: x1=11, x2=2.3, sum=13.3");
}
The 'embed' function parses the input string, extracts the '$' tokens (with
a proper escape mechanism) and return a string of the form 'std.conv.text(
...)' with appropriate arguments so that the string can be mixed in at call
site as above.

Without the  mixin property we have:
mixin("variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed)

With the proposed  mixin property this would simplify to:
"variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed

Which is simpler and easier to read than:

text("variables: x1=",x1,", x2=",x2,", sum=",x1+x2);


There are many other use cases.

If for some reason we can't have  mixin special property, can we at least
have UFCS for mixin, so that we could write:
"variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin
Sep 02 2013
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 3 September 2013 at 04:15:17 UTC, Timothee Cour wrote:

 I'd like to be able to declare a function with a special  mixin 
 property
 that will auto-mixin at call site


Kenji had apparently implemented this:
https://github.com/D-Programming-Language/dmd/pull/459

But pulled out. This recent thread should sum it up:
http://forum.dlang.org/thread/yaasjclvyobpeftgwmke forum.dlang.org

Long story short, you are basically asking for macro. If we
allowed this, than anything could actually be code injection, and
mean anything.


 If for some reason we can't have  mixin special property, can 
 we at least
 have UFCS for mixin, so that we could write:
 "variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin


I'm not sure this us very interesting (why not though), since
nothing would ever come after the mixin.

I'd still rather have "typeof" be UFCS-able:
5.typeof.stringof.writeln();

If we only got to choose 1 that is ;)
Sep 02 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-09-03 07:11, monarch_dodra wrote:


 Kenji had apparently implemented this:
 https://github.com/D-Programming-Language/dmd/pull/459

 But pulled out. This recent thread should sum it up:
 http://forum.dlang.org/thread/yaasjclvyobpeftgwmke forum.dlang.org

 Long story short, you are basically asking for macro. If we
 allowed this, than anything could actually be code injection, and
 mean anything.


Yes, this seems to be another workaround for some kind of AST macros.

-- 
/Jacob Carlborg
Sep 03 2013
prev sibling next sibling parent reply Timothee Cour <thelastmammoth gmail.com> writes:
On Mon, Sep 2, 2013 at 10:11 PM, monarch_dodra <monarchdodra gmail.com>wrote:


 On Tuesday, 3 September 2013 at 04:15:17 UTC, Timothee Cour wrote:


 I'd like to be able to declare a function with a special  mixin property
 that will auto-mixin at call site


 Kenji had apparently implemented this:
 https://github.com/D-**Programming-Language/dmd/pull/**459<https://github.com/D-Programming-Language/dmd/pull/459>

 But pulled out. This recent thread should sum it up:
 http://forum.dlang.org/thread/**yaasjclvyobpeftgwmke forum.**dlang.org<http://forum.dlang.org/thread/yaasjclvyobpeftgwmke forum.dlang.org>



The last comment from Andrei in
https://github.com/D-Programming-Language/dmd/pull/459 is actually rather
supportive of such feature, and seemed to encourage a more thorough
discussion which never happened (AFAIK).
And unlike what is said in the thread above (last comment), the feature
indeed was implemented already, but not merged.




 Long story short, you are basically asking for macro. If we
 allowed this, than anything could actually be code injection, and
 mean anything.



No, only mixin statements and  mixin functions. And a regular function
calling a  mixin function would still be regular, eg not able to access
symbols in enclosing scope by name.
Given that properties can be queried at compile time, there is no
ambiguity. Proper tooling can reveal which statements are mixins if needed
(which doesn't imply that the language readability depends on having an
IDE).



  If for some reason we can't have  mixin special property, can we at least

 have UFCS for mixin, so that we could write:
 "variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin


 I'm not sure this us very interesting (why not though), since
 nothing would ever come after the mixin.


What do you mean by "nothing would ever come after the mixin" ?
You can have: some_string.embed.mixin.writeln;

Simple use case:
or: assert(foo(x)==bar(y), "error with $(foo(x)) and $(bar(y)) ".embed);

I'd still rather have "typeof" be UFCS-able:

 5.typeof.stringof.writeln();

 If we only got to choose 1 that is ;)


Features should be orthogonal unless there's a good reason not to. IMO,
assert, typeof, mixin, etc should all be UFCS-able. But this has been
brought up many times before.
Sep 03 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 3 September 2013 at 08:23:57 UTC, Timothee Cour wrote:

 Long story short, you are basically asking for macro. If we
 allowed this, than anything could actually be code injection, 
 and
 mean anything.



 No, only mixin statements and  mixin functions. And a regular 
 function
 calling a  mixin function would still be regular, eg not able 
 to access
 symbols in enclosing scope by name.


But that sill implies knowing exactly *what* is called, eg 
knowing the documentation of everything that is used, as opposed 
to just knowing "that *looks* like a function call that *does* 
something or other", but not worrying about it more than that.

Allowing implicit mixin adds an entire extra level of 
"dereference" when reading wode.


  If for some reason we can't have  mixin special property, can 
 we at least

 have UFCS for mixin, so that we could write:
 "variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin


 I'm not sure this us very interesting (why not though), since
 nothing would ever come after the mixin.


 What do you mean by "nothing would ever come after the mixin" ?
 You can have: some_string.embed.mixin.writeln;


Hadn't thought of that.
Sep 03 2013
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/3/13 2:11 AM, monarch_dodra wrote:

 On Tuesday, 3 September 2013 at 04:15:17 UTC, Timothee Cour wrote:

 I'd like to be able to declare a function with a special  mixin property
 that will auto-mixin at call site


 Kenji had apparently implemented this:
 https://github.com/D-Programming-Language/dmd/pull/459

 But pulled out. This recent thread should sum it up:
 http://forum.dlang.org/thread/yaasjclvyobpeftgwmke forum.dlang.org

 Long story short, you are basically asking for macro. If we
 allowed this, than anything could actually be code injection, and
 mean anything.


 If for some reason we can't have  mixin special property, can we at least
 have UFCS for mixin, so that we could write:
 "variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin


 I'm not sure this us very interesting (why not though), since
 nothing would ever come after the mixin.

 I'd still rather have "typeof" be UFCS-able:
 5.typeof.stringof.writeln();

 If we only got to choose 1 that is ;)


"Today if anyone writes or sees sees mixin(stuff) they are well warned 
that arbitrary code generation is taking place and are ready to deal 
with the consequences."

When you do "import foo.bar" you are importing arbitrary code... If you 
are not sure what that does, you go look it up. And it's the same for 
implicit mixins. I don't see the issue at all...
Sep 03 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 3 September 2013 at 18:26:57 UTC, Ary Borenszweig 
wrote:

 On 9/3/13 2:11 AM, monarch_dodra wrote:

 On Tuesday, 3 September 2013 at 04:15:17 UTC, Timothee Cour 
 wrote:

 I'd like to be able to declare a function with a special 
  mixin property
 that will auto-mixin at call site


 Kenji had apparently implemented this:
 https://github.com/D-Programming-Language/dmd/pull/459

 But pulled out. This recent thread should sum it up:
 http://forum.dlang.org/thread/yaasjclvyobpeftgwmke forum.dlang.org

 Long story short, you are basically asking for macro. If we
 allowed this, than anything could actually be code injection, 
 and
 mean anything.


 If for some reason we can't have  mixin special property, can 
 we at least
 have UFCS for mixin, so that we could write:
 "variables: x1=$x1, x2=$x2, sum=$(x1+x2)".embed.mixin


 I'm not sure this us very interesting (why not though), since
 nothing would ever come after the mixin.

 I'd still rather have "typeof" be UFCS-able:
 5.typeof.stringof.writeln();

 If we only got to choose 1 that is ;)


 "Today if anyone writes or sees sees mixin(stuff) they are well 
 warned that arbitrary code generation is taking place and are 
 ready to deal with the consequences."

 When you do "import foo.bar" you are importing arbitrary 
 code... If you are not sure what that does, you go look it up. 
 And it's the same for implicit mixins. I don't see the issue at 
 all...


But "import" is also explicit. Do you think it would be a good 
idea that a function be allowed to implicitly import things just 
by calling it? Because that basically what implicit mixin allows.
Sep 03 2013
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/3/13, Ary Borenszweig <ary esperanto.org.ar> wrote:

 When you do "import foo.bar" you are importing arbitrary code...


You are importing symbols. And when you do "foo()" you know you're
calling a function. With the change, you'll never know what foo()
does.

This feature is never going to fly, but people are just going to argue
this forever..
Sep 03 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-09-03 21:05, Andrej Mitrovic wrote:


 You are importing symbols. And when you do "foo()" you know you're
 calling a function. With the change, you'll never know what foo()
 does.

 This feature is never going to fly, but people are just going to argue
 this forever..


With properties you never know if you're invoking a method or accessing 
a field:

foo.data; // call method or access field?

-- 
/Jacob Carlborg
Sep 03 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 3 September 2013 at 19:10:12 UTC, Jacob Carlborg 
wrote:

 On 2013-09-03 21:05, Andrej Mitrovic wrote:


 You are importing symbols. And when you do "foo()" you know 
 you're
 calling a function. With the change, you'll never know what 
 foo()
 does.

 This feature is never going to fly, but people are just going 
 to argue
 this forever..


 With properties you never know if you're invoking a method or 
 accessing a field:

 foo.data; // call method or access field?


Which is exactly why parens-less calls and properties with side 
effects suck.
Sep 03 2013
parent reply Timothee Cour <thelastmammoth gmail.com> writes:
On Tue, Sep 3, 2013 at 12:13 PM, Dicebot <public dicebot.lv> wrote:


 On Tuesday, 3 September 2013 at 19:10:12 UTC, Jacob Carlborg wrote:


 On 2013-09-03 21:05, Andrej Mitrovic wrote:

  You are importing symbols. And when you do "foo()" you know you're

 calling a function. With the change, you'll never know what foo()
 does.

 This feature is never going to fly, but people are just going to argue
 this forever..


 With properties you never know if you're invoking a method or accessing a
 field:

 foo.data; // call method or access field?


 Which is exactly why parens-less calls and properties with side effects
 suck.


that's the whole point, it allows to transparently replace a field access
by a property function call without breaking client code. How else would
you do that?
Sep 03 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 02:26:27 UTC, Timothee Cour 
wrote:

 that's the whole point, it allows to transparently replace a 
 field access
 by a property function call without breaking client code. How 
 else would
 you do that?


You can't replace field access with function call transparently 
if it has side effects. In the end breakage may be even worse. 
For me only replacing  property fields with  property methods 
makes sense (with former prohibited to be taken address of and 
latter forced to be weakly pure).'
Sep 04 2013
parent reply Timothee Cour <thelastmammoth gmail.com> writes:
On Wed, Sep 4, 2013 at 2:23 AM, Dicebot <public dicebot.lv> wrote:


 On Wednesday, 4 September 2013 at 02:26:27 UTC, Timothee Cour wrote:


 that's the whole point, it allows to transparently replace a field access
 by a property function call without breaking client code. How else would
 you do that?


 You can't replace field access with function call transparently if it has
 side effects. In the end breakage may be even worse. For me only replacing
  property fields with  property methods makes sense (with former prohibited
 to be taken address of and latter forced to be weakly pure).'


IIRC, &foo.x with x a property should take address of return value of x(),
which will either fail to compile or do the right thing if it returns a
lvalue. So there's no undefined behavior as far as taking address of is
concerned when replacing field by property.

As for side effects, this is the reason one would go from field access to
property, eg populating some data upon 1st access to a field x. Sure it can
be misused, but I haven't seen a case in practice where it is misused.
Sep 04 2013
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 17:07:33 UTC, Timothee Cour 
wrote:

 IIRC, &foo.x with x a property should take address of return 
 value of x(),
 which will either fail to compile or do the right thing if it 
 returns a
 lvalue.


It is not possible because getters are not guaranteed to return 
lvalues. There is a quite elegant (in my opinion) proposal 
hanging around to allow annotating plain fields with  property so 
that compiler will prohibit doing any operation on them that may 
later break because of transition to actual properties, such as 
taking address.


 As for side effects, this is the reason one would go from field 
 access to
 property, eg populating some data upon 1st access to a field x. 
 Sure it can
 be misused, but I haven't seen a case in practice where it is 
 misused.


I haven't seen a case in practice when using normal method 
instead of a property was really harmful. Also modifying own 
members is legal for weakly pure method as far as I remember (it 
can modify arguments, including hidden `this`), so your case will 
actually work within my restrictions.
Sep 05 2013
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 9/3/13, Jacob Carlborg <doob me.com> wrote:

 With properties you never know if you're invoking a method or accessing
 a field:

 foo.data; // call method or access field?


Yeah but it does something with data on its own side. If this becomes
an implicit mixin, it could do something with code at the call site.
E.g.

auto st = getSecurityToken(...);
updateGui();  // what if this becomes a mixin and ends up reading 'st'
and displaying it on the screen?

It is equivalent to using globals everywhere in your codebase.
Sep 03 2013
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/3/13 5:47 PM, Andrej Mitrovic wrote:

 On 9/3/13, Jacob Carlborg <doob me.com> wrote:

 With properties you never know if you're invoking a method or accessing
 a field:

 foo.data; // call method or access field?


 Yeah but it does something with data on its own side. If this becomes
 an implicit mixin, it could do something with code at the call site.
 E.g.

 auto st = getSecurityToken(...);
 updateGui();  // what if this becomes a mixin and ends up reading 'st'
 and displaying it on the screen?

 It is equivalent to using globals everywhere in your codebase.


Why would anyone do that?

Do you use other people's source code without reading the documentation 
or reading the source code?

How would the author of "updateGui()" know that you named your variable 
"st"?

What if "updateGui()" does "rm -rf /" ?

What if "updateGui()" always does a null pointer dereference?

Ah, the language is too dangerous. I say we should remove function calls 
from the language.
Sep 04 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 14:44:50 UTC, Ary Borenszweig 
wrote:

 Why would anyone do that?


By an accident - mixins are not hygienic.


 Do you use other people's source code without reading the 
 documentation or reading the source code?


Yes, all the time. Documentation is always lacking and reading 
full 10 MLOC code base to tweak a single function is never an 
option.


 How would the author of "updateGui()" know that you named your 
 variable "st"?


He has no idea how you may name your variables, that is the key 
point. Thus he has no idea what safe symbol names to use.


 What if "updateGui()" does "rm -rf /" ?

 What if "updateGui()" always does a null pointer dereference?


You will have a bugged/malicious function in your code base, one 
very easy to detect and fix. Nothing in common with discussed 
problem.
Sep 04 2013
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/4/13 11:54 AM, Dicebot wrote:

 On Wednesday, 4 September 2013 at 14:44:50 UTC, Ary Borenszweig wrote:

 Why would anyone do that?


 By an accident - mixins are not hygienic.


 Do you use other people's source code without reading the
 documentation or reading the source code?


 Yes, all the time. Documentation is always lacking and reading full 10
 MLOC code base to tweak a single function is never an option.


 How would the author of "updateGui()" know that you named your
 variable "st"?


 He has no idea how you may name your variables, that is the key point.
 Thus he has no idea what safe symbol names to use.


So the problem is not that implicit mixin is unsafe. The problem is that 
there's no way to declare a new variable that won't conflict with 
existing variables in the current scope?
Sep 04 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 15:18:10 UTC, Ary Borenszweig 
wrote:

 So the problem is not that implicit mixin is unsafe. The 
 problem is that there's no way to declare a new variable that 
 won't conflict with existing variables in the current scope?


This is the same problem. The key property of mixin is that it is 
unhygienic and invades the caller scope. It is the only entity in 
D that is allowed to do it. Allowing implicit unhygienic 
inclusions is guaranteed to result in accidental symbol clash 
and/or unexpected modification sooner or later.
Sep 04 2013
parent reply Timothee Cour <thelastmammoth gmail.com> writes:
frankly, UFCS mixin would make the use case in the OT bearable.

fun().mixin.writeln is ok
mixin(fun()).writeln is ugly (esp in more complex cases).

so, is there anything against it despite requiring one to implement it?


On Wed, Sep 4, 2013 at 8:29 AM, Dicebot <public dicebot.lv> wrote:


 On Wednesday, 4 September 2013 at 15:18:10 UTC, Ary Borenszweig wrote:


 So the problem is not that implicit mixin is unsafe. The problem is that
 there's no way to declare a new variable that won't conflict with existing
 variables in the current scope?


 This is the same problem. The key property of mixin is that it is
 unhygienic and invades the caller scope. It is the only entity in D that is
 allowed to do it. Allowing implicit unhygienic inclusions is guaranteed to
 result in accidental symbol clash and/or unexpected modification sooner or
 later.
Sep 04 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 17:10:32 UTC, Timothee Cour 
wrote:

 frankly, UFCS mixin would make the use case in the OT bearable.

 fun().mixin.writeln is ok
 mixin(fun()).writeln is ugly (esp in more complex cases).

 so, is there anything against it despite requiring one to 
 implement it?


I guess there is nothing terrible but also nothing extremely 
useful :) For me second snippet is not any uglier.
Sep 05 2013
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 5 September 2013 at 10:36:40 UTC, Dicebot wrote:

 On Wednesday, 4 September 2013 at 17:10:32 UTC, Timothee Cour 
 wrote:

 frankly, UFCS mixin would make the use case in the OT bearable.

 fun().mixin.writeln is ok
 mixin(fun()).writeln is ugly (esp in more complex cases).

 so, is there anything against it despite requiring one to 
 implement it?


 I guess there is nothing terrible but also nothing extremely 
 useful :) For me second snippet is not any uglier.


Stuff like data validation, or lazy initialization, anything 
really data oriented usually benefit from that.

Also, the dichotomy function call/field access is really not that 
clear at the end. Accessing some data may require a function 
call, go through a signal handler, or involve complicated 
operation by the CPU (potentially 2 round trip to memory).

On the other hand, optimizer will remove many function calls 
making them effectively field access. Even if it isn't inlined, a 
simple function call could end up being faster than 2 round trip 
to memory (the stack is hot).
Sep 05 2013
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 5 September 2013 at 10:51:19 UTC, deadalnix wrote:

 Stuff like data validation, or lazy initialization, anything 
 really data oriented usually benefit from that.

 Also, the dichotomy function call/field access is really not 
 that clear at the end. Accessing some data may require a 
 function call, go through a signal handler, or involve 
 complicated operation by the CPU (potentially 2 round trip to 
 memory).

 On the other hand, optimizer will remove many function calls 
 making them effectively field access. Even if it isn't inlined, 
 a simple function call could end up being faster than 2 round 
 trip to memory (the stack is hot).


I guess it was an answer to earlier property comment. Key concern 
here is not actually speed but ease to understand the program - 
it is pretty much the same deal as pure vs non-pure. Non-volatile 
non-shared field access is extremely unlikely to cause any 
side-effects in global program state and same is generally 
expected from something that looks like plain field access.
Sep 05 2013
prev sibling parent Timothee Cour <thelastmammoth gmail.com> writes:
the other part of my post was about UFCS syntax for mixin:
eg: foo.mixin.writeln

is there any reason not to allow it?



On Tue, Sep 3, 2013 at 1:47 PM, Andrej Mitrovic
<andrej.mitrovich gmail.com>wrote:


 On 9/3/13, Jacob Carlborg <doob me.com> wrote:

 With properties you never know if you're invoking a method or accessing
 a field:

 foo.data; // call method or access field?


 Yeah but it does something with data on its own side. If this becomes
 an implicit mixin, it could do something with code at the call site.
 E.g.

 auto st = getSecurityToken(...);
 updateGui();  // what if this becomes a mixin and ends up reading 'st'
 and displaying it on the screen?

 It is equivalent to using globals everywhere in your codebase.
Sep 03 2013
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 9/3/13 4:05 PM, Andrej Mitrovic wrote:

 On 9/3/13, Ary Borenszweig <ary esperanto.org.ar> wrote:

 When you do "import foo.bar" you are importing arbitrary code...


 You are importing symbols. And when you do "foo()" you know you're
 calling a function. With the change, you'll never know what foo()
 does.


Yes you know: you look at the source code, or the documentation.


 This feature is never going to fly, but people are just going to argue
 this forever..


I know, I'm just trying to defend this point of view.
Sep 04 2013
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 4 September 2013 at 14:42:26 UTC, Ary Borenszweig 
wrote:

 On 9/3/13 4:05 PM, Andrej Mitrovic wrote:

 On 9/3/13, Ary Borenszweig <ary esperanto.org.ar> wrote:

 When you do "import foo.bar" you are importing arbitrary 
 code...


 You are importing symbols. And when you do "foo()" you know 
 you're
 calling a function. With the change, you'll never know what 
 foo()
 does.


 Yes you know: you look at the source code, or the documentation.


You don't do it for _every single symbol_. Good good matches 
naive assumptions, this is exactly what allows to read it fast. 
Reading code where you can't make assumptions about anything is a 
transitive (possibly exponential) task. Exactly the hell you get 
with undisciplined C macro usage.
Sep 04 2013
prev sibling parent "Dicebot" <public dicebot.lv> writes:
Was objecting in previous thread and have not changed my mind - 
it is unhygienic and mostly useless.
Sep 03 2013