www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Mixin templates are a pain at best, useless at worst for any

reply Ethan <gooberman gmail.com> writes:
I've come across this before with Binderoo, but now I've got 
really simple use cases.

Rather than having one unmaintainable mess of a file that handles 
everything (for a really egregious example, see 
std.datetime.systime which has the distinction of both its source 
code and documentation being unreadable), I decided to do 

declared in other modules to complete the implementation.

This is all well and good until you get to dealing with function 
overloads.

Exhibit A: https://run.dlang.io/is/a85Lbq

As soon as you have an overload of a function declared in the 
base object you're mixing in to, any other overload mixed in will 
not resolve correctly. Great.

The core use case I derived this from is client/server message 
handling, and I plan on reusing mixins across client/server types 
since message handling will also require variables added in to 
the object to do so. Simple example: Handling Ping and Pong will 
require the Ping sender to start a timer and resolve it on 
receiving Pong. The partial class model is perfect for this, and 
mixins are the closest we have to this.

I submitted a bug with the above code, and it was "helpfully" 
shut down with a link to the documentation and workaround. 
Congratulations. That's not the use case here, and to be quite 
honest this is one of those examples where a #define macro would 
"just work". And I'm firmly of the view that *ANY* example of 
that should be dealt with at a language level in order to 
completely remove the argument of needing a preprocessor.

Exhibit B: https://run.dlang.io/is/s2BJUO

This one fired as soon as Binderoo tried to do its thing: Doing 
an allMembers pass of the object will list your entirely-mixed-in 
overload as a member, but attempting to get that member will 
resolve the symbol to void.

Great. So functions that otherwise completely resolve as a member 
of a class actually don't resolve as a member of a class? Huh? I 
don't even.

To see where I'm going with this, exhibit C: 
https://run.dlang.io/is/KWN9yA

Rather than mixin things one by one and manually handle errors 
and language shortcomings, I'm using helpers to wholesale add 
functionality to my object.

And, honestly, with this method, I am already seeing the 
workaround. Because I've had to do it a ton of times already with 
other templates. Run a search for 'mixin( "import' in Binderoo to 
see how many times I've had to get around the changes to template 
visibility rules. Rather than do that though, now I'll have to 
iterate over every member of a stored mixin and alias them in to 
the surrounding object.

Sigh.

I know that simply posting this will result in a thread of people 
offering workarounds. I don't need or want them. Why? Because the 
next person that tries to do this will have the same problems. 
And rather than the "correct" way to do this be voodoo knowledge 
found by a Google search if you're lucky, I would far prefer this 
thread become a discussion on how to improve mixins so that a DIP 
can be written and resolved. So that the next person can express 
language as simply as possible and have it all Just Work(TM).
Jun 05 2018
next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Tuesday, 5 June 2018 at 10:11:49 UTC, Ethan wrote:


 Exhibit A: https://run.dlang.io/is/a85Lbq


[..snip..]

 I submitted a bug with the above code, and it was "helpfully" 
 shut down with a link to the documentation and workaround.


This looks like the bug report you are referring to:  
https://issues.dlang.org/show_bug.cgi?id=18944

I understand that the specification defines the current behavior 
and provides a workaround, but what's the justification for that 
behavior?  Is it hygiene?

I've been interested in improving mixins for reasons discussed in 
the interpolated strings PR 
(https://github.com/dlang/dmd/pull/7988) and this discussion 
about library-implemented properties 
(https://forum.dlang.org/post/mqveusvzkmkshrzwsgjy forum.dlang.org).

Perhaps there's an opportunity here for a language improvement 
that would help all of these use cases.

Mike
Jun 05 2018
prev sibling next sibling parent reply Ethan <gooberman gmail.com> writes:
On Tuesday, 5 June 2018 at 10:11:49 UTC, Ethan wrote:

 And, honestly, with this method, I am already seeing the 
 workaround. Because I've had to do it a ton of times already 
 with other templates. Run a search for 'mixin( "import' in 
 Binderoo to see how many times I've had to get around the 
 changes to template visibility rules. Rather than do that 
 though, now I'll have to iterate over every member of a stored 
 mixin and alias them in to the surrounding object.


I've had to go nuclear, in fact, to get this working. And resort 
to string mixins.

https://run.dlang.io/is/O5PDaK

So it works. But it doesn't Just Work(TM).
Jun 05 2018
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 5 June 2018 at 11:35:10 UTC, Ethan wrote:

 On Tuesday, 5 June 2018 at 10:11:49 UTC, Ethan wrote:

 And, honestly, with this method, I am already seeing the 
 workaround. Because I've had to do it a ton of times already 
 with other templates. Run a search for 'mixin( "import' in 
 Binderoo to see how many times I've had to get around the 
 changes to template visibility rules. Rather than do that 
 though, now I'll have to iterate over every member of a stored 
 mixin and alias them in to the surrounding object.


 I've had to go nuclear, in fact, to get this working. And 
 resort to string mixins.

 https://run.dlang.io/is/O5PDaK

 So it works. But it doesn't Just Work(TM).


avoid unrolling for-each mate.
you do that by enclosing the Tuple Argument in [Args].
Jun 05 2018
prev sibling next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Tuesday, 5 June 2018 at 10:11:49 UTC, Ethan wrote:

 Exhibit A: https://run.dlang.io/is/a85Lbq

 I submitted a bug with the above code, and it was "helpfully" 
 shut down with a link to the documentation and workaround. 
 Congratulations. That's not the use case here, and to be quite 
 honest this is one of those examples where a #define macro 
 would "just work". And I'm firmly of the view that *ANY* 
 example of that should be dealt with at a language level in 
 order to completely remove the argument of needing a 
 preprocessor.


There's a reason for those rules in the language, namely function 
hijacking. This is an issue we take very seriously, and 
workarounds exists.

As you point out, the suggested workaround is aliasing each 
function in the mixin (you also say you don't want workarounds - 
bear with me, I'm building to a point). This can to some extent 
be encapsulated in a string mixin that's used alongside any 
template mixin:

string introduceSymbols(alias F)()
{
     enum id = __traits(identifier, F);
     return "static foreach (fn; __traits(allMembers, "~id~")) {"~
            "    mixin(`alias `~fn~` = "~id~".`~fn~`;`);"~
            "}";
}

mixin foo!() foo_;
mixin(introduceSymbols!foo_);

This introduces complexity in that each mixin must be given a 
unique name at instantiation, another line must be included with 
every mixin, and the name must be repeated in the second line. 
This situation is less than optimal, and the compiler gives no 
help if you've forgotten a introduceSymbols line, or passed it 
the wrong name.

These issues can be ameliorated as in your other post here, by 
wrapping the template mixin statement as well. This leads to a 
host of other problems - how do you specify arguments to the 
mixin template? How do you deal with aliased mixin templates? 
These issues may possibly be fixed, but it's gonna get ugly.


Can we perhaps do better? Could we somehow say at the 
instantiation point 'please introduce all symbols in this mixin 
into the instantiation scope even if there are hijackings.'?  
Something like `mixin scope foo!();`? That seems possible to me. 
If we write a DIP for it, I've a good feeling about getting that 
into the language.

This issue shows up every now and then, so we know it's a real 
issue, and the workarounds are clunky. At the same time, the 
current behavior is there for a reason, and is very unlikely to 
change.

--
   Simen
Jun 05 2018
parent Ethan <gooberman gmail.com> writes:
On Tuesday, 5 June 2018 at 12:08:58 UTC, Simen Kjærås wrote:

 There's a reason for those rules in the language, namely 
 function hijacking. This is an issue we take very seriously, 
 and workarounds exists.


So serious that there is no meaningful error message?


 These issues can be ameliorated as in your other post here, by 
 wrapping the template mixin statement as well. This leads to a 
 host of other problems - how do you specify arguments to the 
 mixin template? How do you deal with aliased mixin templates? 
 These issues may possibly be fixed, but it's gonna get ugly.


I've posted on these newsgroups literally within the last three 
weeks about dealing with multiple different argument types 
(parsing template parameters) and aliases (__traits( identifier, 
Symbol ) does not match Symbol.stringof if it is an alias). As 
such, that's purely within the realms of "solvable by the 
programmer with new idioms" as I see it.

This thing I'm posting about is solvable by giving up and 
generating a string to treat as code. ie the nuclear option when 
everything else in the language is unsuitable.


 This issue shows up every now and then, so we know it's a real 
 issue, and the workarounds are clunky. At the same time, the 
 current behavior is there for a reason, and is very unlikely to 
 change.


https://dlang.org/spec/template-mixin.html

1. A TemplateMixin takes an arbitrary set of declarations from 
the body of a TemplateDeclaration and inserts them into the 
current context.

Everything you've talked about here directly conflicts with point 
number 1 of the mixin template spec. Point 3 of the spec makes a 
note of the implementation not actually doing the direct 
copy/paste that it mentions and instead embeds it in to a nested 
scope and imports it after the fact. Yet there appears to be 
quite a number of "except for this" clauses that aren't made 
clear. Clearly something has gone wrong somewhere along the way.

So by that token: How many real world dollars have been wasted on 
function hijacking? Is this something a corporate client has been 
adamant about? Bringing up the #define macro analogy again, these 
are clearly similar problems that can happen in C/C++. So is it 
more of an ideological stand than a real-world one?
Jun 05 2018
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 5 June 2018 at 10:11:49 UTC, Ethan wrote:

 As soon as you have an overload of a function declared in the 
 base object you're mixing in to, any other overload mixed in 
 will not resolve correctly. Great.


Yes, it is great, since this lets you selectively override 
behavior from a generic mixin template for a specific use. I like 
this.

But we might be able to change the rule so, for functions, it 
follows the overload rules with arguments instead of just going 
by name. That would let you add functions... but that's 
inconsistent with how D does child class inheritance too (you 
need to alias in overloads there as well), for better or for 
worse.

Perhaps adding something like `alias * = Base.*;` as a feature 
would be good. I kinda hate that. But the idea there would be to 
just add all of Base's things to the overload set. OK that 
basically sucks especially when there's multiple bases. but it 
would be fairly consistent.
Jun 05 2018