www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - "hijackable"/"customizable" keyword for solving the "customized

reply "monarch_dodra" <monarchdodra gmail.com> writes:
A little while ago, Steven Schveighoffer started a thread called 
"D UFCS anti-pattern". The conversation started in regards to 
"how can a generic algorithm be replaced by a custom type". In 
particular (in my original use case) "retro"

The issue just came up again in ".learn" for ".find". It comes up 
regularly with ".put".

To put it simply, the problem is "given the function std.foo, how 
can I implement myObject.foo, and make sure that that is called 
when we call foo?"

There were two options:
1. Make std.foo test "hasMember!(T, "foo")". This would hardly 
scale though, as applying it to *every* algorithm is not 
acceptable
2. For *all* calls to be UFCS style, and ignore the "anti-pattern 
issue". Even then, it means you give no guarantees as to *what* 
is called, depending on how the user writes the code. So not 
acceptable

I just had a crazy idea. "hijackable" keyword (yeah... another 
keyword):

Given a function:
"Ret foo(T input, Args... args)  hijackable"

Then, when the compiler sees:
"foo(input, args);"

It will always forward directly to T.foo if T.foo exists, 
bypassing std.foo entirely.

I think it is a clean and generic way to allow ALL algorithms in 
D much greater customize-ability.

I didn't put *much* more thought than that to it yet, so I may 
have missed something obvious? In particular, "hijack" might not 
be the correct word? " customizable" ?

Please provide feedback/destruction ?
May 14 2014
next sibling parent "w0rp" <devw0rp gmail.com> writes:
I'm going to get into an opinion on member functions and 
overrides here, and it's kind of "anti-classes," so you can 
choose to ignore this post if you don't want to hear about that.

I think this is a bad idea. One of the things that really appeals 
to me about algorithms presented through functions is that I can 
predict the behaviour of them. It will do something generically 
on a category of types. I think any need to change the behaviour 
of an algorithm either means that the algorithm was written 
incorrectly, or really you just have a different algorithm and 
you should probably use another name for it. One of the things 
I've really lost faith in over the years is method overrides, 
becuase of how confusing it has made code I have had to read. 
These days I prefer generic functions that give me one or maybe a 
couple of places to look when something goes wrong.
May 14 2014
prev sibling next sibling parent reply "Meta" <jared771 gmail.com> writes:
On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:

 A little while ago, Steven Schveighoffer started a thread 
 called "D UFCS anti-pattern". The conversation started in 
 regards to "how can a generic algorithm be replaced by a custom 
 type". In particular (in my original use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes 
 up regularly with ".put".

 To put it simply, the problem is "given the function std.foo, 
 how can I implement myObject.foo, and make sure that that is 
 called when we call foo?"

 There were two options:
 1. Make std.foo test "hasMember!(T, "foo")". This would hardly 
 scale though, as applying it to *every* algorithm is not 
 acceptable
 2. For *all* calls to be UFCS style, and ignore the 
 "anti-pattern issue". Even then, it means you give no 
 guarantees as to *what* is called, depending on how the user 
 writes the code. So not acceptable

 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.

 I think it is a clean and generic way to allow ALL algorithms 
 in D much greater customize-ability.

 I didn't put *much* more thought than that to it yet, so I may 
 have missed something obvious? In particular, "hijack" might 
 not be the correct word? " customizable" ?

 Please provide feedback/destruction ?


It's a good idea, but yet another function annotation is pretty 
much a no-go. How bad an idea is it to *always* defer to a member 
function if the object/struct in question has such a function 
defined? I thought that was the case already... I suppose it will 
cause untold amounts of code breakage. Very unfortunate.
May 14 2014
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 14 May 2014 at 23:57:24 UTC, Meta wrote:

 It's a good idea, but yet another function annotation is pretty 
 much a no-go. How bad an idea is it to *always* defer to a 
 member function if the object/struct in question has such a 
 function defined? I thought that was the case already... I 
 suppose it will cause untold amounts of code breakage. Very 
 unfortunate.


UFCS only apply to the method call style, not the the function 
call style, so it's not a matter of priority here - 
`foo(myObject)` will not call the `foo.myObject()` method even if 
there is no `foo` function(in that case it'll just fail).

Having the function call style always defer to a member function 
is a really bad idea - and not just because of code breakage. 
It'll make it impossible to call a function on an object that has 
a method of the same name unless you use an alias or a function 
variable to copy the function - but then you have to make sure 
the alias\variable name is also not taken!

This will be a disaster for anyone who wants to build or use a D 
library that uses templates. Well, how common can that case be?
May 14 2014
parent reply "Meta" <jared771 gmail.com> writes:
On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:

 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?


That's exactly what the proposed  hijackable annotation on a 
function would do, if I understand it correctly.
May 14 2014
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote:

 On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:

 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not 
 taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?


 That's exactly what the proposed  hijackable annotation on a 
 function would do, if I understand it correctly.


Right. The idea though is not to "customize the behavior" of the 
algorithm, but rather "provide a more efficient implementation". 
It shouldn't really break any existing code.

In any case, it should not cause any more than the breakage that 
using UFCS *already* causes... this would just (opt-in by the 
implementation) make sure you get the same behavior with/without 
UFCS.

FYI, there are already a few functions in phobos that delegate to 
member functions "if they exist", and do something generic if 
they don't. "Take" and "put" are some of the more trivial 
examples. "move" will also delegate to "proxyMove". At one point, 
there were also talks of doing the same for popFrontN too.

The issue though is that it's all very hackish, and it makes the 
implementation jump through hoops to achieve said result, and it 
only works if the implementation pre-emptivelly thought about 
doing so.
May 14 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:


 "move" will also delegate to "proxyMove".


This is the correct solution IMO. The principle of least surprise should  
dictate that a.foo should always mean the same thing. All that is required  
to enforce this is to make the "hook" function have a different name.

I think the two mechanisms of overriding default behavior:

1. Use a hook, to define the basic minimum functionality.
2. Implement the same-named method, but you must implement ALL  
functionality (possibly deferring to the global function if necessary).

-Steve
May 15 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer 
wrote:

 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra 
 <monarchdodra gmail.com> wrote:


 "move" will also delegate to "proxyMove".


 This is the correct solution IMO. The principle of least 
 surprise should dictate that a.foo should always mean the same 
 thing. All that is required to enforce this is to make the 
 "hook" function have a different name.


The issue(s) I have with the "hook must have a different name" is:
1. In the algorithm implementation, you must explicitly test for 
the hook, which, if we want to expand on, would turn all our 
algorithms into:
void foo(T)(T t)
{
     static if (is(typeof(t.fooHook())))
         return t.fooHook();
     else
         ...
}
2. The "overriden" hook is only useful if you import the 
corresponding algorithm. So for example, in my 3rd pary library, 
if my type has "findHook", I'd *have* to import 
std.algorithm.find for it to be useful. Unless I want to make a 
direct call to "findHook", which would be ugly...

An example for "2" are range primitives:


 I think the two mechanisms of overriding default behavior:

 1. Use a hook, to define the basic minimum functionality.
 2. Implement the same-named method, but you must implement ALL 
 functionality (possibly deferring to the global function if 
 necessary).

 -Steve


Also: "moveFront"/"moveBack" are other examples of such "globally 
hijack-able" functions.
May 15 2014
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 15 May 2014 13:08:56 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:


 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer wrote:

 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra  
 <monarchdodra gmail.com> wrote:


 "move" will also delegate to "proxyMove".


 This is the correct solution IMO. The principle of least surprise  
 should dictate that a.foo should always mean the same thing. All that  
 is required to enforce this is to make the "hook" function have a  
 different name.


 The issue(s) I have with the "hook must have a different name" is:
 1. In the algorithm implementation, you must explicitly test for the  
 hook, which, if we want to expand on, would turn all our algorithms into:
 void foo(T)(T t)
 {
      static if (is(typeof(t.fooHook())))
          return t.fooHook();
      else
          ...
 }
 2. The "overriden" hook is only useful if you import the corresponding  
 algorithm. So for example, in my 3rd pary library, if my type has  
 "findHook", I'd *have* to import std.algorithm.find for it to be useful.  
 Unless I want to make a direct call to "findHook", which would be ugly...


Obviously, foo is not a good example for hooking. This problem only exists  
if the functionality of the module-level function exceeds that of the  
member. The above does not have any overloads, and basically does one  
thing.

I did consider the fact that you would have to import the module-level  
function in order for it to be part of the API, but that's how the system  
works. It would be nice to say "if you import module x, then a.foo has  
more functionality, if you don't, it's limited to this one aspect," but I  
don't think that's possible. It's not a perfect solution, but the only one  
that makes sense to me.

-Steve
May 15 2014
prev sibling parent reply "Yota" <yotaxp thatGoogleMailThing.com> writes:
On Thursday, 15 May 2014 at 17:08:58 UTC, monarch_dodra wrote:

 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer 
 wrote:

 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra 
 <monarchdodra gmail.com> wrote:


 "move" will also delegate to "proxyMove".


 This is the correct solution IMO. The principle of least 
 surprise should dictate that a.foo should always mean the same 
 thing. All that is required to enforce this is to make the 
 "hook" function have a different name.


 The issue(s) I have with the "hook must have a different name" 
 is:
 1. In the algorithm implementation, you must explicitly test 
 for the hook, which, if we want to expand on, would turn all 
 our algorithms into:
 void foo(T)(T t)
 {
     static if (is(typeof(t.fooHook())))
         return t.fooHook();
     else
         ...
 }
 2. The "overriden" hook is only useful if you import the 
 corresponding algorithm. So for example, in my 3rd pary 
 library, if my type has "findHook", I'd *have* to import 
 std.algorithm.find for it to be useful. Unless I want to make a 
 direct call to "findHook", which would be ugly...


How about a middle ground?  Have the function names be identical, 
and decorate the member version with  proxy or  hook, rather than 
decorating the original definition.  I'd find this to be less 
surprising.
May 16 2014
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 16 May 2014 16:45:28 +0000
Yota via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Thursday, 15 May 2014 at 17:08:58 UTC, monarch_dodra wrote:

 On Thursday, 15 May 2014 at 12:16:52 UTC, Steven Schveighoffer
 wrote:

 On Thu, 15 May 2014 02:05:08 -0400, monarch_dodra
 <monarchdodra gmail.com> wrote:


 "move" will also delegate to "proxyMove".


 This is the correct solution IMO. The principle of least
 surprise should dictate that a.foo should always mean the same
 thing. All that is required to enforce this is to make the
 "hook" function have a different name.


 The issue(s) I have with the "hook must have a different name"
 is:
 1. In the algorithm implementation, you must explicitly test
 for the hook, which, if we want to expand on, would turn all
 our algorithms into:
 void foo(T)(T t)
 {
     static if (is(typeof(t.fooHook())))
         return t.fooHook();
     else
         ...
 }
 2. The "overriden" hook is only useful if you import the
 corresponding algorithm. So for example, in my 3rd pary
 library, if my type has "findHook", I'd *have* to import
 std.algorithm.find for it to be useful. Unless I want to make a
 direct call to "findHook", which would be ugly...


 How about a middle ground?  Have the function names be identical,
 and decorate the member version with  proxy or  hook, rather than
 decorating the original definition.  I'd find this to be less
 surprising.


That sounds like an interesting idea, but it would require adding the concept
to the compiler - though I suppose that that's a downside with monarch_dodra's
original proposal as well. Neither can be done in the library itself. That's
not necessarily the end of the world, but at this point, I'm very hesitant to
support adding another attribute to the language without a really, really good
reason.

This particular problem can be solved by Steven's suggestion of using proxy
functions with different names, which works within the language as it's
currently defined. So, while that might not be an altogether desirable
solution, I'm inclined to believe that it's good enough to make it not worth
adding additional attributes to the language to solve the problem.

- Jonathan M Davis
May 17 2014
prev sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Thursday, 15 May 2014 at 02:07:40 UTC, Meta wrote:

 On Thursday, 15 May 2014 at 01:49:17 UTC, Idan Arye wrote:

 UFCS only apply to the method call style, not the the function 
 call style, so it's not a matter of priority here - 
 `foo(myObject)` will not call the `foo.myObject()` method even 
 if there is no `foo` function(in that case it'll just fail).

 Having the function call style always defer to a member 
 function is a really bad idea - and not just because of code 
 breakage. It'll make it impossible to call a function on an 
 object that has a method of the same name unless you use an 
 alias or a function variable to copy the function - but then 
 you have to make sure the alias\variable name is also not 
 taken!

 This will be a disaster for anyone who wants to build or use a 
 D library that uses templates. Well, how common can that case 
 be?


 That's exactly what the proposed  hijackable annotation on a 
 function would do, if I understand it correctly.


Yes, but there is quite a difference between applying this to 
specific functions where this behavior is desirable and applying 
this to everything.
May 15 2014
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 5/14/14, 3:05 PM, monarch_dodra wrote:

 A little while ago, Steven Schveighoffer started a thread called "D UFCS
 anti-pattern". The conversation started in regards to "how can a generic
 algorithm be replaced by a custom type". In particular (in my original
 use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes up
 regularly with ".put".

 To put it simply, the problem is "given the function std.foo, how can I
 implement myObject.foo, and make sure that that is called when we call
 foo?"


That's the big problem with UFCS.

If you were forced to always use UFCS then this wouldn't be a problem: 
if the type at question provides a more specific implementation *that 
matches the arguments' types*, the compiler takes it. Otherwise, it 
checks for free functions that have that type as a first argument and 
the rest of the arguments.

But once you have an option to invoke it as a free function you loose. 
The "U" of UFCS means "uniform", so in my opinion when you do:

foo(x, y);

then the compiler must first check if the type of "x" defines "foo(y)". 
If so, then it invokes that specialized version. If not, it must check 
if "foo(x, y)" exists somewhere else.

If you do:

x.foo(y);

the compiler would do exactly the same.

This is uniformity: either way I invoke a function, the compiler's logic 
is the same.

But if you can write a "same thing" in two different ways but behave 
differently, that's looking for trouble.

If you don't want the compiler to convert foo(x, y) to x.foo(y), then 
use a qualified name:

bar.baz.foo(x, y);
May 15 2014
parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 5/15/14, 7:54 AM, Ary Borenszweig wrote:

 On 5/14/14, 3:05 PM, monarch_dodra wrote:

 A little while ago, Steven Schveighoffer started a thread called "D UFCS
 anti-pattern". The conversation started in regards to "how can a generic
 algorithm be replaced by a custom type". In particular (in my original
 use case) "retro"

 The issue just came up again in ".learn" for ".find". It comes up
 regularly with ".put".

 To put it simply, the problem is "given the function std.foo, how can I
 implement myObject.foo, and make sure that that is called when we call
 foo?"


 That's the big problem with UFCS.

 If you were forced to always use UFCS then this wouldn't be a problem:
 if the type at question provides a more specific implementation *that
 matches the arguments' types*, the compiler takes it. Otherwise, it
 checks for free functions that have that type as a first argument and
 the rest of the arguments.

 But once you have an option to invoke it as a free function you loose.
 The "U" of UFCS means "uniform", so in my opinion when you do:

 foo(x, y);

 then the compiler must first check if the type of "x" defines "foo(y)".
 If so, then it invokes that specialized version. If not, it must check
 if "foo(x, y)" exists somewhere else.

 If you do:

 x.foo(y);

 the compiler would do exactly the same.

 This is uniformity: either way I invoke a function, the compiler's logic
 is the same.

 But if you can write a "same thing" in two different ways but behave
 differently, that's looking for trouble.

 If you don't want the compiler to convert foo(x, y) to x.foo(y), then
 use a qualified name:

 bar.baz.foo(x, y);


Of course, the "problem" with this is that when you read this code:

foo(x, y);

to understand it, you must first check if there's a "foo" definition 
somewhere in your imported modules that matches. Otherwise, check if x's 
type defines it.

The same is true for this:

x.foo(y);

To understand it, you must check if x's type defines a "foo(y)". If so, 
it's taken. Otherwise, check which of the functions in the imported 
modules matches "foo(x, y)".

This makes it really hard to reason about code: everytime you see 
"foo(x, y)" you must remember which of the modules had that function 
"foo", or if x defines "foo".

In OOP-like land, "foo(x, y)" always means: check the function "foo(x, 
y)". And "x.foo(y)" always means: check the member "foo" of x's type. No 
confusion at all for the reader. But "foo(x, y)" is almost never used 
(global functions), so you never have to remember where methods are 
located by heart. You just go to that type's code and find the method there.
May 15 2014
prev sibling parent reply "Yota" <yotaxp thatGoogleMailThing.com> writes:
On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:
...

 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.


...

What about stuff like this:
   import some.pkg;
   some.pkg.foo(x);

Does this still call x.foo()?  What if 'foo' is an alias for 
'bar', or vice versa?
May 16 2014
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 16 May 2014 at 16:50:37 UTC, Yota wrote:

 On Wednesday, 14 May 2014 at 18:05:44 UTC, monarch_dodra wrote:
 ...

 I just had a crazy idea. "hijackable" keyword (yeah... another 
 keyword):

 Given a function:
 "Ret foo(T input, Args... args)  hijackable"

 Then, when the compiler sees:
 "foo(input, args);"

 It will always forward directly to T.foo if T.foo exists, 
 bypassing std.foo entirely.


 ...

 What about stuff like this:
   import some.pkg;
   some.pkg.foo(x);

 Does this still call x.foo()?


In theory, yes, because "some.pkg" would *be* X.foo, or at the 
very least, a very thin wrapper. You'd get the very same result 
with "static if" based implementation, such as "moveFront", where 
"std.range.moveFront(r)" still just calls "r.moveFront()". Ditto 
for "take", which just calls opSlice.


 What if 'foo' is an alias for 'bar', or vice versa?


An alias shouldn't change anything I think.
May 16 2014