www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Should we add `a * b` for vectors?

reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
Should we add `a * b` to ndslice for 1d vectors?
Discussion at https://github.com/libmir/mir-algorithm/issues/91
Sep 22 2017
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


My position is still that it can't hurt to add a dot product 
function and other arithmetic or linear algebra functions that 
might be used in the future with operator overloading. Ideally 
they could switch to other mir implementations at compile time if 
those have been imported.
Sep 23 2017
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On 23 September 2017 at 03:11, Ilya Yaroshenko via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


I often ponder this sort of question... and while the nerdy bit of my would
love to see this, I can never get past the fact that `is(typeof(a) !=
typeof(a * b))`... I feel that's a big enough reason to put a bullet in the
idea. D has array operations, which look just like 1d ndslices, also SIMD
vectors usually implement operators, but they do element-wise operation, so
now there's a confusion.
Consider, a generic function receives some T; if T is a slice or simd, it
does element-wise stuff, if T is an ndslice, it behaves differently. That
doesn't seem right.
Sep 23 2017
parent Z <me zigzah.com> writes:
On Sunday, 24 September 2017 at 05:24:57 UTC, Manu wrote:

 On 23 September 2017 at 03:11, Ilya Yaroshenko via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:


 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


 I often ponder this sort of question... and while the nerdy bit 
 of my would
 love to see this, I can never get past the fact that 
 `is(typeof(a) !=
 typeof(a * b))`... I feel that's a big enough reason to put a 
 bullet in the
 idea. D has array operations, which look just like 1d ndslices, 
 also SIMD
 vectors usually implement operators, but they do element-wise 
 operation, so
 now there's a confusion.
 Consider, a generic function receives some T; if T is a slice 
 or simd, it
 does element-wise stuff, if T is an ndslice, it behaves 
 differently. That
 doesn't seem right.


Python has a dedicated matrix multiplication operator, that is 
not used by the standard library, but is implemented for e.g. 
numpy arrays.

https://www.python.org/dev/peps/pep-0465/
Sep 24 2017
prev sibling next sibling parent reply Mark <smarksc gmail.com> writes:
On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


Generally I expect that a binary operation denoted by + or * 
would produce an element from the original domain, e.g. 
multiplying two matrices yields a matrix, concatenating two 
strings yields a string, etc. So personally I don't like this 
notation.

Note that in the case of 3-dimensional vectors, people might 
confuse this for the cross product. I would go with dot(a,b) and 
cross(a,b) (if you support it).
Sep 24 2017
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Sep 24, 2017 at 06:18:38PM +0000, Mark via Digitalmars-d wrote:

 On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


 
 Generally I expect that a binary operation denoted by + or * would produce
 an element from the original domain, e.g. multiplying two matrices yields a
 matrix, concatenating two strings yields a string, etc. So personally I
 don't like this notation.
 
 Note that in the case of 3-dimensional vectors, people might confuse this
 for the cross product. I would go with dot(a,b) and cross(a,b) (if you
 support it).


With UFCS, you have the slightly nicer notation a.dot(b) and a.cross(b).


T

-- 
Claiming that your operating system is the best in the world because more
people use it is like saying McDonalds makes the best food in the world. --
Carl B. Constantine
Sep 24 2017
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Sunday, 24 September 2017 at 22:33:48 UTC, H. S. Teoh wrote:

 With UFCS, you have the slightly nicer notation a.dot(b) and 
 a.cross(b).


 T


Below is a link to the operators in Matlab and associated 
functions. mir can include all of those functions, but not all of 
them can be implemented as operators (because D doesn't have 
things like .*). I think it would be annoying to constantly have 
to write plus or mtimes in writing formulas for a linear algebra 
library.

https://www.mathworks.com/help/matlab/matlab_prog/matlab-operators-and-special-characters.html
Sep 24 2017
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 24 September 2017 at 23:59:59 UTC, jmh530 wrote:

 On Sunday, 24 September 2017 at 22:33:48 UTC, H. S. Teoh wrote:

 With UFCS, you have the slightly nicer notation a.dot(b) and 
 a.cross(b).


 T


 Below is a link to the operators in Matlab and associated 
 functions. mir can include all of those functions, but not all 
 of them can be implemented as operators (because D doesn't have 
 things like .*). I think it would be annoying to constantly 
 have to write plus or mtimes in writing formulas for a linear 
 algebra library.

 https://www.mathworks.com/help/matlab/matlab_prog/matlab-operators-and-special-characters.html


There's nothing stopping someone writing a DIP to include ` ` 

characters on the standard keyboard are used I think.
Sep 24 2017
next sibling parent reply user1234 <user1234 12.hu> writes:
On Monday, 25 September 2017 at 01:08:35 UTC, Nicholas Wilson 
wrote:

 There's nothing stopping someone writing a DIP to include ` ` 

 characters on the standard keyboard are used I think.


̣̣§
Sep 24 2017
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Monday, 25 September 2017 at 03:22:01 UTC, user1234 wrote:

 On Monday, 25 September 2017 at 01:08:35 UTC, Nicholas Wilson 
 wrote:

 There's nothing stopping someone writing a DIP to include ` ` 

 characters on the standard keyboard are used I think.


 ̣̣§


That is not on mine.
Sep 24 2017
prev sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 25 September 2017 at 01:08:35 UTC, Nicholas Wilson 
wrote:

 There's nothing stopping someone writing a DIP to include ` ` 



I actually like * for dot product and matrix multiplication...

The issue is that if you use * for these, then you can't use it 
for element-wise multiplication. Numpy arrays (but not matrices) 
use * and / for element-wise multiplication and division. To do 
matrix multiplication with Numpy arrays, you used to have to use 
the dot function. This is they decided to introduce   to the 
language, b/c users didn't want to write dot all over the place.

A very popular language with a wide range of uses had to 
introduce a new operator to the language because of design 
choices of its most popular linear algebra library. So it makes 
sense to think about these issues a bit.


 All other characters on the standard keyboard are used I think.


What about forward slash '\' (for inverse)? That is currently 
used for string literals, but I can't think of a binary use right 
now.
Sep 24 2017
prev sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Sunday, 24 September 2017 at 18:18:38 UTC, Mark wrote:

 Generally I expect that a binary operation denoted by + or * 
 would produce an element from the original domain, e.g. 
 multiplying two matrices yields a matrix, concatenating two 
 strings yields a string, etc. So personally I don't like this 
 notation.


This is true of element-wise operators. + works, - works, but * 
(and by implication /) only has that property for Hadamard/Schur 
products. It also would work for inverse. Even matrix 
multiplication could have A*B produce a matrix, but if A is 1XN 
and B is MX1, then you may as well return the scalar.


 Note that in the case of 3-dimensional vectors, people might 
 confuse this for the cross product. I would go with dot(a,b) 
 and cross(a,b) (if you support it).


I assure you, no one would confuse dot for cross. No language or 
linear algebra library does this. The typical option is matrix 
multiplication for *, but languages like Python and Matlab can't 
do things like have a special version that is dot for vectors and 
matrix multiplication for matrices.
Sep 24 2017
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


I'd say yes.

Atila
Sep 26 2017
parent reply Manu <turkeyman gmail.com> writes:
On 26 September 2017 at 21:41, Atila Neves via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko wrote:


 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


 I'd say yes.

 Atila



Just remember, it's okay to vote no! Even if it makes you a bigoted dick ;)
In this case, I think 'no' is the only reasonable choice.
If this is going to seriously be considered, then ndslice should definitely
be renamed to 'matrix'.

An alternative solution might be to introduce a wrapper of ndslice called
'matrix' that supports matrix mul...?
Sep 26 2017
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Wednesday, 27 September 2017 at 04:59:10 UTC, Manu wrote:

 On 26 September 2017 at 21:41, Atila Neves via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
 wrote:


 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at 
 https://github.com/libmir/mir-algorithm/issues/91


 I'd say yes.

 Atila



 Just remember, it's okay to vote no! Even if it makes you a 
 bigoted dick ;)
 In this case, I think 'no' is the only reasonable choice.
 If this is going to seriously be considered, then ndslice 
 should definitely
 be renamed to 'matrix'.

 An alternative solution might be to introduce a wrapper of 
 ndslice called 'matrix' that supports matrix mul...?


I would prefer outer operator overloading be added to D instead 
of type wrappers. So a user can import a library for operations, 
rather then library of wrappers. --Ilya
Sep 27 2017
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 27 September 2017 at 07:41:23 UTC, Ilya Yaroshenko 
wrote:

 I would prefer outer operator overloading be added to D instead 
 of type wrappers. So a user can import a library for 
 operations, rather then library of wrappers. --Ilya


This might be a step in the right direction. It doesn't need to 
be full-blown extension methods/partial classes. Just the ability 
to treat operator overloading like free-standing functions (with 
power of UFCS). Something like:

struct Foo
{
     int data;
}

Foo opBinary!(string op)(Foo x, Foo y)
{
     return mixin("x.data" ~ op ~ "y.data");
}

That would mean you could also do something like:
import lubeck : opBinary;
Sep 27 2017
parent reply Manu <turkeyman gmail.com> writes:
On 27 September 2017 at 22:01, jmh530 via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 27 September 2017 at 07:41:23 UTC, Ilya Yaroshenko wrote:


 I would prefer outer operator overloading be added to D instead of type
 wrappers. So a user can import a library for operations, rather then
 library of wrappers. --Ilya


 This might be a step in the right direction. It doesn't need to be
 full-blown extension methods/partial classes. Just the ability to treat
 operator overloading like free-standing functions (with power of UFCS).
 Something like:

 struct Foo
 {
     int data;
 }

 Foo opBinary!(string op)(Foo x, Foo y)
 {
     return mixin("x.data" ~ op ~ "y.data");
 }

 That would mean you could also do something like:
 import lubeck : opBinary;


Again, sadly, D has no ADL, and this will be an unmitigated disaster as a
result!
Instantiations of templates will use the default elementwise operator
instead of the one you specified in your module, and nobody will understand
why.
Sep 27 2017
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 27 September 2017 at 23:25:34 UTC, Manu wrote:

 Again, sadly, D has no ADL, and this will be an unmitigated 
 disaster as a
 result!
 Instantiations of templates will use the default elementwise 
 operator
 instead of the one you specified in your module, and nobody 
 will understand
 why.


Argument-dependent lookup? I'm not an expert on C++, but I read 
the wikipedia entry on it. It does seem like an issue worth 
thinking more about

One (hacky) solution is to not have default implementations for 
the all the operators (the focus so far has been on the multiple 
uses of * for element-wise multiply and dot product/matrix 
multiplication). So you have Slice in mir.ndslice.slice. Then 
make a mir.ndslice.operator.elementwise module that has the 
operator overloading for elementwise operations, then a module 
mir.ndslice.operator.linalg (or somewhere in lubeck) with the dot 
product or inverse implementations. The key to making this work 
is that you also need a mid.ndslice.arithmetic or something that 
allows the user to call these functions without operator 
overloading. This way they can put import their default, but if 
they have a function that mixes and matches dot and element-wise 
multiplication, they can do that too.
Sep 27 2017
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 27 September 2017 at 17:41, Ilya Yaroshenko via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 27 September 2017 at 04:59:10 UTC, Manu wrote:


 On 26 September 2017 at 21:41, Atila Neves via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko wrote:

 Should we add `a * b` to ndslice for 1d vectors?

 Discussion at https://github.com/libmir/mir-algorithm/issues/91


 I'd say yes.

 Atila


 Just remember, it's okay to vote no! Even if it makes you a bigoted dick
 ;)
 In this case, I think 'no' is the only reasonable choice.
 If this is going to seriously be considered, then ndslice should
 definitely
 be renamed to 'matrix'.

 An alternative solution might be to introduce a wrapper of ndslice called
 'matrix' that supports matrix mul...?


 I would prefer outer operator overloading be added to D instead of type
 wrappers. So a user can import a library for operations, rather then
 library of wrappers. --Ilya


I don't really think this will work in a reasonable way in D even if it
were added.
D does not have ADL, which will almost certainly lead to _very_ nasty
surprises in behaviour.
Sep 27 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/27/2017 4:21 PM, Manu wrote:

 D does not have ADL,


Thank gawd! :-)


 which will almost certainly lead to _very_ nasty surprises in behaviour.


ADL was always a hack to get around the wretched overloading symbol lookup 
behavior in C++. I see it has somehow morphed into a feature :-( but I see no 
advantage to it over D's approach (the reverse operand lookup scheme).
Sep 27 2017
next sibling parent reply Wyatt <wyatt.epp gmail.com> writes:
On Thursday, 28 September 2017 at 01:58:24 UTC, Walter Bright 
wrote:

 ADL was always a hack to get around the wretched overloading 
 symbol lookup behavior in C++.


Any sufficiently advanced bug is indistinguishable from a 
feature! ;)

-Wyatt
Sep 28 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/28/2017 7:42 AM, Wyatt wrote:

 Any sufficiently advanced bug is indistinguishable from a feature! ;)


D has other ways of doing what ADL does, so I am curious for an example from 
Manu about why it doesn't work.
Sep 30 2017
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 30.09.2017 23:45, Walter Bright wrote:

 ...
 D has other ways of doing what ADL does,


What are those ways? I'm aware of two basic strategies, both suboptimal:

- Every module imports all other modules.
- Proliferation of wrapper types.


 so I am curious for an example from Manu about why it doesn't work.


It's not per se related to operator overloading:

---
module a;
import std.range: isInputRange;
auto sum(R)(R r)if(isInputRange!R){
     typeof(r.front) result;
     for(auto t=r.save;!t.empty;t.popFront())
         result+=t.front;
     return result;
}
---

---
module b;
import a;
import std.range;

void main(){
     int[] a = [1,2,3,4];
     import std.stdio: writeln;
     writeln(a.front); // ok
     writeln(sum(a)); // error, the type is an input range, yet has no front
}
---

I.e., the operations that are supported on the type differ depending on 
the module that it is accessed from. If there are multiple definitions 
of the same name, different modules might not agree which one is being 
referred to. (This is particularly likely for overloaded operators, as 
the set of names is finite and small. This is what Manu means when he 
says it can lead to nasty surprises. This is very plausible, but I don't 
have a good example.)
Oct 02 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/2/2017 4:15 AM, Timon Gehr wrote:

 On 30.09.2017 23:45, Walter Bright wrote:

 ...
 D has other ways of doing what ADL does,


 
 What are those ways? I'm aware of two basic strategies, both suboptimal:
 
 - Every module imports all other modules.
 - Proliferation of wrapper types.


https://dlang.org/spec/operatoroverloading.html#binary

C++ does not have this notion.



 It's not per se related to operator overloading:


ADL was specifically intended to address operator overloading.



 ---
 module a;
 import std.range: isInputRange;
 auto sum(R)(R r)if(isInputRange!R){
      typeof(r.front) result;
      for(auto t=r.save;!t.empty;t.popFront())
          result+=t.front;
      return result;
 }
 ---
 
 ---
 module b;
 import a;
 import std.range;
 
 void main(){
      int[] a = [1,2,3,4];
      import std.stdio: writeln;
      writeln(a.front); // ok
      writeln(sum(a)); // error, the type is an input range, yet has no front
 }
 ---
 
 I.e., the operations that are supported on the type differ depending on the 
 module that it is accessed from. If there are multiple definitions of the same 
 name, different modules might not agree which one is being referred to. (This
is 
 particularly likely for overloaded operators, as the set of names is finite
and 
 small. This is what Manu means when he says it can lead to nasty surprises.
This 
 is very plausible, but I don't have a good example.)


This gets into the anti-hijacking support D has (and C++ does not). If multiple 
modules with declarations of `writeln` are in scope, the compiler attempts a 
match with `writeln` in each module. If there is a match with more than one 
module, an error is issued. The user will then have to specify which one he
wants.

This is specifically designed to prevent nasty surprises. C++ has a big problem 
with ADL in that overloading is not encapsulated and any #included header can 
inadvertently add more overloads that may be entirely unrelated. Any .h can 
crack open any namespace and insert more overloads into it. It's completely 
unhygienic and uncontrollable.

As for the specific example you gave, I get:

a.d(3): Error: no property 'front' for type 'int[]'
a.d(4): Error: no property 'save' for type 'int[]'
b.d(8): Error: template instance a.sum!(int[]) error instantiating
Oct 03 2017
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 3 October 2017 at 19:25:32 UTC, Walter Bright wrote:

 This gets into the anti-hijacking support D has (and C++ does 
 not). If multiple modules with declarations of `writeln` are in 
 scope, the compiler attempts a match with `writeln` in each 
 module. If there is a match with more than one module, an error 
 is issued. The user will then have to specify which one he 
 wants.


Anti-hacking might make it tricky to have operators as 
free-standing functions, but I'm not sure it would be impossible. 
The solution for function overloading is fully qualified names. 
However, this does not make sense operator overloading. Mixing 
operator overloading and D's anti-hijacking, you would basically 
be prevented from having two overloaded versions of an operator. 
This means that if you want multiple versions of operators, you 
would not be able to have the operators as member functions and 
they would have to be free-standing functions, ideally in 
separate modules.

With respect to the earlier discussion of type wrappers, this can 
also be implemented with a template parameters that controls the 
operator-overloading of *, something like below.

struct Foo(bool mtimesOverload = false)
{
     template opBinary(string op = "*")(Foo foo)
     {
         static if (mtimesOverload)
         {
             return mtimes(this, foo);
         }
         else
         {
             return times(this, foo);
         }
     }
}

So you could do a Matrix alias that has mtimesOverload=true.
Oct 03 2017
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/3/17 3:25 PM, Walter Bright wrote:

 This is specifically designed to prevent nasty surprises. C++ has a big 
 problem with ADL in that overloading is not encapsulated and any 
 #included header can inadvertently add more overloads that may be 
 entirely unrelated. Any .h can crack open any namespace and insert more 
 overloads into it. It's completely unhygienic and uncontrollable.


One thing I would like to have control over is how 'this' is passed.

A nice thing about UFCS is I can use it to pass-by-value the faux 'this' 
parameter, whereas I cannot do this with a member function. It makes a 
difference in some cases, especially with const functions.

It also can be cheaper to pass a small struct by value.

None of this is possible with operator overloading. A way out could 
possibly be to alias a non-member function for the operator.

-Steve
Oct 03 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/3/2017 5:24 PM, Steven Schveighoffer wrote:

 It also can be cheaper to pass a small struct by value.


Should not be a problem if the compiler inlines it.
Oct 03 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/3/17 10:00 PM, Walter Bright wrote:

 On 10/3/2017 5:24 PM, Steven Schveighoffer wrote:

 It also can be cheaper to pass a small struct by value.


 
 Should not be a problem if the compiler inlines it.


That's not always possible.

-Steve
Oct 05 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/5/2017 6:13 AM, Steven Schveighoffer wrote:

 On 10/3/17 10:00 PM, Walter Bright wrote:

 On 10/3/2017 5:24 PM, Steven Schveighoffer wrote:

 It also can be cheaper to pass a small struct by value.


 Should not be a problem if the compiler inlines it.


 
 That's not always possible.


Right. But then the question becomes how much more complexity do we want to add 
chasing that last percent of perfection?

For example, right now I'm in my 3rd day of attempting to resolve

   https://issues.dlang.org/show_bug.cgi?id=17635

which is a regression brought about by layers and layers of fixes over time for 
a seemingly simple issue - implicitly converting a unique return from a pure 
function into an immutable.

For another example, it took Martin and I months to implement the new import 
lookup scheme. It turned out to be fairly complicated, and there were many 
regressions. There are probably still issues lurking in it.

We need to keep the language rules simple enough to understand and simple
enough 
to implement, and there will be compromises with that.
Oct 05 2017
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.10.2017 21:40, Walter Bright wrote:


 
 Right. But then the question becomes how much more complexity do we want 
 to add chasing that last percent of perfection?
 
 For example, right now I'm in my 3rd day of attempting to resolve
 
 https://issues.dlang.org/show_bug.cgi?id=17635
 
 which is a regression brought about by layers and layers of fixes over 
 time for a seemingly simple issue - implicitly converting a unique 
 return from a pure function into an immutable.
 ...


Sounds like the code might need a rewrite.


 For another example, it took Martin and I months to implement the new 
 import lookup scheme. It turned out to be fairly complicated, and there 
 were many regressions. There are probably still issues lurking in it.
 ...


Do you have some examples of why it is complicated? (I'm curious whether 
there is a good way to simplify it.)


 We need to keep the language rules simple enough to understand and 
 simple enough to implement, and there will be compromises with that.


It is however also important to not conflate implementation effort in 
DMD due to evolvability issues with complexity of the feature. (Of 
course, pragmatically, it will have some influence on the language 
design, but ideally it should not.)



 Oct 05 2017



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 1:13 PM, Timon Gehr wrote:

 On 05.10.2017 21:40, Walter Bright wrote:


 Right. But then the question becomes how much more complexity do we want to 
 add chasing that last percent of perfection?

 For example, right now I'm in my 3rd day of attempting to resolve

 https://issues.dlang.org/show_bug.cgi?id=17635

 which is a regression brought about by layers and layers of fixes over time 
 for a seemingly simple issue - implicitly converting a unique return from a 
 pure function into an immutable.
 ...


 
 Sounds like the code might need a rewrite.


I did rewrite much of it:

   https://github.com/dlang/dmd/pull/7179
   https://github.com/dlang/dmd/pull/7186

There's a lot of accumulated technical debt in the compiler.



 For another example, it took Martin and I months to implement the new import 
 lookup scheme. It turned out to be fairly complicated, and there were many 
 regressions. There are probably still issues lurking in it.


 Do you have some examples of why it is complicated? (I'm curious whether there 
 is a good way to simplify it.)


Not offhand. It's been too long since I worked on it (and then Martin more or 
less redid it). Martin deserves a lot of credit for that, it was a dirty job
and 
he did it marvelously.



 We need to keep the language rules simple enough to understand and simple 
 enough to implement, and there will be compromises with that.


 It is however also important to not conflate implementation effort in DMD due
to 
 evolvability issues with complexity of the feature. (Of course, pragmatically, 
 it will have some influence on the language design, but ideally it should not.)


Back in the early days of C++, around 1990 or so, it was popular among C++ 
committee members to say things like "compiler implementation difficulty is not 
a consideration." Well, they produced a language design that literally takes 10 
years to implement, and 10-15 years of disastrous compiler problems from all
the 
vendors. It really didn't get straightened out until 2005 or so.

Even for something that's not C++'s fault, the C preprocesser, I scrapped and 
completely rewrote it 4 times, I believe. I think I finally got it right with 
Warp :-)

I've got a lot of scar tissue from "compiler implementation difficulty is not a 
consideration." I don't believe that is a necessary path to a high quality 
language. Compiler complexity is a huge red flag that something is wrong with 
the design of the language.

The compiler implementation should be a joy to read and someone should be able 
to read the spec, read the implementation of the spec, and go yeah, of course, 
this is obviously correct, it's so simple!



 Oct 07 2017



 next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 07.10.2017 09:24, Walter Bright wrote:

 
 I've got a lot of scar tissue from "compiler implementation difficulty 
 is not a consideration." I don't believe that is a necessary path to a 
 high quality language. Compiler complexity is a huge red flag that 
 something is wrong with the design of the language.
 
 The compiler implementation should be a joy to read and someone should 
 be able to read the spec, read the implementation of the spec, and go 
 yeah, of course, this is obviously correct, it's so simple!


I fully agree, my point was more that details of a specific 
implementation should not influence the language design (too much).

What should matter more is compiler implementation difficulty "from 
scratch", assuming all features that need to be supported are known from 
the beginning. We don't really need technical debt in the language 
specification.



 Oct 07 2017



prev sibling  parent  Jacob Carlborg <doob me.com>  writes:


On 2017-10-07 09:24, Walter Bright wrote:


 Even for something that's not C++'s fault, the C preprocesser, I 
 scrapped and completely rewrote it 4 times, I believe. I think I finally 
 got it right with Warp :-)


You mean you got it right with D ;)

-- 
/Jacob Carlborg



 Oct 09 2017



prev sibling next sibling parent reply Joakim <dlang joakim.fea.st>  writes:


On Thursday, 5 October 2017 at 19:40:05 UTC, Walter Bright wrote:

 On 10/5/2017 6:13 AM, Steven Schveighoffer wrote:

 On 10/3/17 10:00 PM, Walter Bright wrote:

 On 10/3/2017 5:24 PM, Steven Schveighoffer wrote:

 It also can be cheaper to pass a small struct by value.


 Should not be a problem if the compiler inlines it.


 
 That's not always possible.


 Right. But then the question becomes how much more complexity 
 do we want to add chasing that last percent of perfection?

 For example, right now I'm in my 3rd day of attempting to 
 resolve

   https://issues.dlang.org/show_bug.cgi?id=17635

 which is a regression brought about by layers and layers of 
 fixes over time for a seemingly simple issue - implicitly 
 converting a unique return from a pure function into an 
 immutable.

 For another example, it took Martin and I months to implement 
 the new import lookup scheme. It turned out to be fairly 
 complicated, and there were many regressions. There are 
 probably still issues lurking in it.

 We need to keep the language rules simple enough to understand 
 and simple enough to implement, and there will be compromises 
 with that.


The impression I have, correct me if I'm wrong, is that users- 
and since you're writing a compiler, your users are developers 
themselves- pay no attention to simplicity and principles and 
expect software to always guess correctly about whatever they 
intended, similar to how dmd offers spelling suggestions for 
other functions when you typo a function name.

Sometimes this is possible in simple, non-invasive ways like 
spelling suggestions, but you will go mad trying to create and 
maintain a complex system that mirrors their contradictory 
expectations.  Without having delved into the complex import 
lookup scheme you and Martin set up, I agree with Timon that 
someone should try to boil it down to some simple principles 
again.



 Oct 05 2017



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 3:44 PM, Joakim wrote:

 Sometimes this is possible in simple, non-invasive ways like spelling 
 suggestions, but you will go mad trying to create and maintain a complex
system 
 that mirrors their contradictory expectations.  Without having delved into
the 
 complex import lookup scheme you and Martin set up, I agree with Timon that 
 someone should try to boil it down to some simple principles again.


It's something we all wish for. But keep in mind that the new import rules
broke 
existing code, and so both lookup systems are still there in the compiler, 
switch selectable. A third system, no, I just can't stomach that.

To be brutally honest, this sort of thing is also a low priority. We've got a 
lot of far more important issues to resolve. Like these:

https://issues.dlang.org/buglist.cgi?bug_severity=regression&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&list_id=217099&query_format=advanced

Everybody likes to work on new features, but this is bread and butter stuff
that 
pays the electric bill that we dare not neglect.



 Oct 07 2017



prev sibling  parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 10/5/17 3:40 PM, Walter Bright wrote:

 On 10/5/2017 6:13 AM, Steven Schveighoffer wrote:

 On 10/3/17 10:00 PM, Walter Bright wrote:

 On 10/3/2017 5:24 PM, Steven Schveighoffer wrote:

 It also can be cheaper to pass a small struct by value.


 Should not be a problem if the compiler inlines it.


 That's not always possible.


 
 Right. But then the question becomes how much more complexity do we want 
 to add chasing that last percent of perfection?


It all depends on how much you value convenience of operators. I can 
already do something like:

a.add(b);

which would "fix" the issue.


 
 For example, right now I'm in my 3rd day of attempting to resolve
 
    https://issues.dlang.org/show_bug.cgi?id=17635
 
 which is a regression brought about by layers and layers of fixes over 
 time for a seemingly simple issue - implicitly converting a unique 
 return from a pure function into an immutable.
 
 For another example, it took Martin and I months to implement the new 
 import lookup scheme. It turned out to be fairly complicated, and there 
 were many regressions. There are probably still issues lurking in it.


The import changes modified how existing code worked. This is not the 
same thing. Of course you need to take more care of features that are 
going to break a lot of projects and make sure they don't. I spent a 
long time fixing the cycle detection, but had to figure out a way to do 
it such that it allowed people to use the old broken detection if they 
wanted to. That was not an easy thing to do, and took about 4 iterations.


 We need to keep the language rules simple enough to understand and 
 simple enough to implement, and there will be compromises with that.


I understand, and agree. I was just noting one reason why allowing 
operator overloading outside the type itself could be useful. It doesn't 
even have to be full-blown UFCS, it could be as simple as aliasing a 
member to an external function, which doesn't require any new changes to 
lookup rules.

In addition, inlining doesn't get around certain requirements for 
calling (e.g. the nice property of being able to implicitly cast away 
const).

-Steve



 Oct 06 2017



prev sibling next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Tuesday, 3 October 2017 at 19:25:32 UTC, Walter Bright wrote:

 This is specifically designed to prevent nasty surprises. C++ 
 has a big problem with ADL in that overloading is not 
 encapsulated and any #included header can inadvertently add 
 more overloads that may be entirely unrelated. Any .h can crack 
 open any namespace and insert more overloads into it. It's 
 completely unhygienic and uncontrollable.

 As for the specific example you gave, I get:

 a.d(3): Error: no property 'front' for type 'int[]'
 a.d(4): Error: no property 'save' for type 'int[]'
 b.d(8): Error: template instance a.sum!(int[]) error 
 instantiating


But you can't deny our solution eats expressive power: If you 
don't want to change code you're importing, you have to write a 
wrapper type for int[] here. Alias this helps, but because save() 
and slicing operators have to return the type of this, there's 
still manual work to do if you want Phobos algorithms to utilize 
it's random access.

It may be that ADL or something similar would cause too much 
trouble to be worth it, don't know about that. But what I'm 
saying that we definitely have a considerable problem here and it 
would solve it.



 Oct 04 2017



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 10/4/2017 2:28 AM, Dukc wrote:

 But you can't deny our solution eats expressive power: If you don't want to 
 change code you're importing, you have to write a wrapper type for int[] here. 


Please present an example.



 Oct 04 2017



 next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com>  writes:


On Wednesday, 4 October 2017 at 17:56:16 UTC, Walter Bright wrote:

 On 10/4/2017 2:28 AM, Dukc wrote:

 But you can't deny our solution eats expressive power: If you 
 don't want to change code you're importing, you have to write 
 a wrapper type for int[] here.


 Please present an example.


I think that was the point of Timon's example. If you have a 
module A that implements a range algorithm, a module B that 
defines a range-like type (but actually its member function not 
matching the exact signature of range primitives), you (module C) 
as user of modules A and B are not able to provide range 
primitive wrapper functions for the type defined in module B 
(which you can't modify). So you can't use A's range algorithm on 
type defined in B.

ADL solves this problem (adapting third-party libraries to your 
needs).

Since D's modules are closed (can't be extended from the outside 
like namespaces), if we want to support some form of ADL (the 
primary use-case being algorithm libraries), we would probably 
need to introduce some form of open for extension scopes like 
namespaces. Or change extern (C++, namespace) to behave like 
people coming from C++ may expect.



 Oct 04 2017



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 10/4/2017 3:35 PM, Petar Kirov [ZombineDev] wrote:

 On Wednesday, 4 October 2017 at 17:56:16 UTC, Walter Bright wrote:

 On 10/4/2017 2:28 AM, Dukc wrote:

 But you can't deny our solution eats expressive power: If you don't want to 
 change code you're importing, you have to write a wrapper type for int[] here.


 Please present an example.


 
 I think that was the point of Timon's example.


An example would be appreciated. Timon's example requires guesswork as to what 
he intended, because it does not compile in ways unrelated to his point.



 Oct 04 2017



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2017-10-05 00:59, Walter Bright wrote:


 An example would be appreciated. Timon's example requires guesswork as 
 to what he intended, because it does not compile in ways unrelated to 
 his point.


It's supposed to not compile, because D doesn't have ADL.

$ cat foo.d
module foo;
import std.range: isInputRange;

auto sum(R)(R r)if(isInputRange!R){
     typeof(r.front) result;
     for(auto t=r.save;!t.empty;t.popFront())
         result+=t.front;
     return result;
}

$ cat main.d
import foo;
import std.range;

void main(){
     int[] a = [1,2,3,4];
     import std.stdio: writeln;
     writeln(a.front); // ok
     writeln(sum(a)); // error, the type is an input range, yet has no front
}

$ dmd foo.d main.d
foo.d(5): Error: no property 'front' for type 'int[]'
foo.d(6): Error: no property 'save' for type 'int[]'
main.d(8): Error: template instance foo.sum!(int[]) error instantiating

Adding the following line to the "foo" module fixes the problem:

import std.range : front, save, empty, popFront;

  $ cat foo.d
module foo;
import std.range: isInputRange;
import std.range : front, save, empty, popFront;
auto sum(R)(R r)if(isInputRange!R){
     typeof(r.front) result;
     for(auto t=r.save;!t.empty;t.popFront())
         result+=t.front;
     return result;
}

$ dmd foo.d -run main.d
1
10

-- 
/Jacob Carlborg



 Oct 04 2017



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/4/2017 11:44 PM, Jacob Carlborg wrote:

 On 2017-10-05 00:59, Walter Bright wrote:
 

 An example would be appreciated. Timon's example requires guesswork as to what 
 he intended, because it does not compile in ways unrelated to his point.


 
 It's supposed to not compile, because D doesn't have ADL.


If that is what Timon intended, ok. It wasn't clear to me. But I'm left 
wondering what the issue variously described as suboptimal, nasty, no solution, 
etc., is.

The worst I can see is if you didn't import the relevant modules, you'll get a 
compiler error.



 Oct 05 2017



prev sibling next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Wednesday, 4 October 2017 at 17:56:16 UTC, Walter Bright wrote:

 Please present an example.


Let's say you're importing a C library which defines random 
function generators. They may be more random than Phobos rngs, 
they might be crytpo secure or whatever so you want to use them. 
It could be like:

module crng;

extern(c) struct Crng
{   int seedA;
     int seedB;
     ubyte[] restOfTheData;
}

extern(c) int current(Crng*);
extern(c) void toNextValue(Crng*);
extern(c) Crng initByTime(int unixTime);
//...

But you also want a probos-style range interface for them. You 
have to wrap the c struct into a new struct type:

struct Crng
{   crng.Crng _impl;
     alias _impl this;
     auto front(){return current(*_impl);}
     void popFront(){toNextValue(*_impl);}
     //...
}

and you have to rewrite many wrappers for Crng functions despite 
the alias this because they either require return Crng or gequire 
a pointer to one. This needs to be defined manually for example:

Crng initByTime(int time){return Crng(crng.initByTime(time))};

With ADL it would be enough to extend the original struct with 
range primitives:

auto front(Crng range){return current(*range);}
void popFront(ref Crng range){toNextValue(*range);}
//...

With current semantics the latter example will work only with 
locally defined range function templates. Not with Phobos ones, 
because they cannot see the extension functions.

Note, I do not have a D compiler in where I posted these from so 
they're not checked for compilation errors.



 Oct 05 2017



 next sibling parent  Dukc <ajieskola gmail.com>  writes:


On Thursday, 5 October 2017 at 08:27:14 UTC, Dukc wrote:

 and you have to rewrite many wrappers for Crng functions 
 despite the alias this because they either require return Crng 
 or gequire a pointer to one.


Of course, If we could find a way to automate this universally, 
there would be much less if any need for ADL.



 Oct 05 2017



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


Seems to me just add:

   public import crng;

to the module defining struct Crng.



 Oct 05 2017



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 05.10.2017 11:21, Walter Bright wrote:

 Seems to me just add:
 
    public import crng;
 
 to the module defining struct Crng.


(That module is crng itself.)

The "fix" is to add

     public import crng;

to _std.range_.



 Oct 05 2017



 next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Thursday, 5 October 2017 at 09:26:44 UTC, Timon Gehr wrote:

 (That module is crng itself.)


Depends on if he meant the extern (c) Crng or the 
range-implementing Crng. But would still not work, unless i have 
misunderstood something.


 The "fix" is to add

     public import crng;

 to _std.range_.


Yes, would work. I don't think it needs explanation why it's, as 
you said, more like a "fix" than a fix.



 Oct 05 2017



  parent  Dukc <ajieskola gmail.com>  writes:


On Thursday, 5 October 2017 at 09:39:58 UTC, Dukc wrote:

 On Thursday, 5 October 2017 at 09:26:44 UTC, Timon Gehr wrote:

 The "fix" is to add

     public import crng;

 to _std.range_.


 Yes, would work. I don't think it needs explanation why it's, 
 as you said, more like a "fix" than a fix.


Correction: Wouldn't quite do it, std.range would need to import 
the extension functions to crng, not it's C api. And it needn't 
to be a public import, if you also import it to it's child 
modules it happens to import. Otherwise correct.



 Oct 05 2017



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


I suggest we continue where I made a more extensive reply to you.



 Oct 05 2017



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


After reading Timon's message, I understand your point a lot better. See my 
reply to Timon.


 and you have to rewrite many wrappers for Crng functions despite the alias 


this because they either require return Crng or gequire a pointer to one.

This is a good point. Let me think about it.



 Oct 05 2017



prev sibling  parent reply nkm1 <t4nk074 openmailbox.org>  writes:


On Thursday, 5 October 2017 at 08:27:14 UTC, Dukc wrote:

 and you have to rewrite many wrappers for Crng functions 
 despite the alias this because they either require return Crng 
 or gequire a pointer to one. This needs to be defined manually 
 for example:

 Crng initByTime(int time){return Crng(crng.initByTime(time))};

 With ADL it would be enough to extend the original struct with 
 range primitives:

 auto front(Crng range){return current(*range);}
 void popFront(ref Crng range){toNextValue(*range);}
 //...

 With current semantics the latter example will work only with 
 locally defined range function templates. Not with Phobos ones, 
 because they cannot see the extension functions.


Am I missing something? You can already extend the original 
struct:

extern(c) struct Crng
{   int seedA;
     int seedB;
     ubyte[] restOfTheData;

     extern (D) {
         // or without extern (D)...
         auto front() { return current(&this); }
         void popFront() { toNextValue(&this); }
     }
}

What does it matter if you put your extension functions inside 
the struct or outside of it? (same question to Timon). Unless you 
also propose "extending" the modules themselves (like reopening 
namespaces in C++? This is a whole another can of worms...)
Looks like what most people want are extension methods, not ADL?



 Oct 05 2017



 next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Thursday, 5 October 2017 at 10:07:31 UTC, nkm1 wrote:

 Am I missing something? You can already extend the original 
 struct:

 extern(c) struct Crng
 {   int seedA;
     int seedB;
     ubyte[] restOfTheData;

     extern (D) {
         // or without extern (D)...
         auto front() { return current(&this); }
         void popFront() { toNextValue(&this); }
     }
 }


Does that work? If so, good. But still not optimal, because you 
should be able to extend the functionality of code without 
changing it.


 Looks like what most people want are extension methods, not ADL?


We already have extension methods, but rather we want extension 
methods that work with other extension methods they don't know 
of. But I think that's what you meant and in this case the answer 
at least in my case is yes.

Unrelated: Since when have C struct had array members? I'm not so 
smart in my examples X).



 Oct 05 2017



 next sibling parent reply nkm1 <t4nk074 openmailbox.org>  writes:


On Thursday, 5 October 2017 at 10:23:17 UTC, Dukc wrote:

 Does that work? If so, good. But still not optimal, because you 
 should be able to extend the functionality of code without 
 changing it.


It works, yes. The point is, additional methods in the struct 
body, and free standing functions outside of the body, but in the 
same module, is basically the same thing in D (I believe Andrei 
already mentioned that).


 Looks like what most people want are extension methods, not 
 ADL?


 We already have extension methods, but rather we want extension 
 methods that work with other extension methods they don't know 
 of. But I think that's what you meant and in this case the 
 answer at least in my case is yes.


Ah, right. Well, anyway, that's not ADL by itself, since ADL only 
looks in the namespace of the argument type (so it won't find 
your extension methods if they're in some different 
namespace/module).


 Unrelated: Since when have C struct had array members? I'm not 
 so smart in my examples X).


Isn't that the "struct hack" (aka "flexible array member")? :)



 Oct 05 2017



 next sibling parent  nkm1 <t4nk074 openmailbox.org>  writes:


On Thursday, 5 October 2017 at 11:22:27 UTC, nkm1 wrote:

 Isn't that the "struct hack" (aka "flexible array member")? :)


(I mean, it would be in C :)



 Oct 05 2017



prev sibling  parent  Dukc <ajieskola gmail.com>  writes:


On Thursday, 5 October 2017 at 11:22:27 UTC, nkm1 wrote:


 It works, yes. The point is, additional methods in the struct 
 body, and free standing functions outside of the body, but in 
 the same module, is basically the same thing in D (I believe 
 Andrei already mentioned that).


But I meant extending the struct remotely from a different module.


 Ah, right. Well, anyway, that's not ADL by itself, since ADL 
 only looks in the namespace of the argument type (so it won't 
 find your extension methods if they're in some different 
 namespace/module).


Can well be, I don't know ADL precisely. Probably it should not 
be used as-is then, if at all.



 Oct 05 2017



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 3:23 AM, Dukc wrote:

 Since when have C struct had array members?


Since the dawn of time :-)

The problems come from when you use a D only type as a function parameter type 
in a C++ function. Then, the C++ function mangler fails because, well, the C++ 
community has sadly neglected to add D types to the C++ ABI :-)

When cases like that crop up, I switch it to C mangling, which means no 
mangling, so no problems! and hopefully overloading isn't involved.
(Overloading 
isn't supported with C functions, because overloading relies on name mangling 
and C functions are not mangled.)



 Oct 05 2017



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05.10.2017 12:07, nkm1 wrote:

 Am I missing something? You can already extend the original struct:
 
 extern(c) struct Crng
 {   int seedA;
      int seedB;
      ubyte[] restOfTheData;
 
      extern (D) {
          // or without extern (D)...
          auto front() { return current(&this); }
          void popFront() { toNextValue(&this); }
      }
 }
 
 What does it matter if you put your extension functions inside the 
 struct or outside of it? (same question to Timon)


For example:

- Methods force the 'this' argument to be references (for structs) or 
values (for classes).

- Adding functionality to a type in the module where it is defined is 
not the only use case. Wrapper types can be more cumbersome to use than 
the original type.



 Oct 05 2017



prev sibling  parent  Dukc <ajieskola gmail.com>  writes:


On Wednesday, 4 October 2017 at 17:56:16 UTC, Walter Bright wrote:

 Please present an example.


Let's say you're importing a C library which defines random 
function generators. They may be more random than Phobos rngs, 
they might be crytpo secure or whatever so you want to use them. 
It could be like:

module crng;

extern(c) struct Crng
{   int seedA;
     int seedB;
     ubyte[] restOfTheData;
}

extern(c) int current(Crng*);
extern(c) void toNextValue(Crng*);
extern(c) Crng initByTime(int unixTime);
//...

But you also want a phobos-style range interface for them. You 
have to wrap the c struct into a new struct type:

struct Crng
{   crng.Crng _impl;
     alias _impl this;
     auto front(){return current(*_impl);}
     void popFront(){toNextValue(*_impl);}
     //...
}

and you have to rewrite many wrappers for Crng functions despite 
the alias this because they either require return Crng or gequire 
a pointer to one. This needs to be defined manually for example:

Crng initByTime(int time){return Crng(crng.initByTime(time))};

With ADL it would be enough to extend the original struct with 
range primitives:

auto front(Crng range){return current(*range);}
void popFront(ref Crng range){toNextValue(*range);}
//...

With current semantics the latter example will work only with 
locally defined range function templates. Not with Phobos ones, 
because they cannot see the extension functions.

Note, I do not have a D compiler in where I posted these from so 
they're not checked for compilation errors.



 Oct 05 2017



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 03.10.2017 21:25, Walter Bright wrote:

 On 10/2/2017 4:15 AM, Timon Gehr wrote:

 On 30.09.2017 23:45, Walter Bright wrote:

 ...
 D has other ways of doing what ADL does,


 What are those ways? I'm aware of two basic strategies, both suboptimal:

 - Every module imports all other modules.
 - Proliferation of wrapper types.


 
 https://dlang.org/spec/operatoroverloading.html#binary
 
 C++ does not have this notion.
 
 

 It's not per se related to operator overloading:


 
 ADL was specifically intended to address operator overloading.
 ...


It's easy to explain why: In C++, operators are the _only_ functions 
that have UFCS.

This is in stark contrast to D, where all functions _but_ operators have 
UFCS.

The proposal was allow UFCS also for overloaded operators.

Hence, this discussion is about UFCS. These are not really operator 
overloading issues.


 

 ---
 module a;
 import std.range: isInputRange;
 auto sum(R)(R r)if(isInputRange!R){
      typeof(r.front) result;
      for(auto t=r.save;!t.empty;t.popFront())
          result+=t.front;
      return result;
 }
 ---

 ---
 module b;
 import a;
 import std.range;

 void main(){
      int[] a = [1,2,3,4];
      import std.stdio: writeln;
      writeln(a.front); // ok
      writeln(sum(a)); // error, the type is an input range, yet has no 
 front
 }
 ---

 I.e., the operations that are supported on the type differ depending 
 on the module that it is accessed from. If there are multiple 
 definitions of the same name, different modules might not agree which 
 one is being referred to. (This is particularly likely for overloaded 
 operators, as the set of names is finite and small. This is what Manu 
 means when he says it can lead to nasty surprises. This is very 
 plausible, but I don't have a good example.)


 
 This gets into the anti-hijacking support D has (and C++ does not). If 
 multiple modules with declarations of `writeln` are in scope, the 
 compiler attempts a match with `writeln` in each module. If there is a 
 match with more than one module, an error is issued. The user will then 
 have to specify which one he wants.
 ...


UFCS allows hijacking. For an example, see:
https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85

Commenting out 'private' caused a stack overflow.


 This is specifically designed to prevent nasty surprises. C++ has a big 
 problem with ADL in that overloading is not encapsulated and any 
 #included header can inadvertently add more overloads that may be 
 entirely unrelated. Any .h can crack open any namespace and insert more 
 overloads into it. It's completely unhygienic and uncontrollable.
 
 As for the specific example you gave, I get:
 
 a.d(3): Error: no property 'front' for type 'int[]'
 a.d(4): Error: no property 'save' for type 'int[]'
 b.d(8): Error: template instance a.sum!(int[]) error instantiating


(There is a comment in the code noting that it will not compile.)

The intention of the code was to demonstrate that a type can pass 
isInputRange in the same module in which it does not support front. This 
is an example of surprising name lookup behavior.

Of course there is also the opposite problem. You can have a type that 
supports all range primitives via UFCS but does not pass isInputRange, 
because Phobos does not import the module where the primitives are 
defined. (This particular case is sometimes solved by ADL, sometimes not.)

---

struct Iota{ private int a,b; }
auto iota(int a,int b){ return Iota(a,b); }

 property int front(Iota i){ return i.a; }
 property bool empty(Iota i){ return i.a>=i.b; }
void popFront(ref Iota i){ ++i.a; }

void main(){
     import std.stdio;
     for(auto r=iota(0,10);!r.empty;r.popFront){ // ok
         writeln(r.front);
     }
     import std.algorithm;
     iota(0,10).each!writeln; // error
     foreach(i;iota(0,10)) writeln(i); // error
}

---

If I copy-paste portions of std.range into the main module of the above 
program instead of importing, I will be able to use my custom type with 
those Phobos ranges.



 Oct 05 2017



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 2:13 AM, Timon Gehr wrote:

 It's easy to explain why: In C++, operators are the _only_ functions that have 
 UFCS.
 
 This is in stark contrast to D, where all functions _but_ operators have UFCS.
 
 The proposal was allow UFCS also for overloaded operators.
 
 Hence, this discussion is about UFCS. These are not really operator
overloading 
 issues.


Ok, but I'm not sure what the proposal was.



 UFCS allows hijacking. For an example, see:
 https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85


That may be a bug in the compiler. Can you produce a small test case? I know 
that some of the UFCS code was written without regard to hijacking.



 (There is a comment in the code noting that it will not compile.)


Right, but I wasn't sure what errors you expected to see. I hate to assume with 
these sorts of things, as people often post examples with errors they didn't 
intend. This is why I ask for examples, and it's nice to also point out the
errors.



 The intention of the code was to demonstrate that a type can pass isInputRange 
 in the same module in which it does not support front. This is an example of 
 surprising name lookup behavior.


I submit it is surprising only if you're used to ADL :-)

ADL in C++ makes lookups bafflingly complex, and I doubt many C++ people 
actually understand it. It's like C++ overload resolution, nobody understands 
it, not even me (and I implemented it to the letter of the spec!). Some will 
argue it "just works", and no understanding is necessary, and I suppose that's 
fine until one gets an incomprehensible compiler error.

D's name lookup rules were quite simple, and deliberately set up that way. 
Unfortunately, most people thought they were unintuitive. It turns out that 
simple algorithms are not intuitive, and we now have a fairly complex lookup 
system. Martin and I implemented it, and probably neither of us completely 
understands it. I find it regrettable that things have gotten to that state.



 Of course there is also the opposite problem. You can have a type that
supports 
 all range primitives via UFCS but does not pass isInputRange, because Phobos 
 does not import the module where the primitives are defined. (This particular 
 case is sometimes solved by ADL, sometimes not.)
 
 ---
 
 struct Iota{ private int a,b; }
 auto iota(int a,int b){ return Iota(a,b); }
 
  property int front(Iota i){ return i.a; }
  property bool empty(Iota i){ return i.a>=i.b; }
 void popFront(ref Iota i){ ++i.a; }
 
 void main(){
      import std.stdio;
      for(auto r=iota(0,10);!r.empty;r.popFront){ // ok
          writeln(r.front);
      }
      import std.algorithm;
      iota(0,10).each!writeln; // error
      foreach(i;iota(0,10)) writeln(i); // error
 }
 
 ---
 
 If I copy-paste portions of std.range into the main module of the above
program 
 instead of importing, I will be able to use my custom type with those Phobos 
 ranges.


I understand this one, as it helpfully says the expected error :-) thank you.

I suggest for this case writing a wrapper type for Iota, with 
front/empty/popFront as members of that wrapper, then it should be good to go. 
It's hardly any more work than writing the free functions. Doing this kind of 
wrapper doesn't work for operator overloading, which brings us back to ADL is 
for operator overloading.



 Oct 05 2017



 next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 05.10.2017 11:52, Walter Bright wrote:

 On 10/5/2017 2:13 AM, Timon Gehr wrote:

 It's easy to explain why: In C++, operators are the _only_ functions 
 that have UFCS.

 This is in stark contrast to D, where all functions _but_ operators 
 have UFCS.

 The proposal was allow UFCS also for overloaded operators.

 Hence, this discussion is about UFCS. These are not really operator 
 overloading issues.


 
 Ok, but I'm not sure what the proposal was.
 
 

 UFCS allows hijacking. For an example, see:
 https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85


 
 That may be a bug in the compiler. Can you produce a small test case?


struct S{
     // string foo(){ return "hijacked!"; } // uncomment to hijack
}

string foo(S s){ return "not hijacked!"; }

void main(){
     S s;
     import std.stdio;
     writeln(s.foo());
}



 I know that some of the UFCS code was written without regard to hijacking.
 ...


I think it is by design. Lookup first tries to find members of the type, 
and only if they don't exist applies UFCS lookup. Therefore, if members 
are added to the type or made visible, this may hijack existing UFCS usages.

The way to fix it is to do UFCS lookup always and then error out if both 
UFCS and member lookup match, but there is probably quite some code 
relying on the fact that you can provide a custom implementation of a 
general UFCS function by just adding a member. Also, with the hijacking 
error if you actually meant the member function the only way out I see 
is to use __traits(getMember, ...) for disambiguation.

(BTW: Local imports allow hijacking too. Maybe this one could be fixed?)


 

 The intention of the code was to demonstrate that a type can pass 
 isInputRange in the same module in which it does not support front. 
 This is an example of surprising name lookup behavior.


 
 I submit it is surprising only if you're used to ADL :-)
 ...


I'm not used to ADL. I think it is surprising because input ranges 
support front. ;) (It's not surprising to _me_, I'm rather familiar with 
D's features, especially those added before last year or so. My 
statement is more that it could be surprising to many, and that it is 
not necessarily reasonable to blame them for being surprised.)


 ...
 
 D's name lookup rules were quite simple, and deliberately set up that 
 way. Unfortunately, most people thought they were unintuitive. It turns 
 out that simple algorithms are not intuitive, and we now have a fairly 
 complex lookup system. Martin and I implemented it, and probably neither 
 of us completely understands it. I find it regrettable that things have 
 gotten to that state.
 ...


It might make sense to sit down at some point and see what goals the 
complex rules try to achieve and then come up with simpler rules that 
achieve the same (or better) goals.


 

 Of course there is also the opposite problem. You can have a type that 
 supports all range primitives via UFCS but does not pass isInputRange, 
 because Phobos does not import the module where the primitives are 
 defined. (This particular case is sometimes solved by ADL, sometimes 
 not.)
  ...


 
 I suggest for this case writing a wrapper type for Iota, with 
 front/empty/popFront as members of that wrapper, then it should be good 
 to go. It's hardly any more work than writing the free functions.


One thing that currently works is having the following string constant 
in a util.d file:

enum ufcs_=q{
     private{
         import std.typecons: Proxy;
         struct UFCS(T){ T payload; mixin Proxy!payload; }
         auto ufcs(T)(T arg) trusted{ return *cast(UFCS!T*)&arg; }
     }
};

Then, the following code compiles and runs:

import util: ufcs_;
mixin(ufcs_);

struct Iota{ private int a,b; }
auto iota(int a,int b){ return Iota(a,b); }

 property int front(Iota i){ return i.a; }
 property bool empty(Iota i){ return i.a>=i.b; }
void popFront(ref Iota i){ ++i.a; }

void main(){
     import std.algorithm, std.stdio;
     iota(0,10).ufcs.each!writeln; // ok
}

This exploits what seems to be a serious encapsulation-breaking bug in 
std.typecons.Proxy in order to pick up all UFCS functions visible from 
the current module, but a safe version of this could be made.


 Doing 
 this kind of wrapper doesn't work for operator overloading, which brings 
 us back to ADL is for operator overloading.


Why does it not work for operator overloading?



 Oct 05 2017



 next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 4:26 AM, Timon Gehr wrote:

 UFCS allows hijacking. For an example, see:
 https://github.com/tgehr/d-compiler/pull/1#discussion-diff-89697186L85


 That may be a bug in the compiler. Can you produce a small test case?


 
 struct S{
      // string foo(){ return "hijacked!"; } // uncomment to hijack
 }
 
 string foo(S s){ return "not hijacked!"; }
 
 void main(){
      S s;
      import std.stdio;
      writeln(s.foo());
 }


Thank you:

  https://issues.dlang.org/show_bug.cgi?id=17879



 Oct 05 2017



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 4:26 AM, Timon Gehr wrote:

 I know that some of the UFCS code was written without regard to hijacking.
 ...


 
 I think it is by design. Lookup first tries to find members of the type, and 
 only if they don't exist applies UFCS lookup. Therefore, if members are added
to 
 the type or made visible, this may hijack existing UFCS usages.


It may have been done that way to avoid breaking existing code.


 The way to fix it is to do UFCS lookup always and then error out if both UFCS 
 and member lookup match, but there is probably quite some code relying on the 
 fact that you can provide a custom implementation of a general UFCS function
by 
 just adding a member. Also, with the hijacking error if you actually meant the 
 member function the only way out I see is to use __traits(getMember, ...) for 
 disambiguation.
 
 (BTW: Local imports allow hijacking too. Maybe this one could be fixed?)


I thought that was taken care of when the whole import lookup mechanism was 
redone a year or two ago.



 The intention of the code was to demonstrate that a type can pass 
 isInputRange in the same module in which it does not support front. This is 
 an example of surprising name lookup behavior.


 I submit it is surprising only if you're used to ADL :-)
 ...


 
 I'm not used to ADL. I think it is surprising because input ranges support 
 front. ;) (It's not surprising to _me_, I'm rather familiar with D's features, 
 especially those added before last year or so. My statement is more that it 
 could be surprising to many, and that it is not necessarily reasonable to
blame 
 them for being surprised.)


When I learn a new language, I am sometimes surprised at the depth of my 
preconceived notions about how they work that turns out to be very specific to
a 
language I am very used to.



 D's name lookup rules were quite simple, and deliberately set up that way. 
 Unfortunately, most people thought they were unintuitive. It turns out that 
 simple algorithms are not intuitive, and we now have a fairly complex lookup 
 system. Martin and I implemented it, and probably neither of us completely 
 understands it. I find it regrettable that things have gotten to that state.
 ...


 
 It might make sense to sit down at some point and see what goals the complex 
 rules try to achieve and then come up with simpler rules that achieve the same 
 (or better) goals.


There wasn't a lack of discussion about the import lookup rules :-)



 One thing that currently works is having the following string constant in a 
 util.d file:
 
 enum ufcs_=q{
      private{
          import std.typecons: Proxy;
          struct UFCS(T){ T payload; mixin Proxy!payload; }
          auto ufcs(T)(T arg) trusted{ return *cast(UFCS!T*)&arg; }
      }
 };
 
 Then, the following code compiles and runs:
 
 import util: ufcs_;
 mixin(ufcs_);
 
 struct Iota{ private int a,b; }
 auto iota(int a,int b){ return Iota(a,b); }
 
  property int front(Iota i){ return i.a; }
  property bool empty(Iota i){ return i.a>=i.b; }
 void popFront(ref Iota i){ ++i.a; }
 
 void main(){
      import std.algorithm, std.stdio;
      iota(0,10).ufcs.each!writeln; // ok
 }
 
 This exploits what seems to be a serious encapsulation-breaking bug in 
 std.typecons.Proxy in order to pick up all UFCS functions visible from the 
 current module, but a safe version of this could be made.


Nice


 Doing this kind of wrapper doesn't work for operator overloading, which brings 
 us back to ADL is for operator overloading.


 
 Why does it not work for operator overloading?


3+s



 Oct 05 2017



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 06.10.2017 03:01, Walter Bright wrote:

 On 10/5/2017 4:26 AM, Timon Gehr wrote:

 I know that some of the UFCS code was written without regard to 
 hijacking.
 ...


 I think it is by design. Lookup first tries to find members of the 
 type, and only if they don't exist applies UFCS lookup. Therefore, if 
 members are added to the type or made visible, this may hijack 
 existing UFCS usages.


 
 It may have been done that way to avoid breaking existing code.
 


I think so. It can break new code though. ;)


 The way to fix it is to do UFCS lookup always and then error out if 
 both UFCS and member lookup match, but there is probably quite some 
 code relying on the fact that you can provide a custom implementation 
 of a general UFCS function by just adding a member. Also, with the 
 hijacking error if you actually meant the member function the only way 
 out I see is to use __traits(getMember, ...) for disambiguation.

 (BTW: Local imports allow hijacking too. Maybe this one could be fixed?)


 
 I thought that was taken care of when the whole import lookup mechanism 
 was redone a year or two ago.
 ...


https://issues.dlang.org/show_bug.cgi?id=17589

I think the best fix is what I quickly outline at the end of the issue. 
It is originally a proposal of deadalnix:
https://issues.dlang.org/show_bug.cgi?id=10378#c9


 ...
 

 D's name lookup rules were quite simple, and deliberately set up that 
 way. Unfortunately, most people thought they were unintuitive. It 
 turns out that simple algorithms are not intuitive, and we now have a 
 fairly complex lookup system. Martin and I implemented it, and 
 probably neither of us completely understands it. I find it 
 regrettable that things have gotten to that state.
 ...


 It might make sense to sit down at some point and see what goals the 
 complex rules try to achieve and then come up with simpler rules that 
 achieve the same (or better) goals.


 
 There wasn't a lack of discussion about the import lookup rules :-)
 ...


The only related issue of which I was aware was visibility of private 
symbols. (I.e. private symbols should not cause conflicts with public 
symbols in other modules.)


 ... >>> Doing this kind of wrapper doesn't work for operator overloading,

 which brings us back to ADL is for operator overloading.


 Why does it not work for operator overloading?


 
 3+s


typeof(s) could overload opBinaryRight, no?



 Oct 09 2017



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 09.10.2017 17:41, Timon Gehr wrote:

 There wasn't a lack of discussion about the import lookup rules :-)
 ...


 
 The only related issue of which I was aware was visibility of private 
 symbols. (I.e. private symbols should not cause conflicts with public 
 symbols in other modules.)


(Besides local imports, that is.)



 Oct 09 2017



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 05.10.2017 11:52, Walter Bright wrote:

 On 10/5/2017 2:13 AM, Timon Gehr wrote:

 It's easy to explain why: In C++, operators are the _only_ functions 
 that have UFCS.

 This is in stark contrast to D, where all functions _but_ operators 
 have UFCS.

 The proposal was allow UFCS also for overloaded operators.

 Hence, this discussion is about UFCS. These are not really operator 
 overloading issues.


 
 Ok, but I'm not sure what the proposal was.


I forgot to answer to this.



 On 27 September 2017 at 17:41, Ilya Yaroshenko via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 I would prefer outer operator overloading be added to D instead of type
wrappers. So a user can import a library for operations, rather then library of
wrappers. --Ilya
 


"outer operator overloading" is UFCS for operators. I.e.:

struct S{ int x; }
S opBinary(string op:"+")(S a,S b){ return S(a.x+b.x); }

void main(){
     auto s=S(3), t=S(4);
     import std.stdio;
     writeln(s+t); // S(7)
}

Starting from:

s+t

It rewritten to (as per the spec):
s.opBinary!"+"(t)

and then UFCS is applied (as per the spec):
opBinary!"+"(s,t)


I'm very much in favor of this. Also, those rewrites should be 
consistently applied for all types, even built-ins (the compiler 
implementation can be more complex, but the language rules would be 
simplified).
One immediate benefit would be that opCmp could be reliably used for all 
types that support comparison, for example 2.opCmp(3).
Another benefit would be that operators such as += can reassign class 
references, for example when a value type is implemented as a unique 
reference to immutable data.



 Oct 05 2017



 next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/5/2017 4:36 AM, Timon Gehr wrote:

 I forgot to answer to this.


Thanks for the explanation!



 Oct 05 2017



prev sibling  parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Thursday, October 05, 2017 13:36:23 Timon Gehr via Digitalmars-d wrote:

 On 27 September 2017 at 17:41, Ilya Yaroshenko via Digitalmars-d
 <digitalmars-d puremagic.com> wrote: I would prefer outer operator
 overloading be added to D instead of type wrappers. So a user can
 import a library for operations, rather then library of wrappers.
 --Ilya


 "outer operator overloading" is UFCS for operators. I.e.:

 struct S{ int x; }
 S opBinary(string op:"+")(S a,S b){ return S(a.x+b.x); }

 void main(){
      auto s=S(3), t=S(4);
      import std.stdio;
      writeln(s+t); // S(7)
 }

 Starting from:

 s+t

 It rewritten to (as per the spec):
 s.opBinary!"+"(t)

 and then UFCS is applied (as per the spec):
 opBinary!"+"(s,t)


 I'm very much in favor of this. Also, those rewrites should be
 consistently applied for all types, even built-ins (the compiler
 implementation can be more complex, but the language rules would be
 simplified).
 One immediate benefit would be that opCmp could be reliably used for all
 types that support comparison, for example 2.opCmp(3).
 Another benefit would be that operators such as += can reassign class
 references, for example when a value type is implemented as a unique
 reference to immutable data.


Being able to do 2.opCmp(3) would be pretty cool, but I'm still convinced
that allowing for operators to be overloaded outside of the type is a
terrible idea. It's far cleaner for them to be tied to the type - especially
when you consider that it's not possible to differentiate between
conflicting overloadeded operators. And having them declared outside of the
type just opens up all of the problems that were just being complained about
in this thread with templated code not being able to access free functions
that weren't imported in the module that it's in.

- Jonathan M Davis



 Oct 05 2017



 next sibling parent reply Guillaume Boucher <guillaume.boucher.d outlook.com>  writes:


On Thursday, 5 October 2017 at 22:04:10 UTC, Jonathan M Davis 
wrote:

 It's far cleaner for them to be tied to the type - especially 
 when you consider that it's not possible to differentiate 
 between conflicting overloadeded operators.


In other words, we lose global uniqueness of operators if we were 
to allow free functions to implement operators.

Knowing which function is called when we see its name is very 
important for reading code.  That's why we have those anti 
hijacking rules: they disallow cases where the compiler knows 
that the call can be misleading (or can silently break existing 
code).  Another, more tricky case is when there are two functions 
with the same name in the project you are working on, but only 
one of them is being imported.  If you read the code, you are 
unsure which one is called.  The anti-hijacking rules won't work 
in that case.
Fortunately, a good naming scheme avoids those problems so they 
are not a big problem in practice.

Those problems will pop up, however, if we allow them for 
operators.  Without further regulations, different 
implementations for operators are almost guaranteed, which will 
lead to uncertainty and distrust in code using operators.



Besides, I don't how it should work without changes to lookup 
rules.  How should sum() be able to use a +-operator defined in 
an imported module?  Not even ADL is helping here.

In my opinion, operators should only be defined in the module 
defining the type.



 Oct 06 2017



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Friday, 6 October 2017 at 10:56:06 UTC, Guillaume Boucher 
wrote:

 Knowing which function is called when we see its name is very 
 important for reading code.  That's why we have those anti 
 hijacking rules: they disallow cases where the compiler knows 
 that the call can be misleading (or can silently break existing 
 code).  Another, more tricky case is when there are two 
 functions with the same name in the project you are working on, 
 but only one of them is being imported.  If you read the code, 
 you are unsure which one is called.  The anti-hijacking rules 
 won't work in that case.
 Fortunately, a good naming scheme avoids those problems so they 
 are not a big problem in practice.


With respect to overloaded operators, I was trying to make the 
point above that the anti-hijacking would stay in place. If you 
wanted to offer two different options for operators, you could 
put them in separate modules so that you could import one or the 
other, but not both (which should give an error). I admit it's 
not super convenient, but it's an option.



 Oct 06 2017



prev sibling next sibling parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 10/5/17 6:04 PM, Jonathan M Davis wrote:

 On Thursday, October 05, 2017 13:36:23 Timon Gehr via Digitalmars-d wrote:

 On 27 September 2017 at 17:41, Ilya Yaroshenko via Digitalmars-d
 <digitalmars-d puremagic.com> wrote: I would prefer outer operator
 overloading be added to D instead of type wrappers. So a user can
 import a library for operations, rather then library of wrappers.
 --Ilya


 "outer operator overloading" is UFCS for operators. I.e.:

 struct S{ int x; }
 S opBinary(string op:"+")(S a,S b){ return S(a.x+b.x); }

 void main(){
       auto s=S(3), t=S(4);
       import std.stdio;
       writeln(s+t); // S(7)
 }

 Starting from:

 s+t

 It rewritten to (as per the spec):
 s.opBinary!"+"(t)

 and then UFCS is applied (as per the spec):
 opBinary!"+"(s,t)


 I'm very much in favor of this. Also, those rewrites should be
 consistently applied for all types, even built-ins (the compiler
 implementation can be more complex, but the language rules would be
 simplified).
 One immediate benefit would be that opCmp could be reliably used for all
 types that support comparison, for example 2.opCmp(3).
 Another benefit would be that operators such as += can reassign class
 references, for example when a value type is implemented as a unique
 reference to immutable data.


 
 Being able to do 2.opCmp(3) would be pretty cool, but I'm still convinced
 that allowing for operators to be overloaded outside of the type is a
 terrible idea. It's far cleaner for them to be tied to the type - especially
 when you consider that it's not possible to differentiate between
 conflicting overloadeded operators.


But UFCS already has all these "problems", and yet it's one of the most 
useful features of D.


 And having them declared outside of the
 type just opens up all of the problems that were just being complained about
 in this thread with templated code not being able to access free functions
 that weren't imported in the module that it's in.


That is a problem, but no different from any other UFCS scheme.

Indeed, providing a way to alias "official" UFCS functions into the 
namespace of the type itself would be useful.

-Steve



 Oct 06 2017



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 06.10.2017 00:04, Jonathan M Davis wrote:

 I'm very much in favor of this. Also, those rewrites should be
 consistently applied for all types, even built-ins (the compiler
 implementation can be more complex, but the language rules would be
 simplified).
 One immediate benefit would be that opCmp could be reliably used for all
 types that support comparison, for example 2.opCmp(3).
 Another benefit would be that operators such as += can reassign class
 references, for example when a value type is implemented as a unique
 reference to immutable data.


 Being able to do 2.opCmp(3) would be pretty cool, but I'm still convinced
 that allowing for operators to be overloaded outside of the type is a
 terrible idea. It's far cleaner for them to be tied to the type


It's far from clean because it enforces a certain calling convention.


 - especially
 when you consider that it's not possible to differentiate between
 conflicting overloadeded operators.


This is just not true. D has enough mechanisms for differentiating 
between conflicting overloads.


 And having them declared outside of the
 type just opens up all of the problems that were just being complained about
 in this thread with templated code not being able to access free functions
 that weren't imported in the module that it's in.


Those are UFCS issues, but I'd rather have UFCS than no UFCS.



 Oct 06 2017



  parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, October 06, 2017 21:58:04 Timon Gehr via Digitalmars-d wrote:

 On 06.10.2017 00:04, Jonathan M Davis wrote:

 I'm very much in favor of this. Also, those rewrites should be
 consistently applied for all types, even built-ins (the compiler
 implementation can be more complex, but the language rules would be
 simplified).
 One immediate benefit would be that opCmp could be reliably used for
 all
 types that support comparison, for example 2.opCmp(3).
 Another benefit would be that operators such as += can reassign class
 references, for example when a value type is implemented as a unique
 reference to immutable data.


 Being able to do 2.opCmp(3) would be pretty cool, but I'm still
 convinced
 that allowing for operators to be overloaded outside of the type is a
 terrible idea. It's far cleaner for them to be tied to the type


 It's far from clean because it enforces a certain calling convention.


It forces them to actually be designed with the type and be easily located
with the type. Would you want a programmer to be able to go and implement
opBinary!"+" for strings? I sure wouldn't. And I don't want anyone doing
that for user-defined types that they didn't define either.


 - especially
 when you consider that it's not possible to differentiate between
 conflicting overloadeded operators.


 This is just not true. D has enough mechanisms for differentiating
 between conflicting overloads.


It isn't possible to differentiate when you do foo + bar, though I suppose
that it would then be possible to differentiate if you called
foo.opBinary!"+"(bar). However, that completely defeats the purpose of
overloaded operators.


 And having them declared outside of the
 type just opens up all of the problems that were just being complained
 about in this thread with templated code not being able to access free
 functions that weren't imported in the module that it's in.


 Those are UFCS issues, but I'd rather have UFCS than no UFCS.


And I'd rather never see operators overloaded by anything but the type
they're for.

- Jonathan M Davis



 Oct 06 2017



 next sibling parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Friday, 6 October 2017 at 20:36:47 UTC, Jonathan M Davis wrote:

 It forces them to actually be designed with the type and be 
 easily located with the type. Would you want a programmer to be 
 able to go and implement opBinary!"+" for strings? I sure 
 wouldn't. And I don't want anyone doing that for user-defined 
 types that they didn't define either.


Tangentially (I admit), there's nothing stopping you from below:

struct MyString
{
     string mystring;
     alias mystring this;
     string opBinary(string op)(string rhs)
         if(op == "+")
     {
         return mystring ~ rhs;
     }
}

void main()
{
     MyString x = MyString("foo");
     string y = "bar";
     auto z = x + y;
     assert(z == "foobar");
}



 Oct 06 2017



  parent  Jonathan M Davis <newsgroup.d jmdavisprog.com>  writes:


On Friday, October 06, 2017 21:05:05 jmh530 via Digitalmars-d wrote:

 On Friday, 6 October 2017 at 20:36:47 UTC, Jonathan M Davis wrote:

 It forces them to actually be designed with the type and be
 easily located with the type. Would you want a programmer to be
 able to go and implement opBinary!"+" for strings? I sure
 wouldn't. And I don't want anyone doing that for user-defined
 types that they didn't define either.


 Tangentially (I admit), there's nothing stopping you from below:

 struct MyString
 {
      string mystring;
      alias mystring this;
      string opBinary(string op)(string rhs)
          if(op == "+")
      {
          return mystring ~ rhs;
      }
 }

 void main()
 {
      MyString x = MyString("foo");
      string y = "bar";
      auto z = x + y;
      assert(z == "foobar");
 }


Yes, but then at least it's a type that you've defined to be dumb like that
rather that built-in's or someone else's type being hijacked to behave
badly. It's not like we're going to stop every stupid thing that someone
could do (and if we did, it would severely limit our ability to do
intelligent things), but I see no value in allowing 3rd parties to tack on
overloaded operators onto types that they do not control.

- Jonathan M Davis



 Oct 06 2017



prev sibling  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Friday, 6 October 2017 at 20:36:47 UTC, Jonathan M Davis wrote:

 It forces them to actually be designed with the type and be 
 easily located with the type. Would you want a programmer to be 
 able to go and implement opBinary!"+" for strings? I sure 
 wouldn't. And I don't want anyone doing that for user-defined 
 types that they didn't define either.


Hmm, you could think of it like the current default is that you 
cannot do outer operator overloading. We have  disable this(); 
for disabling default construction. What about something like
 outer T opBinary(string op)(T)
     if(op == "+")
which means that it allows opBinary to be defined outside the 
struct/class. The default would be that it's not allowed. And it 
would give a hint to somebody that they would need to look for 
that method elsewhere?



 Oct 06 2017



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 10/02/2017 07:15 AM, Timon Gehr wrote:

 If there are multiple definitions of the same name, different modules 
 might not agree which one is being referred to. (This is particularly 
 likely for overloaded operators, as the set of names is finite and 
 small. This is what Manu means when he says it can lead to nasty 
 surprises. This is very plausible, but I don't have a good example.)


Indeed a good example would strengthen the argument considerably. -- Andrei



 Oct 04 2017



prev sibling next sibling parent reply Manu <turkeyman gmail.com>  writes:


On 28 September 2017 at 11:58, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 9/27/2017 4:21 PM, Manu wrote:


 D does not have ADL,


 Thank gawd! :-)

 which will almost certainly lead to _very_ nasty surprises in behaviour.


 ADL was always a hack to get around the wretched overloading symbol lookup
 behavior in C++. I see it has somehow morphed into a feature :-( but I see
 no advantage to it over D's approach (the reverse operand lookup scheme).


Whether you like it or not, the lack of ADL will make the idea of operator
overloading a disaster.

Don't get me wrong, I'm not married to ADL, I'm completely ambivalent on
its design. As I see, it just makes a fairly important set of problems work
in a predictable and useful manner, and I haven't encountered issues with
it before.
The problem is, D offers no solution to the important issues that ADL
addresses, and I think that's a much worse situation than the existence of
ADL.



 Sep 29 2017



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 9/29/2017 7:17 PM, Manu wrote:

 Whether you like it or not, the lack of ADL will make the idea of operator 
 overloading a disaster.


Please present an example.



 Sep 30 2017



prev sibling  parent reply Enamex <enamex+d outlook.com>  writes:


On Thursday, 28 September 2017 at 01:58:24 UTC, Walter Bright 
wrote:

 On 9/27/2017 4:21 PM, Manu wrote:
 [...]
 but I see no advantage to it over D's approach (the reverse 
 operand lookup scheme).


I couldn't find anything focused on D's overloading resolution 
scheme (or anything with the specific term "reverse operand 
lookup"). Can you elaborate?



 Oct 03 2017



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 10/3/2017 12:58 AM, Enamex wrote:

 I couldn't find anything focused on D's overloading resolution scheme (or 
 anything with the specific term "reverse operand lookup"). Can you elaborate?


See my reply to Timon.



 Oct 03 2017



prev sibling  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 27 September 2017 at 04:59:10 UTC, Manu wrote:

 An alternative solution might be to introduce a wrapper of 
 ndslice called 'matrix' that supports matrix mul...?


That's exactly how Numpy works with array and matrix types. It's 
confusing. That being said, mir has 
Contiguous/Universal/Canonical, which are also a little 
confusing...



 Sep 27 2017



prev sibling next sibling parent  tn <no email.com>  writes:


On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


If it is for element-wise product, then possibly yes.
If it is for dot product (as suggested by the github issue), then 
definitely no.



 Sep 27 2017



prev sibling  parent  John Colvin <john.loughran.colvin gmail.com>  writes:


On Friday, 22 September 2017 at 17:11:56 UTC, Ilya Yaroshenko 
wrote:

 Should we add `a * b` to ndslice for 1d vectors?
 Discussion at https://github.com/libmir/mir-algorithm/issues/91


Unless it's always just simple element-wise, make it a different 
type. N-dimensional rectangular data structures are only 
sometimes matrices.

Also, if you did two different types I have concerns about code 
that mixed the two, it would become quite unclear which ones were 
matrix operations and which were element-wise.



 Sep 27 2017