www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - templating opEquals/opCmp (e.g. for DSL/expression templates)

reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
With opBinary and opUnary it is possible to create types that 
span the binary and unary operators. However opEquals can only be 
used to types that compare for equality not inequality, since a 
== b -> a.opEquals(b) can return whatever type it likes but a == 
b -> !(a.opEquals(b)) returns a bool. opCmp can't be used at all 
since a <=> b -> a.opCmp(b) <=> 0 which evaluates to a bool.

I propose that in addition to the current (non-template) forms of 
opCmp/opEquals

struct A { bool opEquals(A rhs); int opCmp(A rhs); }

we allow

struct B
{
     T opEquals(string op)(B rhs); // op is "<" "<=" etc.
     T opCmp(string op)(B rhs);    // op is "==" "!="
}

where T is any arbitrary type.

see also 
https://github.com/k3kaimu/dranges/blob/master/source/dranges/_lambda.d
Feb 11 2019
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, February 11, 2019 9:15:34 PM MST Nicholas Wilson via Digitalmars-
d wrote:

 With opBinary and opUnary it is possible to create types that
 span the binary and unary operators. However opEquals can only be
 used to types that compare for equality not inequality, since a
 == b -> a.opEquals(b) can return whatever type it likes but a ==
 b -> !(a.opEquals(b)) returns a bool. opCmp can't be used at all
 since a <=> b -> a.opCmp(b) <=> 0 which evaluates to a bool.

 I propose that in addition to the current (non-template) forms of
 opCmp/opEquals

 struct A { bool opEquals(A rhs); int opCmp(A rhs); }

 we allow

 struct B
 {
      T opEquals(string op)(B rhs); // op is "<" "<=" etc.
      T opCmp(string op)(B rhs);    // op is "==" "!="
 }

 where T is any arbitrary type.

 see also
 https://github.com/k3kaimu/dranges/blob/master/source/dranges/_lambda.d


Weren't D's overloaded operators designed specifically with the idea that
they _wouldn't_ support stuff like expression templates? They're supposed to
emulate the built-in types with how normal equality and the like works, not
be used for creating new syntax.

- Jonathan M Davis
Feb 11 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 February 2019 at 04:42:39 UTC, Jonathan M Davis 
wrote:

 Weren't D's overloaded operators designed specifically with the 
 idea that they _wouldn't_ support stuff like expression 
 templates?


Well you can but only with non-relational operators. IIRC they 
were designed s.t. you only needed to define one function to get 
a suite of operators (c.f. C++).


 They're supposed to emulate the built-in types with how normal 
 equality and the like works, not be used for creating new 
 syntax.


Well the syntax is still the same, just the meaning is different.
Feb 11 2019
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, February 11, 2019 11:07:43 PM MST Nicholas Wilson via 
Digitalmars-d wrote:

 They're supposed to emulate the built-in types with how normal
 equality and the like works, not be used for creating new
 syntax.


 Well the syntax is still the same, just the meaning is different.


Having the meaning be different is precisely the problem. It's using the
syntax for something _other_ than what it's intended for. It's just supposed
to be a way to have user-defined types emulate the operations of built-in
types. Walter has mentioned time and time again how he thinks that stuff
like << in C++ where they made the operators mean something completely
different than what they mean for built-in types was a big mistake, and
personally, I agree. If it were up to me, it wouldn't even be legal to have
opEquals return anything other than bool. In fact, I actually thought that
we'd fixed it at one point so that it _did_ have to be bool, but I saw a
post not that long ago where someone had an example that compiled where it
returned something else. So, I guess that the compiler isn't quite that
strict at the moment. Either way, I think that Walter has made it pretty
clear that overloaded operators are not intended for stuff like DSLs or
expression templates, and if you try to alter overloaded operators to make
them work better for expression templates or the like, I would fully expect
him to veto it. But I guess that we'll see his response soon enough if you
try to push this.

- Jonathan M Davis
Feb 12 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 February 2019 at 10:37:22 UTC, Jonathan M Davis 
wrote:

 Having the meaning be different is precisely the problem. It's 
 using the syntax for something _other_ than what it's intended 
 for. It's just supposed to be a way to have user-defined types 
 emulate the operations of built-in types. Walter has mentioned 
 time and time again how he thinks that stuff like << in C++ 
 where they made the operators mean something completely 
 different than what they mean for built-in types was a big 
 mistake, and personally, I agree.


I agree using << for I/O is stupid.


 Either way, I think that Walter has made it pretty clear that 
 overloaded operators are not intended for stuff like DSLs or 
 expression templates, and if you try to alter overloaded 
 operators to make them work better for expression templates or 
 the like, I would fully expect him to veto it.


Link?


 But I guess that we'll see his response soon enough if you try 
 to push this.


The use case is e.g. building a DB query predicate where == still 
means equality not e.g. I/O, it just isn't evaluated directly. 
The fact is you can already use it for arithmetic to build e.g. 
linear algebra expression templates, and you can use == 
predicates but not != or any of the ordering comparisons is 
inconsistent arbitrary and annoying.
Feb 12 2019
next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 12 February 2019 at 11:35:28 UTC, Nicholas Wilson 
wrote:

 On Tuesday, 12 February 2019 at 10:37:22 UTC, Jonathan M Davis 
 wrote:

 Having the meaning be different is precisely the problem. It's 
 using the syntax for something _other_ than what it's intended 
 for. It's just supposed to be a way to have user-defined types 
 emulate the operations of built-in types. Walter has mentioned 
 time and time again how he thinks that stuff like << in C++ 
 where they made the operators mean something completely 
 different than what they mean for built-in types was a big 
 mistake, and personally, I agree.


 I agree using << for I/O is stupid.


I've never understood this argument. I've never met anyone who 
was confused by this. Personally, I read it as "shove". C++20 
ranges use `|` as a pipe operator instead of bitwise or, and not 
only does it make sense, I don't think anybody's complaining 
about that either.


 The use case is e.g. building a DB query predicate where == 
 still means equality not e.g. I/O, it just isn't evaluated 
 directly. The fact is you can already use it for arithmetic to 
 build e.g. linear algebra expression templates, and you can use 
 == predicates but not != or any of the ordering comparisons is 
 inconsistent arbitrary and annoying.


Agreed.
Feb 12 2019
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2019-02-12 12:35, Nicholas Wilson wrote:


 The use case is e.g. building a DB query predicate where == still means 
 equality not e.g. I/O, it just isn't evaluated directly. The fact is you 
 can already use it for arithmetic to build e.g. linear algebra 
 expression templates, and you can use == predicates but not != or any of 
 the ordering comparisons is inconsistent arbitrary and annoying.


I like it. I would like to be able to do this with DB queries as well. 
The alternative would be AST macros, but that is not likely to be 
accepted either.

-- 
/Jacob Carlborg
Feb 12 2019
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, February 12, 2019 4:35:28 AM MST Nicholas Wilson via 
Digitalmars-d wrote:

 On Tuesday, 12 February 2019 at 10:37:22 UTC, Jonathan M Davis
 wrote:

 Either way, I think that Walter has made it pretty clear that
 overloaded operators are not intended for stuff like DSLs or
 expression templates, and if you try to alter overloaded
 operators to make them work better for expression templates or
 the like, I would fully expect him to veto it.


 Link?


He's said as much on several occasions over the years, but here's a link to
one such comment:

https://forum.dlang.org/post/nsffdd$1h6m$1 digitalmars.com

- Jonathan M Davis
Feb 12 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 February 2019 at 12:35:50 UTC, Jonathan M Davis 
wrote:

 On Tuesday, February 12, 2019 4:35:28 AM MST Nicholas Wilson

 Link?


 He's said as much on several occasions over the years, but 
 here's a link to one such comment:

 https://forum.dlang.org/post/nsffdd$1h6m$1 digitalmars.com


Thanks


 The limitations are deliberate based on the idea that 
 comparison operators need to be consistent and predictable, and 
 should have a close relationship to the mathematical meaning of 
 the operators. Overloading <= to mean something other than 
 "less than or equal" is considered poor style in D, and the 
 same goes for the other arithmetic operators.


I don't think that sentiment changes under the use of <= for 
symbolic less-than as opposed to logical less-than.


 The use of them to create DSLs (a technique called "expression 
 templates"
 in C++) is discouraged in D, for several reasons. The 
 recommended way to create DSLs in D is to parse strings using 
 CTFE. An excellent example of that is the std.regex package.


Regex fundamentally deals with strings so is a poor choice for 
comparison (i.e. strings is the right way to do it for regexen, 
but not for everything). My use case is "give me a form of code 
that is symbolically the same that I can execute later".


 There are no plans to change this.


Oh well, more things to discuss at dconf. That whole thread seems 
to be Walter either being stubborn or not getting it (or both).
Feb 12 2019
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2019-02-12 14:22, Nicholas Wilson wrote:


 Oh well, more things to discuss at dconf. That whole thread seems to be 
 Walter either being stubborn or not getting it (or both).


Yes, exactly. Why can we overload some operators but not all? Even if a 
language doesn't support operator overloading it's possible to abuse:

int add(int a, int b)
{
     return a - b;
}

-- 
/Jacob Carlborg
Feb 12 2019
prev sibling parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson 
wrote:

 Oh well, more things to discuss at dconf. That whole thread 
 seems to be Walter either being stubborn or not getting it (or 
 both).


Or your proposal has drawbacks you're underestimating.

This community really needs to stop defaulting to "We could 
totally use my awesome feature if Walter wasn't so stubborn" 
whenever discussing language changes.
Feb 12 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 February 2019 at 23:24:54 UTC, Olivier FAURE wrote:

 On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson 
 wrote:

 Oh well, more things to discuss at dconf. That whole thread 
 seems to be Walter either being stubborn or not getting it (or 
 both).


 Or your proposal has drawbacks you're underestimating.


... go on. I don't see any drawbacks: its backwards compatible, 
intuitive (behaves like opBinary), useful. We should be making 
design decisions based on merit, not dismissing them not 
potential for abuse.


 This community really needs to stop defaulting to "We could 
 totally use my awesome feature if Walter wasn't so stubborn" 
 whenever discussing language changes.


Its because he always seems to do that:
     1) extern(C++, "strings")
     2) extern-std=c++nn
     3) dip1000(!) because the documentation is frankly not up to 
scratch.
     4) extern(C++) T[]
     5) dip1015
     ...

wherein
     1) he finally capitulated
     2) a (not particularly satisfactory) compromise was reached
     3) I _finally_ got him to update the spec. The docs are still 
shit.
     4) He believed that people want a std::vector interface (fair 
enough), but has has not responded since when it was pointed out 
that the interface is compatible with std::vector under C++'s 
implicit conversions
     5) fully 100% of the community thought the outcome of that 
DIP is wrong and that his reasoning is flawed.

Its a real pain working with him when he doesn't like what you're 
doing, and frankly needs to change. Sometimes he has good 
insight, but more and more I feel that he acts more like a 
gatekeeper and less like a leader.
Feb 12 2019
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, February 12, 2019 5:09:19 PM MST Nicholas Wilson via 
Digitalmars-d wrote:

 On Tuesday, 12 February 2019 at 23:24:54 UTC, Olivier FAURE wrote:

 On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson

 wrote:

 Oh well, more things to discuss at dconf. That whole thread
 seems to be Walter either being stubborn or not getting it (or
 both).


 Or your proposal has drawbacks you're underestimating.


 ... go on. I don't see any drawbacks: its backwards compatible,
 intuitive (behaves like opBinary), useful. We should be making
 design decisions based on merit, not dismissing them not
 potential for abuse.


Except that what you're proposing is precisely so that you can more easily
abuse certain operators when overloading them. IMHO, if an operator is being
overloaded in a manner that doesn't fit what you get with the basic
arithmetic types, then that's bad practice. I know that some folks like to
do it (especially in C++), but that's one of the big reasons that some
languages (like Java) chose to not have operator overloading at all. People
kept using them to mean things that didn't match their meaning for the
built-in types. Such operator overloading abuse results in code that's
harder to understand and maintain, because the operators are not being used
with their normal meaning. It also doesn't play well with generic code,
because such code is going to assume the normal arithmetic meanings for
those operators, and so having a type that uses them differently with such
code will tend to result in incorrect behavior (or in some cases, the code
won't even compile).

We can't completely eliminate operator overloading abuse (e.g. someone doing
something dumb like making + mean -), but just because it's possible to
abuse one operator doesn't mean that we should make easier to abuse another.

- Jonathan M Davis
Feb 12 2019
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 00:24:48 UTC, Jonathan M Davis 
wrote:

 Except that what you're proposing is precisely so that you can 
 more easily abuse


Use, not abuse.


 certain operators when overloading them. IMHO, if an operator 
 is being overloaded in a manner that doesn't fit what you get 
 with the basic arithmetic types, then that's bad practice.


Yes, thats called abuse.


 People kept using [C++ operators] to mean things that didn't 
 match their meaning for the built-in types. Such operator 
 overloading abuse results in code that's harder to understand 
 and maintain, because the operators are not being used with 
 their normal meaning.


That falls down to convention. I personally think << for I/O is 
silly, but the thing that makes it work is that it is 
consistently applied.


 It also doesn't play well with generic code, because such code 
 is going to assume the normal arithmetic meanings for those 
 operators, and so having a type that uses them differently with 
 such code will tend to result in incorrect behavior (or in some 
 cases, the code won't even compile).


Actually it plays excellently with generic code. If I have a 
symbolic execution engine and I use it with a generic function, 
what results is a symbolic function, for free! You are then free 
to do what ever you please with, e.g. turn it into a DB query, 
send it off to a graph execution engine, optimise it, evaluate it 
at a later time, ... .


 We can't completely eliminate operator overloading abuse (e.g. 
 someone doing something dumb like making + mean -), but just 
 because it's possible to abuse one operator doesn't mean that 
 we should make easier to abuse another.


It doesn't mean we shouldn't either, it bears no correlation. 
What we want to prevent is abuse, not potential for abuse. We 
already have massive potential for abuse, goto, operator 
overloading, mixins, etc., we have almost no _actual_ abuse. D is 
designed for someone who is sensible and knows how to use 
discretion.

Yes this proposal increases the potential for abuse. How much of 
that potential will actually materialise? History suggests not 
much, and that should be reflected in the decision making.
Feb 12 2019
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 12, 2019 at 11:24:54PM +0000, Olivier FAURE via Digitalmars-d wrote:

 On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson wrote:

 Oh well, more things to discuss at dconf. That whole thread seems to
 be Walter either being stubborn or not getting it (or both).


 
 Or your proposal has drawbacks you're underestimating.
 
 This community really needs to stop defaulting to "We could totally
 use my awesome feature if Walter wasn't so stubborn" whenever
 discussing language changes.


Frankly, I think it's a very good thing that in D comparison operators
are confined to opEquals/opCmp.  If anything, I would vote for enforcing
opEquals to return bool and bool only.

The reason for this is readability and maintainability.  Symbols like <=
or == should mean one and only one thing in the language, and should not
be subject to random overloaded interpretations. Being built-in
operators, they are used universally in the language, and any
inconsistency in semantics hurts readability, comprehension, and
maintainability, such as C++'s free-for-all operator overloading, where
any piece of syntax can have wildly-divergent interpretations (even
completely different parse trees) depending on what came before.  For
example, recently I came up with a C++ monstrosity where the lines:

        fun<A, B>(a, b);
        gun<T, U>(a, b);

have wildly-different, completely unrelated *parse trees*, as an
illustration of how unreadable C++ can become.

Yes, it's extremely flexible, yes it's extremely powerful and can
express literally *whatever* you want it to express. It's also
completely unreadable and unmaintainable, because the surface structure
of the code text becomes completely detached from the actual underlying
semantics. I don't even want to imagine what debugging such kind of code
must be like.

Operator overloading should be reserved for objects that behave like
arithmetical entities in some way.  And especially comparison operators
should not have any other meaning than the standard meaning.  It should
be illegal to make == and <= mean something completely unrelated to each
other.

If you need to redefine comparison operators, what you want is really a
DSL wherein you can define operators to mean whatever you want.  The
usual invocation of such a DSL as a compile-time argument, say something
like this:

	myDsl!'a <= b'

contains one often overlooked, but very important element: the
identifier `myDsl`, that sets it apart from other uses of `<=`, and
clearly identifies which interpretation should be ascribed to the
symbols found in the template argument.

See, the thing is, when you see a random expression with arithmetical
operators in it, the expected semantics is the computation of some kind
of arithmetic objects producing an arithmetical result -- because that's
what such expressions mean in general, in the language.  It's abusive to
overload that to mean something else entirely -- because there is no
warning sign to the reader of the code that something different is
happening. When you see a line like:

        fun<A, B>(a, b);

and then somewhere else a line like:

        gun<T, U>(a, b);

the actual behaviour of the code should be similar enough that you can
correctly guess the semantics.  It should not be that the first line
instantiates and calls a template function, whereas the second is
evaluated as an expression with a bunch of overloaded comma and
comparison operators.

Similarly, it should not be the case that:

	auto x = a <= b;

evaluates a comparison expression and assigns a boolean value to x,
whereas:

	auto y = p <= q;

creates an expression object capturing p and q, that needs to be called
later before it yields a boolean value.

With a string DSL, that little identifier `myDsl` (or whatever
identifier you choose for this purpose) serves as a cue to the reader of
the code that something special is happening here.  For example:

	auto y = deferred!`p <= q`;

immediately tells the reader of the code that the <= is to be understood
with a different meaning than the usual <= operator. Just as an
expression like:

	auto dg = (p, q) => p <= q;

by virtue of its different syntax tells the reader that the expression
`p <= q` isn't being evaluated here and now, as it otherwise would be.

The presence of such visual cues is good, and is the way things should
be done.

It should not be that something that looks like an expression, evaluated
here and now, should turn out to do something else. That kind of
free-for-all, can-mean-literally-anything semantics makes code
unreadable, unmaintainable, and a ripe breeding ground for bugs --
someone (i.e., yourself after 3 months) will inevitably forget (or not
know) the special meaning of <= in this particular context and write
wrong code for it.


P.S. And as a bonus, a string DSL gives you the freedom to employ
operators not found among the built-in D operators, for example:

	auto result = eval!`(f∘g)(√x ± √y)`;

And if you feel the usual strings literals are too cumbersome to use for
long expressions, there's always the under-used token strings to the
rescue:

	auto result = eval!q{
		( (f ∘ g)(√(x + y) ± √(x - y)) ) / |x|·|y| +
		2.0 * ∫ f(x)·dx
	};

The `eval` tells the reader that something special is happening here,
and also provides a name by which the reader can search for the
definition of the template that processes this expression, and thereby
learn what it means.

Without this little identifier `eval`, it would be anyone's guess as to
what the code is trying to do. Throw in C++-style SFINAE and Koenig
lookup, and what ought to be a 10-second source tree search for an
identifier easily turns into a 6-hour hair-pulling session of trying to
understand exactly which obscure rules the C++ compiler applied to
resolve those operators to which symbol(s) defined in which obscure
files buried deep in the source tree.


T

-- 
Frank disagreement binds closer than feigned agreement.
Feb 12 2019
next sibling parent reply Rubn <where is.this> writes:
On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh wrote:

 On Tue, Feb 12, 2019 at 11:24:54PM +0000, Olivier FAURE via 
 Digitalmars-d wrote:

 On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson 
 wrote:

 Oh well, more things to discuss at dconf. That whole thread 
 seems to be Walter either being stubborn or not getting it 
 (or both).


 
 Or your proposal has drawbacks you're underestimating.
 
 This community really needs to stop defaulting to "We could 
 totally use my awesome feature if Walter wasn't so stubborn" 
 whenever discussing language changes.


 Frankly, I think it's a very good thing that in D comparison 
 operators are confined to opEquals/opCmp.  If anything, I would 
 vote for enforcing opEquals to return bool and bool only.

 The reason for this is readability and maintainability.  
 Symbols like <= or == should mean one and only one thing in the 
 language, and should not be subject to random overloaded 
 interpretations. Being built-in operators, they are used 
 universally in the language, and any inconsistency in semantics 
 hurts readability, comprehension, and maintainability, such as 
 C++'s free-for-all operator overloading, where any piece of 
 syntax can have wildly-divergent interpretations (even 
 completely different parse trees) depending on what came 
 before.  For example, recently I came up with a C++ monstrosity 
 where the lines:

         fun<A, B>(a, b);
         gun<T, U>(a, b);

 have wildly-different, completely unrelated *parse trees*, as 
 an illustration of how unreadable C++ can become.

 Yes, it's extremely flexible, yes it's extremely powerful and 
 can express literally *whatever* you want it to express. It's 
 also completely unreadable and unmaintainable, because the 
 surface structure of the code text becomes completely detached 
 from the actual underlying semantics. I don't even want to 
 imagine what debugging such kind of code must be like.

 Operator overloading should be reserved for objects that behave 
 like arithmetical entities in some way.  And especially 
 comparison operators should not have any other meaning than the 
 standard meaning.  It should be illegal to make == and <= mean 
 something completely unrelated to each other.

 If you need to redefine comparison operators, what you want is 
 really a DSL wherein you can define operators to mean whatever 
 you want.  The usual invocation of such a DSL as a compile-time 
 argument, say something like this:

 	myDsl!'a <= b'

 contains one often overlooked, but very important element: the 
 identifier `myDsl`, that sets it apart from other uses of `<=`, 
 and clearly identifies which interpretation should be ascribed 
 to the symbols found in the template argument.

 See, the thing is, when you see a random expression with 
 arithmetical operators in it, the expected semantics is the 
 computation of some kind of arithmetic objects producing an 
 arithmetical result -- because that's what such expressions 
 mean in general, in the language.  It's abusive to overload 
 that to mean something else entirely -- because there is no 
 warning sign to the reader of the code that something different 
 is happening. When you see a line like:

         fun<A, B>(a, b);

 and then somewhere else a line like:

         gun<T, U>(a, b);

 the actual behaviour of the code should be similar enough that 
 you can correctly guess the semantics.  It should not be that 
 the first line instantiates and calls a template function, 
 whereas the second is evaluated as an expression with a bunch 
 of overloaded comma and comparison operators.

 Similarly, it should not be the case that:

 	auto x = a <= b;

 evaluates a comparison expression and assigns a boolean value 
 to x, whereas:

 	auto y = p <= q;

 creates an expression object capturing p and q, that needs to 
 be called later before it yields a boolean value.

 With a string DSL, that little identifier `myDsl` (or whatever 
 identifier you choose for this purpose) serves as a cue to the 
 reader of the code that something special is happening here.  
 For example:

 	auto y = deferred!`p <= q`;

 immediately tells the reader of the code that the <= is to be 
 understood with a different meaning than the usual <= operator. 
 Just as an expression like:

 	auto dg = (p, q) => p <= q;

 by virtue of its different syntax tells the reader that the 
 expression `p <= q` isn't being evaluated here and now, as it 
 otherwise would be.

 The presence of such visual cues is good, and is the way things 
 should be done.

 It should not be that something that looks like an expression, 
 evaluated here and now, should turn out to do something else. 
 That kind of free-for-all, can-mean-literally-anything 
 semantics makes code unreadable, unmaintainable, and a ripe 
 breeding ground for bugs -- someone (i.e., yourself after 3 
 months) will inevitably forget (or not know) the special 
 meaning of <= in this particular context and write wrong code 
 for it.


 P.S. And as a bonus, a string DSL gives you the freedom to 
 employ operators not found among the built-in D operators, for 
 example:

 	auto result = eval!`(f∘g)(√x ± √y)`;

 And if you feel the usual strings literals are too cumbersome 
 to use for long expressions, there's always the under-used 
 token strings to the rescue:

 	auto result = eval!q{
 		( (f ∘ g)(√(x + y) ± √(x - y)) ) / |x|·|y| +
 		2.0 * ∫ f(x)·dx
 	};

 The `eval` tells the reader that something special is happening 
 here, and also provides a name by which the reader can search 
 for the definition of the template that processes this 
 expression, and thereby learn what it means.

 Without this little identifier `eval`, it would be anyone's 
 guess as to what the code is trying to do. Throw in C++-style 
 SFINAE and Koenig lookup, and what ought to be a 10-second 
 source tree search for an identifier easily turns into a 6-hour 
 hair-pulling session of trying to understand exactly which 
 obscure rules the C++ compiler applied to resolve those 
 operators to which symbol(s) defined in which obscure files 
 buried deep in the source tree.


 T


Always hear that D is somehow better than C++ for operators but 
it isn't in quite a few places already.

     int a;
     auto c = (a) <= 0; // ok
     auto d = (a) => 0; // not ok

For some reason Walter thought in D if you overload the "+" 
operator you couldn't make it not commutative?? Still never got a 
reply to that, so I'll just assume he didn't know what 
commutative was. Yes you can make "a + b != b + a" be true quite 
easily.

Then you have things like "min" where you can do:

     foo( a /min/ b );

To get the "min" value between a and b. I guess you could use 
this as an example of why not to allow. But at the same time, 
we're already pretty much there. That includes "==" operator in 
that. So the comparisons operators aren't even consistent.
Feb 12 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 01:24:45 UTC, Rubn wrote:

 Always hear that D is somehow better than C++ for operators but 
 it isn't in quite a few places already.

     int a;
     auto c = (a) <= 0; // ok
     auto d = (a) => 0; // not ok


Oooh. That one's nasty! Technically unambiguous, but nasty!


 For some reason Walter thought in D if you overload the "+" 
 operator you couldn't make it not commutative?? Still never got 
 a reply to that, so I'll just assume he didn't know what 
 commutative was. Yes you can make "a + b != b + a" be true 
 quite easily.


With opBinary doing something completely different to 
opBinaryRight? Anyway that falls under the category of deliberate 
abuse, something we should not be considering, if people do that 
kind of thing then they should get what they deserve.


 Then you have things like "min" where you can do:

     foo( a /min/ b );

 To get the "min" value between a and b. I guess you could use 
 this as an example of why not to allow. But at the same time, 
 we're already pretty much there. That includes "==" operator in 
 that. So the comparisons operators aren't even consistent.


Indeed.
Feb 12 2019
parent reply Rubn <where is.this> writes:
On Wednesday, 13 February 2019 at 02:00:32 UTC, Nicholas Wilson 
wrote:

 On Wednesday, 13 February 2019 at 01:24:45 UTC, Rubn wrote:

 Always hear that D is somehow better than C++ for operators 
 but it isn't in quite a few places already.

     int a;
     auto c = (a) <= 0; // ok
     auto d = (a) => 0; // not ok


 Oooh. That one's nasty! Technically unambiguous, but nasty!


 For some reason Walter thought in D if you overload the "+" 
 operator you couldn't make it not commutative?? Still never 
 got a reply to that, so I'll just assume he didn't know what 
 commutative was. Yes you can make "a + b != b + a" be true 
 quite easily.


 With opBinary doing something completely different to 
 opBinaryRight? Anyway that falls under the category of 
 deliberate abuse, something we should not be considering, if 
 people do that kind of thing then they should get what they 
 deserve.


There are valid deliberate uses for this. For scalar types 
multiplication might be commutative, but for matrix 
multiplication it is not commutative. What good would it be to 
not be able to use an operator that denotes multiplication in the 
way it is intended to be used regarding the two types that are 
being multiplied. If you have a poorly named function it is in 
effect the same thing as misusing an operator. It'd be the same 
as misnaming a function, you read the function it doesn't do what 
the name suggests you end up having to look it up and you know 
for next time. Arguably it'd be easier to detect, if you say a 
class named "Magic" and it is multiplied by an integer, it makes 
you wonder what it is doing. But if you have have a function in 
"Magic" called "deleteUniverse()" what do you think that function 
is going to do? Oh when you look inside you find an empty 
function with a comment "TODO: figure out if this is possible". 
Which is worse in this situation? I'd argue the misnamed function 
is much more dangerous, at least if I see an operator being used 
with a custom type I know I should lookup what the operator 
means. There's also no good way to prevent poorly named functions.
Feb 13 2019
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 21:01:46 UTC, Rubn wrote:

 On Wednesday, 13 February 2019 at 02:00:32 UTC, Nicholas Wilson

 With opBinary doing something completely different to 
 opBinaryRight? Anyway that falls under the category of 
 deliberate abuse, something we should not be considering, if 
 people do that kind of thing then they should get what they 
 deserve.


 There are valid deliberate uses for this. For scalar types 
 multiplication might be commutative, but for matrix 
 multiplication it is not commutative.


Indeed, but you're example was with + ;). we could probably do a 
python and add   as a binary operator for that kind of thing 



 Which is worse in this situation? I'd argue the misnamed 
 function is much more dangerous, at least if I see an operator 
 being used with a custom type I know I should lookup what the 
 operator means. There's also no good way to prevent poorly 
 named functions.


Indeed.
Feb 13 2019
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 13, 2019 at 01:24:45AM +0000, Rubn via Digitalmars-d wrote:
[...]

 Always hear that D is somehow better than C++ for operators but it
 isn't in quite a few places already.
 
     int a;
     auto c = (a) <= 0; // ok
     auto d = (a) => 0; // not ok


Haha... yeah, I *thought* the choice of => for lambda syntax was not a
good idea.  But people seemed to love it at the time, what can I say?
:-/



 For some reason Walter thought in D if you overload the "+" operator
 you couldn't make it not commutative?? Still never got a reply to
 that, so I'll just assume he didn't know what commutative was. Yes you
 can make "a + b != b + a" be true quite easily.


You don't need operator overloading for that; '+' is already
non-commutative with IEEE 754 floating-point. In fact, most of IEEE 754
arithmetic is non-commutative, even though it appears to be in most
cases.



 Then you have things like "min" where you can do:
 
     foo( a /min/ b );
 
 To get the "min" value between a and b. I guess you could use this as
 an example of why not to allow.


Yes, that's a fine example of operator overloading abuse.



 But at the same time, we're already pretty much there. That includes
 "==" operator in that. So the comparisons operators aren't even
 consistent.


[...]

Don't get me wrong, the situation in D isn't perfect, but imperfection
shouldn't be a reason to open the door to worse things.


T

-- 
If you look at a thing nine hundred and ninety-nine times, you are perfectly
safe; if you look at it the thousandth time, you are in frightful danger of
seeing it for the first time. -- G. K. Chesterton
Feb 12 2019
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 02:37:50 UTC, H. S. Teoh wrote:

 On Wed, Feb 13, 2019 at 01:24:45AM +0000, Rubn via 
 Digitalmars-d wrote: [...]

 Then you have things like "min" where you can do:
 
     foo( a /min/ b );
 
 To get the "min" value between a and b. I guess you could use 
 this as an example of why not to allow.


 Yes, that's a fine example of operator overloading abuse.



 But at the same time, we're already pretty much there. That 
 includes "==" operator in that. So the comparisons operators 
 aren't even consistent.


 [...]

 Don't get me wrong, the situation in D isn't perfect, but 
 imperfection shouldn't be a reason to open the door to worse 
 things.


Imperfection should be a reason for adding things to make it 
better. That they can be abused and nobody does because that 
would be a stupid thing to do is of no relevance and certainly 
not a reason to not do it.
Feb 12 2019
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 13 February 2019 at 02:49:21 UTC, Nicholas Wilson 
wrote:

 [snip]

 Imperfection should be a reason for adding things to make it 
 better. That they can be abused and nobody does because that 
 would be a stupid thing to do is of no relevance and certainly 
 not a reason to not do it.


This kind of gets in to the opt-in vs. opt-out argument. If there 
were a attribute that was something like 
 enable("extended-operator-overloading") that would allow more 
operators to be overloaded only within those blocks, then it 
allows the default to be something restricted but gives users the 
option to make something different if needed.
Feb 13 2019
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 13 February 2019 at 02:37:50 UTC, H. S. Teoh wrote:

 [snip}

 Haha... yeah, I *thought* the choice of => for lambda syntax 
 was not a good idea.  But people seemed to love it at the time, 
 what can I say? :-/


What do you prefer instead? Can C++'s -> cause similar 
ambiguities?
Feb 13 2019
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 13, 2019 at 12:12:17PM +0000, jmh530 via Digitalmars-d wrote:

 On Wednesday, 13 February 2019 at 02:37:50 UTC, H. S. Teoh wrote:

 [snip}
 
 Haha... yeah, I *thought* the choice of => for lambda syntax was not
 a good idea.  But people seemed to love it at the time, what can I
 say?  :-/
 


 
 What do you prefer instead? Can C++'s -> cause similar ambiguities?


I'll admit, I've never really thought too hard about it.  But the main
idea is to choose something that can't be confused for something else.
Since -> isn't a token in D, it seems to be a viable candidate. But it
might be confusing for the, *ahem*, droves of C++ coders who just can't
wait to migrate to D. ;-)


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi
Feb 13 2019
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 16:07:06 UTC, H. S. Teoh wrote:

 On Wed, Feb 13, 2019 at 12:12:17PM +0000, jmh530 via 
 Digitalmars-d wrote:

 On Wednesday, 13 February 2019 at 02:37:50 UTC, H. S. Teoh 
 wrote:

 [snip}
 
 Haha... yeah, I *thought* the choice of => for lambda syntax 
 was not a good idea.  But people seemed to love it at the 
 time, what can I say?  :-/
 


 
 What do you prefer instead? Can C++'s -> cause similar 
 ambiguities?


 I'll admit, I've never really thought too hard about it.  But 
 the main idea is to choose something that can't be confused for 
 something else. Since -> isn't a token in D, it seems to be a 
 viable candidate. But it might be confusing for the, *ahem*, 
 droves of C++ coders who just can't wait to migrate to D. ;-)


-> is overloaded in C++ as:
  a->b as (*a).b
  a->b an overloadable operator
  a-->b ("goes to")
auto func() -> T (trailing return)
and I'm sure some others that I've missed.

Being confused is par for the course, I think they'd be right at 
home ;)
Feb 13 2019
parent reply Olivier FAURE <couteaubleu gmail.com> writes:
On Thursday, 14 February 2019 at 01:54:17 UTC, Nicholas Wilson 
wrote:

  a-->b ("goes to")


What? Pretty sure that parses to (a--) > b
Feb 14 2019
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 14 February 2019 at 17:42:48 UTC, Olivier FAURE 
wrote:

 On Thursday, 14 February 2019 at 01:54:17 UTC, Nicholas Wilson 
 wrote:

  a-->b ("goes to")


 What? Pretty sure that parses to (a--) > b


Yes, that's the joke.
Feb 14 2019
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 13, 2019 at 01:24:45AM +0000, Rubn via Digitalmars-d wrote:
[...]

     int a;
     auto c = (a) <= 0; // ok
     auto d = (a) => 0; // not ok


[...]

Argh, I just realized that this example is invalid: the
greater-than-or-equal operator is >=, not =>.

There is no ambiguity.

mouth.open
     .insert(foot);


T

-- 
Tech-savvy: euphemism for nerdy.
Feb 13 2019
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh wrote:

 Frankly, I think it's a very good thing that in D comparison 
 operators are confined to opEquals/opCmp.


So do I. Many more objects have partial ordering than arithmetic, 
having opCmp under opBinary would be annoying.


 If anything, I would vote for enforcing opEquals to return bool 
 and bool only.


That would be a backwards incompatible change, like it or not.


 The reason for this is readability and maintainability.  
 Symbols like <= or == should mean one and only one thing in the 
 language, and should not be subject to random overloaded 
 interpretations. Being built-in operators, they are used 
 universally in the language, and any inconsistency in semantics 
 hurts readability, comprehension, and maintainability, such as 
 C++'s free-for-all operator overloading, where any piece of 
 syntax can have wildly-divergent interpretations (even 
 completely different parse trees) depending on what came 
 before.  For example, recently I came up with a C++ monstrosity 
 where the lines:

         fun<A, B>(a, b);
         gun<T, U>(a, b);

 have wildly-different, completely unrelated *parse trees*, as 
 an illustration of how unreadable C++ can become.

 Yes, it's extremely flexible, yes it's extremely powerful and 
 can express literally *whatever* you want it to express. It's 
 also completely unreadable and unmaintainable, because the 
 surface structure of the code text becomes completely detached 
 from the actual underlying semantics. I don't even want to 
 imagine what debugging such kind of code must be like.


Thank goodness we use ! for template and don't have `,` available 
for overloading!


 Operator overloading should be reserved for objects that behave 
 like arithmetical entities in some way.


Like symbolic math.


 And especially comparison operators should not have any other 
 meaning than the standard meaning.  It should be illegal to 
 make == and <= mean something completely unrelated to each 
 other.


They do already mean something completely different, <= is an 
ordering, == is equality. Yes it would be bad for (a <= b) == (a 
== b) to be false. I'm sure you could already achieve that 
outcome, would you though? Of course not, it'd be stupid.


 If you need to redefine comparison operators, what you want is 
 really a DSL wherein you can define operators to mean whatever 
 you want.


Yes.


 The usual invocation of such a DSL as a compile-time argument, 
 say something like this:

 	myDsl!'a <= b'

 contains one often overlooked, but very important element: the 
 identifier `myDsl`, that sets it apart from other uses of `<=`, 
 and clearly identifies which interpretation should be ascribed 
 to the symbols found in the template argument.


Its also much uglier and does not commute with things that use <= 
i.e. generic functions.


 See, the thing is, when you see a random expression with 
 arithmetical operators in it, the expected semantics is the 
 computation of some kind of arithmetic objects producing an 
 arithmetical result -- because that's what such expressions 
 mean in general, in the language.  It's abusive to overload 
 that to mean something else entirely -- because there is no 
 warning sign to the reader of the code that something different 
 is happening.


Yes thats the use/abuse distinction, see my other post.

When you see a line like:

         fun<A, B>(a, b);

 and then somewhere else a line like:

         gun<T, U>(a, b);

 the actual behaviour of the code should be similar enough that 
 you can correctly guess the semantics.  It should not be that 
 the first line instantiates and calls a template function, 
 whereas the second is evaluated as an expression with a bunch 
 of overloaded comma and comparison operators.


Indeed! That is not what is being proposed at all!


 Similarly, it should not be the case that:

 	auto x = a <= b;

 evaluates a comparison expression and assigns a boolean value 
 to x, whereas:

 	auto y = p <= q;

 creates an expression object capturing p and q, that needs to 
 be called later before it yields a boolean value.


No. auto y = p <= q; should not e.g. open a socket (you could 
probably do that already with an impure opCmp). Being able to 
capture the expression `p <= q` is the _entire point_ of the 
proposal.


 With a string DSL, that little identifier `myDsl` (or whatever 
 identifier you choose for this purpose) serves as a cue to the 
 reader of the code that something special is happening here.  
 For example:

 	auto y = deferred!`p <= q`;

 immediately tells the reader of the code that the <= is to be 
 understood with a different meaning than the usual <= operator.


Its also much uglier and does not commute with things that use <= 
i.e. generic functions.


 Just as an expression like:

 	auto dg = (p, q) => p <= q;

 by virtue of its different syntax tells the reader that the 
 expression `p <= q` isn't being evaluated here and now, as it 
 otherwise would be.


Can't do symbolic computation with that.


 The presence of such visual cues is good, and is the way things 
 should be done.

 It should not be that something that looks like an expression, 
 evaluated here and now, should turn out to do something else. 
 That kind of free-for-all, can-mean-literally-anything 
 semantics makes code unreadable, unmaintainable, and a ripe 
 breeding ground for bugs -- someone (i.e., yourself after 3 
 months) will inevitably forget (or not know) the special 
 meaning of <= in this particular context and write wrong code 
 for it.


Type autocompletion will tell you the result of  p <= q; which at 
that point if it is still unclear you have bigger problems. In 
generic code you have no choice but to assume that p <= q; is a 
comparison, if someone is using that with a symbolic engine then 
the meaning doesn't change.


 P.S. And as a bonus, a string DSL gives you the freedom to 
 employ operators not found among the built-in D operators, for 
 example:

 	auto result = eval!`(f∘g)(√x ± √y)`;

 And if you feel the usual strings literals are too cumbersome 
 to use for long expressions, there's always the under-used 
 token strings to the rescue:

 	auto result = eval!q{
 		( (f ∘ g)(√(x + y) ± √(x - y)) ) / |x|·|y| +
 		2.0 * ∫ f(x)·dx
 	};

 The `eval` tells the reader that something special is happening 
 here, and also provides a name by which the reader can search 
 for the definition of the template that processes this 
 expression, and thereby learn what it means.

 Without this little identifier `eval`, it would be anyone's 
 guess as to what the code is trying to do.


I would expect it the compute the tuple
(f(g(sqrt(x+y)) + sqrt(x-y)/(abs(x).dot(abs(y)) + 2*integrate(f),
(f(g(sqrt(x+y)) - sqrt(x+y)/(abs(x).dot(abs(y)) + 2*integrate(f)

(you missed the bounds on the integral and x is ambiguous in the 
integral)

What else would I think it would do? If that guess is wrong then 
the person has abused the operators, if its correct that thats a 
win. I'd bet money you could do just that in Julia. I'm not 
suggesting we go that far but they would definitely consider that 
a feature.


 Throw in C++-style SFINAE and Koenig lookup, and what ought to 
 be a 10-second source tree search for an identifier easily 
 turns into a 6-hour hair-pulling session of trying to 
 understand exactly which obscure rules the C++ compiler applied 
 to resolve those operators to which symbol(s) defined in which 
 obscure files buried deep in the source tree.


Its a good thing we don't have SFINAE and Koenig lookup.
Feb 12 2019
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Feb 13, 2019 at 01:50:21AM +0000, Nicholas Wilson via Digitalmars-d
wrote:

 On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh wrote:

 Frankly, I think it's a very good thing that in D comparison
 operators are confined to opEquals/opCmp.


 
 So do I. Many more objects have partial ordering than arithmetic,
 having opCmp under opBinary would be annoying.


Actually, contrary to what Andrei has claimed in the past, opCmp (as
currently implemented) does NOT allow for a consistent definition of a
partial ordering.  I know because I've tried to do it with an integer
set type.  The problem is that when opCmp returns 0, it's ambiguous
whether it means "incomparable" or "equal".  This makes it impossible to
make <= equivalent to the is-subset predicate. For example, here's a
prospective implementation:

	struct IntSet {
		Impl impl;
		int opCmp(in IntSet b) {
			bool isSubset = impl.isSubsetOf(b.impl);
			bool isSuperset = b.impl.isSubsetOf(impl);

			if (isSubset && isSuperset)
				return 0;
			else if (isSubset)
				return -1;
			else if (isSuperset)
				return 1;
			else
				return 0; // incomparable
		}
		bool opEquals(in IntSet b) {
			return impl.isSubsetOf(b.impl) &&
				b.impl.isSubsetOf(impl);
		}
	}

Efficiency concerns aside (should not need to compute two-way subset
relation just to determine <=, for example), this looks good, right?

Nope.  Suppose s is a subset of t.  Then opCmp would return -1, and the
predicate s <= t would be true, because s <= t lowers to:

	s.opCmp(t) <= 0

Now suppose s and t are disjoint (i.e., incomparable). According to the
spec, opCmp should return 0 in this case.  But then:

	assert(!(s <= t));

fails, because opCmp returns 0.  So we cannot distinguish between s
being a subset of t vs. s and t being incomparable by using the <=
operator.  Similarly for >=.  Therefore, s <= t cannot represent the
is-subset operation.

We can try to redefine opCmp to return something different from what's
outlined above, but we'd run into other problems.

tl;dr: opCmp does NOT support partial orderings in any real sense (you
have to accept that <= and >= are true for incomparable elements!). Only
linear orderings work correctly in all cases.



 If anything, I would vote for enforcing opEquals to return bool and
 bool only.


 
 That would be a backwards incompatible change, like it or not.


I know.  That's why I haven't proposed it yet. :-D



 For example, recently I came up with a C++ monstrosity where the
 lines:
 
         fun<A, B>(a, b);
         gun<T, U>(a, b);
 
 have wildly-different, completely unrelated *parse trees*, as an
 illustration of how unreadable C++ can become.




[...]

 Thank goodness we use ! for template and don't have `,` available for
 overloading!


Yeah, I was very happy when we finally deprecated the evil comma
operator. Good riddance.



 Operator overloading should be reserved for objects that behave like
 arithmetical entities in some way.


 
 Like symbolic math.


I know what you're trying to drive at, but there's a difference
arithmetic in code vs. arithmetic in math. An expression in code is
generally assumed to execute when control flow reaches that line of
code. It's unexpected behaviour for what looks like an assignment of the
result of an expression to instead assign the expression itself as
something to be evaluated later.



 And especially comparison operators should not have any other
 meaning than the standard meaning.  It should be illegal to make ==
 and <= mean something completely unrelated to each other.


 
 They do already mean something completely different, <= is an
 ordering, == is equality. Yes it would be bad for (a <= b) == (a == b)
 to be false.  I'm sure you could already achieve that outcome, would
 you though? Of course not, it'd be stupid.


What I meant was that == and <= should behave consistently with each
other.  Ideally both <= and == should be handled by the same function.
Then consistency would be guaranteed. But for practical reasons we
separate them, one reason being that often equality is much cheaper to
compute than linear ordering.


[...]

 The usual invocation of such a DSL as a compile-time argument, say
 something like this:
 
 	myDsl!'a <= b'
 
 contains one often overlooked, but very important element: the
 identifier `myDsl`, that sets it apart from other uses of `<=`, and
 clearly identifies which interpretation should be ascribed to the
 symbols found in the template argument.


 
 Its also much uglier and does not commute with things that use <= i.e.
 generic functions.


Ugliness is debatable; I find that the identifier serves as a good
marker to indicate departure from usual semantics.

As for not commuting with generic functions, do you have a specific
example in mind?  Because all the obvious (to me) examples I can think
of are cases where I think would be better off *not* working with
generic code, because the generic code expects one set of semantics but
overloading opCmp/opEquals to return arbitrary objects produce a
different set of semantics.


[...]

 Similarly, it should not be the case that:
 
 	auto x = a <= b;
 
 evaluates a comparison expression and assigns a boolean value to x,
 whereas:
 
 	auto y = p <= q;
 
 creates an expression object capturing p and q, that needs to be
 called later before it yields a boolean value.


 
 No. auto y = p <= q; should not e.g. open a socket (you could probably
 do that already with an impure opCmp). Being able to capture the
 expression `p <= q` is the _entire point_ of the proposal.


And that's the point I disagree on.  The current expectation of a line
of code like `auto y = p <= q;` is that typeof(y) == bool.  For it to
return a type other than bool is unexpected behaviour, and leads to
subtle differences in semantics that will likely result in bugs.

Explicitly marking it, e.g., as `auto y = symbolic!"p <= q";` makes it
much more obvious what's really going on with the code.  Yes it's ugly,
but at the same time this ugliness is IMO necessary warning for the
would-be code maintainer to understand that the semantics depart from
the usual expectations.


[...]

 P.S. And as a bonus, a string DSL gives you the freedom to employ
 operators not found among the built-in D operators, for example:
 
 	auto result = eval!`(f∘g)(√x ± √y)`;
 
 And if you feel the usual strings literals are too cumbersome to use
 for long expressions, there's always the under-used token strings to
 the rescue:
 
 	auto result = eval!q{
 		( (f ∘ g)(√(x + y) ± √(x - y)) ) / |x|·|y| +
 		2.0 * ∫ f(x)·dx
 	};
 
 The `eval` tells the reader that something special is happening
 here, and also provides a name by which the reader can search for
 the definition of the template that processes this expression, and
 thereby learn what it means.
 
 Without this little identifier `eval`, it would be anyone's guess as
 to what the code is trying to do.


 
 I would expect it the compute the tuple
 (f(g(sqrt(x+y)) + sqrt(x-y)/(abs(x).dot(abs(y)) + 2*integrate(f),
 (f(g(sqrt(x+y)) - sqrt(x+y)/(abs(x).dot(abs(y)) + 2*integrate(f)
 
 (you missed the bounds on the integral and x is ambiguous in the integral)
 
 What else would I think it would do? If that guess is wrong then the person
 has abused the operators, if its correct that thats a win. I'd bet money you
 could do just that in Julia. I'm not suggesting we go that far but they
 would definitely consider that a feature.


[...]

My point was that there are no such operators as ∘ or ± in D, so even if
you had free-for-all operator overloading you couldn't implement such an
expression.  Using a DSL frees you from that constraint.

And because there are no such operators in D, they have no standard
meaning defined by the language, so different authors likely have
different definitions for them. So even in the case where you *could*
add arbitrary operators to the language, you'll end up with a mess as
soon as your code imports two libraries that happen to overload some of
the same custom operators -- it would be hard to tell which set of
semantics apply to the operators in any given piece of code; you'd have
to study the context to be sure.  Having the `eval` or whatever else
prefixed identifier in front of the DSL resolves that question
instantly.


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi
Feb 13 2019
next sibling parent Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Wednesday, 13 February 2019 at 18:42:43 UTC, H. S. Teoh wrote:

 On Wed, Feb 13, 2019 at 01:50:21AM +0000, Nicholas Wilson via 
 Digitalmars-d wrote:

 On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh 
 wrote:

 Frankly, I think it's a very good thing that in D comparison 
 operators are confined to opEquals/opCmp.


 
 So do I. Many more objects have partial ordering than 
 arithmetic, having opCmp under opBinary would be annoying.


 Actually, contrary to what Andrei has claimed in the past, 
 opCmp (as currently implemented) does NOT allow for a 
 consistent definition of a partial ordering.  I know because 
 I've tried to do it with an integer set type.  The problem is 
 that when opCmp returns 0, it's ambiguous whether it means 
 "incomparable" or "equal".  This makes it impossible to make <= 
 equivalent to the is-subset predicate. For example, here's a 
 prospective implementation:

 	struct IntSet {
 		Impl impl;
 		int opCmp(in IntSet b) {
 			bool isSubset = impl.isSubsetOf(b.impl);
 			bool isSuperset = b.impl.isSubsetOf(impl);

 			if (isSubset && isSuperset)
 				return 0;
 			else if (isSubset)
 				return -1;
 			else if (isSuperset)
 				return 1;
 			else
 				return 0; // incomparable
 		}


opCmp is allowed to return float, making it possible to 
distinguish all 4 possible values
float opCmp(in IntSet b) {
    bool isSubset = impl.isSubsetOf(b.impl);
    bool isSuperset = b.impl.isSubsetOf(impl);

    if (isSubset && isSuperset)
       return 0;
    else if (isSubset)
       return -1;
    else if (isSuperset)
       return 1;
    else
       return float.init; // incomparable - NaN
}

This works fine, I've used it in several types in my libraries.
Feb 13 2019
prev sibling next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 18:42:43 UTC, H. S. Teoh wrote:

 On Wed, Feb 13, 2019 at 01:50:21AM +0000, Nicholas Wilson via 
 Digitalmars-d wrote:

 On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh 
 wrote:

 Frankly, I think it's a very good thing that in D comparison 
 operators are confined to opEquals/opCmp.


 
 So do I. Many more objects have partial ordering than 
 arithmetic, having opCmp under opBinary would be annoying.


 [opCmp doesn't actually work for partial ordering]


This proposal would actually let you match

<  : subset
<= : subset or equal
...

and would say, let you return a tuple of (result of predicate, 
are sets disjoint)

auto sub = setA < setB;
if (sub.pred && !sub.disjoint) { ... }


 Operator overloading should be reserved for objects that 
 behave like arithmetical entities in some way.


 
 Like symbolic math.


 I know what you're trying to drive at, but there's a difference 
 arithmetic in code vs. arithmetic in math.


What difference? Graph execution of math math on a nice library 
looks like code math and I don't see anyone in PyTorch 
complaining.


 An expression in code is generally assumed to execute when 
 control flow reaches that line of code.


async says hello.


 It's unexpected behaviour for what looks like an assignment of 
 the result of an expression to instead assign the expression 
 itself as something to be evaluated later.


You are likely missing the context of the expression, e.g. a DB 
query, or that you are dealing with symbolic math. Or you don't 
(and can't) care because you are in generic code.


 What I meant was that == and <= should behave consistently with 
 each other.  Ideally both <= and == should be handled by the 
 same function. Then consistency would be guaranteed. But for 
 practical reasons we separate them, one reason being that often 
 equality is much cheaper to compute than linear ordering.


Also manyfold more objects can be compared for (in)equality than 
can be for ordering.
Feb 13 2019
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 13 February 2019 at 18:42:43 UTC, H. S. Teoh wrote:

 On Wed, Feb 13, 2019 at 01:50:21AM +0000, Nicholas Wilson via 
 Digitalmars-d wrote:

 On Wednesday, 13 February 2019 at 00:56:48 UTC, H. S. Teoh 
 wrote:

 The usual invocation of such a DSL as a compile-time 
 argument, say something like this:
 
 	myDsl!'a <= b'
 
 contains one often overlooked, but very important element: 
 the identifier `myDsl`, that sets it apart from other uses 
 of `<=`, and clearly identifies which interpretation should 
 be ascribed to the symbols found in the template argument.


 
 Its also much uglier and does not commute with things that use 
 <= i.e. generic functions.


 Ugliness is debatable; I find that the identifier serves as a 
 good marker to indicate departure from usual semantics.


All of the examples in the thread are context-free. The 
distinction between eval now to a bool vs other uses should be a 
lot more pronounced with the context of the surrounding code. Or 
you simply just don't care.


 As for [myDsl!'<'] not commuting with generic functions, do you 
 have a specific example in mind?  Because all the obvious (to 
 me) examples I can think of are cases where I think would be 
 better off *not* working with generic code, because the generic 
 code expects one set of semantics but overloading 
 opCmp/opEquals to return arbitrary objects produce a different 
 set of semantics.


Math kernels. kern!float would be a scalar form, kern!float4 
would be vectorised, kern!Expression would be something you could 
pass to e.g. TensorFlow.


 No. auto y = p <= q; should not e.g. open a socket (you could 
 probably do that already with an impure opCmp). Being able to 
 capture the expression `p <= q` is the _entire point_ of the 
 proposal.


 And that's the point I disagree on.  The current expectation of 
 a line of code like `auto y = p <= q;` is that typeof(y) == 
 bool.  For it to return a type other than bool is unexpected 
 behaviour,


Such is the nature of change.


 and leads to subtle differences in semantics


Which should be obvious from the context of the code, or you 
simply don't care about the distinction.


 that will likely result in bugs.


I don't think so. Compile errors yes, because types won't match. 
But bugs?


 Explicitly marking it, e.g., as `auto y = symbolic!"p <= q";` 
 makes it much more obvious what's really going on with the code.


Again these examples are context free: You know the types of p 
and q.
and if you don't (generic code) then you simply don't care. Also 
if you have a page full of `symbolic!"blah"` you're pretty 
quickly going to lose the forest for the trees, especially when 
you have opBinary overloaded to symbolics without 
`symbolic!"blah"`.


 Yes it's ugly, but at the same time this ugliness is IMO 
 necessary warning for the would-be code maintainer to 
 understand that the semantics depart from the usual 
 expectations.


You have comments for that.


 My point was that there are no such operators as ∘ or ± in D, 
 so even if you had free-for-all operator overloading you 
 couldn't implement such an expression.  Using a DSL frees you 
 from that constraint.

 And because there are no such operators in D, they have no 
 standard meaning defined by the language, so different authors 
 likely have different definitions for them.


Most of those operators have standard definitions in math which 
makes that point moot. And if you're making up operators 
willy-nilly then you should expect to end up with an 
unmaintainable mess. I'm not catering for those kind of people.


 So even in the case where you *could* add arbitrary operators 
 to the language, you'll end up with a mess as soon as your code 
 imports two libraries that happen to overload some of the same 
 custom operators -- it would be hard to tell which set of 
 semantics apply to the operators in any given piece of code; 
 you'd have to study the context to be sure.


All you need to know are the types involved and more usefully 
what module they come from, which any decent editor with plugin 
should be able to tell you.


 Having the `eval` or whatever else prefixed identifier in front 
 of the DSL resolves that question instantly.


Yes but does it need to be there? (aside the missing bounds and 
ambiguity of the integral) I'd be surprised if that wasn't valid 
Julia code. Now I'm not suggesting we go that far, having 
arbitrary unicode operators would be a nightmare, what with 
normalisation and all (not to mention trying to ender the damn 
things on a keyboard).
Feb 13 2019
prev sibling parent Rubn <where is.this> writes:
On Tuesday, 12 February 2019 at 23:24:54 UTC, Olivier FAURE wrote:

 On Tuesday, 12 February 2019 at 13:22:12 UTC, Nicholas Wilson 
 wrote:

 Oh well, more things to discuss at dconf. That whole thread 
 seems to be Walter either being stubborn or not getting it (or 
 both).


 Or your proposal has drawbacks you're underestimating.

 This community really needs to stop defaulting to "We could 
 totally use my awesome feature if Walter wasn't so stubborn" 
 whenever discussing language changes.


What doesn't explain his view point, at all most of the time. He 
just says no or says nothing. When you don't say anything without 
an explanation of why, it'll only natural to call that stubborn. 
An explanation you can have a discussion against, someone saying 
"no" not really.
Feb 12 2019
prev sibling next sibling parent aliak <something something.com> writes:
On Tuesday, 12 February 2019 at 04:15:34 UTC, Nicholas Wilson 
wrote:

 With opBinary and opUnary it is possible to create types that 
 span the binary and unary operators. However opEquals can only 
 be used to types that compare for equality not inequality, 
 since a == b -> a.opEquals(b) can return whatever type it likes 
 but a == b -> !(a.opEquals(b)) returns a bool. opCmp can't be 
 used at all since a <=> b -> a.opCmp(b) <=> 0 which evaluates 
 to a bool.

 I propose that in addition to the current (non-template) forms 
 of opCmp/opEquals

 struct A { bool opEquals(A rhs); int opCmp(A rhs); }

 we allow

 struct B
 {
     T opEquals(string op)(B rhs); // op is "<" "<=" etc.
     T opCmp(string op)(B rhs);    // op is "==" "!="
 }

 where T is any arbitrary type.

 see also 
 https://github.com/k3kaimu/dranges/blob/master/source/dranges/_lambda.d


This looks super cool!!

Can a workaround for the opEquals be that a.opEquals(b) produces 
a ProxyType and then ProxyType.opUnary gets you to where you want 
to go?
Feb 11 2019
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12.02.19 05:15, Nicholas Wilson wrote:

 With opBinary and opUnary it is possible to create types that span the 
 binary and unary operators. However opEquals can only be used to types 
 that compare for equality not inequality, since a == b -> a.opEquals(b) 
 can return whatever type it likes but a == b -> !(a.opEquals(b)) returns 
 a bool. opCmp can't be used at all since a <=> b -> a.opCmp(b) <=> 0 
 which evaluates to a bool.
 
 I propose that in addition to the current (non-template) forms of 
 opCmp/opEquals
 
 struct A { bool opEquals(A rhs); int opCmp(A rhs); }
 
 we allow
 
 struct B
 {
      T opEquals(string op)(B rhs); // op is "<" "<=" etc.
      T opCmp(string op)(B rhs);    // op is "==" "!="
 }
 
 where T is any arbitrary type.
 
 see also 
 https://github.com/k3kaimu/dranges/blob/master/source/dranges/_lambda.d


Ideally those would both be cases of opBinary, but I guess your proposal 
has better backwards compatibility.
Feb 11 2019
prev sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 12 February 2019 at 04:15:34 UTC, Nicholas Wilson 
wrote:

 With opBinary and opUnary it is possible to create types that 
 span the binary and unary operators. However opEquals can only 
 be used to types that compare for equality not inequality, 
 since a == b -> a.opEquals(b) can return whatever type it likes 
 but a == b -> !(a.opEquals(b)) returns a bool. opCmp can't be 
 used at all since a <=> b -> a.opCmp(b) <=> 0 which evaluates 
 to a bool.

 I propose that in addition to the current (non-template) forms 
 of opCmp/opEquals

 struct A { bool opEquals(A rhs); int opCmp(A rhs); }

 we allow

 struct B
 {
     T opEquals(string op)(B rhs); // op is "<" "<=" etc.
     T opCmp(string op)(B rhs);    // op is "==" "!="
 }

 where T is any arbitrary type.

 see also 
 https://github.com/k3kaimu/dranges/blob/master/source/dranges/_lambda.d


Any argument about this should consider what was necessary to 
make std.typecons.Proxy work properly for floating point.

https://github.com/dlang/phobos/pull/3927

and the original forum discussion

https://forum.dlang.org/post/vfgawgvzzfgjhshavmlq forum.dlang.org
Feb 13 2019