www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - opConcatAll?

reply Steven Schveighoffer <schveiguy gmail.com> writes:
If I have an array:

int[] arr = [1, 2, 3];

And I use it in a concatenation chain:

auto newArr = [0] ~ arr ~ [4, 5, 6];

The compiler calls a single function to allocate these together 
(_d_arraycatnTX).

However, if I define a struct instead:

struct S
{
    int [] x;
    S opBinary(string s : "~")(S other) {return S(x ~ other.x); }
}

Now, if I do:

S arr = S([1, 2, 3]);

auto newArr = S([0]) ~ arr ~ S([4, 5, 6]);

I get one call PER operator, in other words, it gets translated to:

S([0])opBinary!"~"(arr).opBinary!"~"(S([4, 5, 6]));

Which means one separate allocation for each field. If you have a lot of 
these all put together, it could add up to a lot of allocations, with 
most of the allocations as temporaries.

What about an opConcatAll (or other possible name), which can accept all 
the following concatenations as parameters?

in other words, given:

a ~ b ~ c ~ d

will translate to

a.opConcatAll(b, c, d)

if possible.


Would that make sense?

-Steve
Jul 02 2020
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer 
wrote:

 If I have an array:

 int[] arr = [1, 2, 3];

 And I use it in a concatenation chain:

 auto newArr = [0] ~ arr ~ [4, 5, 6];

 [snip]


With a few small changes, you could just as easily have written

int[] newArr;
newArr[] = [0, 0, 0] + arr[] + [4, 5, 6];

which is similar to the traditional type of problem that 
expression templates were designed to solve.
Jul 02 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/2/20 12:20 PM, jmh530 wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 If I have an array:

 int[] arr = [1, 2, 3];

 And I use it in a concatenation chain:

 auto newArr = [0] ~ arr ~ [4, 5, 6];

 [snip]


 
 With a few small changes, you could just as easily have written
 
 int[] newArr;
 newArr[] = [0, 0, 0] + arr[] + [4, 5, 6];
 
 which is similar to the traditional type of problem that expression 
 templates were designed to solve.


Hm.. I think you misunderstand the example.

-Steve
Jul 02 2020
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 2 July 2020 at 18:40:21 UTC, Steven Schveighoffer 
wrote:

 [snip]
 
 With a few small changes, you could just as easily have written
 
 int[] newArr;
 newArr[] = [0, 0, 0] + arr[] + [4, 5, 6];
 
 which is similar to the traditional type of problem that 
 expression templates were designed to solve.


 Hm.. I think you misunderstand the example.

 -Steve


I think I understand the example, but I may not have made my 
point clear enough...

You are concerned about a ~ b ~ c and avoiding unnecessary 
allocations.

Expression templates are concerned with problems like a + b + c 
and avoiding the use of unnecessary temporaries.

They seem like similar problems to me. Temporaries and 
allocations are obviously different, but the structure of the 
problem is similar. You yourself note that "most of the 
allocations as temporaries".

If my point makes sense, then the implication is that this is a 
problem that is not unique to concatenation and also applies to 
the mathematical operators, in their own way.
Jul 02 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/2/20 3:18 PM, jmh530 wrote:

 On Thursday, 2 July 2020 at 18:40:21 UTC, Steven Schveighoffer wrote:

 [snip]

 With a few small changes, you could just as easily have written

 int[] newArr;
 newArr[] = [0, 0, 0] + arr[] + [4, 5, 6];

 which is similar to the traditional type of problem that expression 
 templates were designed to solve.


 Hm.. I think you misunderstand the example.


 
 I think I understand the example, but I may not have made my point clear 
 enough...
 
 You are concerned about a ~ b ~ c and avoiding unnecessary allocations.
 
 Expression templates are concerned with problems like a + b + c and 
 avoiding the use of unnecessary temporaries.
 
 They seem like similar problems to me. Temporaries and allocations are 
 obviously different, but the structure of the problem is similar. You 
 yourself note that "most of the allocations as temporaries".
 
 If my point makes sense, then the implication is that this is a problem 
 that is not unique to concatenation and also applies to the mathematical 
 operators, in their own way.


OK, I see what you are saying. And yes, a generalized mechanism to 
handle this would be useful for both avoiding allocations and avoiding 
temporaries (among other things).

Nick's idea is pretty good for both.

-Steve
Jul 02 2020
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 2 July 2020 at 20:29:59 UTC, Steven Schveighoffer 
wrote:

 [snip]

 OK, I see what you are saying. And yes, a generalized mechanism 
 to handle this would be useful for both avoiding allocations 
 and avoiding temporaries (among other things).

 Nick's idea is pretty good for both.

 -Steve


Nick's idea only handles the case where all the operators are the 
same, e.g. a + b + c and not a + b - c.
Jul 02 2020
parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 2 July 2020 at 20:33:47 UTC, jmh530 wrote:

 [snip]

 Nick's idea only handles the case where all the operators are 
 the same, e.g. a + b + c and not a + b - c.


Oh, sorry, I didn't see that last bit. Nevermind.
Jul 02 2020
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer 
wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


Sounds good, although I think multiple other operations beside 
concatenation could benefit from being intercepted:

opNary(string op, T...)(T args)

E.g. perhaps BigInt can avoid one or more reallocations for >2 
args. The compiler would try opNary first in that case, and 
fallback to successive calls to opBinary as now.

In fact, maybe some types could benefit from intercepting 
different operations at once:

opNary(string[] ops, T...)(T args)
Jul 02 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/2/20 3:28 PM, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 
 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:
 
 opNary(string op, T...)(T args)
 
 E.g. perhaps BigInt can avoid one or more reallocations for >2 args. The 
 compiler would try opNary first in that case, and fallback to successive 
 calls to opBinary as now.
 
 In fact, maybe some types could benefit from intercepting different 
 operations at once:
 
 opNary(string[] ops, T...)(T args)


That would be awesome, and cover my case!

-Steve
Jul 02 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 2 July 2020 at 20:26:15 UTC, Steven Schveighoffer 
wrote:

 On 7/2/20 3:28 PM, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer 
 wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 
 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:
 
 opNary(string op, T...)(T args)
 
 E.g. perhaps BigInt can avoid one or more reallocations for >2 
 args. The compiler would try opNary first in that case, and 
 fallback to successive calls to opBinary as now.
 
 In fact, maybe some types could benefit from intercepting 
 different operations at once:
 
 opNary(string[] ops, T...)(T args)


 That would be awesome, and cover my case!

 -Steve


We kind of have this already, but a bit closer to what jmh was 
describing earlier.
Though it's just an implementation detail inside druntime for 
implementing array ops:

https://github.com/dlang/druntime/blob/v2.092.1/src/core/internal/array/operations.d#L15-L36

I think that it would be pretty cool if this technique could be 
applied to user-defined types (e.g. allow library authors to 
provide their own `arrayOp` implementation of their types).
Jul 03 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/3/20 3:17 AM, Petar Kirov [ZombineDev] wrote:

 On Thursday, 2 July 2020 at 20:26:15 UTC, Steven Schveighoffer wrote:

 On 7/2/20 3:28 PM, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)

 E.g. perhaps BigInt can avoid one or more reallocations for >2 args. 
 The compiler would try opNary first in that case, and fallback to 
 successive calls to opBinary as now.

 In fact, maybe some types could benefit from intercepting different 
 operations at once:

 opNary(string[] ops, T...)(T args)


 That would be awesome, and cover my case!

 -Steve


 
 We kind of have this already, but a bit closer to what jmh was 
 describing earlier.
 Though it's just an implementation detail inside druntime for 
 implementing array ops:
 
 https://github.com/dlang/druntime/blob/v2.092.1/src/core/internal/array/
perations.d#L15-L36 
 
 
 I think that it would be pretty cool if this technique could be applied 
 to user-defined types (e.g. allow library authors to provide their own 
 `arrayOp` implementation of their types).


How are parens handled?
Jul 03 2020
next sibling parent Aliak <something something.com> writes:
On Friday, 3 July 2020 at 15:39:08 UTC, Andrei Alexandrescu wrote:

 On 7/3/20 3:17 AM, Petar Kirov [ZombineDev] wrote:

 On Thursday, 2 July 2020 at 20:26:15 UTC, Steven Schveighoffer 
 wrote:

 On 7/2/20 3:28 PM, Nick Treleaven wrote:

 [...]


 That would be awesome, and cover my case!

 -Steve


 
 We kind of have this already, but a bit closer to what jmh was 
 describing earlier.
 Though it's just an implementation detail inside druntime for 
 implementing array ops:
 
 https://github.com/dlang/druntime/blob/v2.092.1/src/core/internal/array/operations.d#L15-L36
 
 
 I think that it would be pretty cool if this technique could 
 be applied to user-defined types (e.g. allow library authors 
 to provide their own `arrayOp` implementation of their types).


 How are parens handled?


By multiple consecutive lowerings in order of parens?

(a op b) op c => t.op(a, b).op(c)
Jul 03 2020
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/20 11:39 AM, Andrei Alexandrescu wrote:

 On 7/3/20 3:17 AM, Petar Kirov [ZombineDev] wrote:

 On Thursday, 2 July 2020 at 20:26:15 UTC, Steven Schveighoffer wrote:

 On 7/2/20 3:28 PM, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)

 E.g. perhaps BigInt can avoid one or more reallocations for >2 args. 
 The compiler would try opNary first in that case, and fallback to 
 successive calls to opBinary as now.

 In fact, maybe some types could benefit from intercepting different 
 operations at once:

 opNary(string[] ops, T...)(T args)


 That would be awesome, and cover my case!


 We kind of have this already, but a bit closer to what jmh was 
 describing earlier.
 Though it's just an implementation detail inside druntime for 
 implementing array ops:

 https://github.com/dlang/druntime/blob/v2.092.1/src/core/internal/array/
perations.d#L15-L36 


 I think that it would be pretty cool if this technique could be 
 applied to user-defined types (e.g. allow library authors to provide 
 their own `arrayOp` implementation of their types).


 
 How are parens handled?


There are possibilities. The array vector operations seem to be passed 
in RPN, negating the need for parentheses handling (maybe the compiler 
translates the expression for you?)

Another possibility is just to stop at parentheses boundaries. e.g.:

(a + b + c) * d => a.opNary!(["+", "+"])(b, c).opBinary!"*"(d);

Which isn't as powerful, but still helpful in many cases.

-Steve
Jul 03 2020
prev sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 3 July 2020 at 15:39:08 UTC, Andrei Alexandrescu wrote:

 On 7/3/20 3:17 AM, Petar Kirov [ZombineDev] wrote:

 [..]
 
 We kind of have this already, but a bit closer to what jmh was 
 describing earlier.
 Though it's just an implementation detail inside druntime for 
 implementing array ops:
 
 https://github.com/dlang/druntime/blob/v2.092.1/src/core/internal/array/operations.d#L15-L36
 
 
 I think that it would be pretty cool if this technique could 
 be applied to user-defined types (e.g. allow library authors 
 to provide their own `arrayOp` implementation of their types).


 How are parens handled?


Parens don't need special handling since they're already handled 
by the dmd parser. The frontend simply traverses the expression 
tree for the array op and outputs it in reverse polish notation 
here: 
https://github.com/dlang/dmd/blob/v2.092.1/src/dmd/arrayop.d#L180-L255

For reference, here's the PR by Martin Nowak, who implemented 
this: https://github.com/dlang/dmd/pull/7032
Jul 03 2020
prev sibling next sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Thursday, 2 July 2020 at 19:28:15 UTC, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer 
 wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)

 E.g. perhaps BigInt can avoid one or more reallocations for >2 
 args. The compiler would try opNary first in that case, and 
 fallback to successive calls to opBinary as now.

 In fact, maybe some types could benefit from intercepting 
 different operations at once:

 opNary(string[] ops, T...)(T args)


One typical case that this would do great for is BigInt modular 
exponentiation (a ^^ b) % c.

One problem with this idea is with order of operations - there's 
no way to distinguish (a*b)+c from a*(b+c).

I wrote[0] a suggestion two years ago for something I called 
rvalue types, which might be interesting for the use cases 
described here. Essentially, it's a type for a temporary value 
that will decay to a regular value whenever it's assigned to 
something - somewhat like an alias this that is preferred to 
using the actual type.

--
   Simen

[0]: 
https://forum.dlang.org/post/sfvqxnnibgbuebbxweqb forum.dlang.org
Jul 02 2020
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.07.20 21:28, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 
 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:
 
 opNary(string op, T...)(T args)
 
 E.g. perhaps BigInt can avoid one or more reallocations for >2 args. The 
 compiler would try opNary first in that case, and fallback to successive 
 calls to opBinary as now.
 
 In fact, maybe some types could benefit from intercepting different 
 operations at once:
 
 opNary(string[] ops, T...)(T args)


Maybe it could pass a single fully parenthesized expression instead of 
an array of operators.

But given D's philosophy of crippling operator overloading in the name 
of prevention of "abuse", something like this will likely never be added.
Jul 03 2020
prev sibling next sibling parent reply user1234 <user1234 12.de> writes:
On Thursday, 2 July 2020 at 19:28:15 UTC, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer 
 wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)

 E.g. perhaps BigInt can avoid one or more reallocations for >2 
 args. The compiler would try opNary first in that case, and 
 fallback to successive calls to opBinary as now.

 In fact, maybe some types could benefit from intercepting 
 different operations at once:

 opNary(string[] ops, T...)(T args)


Wow, that looked cool the first 3 secons but now I wander if  you 
really want to let the user break the rules of precedence ?
Jul 03 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/20 11:25 AM, user1234 wrote:

 On Thursday, 2 July 2020 at 19:28:15 UTC, Nick Treleaven wrote:

 On Thursday, 2 July 2020 at 15:47:39 UTC, Steven Schveighoffer wrote:

 a ~ b ~ c ~ d

 will translate to

 a.opConcatAll(b, c, d)


 Sounds good, although I think multiple other operations beside 
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)

 E.g. perhaps BigInt can avoid one or more reallocations for >2 args. 
 The compiler would try opNary first in that case, and fallback to 
 successive calls to opBinary as now.

 In fact, maybe some types could benefit from intercepting different 
 operations at once:

 opNary(string[] ops, T...)(T args)


 
 Wow, that looked cool the first 3 secons but now I wander if  you really 
 want to let the user break the rules of precedence ?


Meh, the user can do anything he wants already. One can get around 
precedence by deferring calls.

It is true, though, that I didn't think of precedence when I first 
posted -- I just wanted to avoid some allocations.

-Steve
Jul 03 2020
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 3 July 2020 at 17:13:32 UTC, Steven Schveighoffer 
wrote:

 [snip]

 Meh, the user can do anything he wants already. One can get 
 around precedence by deferring calls.

 It is true, though, that I didn't think of precedence when I 
 first posted -- I just wanted to avoid some allocations.

 -Steve


I see no reason why Aliak's suggestion above with respect to 
parentheses cannot be also used to ensure precedence is followed. 
For instance, if you have a + b * c, then it gets re-written to a 
+ (b * c), which gets re-written to something like 
a.op1(b.op2(c)).
Jul 03 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/20 1:54 PM, jmh530 wrote:

 On Friday, 3 July 2020 at 17:13:32 UTC, Steven Schveighoffer wrote:

 [snip]

 Meh, the user can do anything he wants already. One can get around 
 precedence by deferring calls.

 It is true, though, that I didn't think of precedence when I first 
 posted -- I just wanted to avoid some allocations.


 
 I see no reason why Aliak's suggestion above with respect to parentheses 
 cannot be also used to ensure precedence is followed. For instance, if 
 you have a + b * c, then it gets re-written to a + (b * c), which gets 
 re-written to something like a.op1(b.op2(c)).


That's what's already done today.

The idea here is to pass all the operations and parameters at once. 
There are actually advantages in some cases to have the entire 
expression, as some things could be optimized by reordering operations 
(in a valid way).

The further question of this subthread is -- can we loosen some of the 
precedence rules so this provides more capabilities, or would it be 
crippled by enforcing precedence rules?

-Steve
Jul 03 2020
parent reply Ben Jones <fake fake.fake> writes:
On Friday, 3 July 2020 at 18:07:08 UTC, Steven Schveighoffer 
wrote:

 That's what's already done today.

 The idea here is to pass all the operations and parameters at 
 once. There are actually advantages in some cases to have the 
 entire expression, as some things could be optimized by 
 reordering operations (in a valid way).

 The further question of this subthread is -- can we loosen some 
 of the precedence rules so this provides more capabilities, or 
 would it be crippled by enforcing precedence rules?

 -Steve


I think grouping all operations with the same precedence using 
your new nAryOp idea is a good compromise.  If you need to mess 
with precedence rules, then you can write expression templates to 
do so.

I imagine the most common case is string concatenation with a 
bunch of strings which that approach would handle.

Example:

a + b - c*d/e would become

a.opNaray!(["+", "-"])(a, b, c.opNary!(["*", "/"])(c, d, e))
Jul 03 2020
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 03.07.20 20:39, Ben Jones wrote:

 
 I think grouping all operations with the same precedence using your new 
 nAryOp idea is a good compromise.  If you need to mess with precedence 
 rules, then you can write expression templates to do so.


Walter's stance on expression templates is that they are operator 
overloading abuse.
Jul 03 2020
prev sibling parent user1234 <user1234 12.de> writes:
On Friday, 3 July 2020 at 17:13:32 UTC, Steven Schveighoffer 
wrote:

 On 7/3/20 11:25 AM, user1234 wrote:

 On Thursday, 2 July 2020 at 19:28:15 UTC, Nick Treleaven wrote:

 [...]


 
 Wow, that looked cool the first 3 secons but now I wander if  
 you really want to let the user break the rules of precedence ?


 Meh, the user can do anything he wants already. One can get 
 around precedence by deferring calls.

 It is true, though, that I didn't think of precedence when I 
 first posted -- I just wanted to avoid some allocations.

 -Steve


Your suggestion has no issue with precedence, it's only the 
version proposed by Nick, as it allows to mix several operators.
Jul 03 2020
prev sibling next sibling parent user1234 <user1234 12.de> writes:
On Thursday, 2 July 2020 at 19:28:15 UTC, Nick Treleaven wrote:

 In fact, maybe some types could benefit from intercepting 
 different operations at once:

 opNary(string[] ops, T...)(T args)


What I'll say is off topic but what I wanted once is

   opDispatch(string[] members, T...)(T args)

To solve a whole identifier chain ine one shot.
Jul 03 2020
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 7/2/20 12:28 PM, Nick Treleaven wrote:


 Sounds good, although I think multiple other operations beside
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)


Given the confusions raised elsewhere in this thread, I think that 
syntax gives the biggest benefit; defined for a single operator. And we 
don't even need a new keyword:

   opBinary(string op, T...)(T args)

which would solve Steve's original issue. Of course, doing the expected 
thing and the associativity issue would still be left to the 
implementer. For example, in order to stay consistent with the rest of 
the type system, the programmer should handle b and c before a below 
because ^^ is right-associative:

   a ^^ b ^^ c

Well, we have to trust the programmer anyway.

So, I propose an extension to opBinary to optionally take multiple 
arguments.

Ali
Jul 03 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/20 3:06 PM, Ali Çehreli wrote:

 On 7/2/20 12:28 PM, Nick Treleaven wrote:
 
  > Sounds good, although I think multiple other operations beside
  > concatenation could benefit from being intercepted:
  >
  > opNary(string op, T...)(T args)
 
 Given the confusions raised elsewhere in this thread, I think that 
 syntax gives the biggest benefit; defined for a single operator. And we 
 don't even need a new keyword:
 
    opBinary(string op, T...)(T args)


Just nitpicking, but opBinary is not a keyword, so neither would opNary be.


 
 which would solve Steve's original issue. Of course, doing the expected 
 thing and the associativity issue would still be left to the 
 implementer. For example, in order to stay consistent with the rest of 
 the type system, the programmer should handle b and c before a below 
 because ^^ is right-associative:
 
    a ^^ b ^^ c
 
 Well, we have to trust the programmer anyway.


Any non-commutative operations rely on order of operations. Division, 
subtraction, etc.


 So, I propose an extension to opBinary to optionally take multiple 
 arguments.


It would be kind of weird/unfortunate for this to support a + b + c, but 
not a + b - c.

We could still do it all with opBinary, as:

opBinary(string[] ops, Args...)(Args args)

And either enforce precedence by reordering the operations (like Petar 
suggests) into RPN, or give up when precedence isn't strictly 
left-to-right. One can always break up the operations into 
subexpressions and sub-calls to opBinary/opNary.

I think there are sufficient cases which could benefit from this to 
warrant a consideration for making a DIP. What I'm really more concerned 
about is pushback on "operator abuse", which clearly this could allow. I 
wouldn't go forward without some feedback from Walter.

-Steve
Jul 03 2020
parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 3 July 2020 at 20:27:10 UTC, Steven Schveighoffer 
wrote:

 It would be kind of weird/unfortunate for this to support a + b 
 + c, but not a + b - c.

 We could still do it all with opBinary, as:

 opBinary(string[] ops, Args...)(Args args)

 And either enforce precedence by reordering the operations 
 (like Petar suggests) into RPN, or give up when precedence 
 isn't strictly left-to-right. One can always break up the 
 operations into subexpressions and sub-calls to opBinary/opNary.


What if the compiler calls opBinaryTemp when it detects an 
expression that would be used in another opBinary call? Given:

S s1, s2, s3;

`s1 ~ s2 ~ s3` would be lowered to:
`s1.opBinaryTemp!"~"(s2).opBinary!"~"(s3)`

Concatenation is commutative, but for other operations brackets 
would be handled naturally by the compiler:

`s1 * (s2 + s3)` lowers to:
`s1.opBinary!"*"(s2.opBinaryTemp!"+"(s3))`

The following code would support either 1 concatenation, or 2 
concatenations being done at once:

struct S {
     int[] data;

     struct Expr {
         S left, right;

         // left ~ right ~ rhs
         auto opBinary(string op : "~")(S rhs) {
             size_t lhsLen = left.data.len + right.data.len;
             auto r = new int[lhsLen + rhs.data.len];

             r[0..left.data.len] = left.data;
             r[left.data.len..$][0..right.data.len] = right.data;
             r[lhsLen..$][0..rhs.data.len] = rhs.data;
             return r;
         }
     }
     auto opBinaryTemp(string op : "~")(S rhs) {
         return Expr(this, right);
     }
     // this ~ rhs
     auto opBinary(string op : "~")(S rhs) {
         return S(data ~ rhs.data);
     }
}

Maybe this pattern could be extended to work with 
arbitrary-length sequences of concatenation. Perhaps it could 
also be extended to work with differing `op` kinds.

Although Expr is essentially an expression template, its use is 
controlled and the above code doesn't need `alias this`.
Jul 04 2020
parent Nick Treleaven <nick geany.org> writes:
On Saturday, 4 July 2020 at 12:37:23 UTC, Nick Treleaven wrote:

 What if the compiler calls opBinaryTemp when it detects an 
 expression that would be used in another opBinary call? Given:

 S s1, s2, s3;

 `s1 ~ s2 ~ s3` would be lowered to:
 `s1.opBinaryTemp!"~"(s2).opBinary!"~"(s3)`


The lowering actually compiles now:
https://github.com/ntrel/stuff/tree/master/optemp


 Maybe this pattern could be extended to work with 
 arbitrary-length sequences of concatenation.


 From the above link:

struct S {
     int[] data;

     static struct Expr(T...)
     {
         T items;

         static foreach (E; T)
             static assert(is(E == S));

         // concat(items) ~ rhs
         S opBinary(string op : "~")(S rhs) {
             auto len = rhs.data.length;
             foreach (s; items)
                 len += s.data.length;
             auto r = new int[len];

             size_t i;
             foreach (s; items) {
                 r[i..$][0..s.data.length] = s.data;
                 i += s.data.length;
             }
             r[i..$] = rhs.data;
             return S(r);
         }
         auto opBinaryTemp(string op : "~")(S rhs) {
             return expr(items, rhs);
         }
     }
     private static expr(T...)(T args) {
         return Expr!T(args);
     }
     auto opBinaryTemp(string op : "~")(S rhs) {
         return expr(this, rhs);
     }
     // this ~ rhs
     S opBinary(string op : "~")(S rhs) {
         return expr(this) ~ rhs;
     }
}

void main()
{
     import std.stdio;

     auto s1 = S([1,10]);
     auto s2 = S([2,22]);
     auto s3 = S([3,-3]);
     //s1 ~ s2
     s1.opBinary!"~"(s2).data.writeln;
     //s1 ~ s2 ~ s3
     s1.opBinaryTemp!"~"(s2).opBinary!"~"(s3).data.writeln;
     //s1 ~ s2 ~ s3 ~ s1
     
s1.opBinaryTemp!"~"(s2).opBinaryTemp!"~"(s3).opBinary!"~"(s1).data.writeln;
}
Jul 05 2020
prev sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Friday, 3 July 2020 at 19:06:21 UTC, Ali Çehreli wrote:

 On 7/2/20 12:28 PM, Nick Treleaven wrote:


 Sounds good, although I think multiple other operations beside
 concatenation could benefit from being intercepted:

 opNary(string op, T...)(T args)


 Given the confusions raised elsewhere in this thread, I think 
 that syntax gives the biggest benefit; defined for a single 
 operator. And we don't even need a new keyword:

   opBinary(string op, T...)(T args)


That already has a defined behavior:

alias AliasSeq(T...) = T;

struct S {
     AliasSeq!(int, int, int) fields;
     void opBinary(string op, T...)(T args) {
         pragma(msg, T); // (int, int, int)
     }
}

unittest {
     S s;
     s + s.fields;
}

--
   Simen
Jul 03 2020