## digitalmars.D - A possible solution for the opIndexXxxAssign morass

• Andrei Alexandrescu (20/20) Oct 13 2009 Right now we're in trouble with operators: opIndex and opIndexAssign
• Don (7/44) Oct 13 2009 Well timed. I just wrote this operator overloading proposal, part 1.
• Robert Jacques (20/47) Oct 13 2009 I've mentioned this problem before, in relation to multi-dimensional
• Bill Baxter (17/23) Oct 13 2009 I think it's a good start.
• Lars T. Kyllingstad (14/61) Oct 14 2009 I like the idea of enforcing relationships between operators. In fact, I...
• Don (14/78) Oct 14 2009 Unfortunately, the last one doesn't work for reals. a*b could overflow
• Lars T. Kyllingstad (7/88) Oct 14 2009 But the case x*2 --> x+x would also likely be the most common in terms
• Bill Baxter (16/77) Oct 14 2009 th
• Andrei Alexandrescu (9/12) Oct 14 2009 I'm not sure multiplication is generally commutative (e.g. in linear
• Robert Jacques (5/18) Oct 14 2009 When a is a scaler, a * x <=> x * a generally holds. It's only when
• Denis Koroskin (3/27) Oct 14 2009 It's commutativity (http://en.wikipedia.org/wiki/Commutativity)
• Don (24/38) Oct 14 2009 It only applies a is an int or real. Its purpose is to allow
• Steven Schveighoffer (18/38) Oct 15 2009 Oh, I didn't realize that's what you meant. I thought that opXxxAssign ...
• Steven Schveighoffer (12/31) Oct 13 2009 I'm guessing you meant opAssign here, or meant to write +=?
• Andrei Alexandrescu (8/34) Oct 13 2009 Great. Indeed the proposed solution leaves a[b][c] += d problematic, and...
• Robert Jacques (5/33) Oct 13 2009 Well, that last case I'd prefer handled by something more generic, like ...
• Bill Baxter (8/46) Oct 13 2009 Huh? It didn't sound to me like it would get rid of anything, except
• Andrei Alexandrescu (7/14) Oct 13 2009 That's a good point. But something is inherently problematic about name
• Bill Baxter (12/26) Oct 13 2009 unt
• Robert Jacques (5/18) Oct 13 2009 A distinct type for a..b is needed to support the mixed slicing and inde...
• Michel Fortin (19/51) Oct 13 2009 I'd rewrite it as opIndexAddAssign(b, c); That way you can also rewrite:
• JC (5/42) Oct 13 2009 This idea along with a slice overload would save me a lot of pain and
• Bill Baxter (4/8) Oct 13 2009 It won't be implemented in D1. Stability -- it's the beauty and the
• bearophile (6/8) Oct 14 2009 Such slice also needs a way to specify the end of the enclosing interval...
• Kagamin (2/5) Oct 14 2009 Dollar is just a synonym for length, isn't it?
• Lars T. Kyllingstad (4/11) Oct 14 2009 Yes, but if opIndex and opSlice take multiple indices (like in a matrix)...
• Kagamin (2/4) Oct 14 2009 size_t length(size_t idx);
• Andrei Alexandrescu (5/19) Oct 14 2009 I think the compiler should rewrite \$ to __currentarray.length in unary
• Robert Jacques (3/8) Oct 14 2009 User types can also override Dollar, though I don't remember off the top...
• Jason House (6/12) Oct 14 2009 I would hope that *= += /= and friends could all be handled efficiently ...
• Bill Baxter (63/75) Oct 14 2009 ntly with one function written by the programmer. As I see it, there are...
• Jason House (2/28) Oct 14 2009 The only issue with templates is that they're never virtual
• Andrei Alexandrescu (6/33) Oct 14 2009 You can make virtuals out of templates, but not templates out of
• Robert Jacques (5/43) Oct 14 2009 I've done something similar for a SmallVec struct. Most of the operator ...
• Fawzi Mohamed (15/62) Oct 15 2009 I would really like a solution to all the overloading ops, as I missed
• Robert Jacques (2/63) Oct 15 2009 Would you like some example code?
• Fawzi Mohamed (57/74) Oct 15 2009 I suppose you would like it ;)
• Fawzi Mohamed (3/86) Oct 15 2009 mmmh I mixed up a bit the ref returning and pointer returning case...
• Bill Baxter (80/151) Oct 14 2009 ently with one function written by the programmer. As I see it, there ar...
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei
```
Oct 13 2009
Don <nospam nospam.com> writes:
```Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to
do this is to make a slice into a type of index.
```
Oct 13 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Tue, 13 Oct 2009 11:56:36 -0400, Don <nospam nospam.com> wrote:

Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like
a[b] += c;
with reasonable expressiveness and efficiency.
Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:
a += b
gets rewritten as
a[b] += c
as
There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:
a[b1, b2, ..., bn] = c
gets rewritten as
What do you think? I may be missing some important cases or threats.
Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to
do this is to make a slice into a type of index.

I've mentioned this problem before, in relation to multi-dimensional
arrays:

// Slice a row out of an Matrix
row0 = myMatrix[0,0..\$];

So basically, opIndex and opSlice need to merge to support this use case.
I've always ended up doing this with using size_t[2] or size_t[3] (for
slicing with strides) when I've coded Nd-arrays, though this is a bit
clunky. However, a while ago someone mentioned that tuple, though
cool/useful/etc wasn't being used as much (compared to other languages)
because of a lack of syntactic sugar. Which gave me the idea of using the
.. operator to be syntactic sugar for tuple, as it would solve two birds
with one stone. (Maybe three, if you count MVR)

Also needed is an extension of the opDollar to return different values
based on the index:
opDollar(size_t index);

P.S. There's also at least one template bug blocking Nd-arrays and small
vector types: (http://d.puremagic.com/issues/show_bug.cgi?id=2257).
P.S.S. Another template issue is that templating both opX and opX_r
generally results in an overload conflict.
```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 8:56 AM, Don <nospam nospam.com> wrote:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to do
this is to make a slice into a type of index.

I think it's a good start.

In the list of properties, you should probably mention that 'a' is a scalar.

But I wonder how the rules involving scalars could be enforced, given
that it's possible to define new scalar types.  One would have to tell
the compiler somehow which types are scalars relative to the type
being defined.

The ++ operators don't make sense for many of the types you listed.
Maybe that should be broken out.  Actually three of the things really
stand out as computer-isms that don't really belong with the other
mathematical properties:
x =y <==> x = x   y
x++ <==> ++x
x =y returns x

Also, you can add Clifford algebra, Grassmann algebra, and geometric
algebra to the list of things where the mathematical properties hold.

--bb
```
Oct 13 2009
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
```Don wrote:
Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to
do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I
think we can take it even further, and require that operator overloading
undefined behaviour. For example, if n is an integer, a and b are
scalars, and x and y are general types, the compiler should be free to
rewrite

n*x  <-->  x + x + ... + x    <-->  2*x + 2*x + ...
x^^n  <-->  x * x * ... * x    <-->  x^^2 * x^^2 * ...
x/a + y/b  <-->  (b*x + a*y)/(a*b)

and so on, based on what it finds to be the most efficient operations.
(Note how I snuck my favourite suggestion for an exponentiation operator
in there. I *really* want that.)

-Lars
```
Oct 14 2009
Don <nospam nospam.com> writes:
```Lars T. Kyllingstad wrote:
Don wrote:
Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to
do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I
think we can take it even further, and require that operator overloading
undefined behaviour. For example, if n is an integer, a and b are
scalars, and x and y are general types, the compiler should be free to
rewrite

n*x  <-->  x + x + ... + x    <-->  2*x + 2*x + ...
x^^n  <-->  x * x * ... * x    <-->  x^^2 * x^^2 * ...
x/a + y/b  <-->  (b*x + a*y)/(a*b)

and so on, based on what it finds to be the most efficient operations.

Unfortunately, the last one doesn't work for reals. a*b could overflow
or underflow.
x/ real.max + y / real.max   is exactly 2.0 if x and y are both real.max
But
(real.max * x + real.max *y)/(real.max * real.max) is infinity/infinity
= NaN.

The others don't always work in general, either. I'm worried about
decimal floats. Say n==10, then it's an exact operation; but addition
isn't exact. It always works for n==2, since there's at most one
roundoff in both cases.

But I do feel that with floating-point, we've lost so many identities,
that we must preserve every one which we have left.

(Note how I snuck my favourite suggestion for an exponentiation operator
in there. I *really* want that.)

I want it too. Heck, I might even make a patch for it <g>.
```
Oct 14 2009
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
```Don wrote:
Don wrote:
Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way
to do this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact,
I think we can take it even further, and require that operator
else leads to undefined behaviour. For example, if n is an integer, a
and b are scalars, and x and y are general types, the compiler should
be free to rewrite

n*x  <-->  x + x + ... + x    <-->  2*x + 2*x + ...
x^^n  <-->  x * x * ... * x    <-->  x^^2 * x^^2 * ...
x/a + y/b  <-->  (b*x + a*y)/(a*b)

and so on, based on what it finds to be the most efficient operations.

Unfortunately, the last one doesn't work for reals. a*b could overflow
or underflow.
x/ real.max + y / real.max   is exactly 2.0 if x and y are both real.max
But
(real.max * x + real.max *y)/(real.max * real.max) is infinity/infinity
= NaN.

Good point. I am thinking like a mathematician, not a programmer. :)

The others don't always work in general, either. I'm worried about
decimal floats. Say n==10, then it's an exact operation; but addition
isn't exact. It always works for n==2, since there's at most one
roundoff in both cases.

But the case x*2 --> x+x would also likely be the most common in terms
of optimisation, right?

But I do feel that with floating-point, we've lost so many identities,
that we must preserve every one which we have left.

(Note how I snuck my favourite suggestion for an exponentiation
operator in there. I *really* want that.)

I want it too. Heck, I might even make a patch for it <g>.

If you do, make sure to announce it loudly and clearly on the NG. Don't
want to miss it. ;)

-Lars
```
Oct 14 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Wed, Oct 14, 2009 at 12:48 AM, Lars T. Kyllingstad
<public kyllingen.nospamnet> wrote:
Don wrote:
Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] +=3D c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading wi=

th
the existing operator names. Consider:

a +=3D b

gets rewritten as

a[b] +=3D c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] =3D c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

The indexing thing was something I didn't have a solution for.

BTW we need to deal with slices as well as indexes. I think the way to d=

o
this is to make a slice into a type of index.

I like the idea of enforcing relationships between operators. In fact, I
think we can take it even further, and require that operator overloading =

in
undefined behaviour. For example, if n is an integer, a and b are scalars=

,
and x and y are general types, the compiler should be free to rewrite

=A0 =A0 =A0 =A0 n*x =A0<--> =A0x + x + ... + x =A0 =A0<--> =A02*x + 2*x +=

...
=A0 =A0 =A0 =A0x^^n =A0<--> =A0x * x * ... * x =A0 =A0<--> =A0x^^2 * x^^2=

* ...
=A0 x/a + y/b =A0<--> =A0(b*x + a*y)/(a*b)

and so on, based on what it finds to be the most efficient operations. (N=

ote
how I snuck my favourite suggestion for an exponentiation operator in the=

re.
I *really* want that.)

You have to be careful when you go rewriting mathematical expressions
on the computer, though.  The numerical error for two mathematically
identical expressions can be quite different when evaluated in finite
precision arithmetic.

I'd love an exponentiation operator, too.

--bb
```
Oct 14 2009
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Don wrote:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear
algebra it isn't). So why should a * x be interchangeable with x * a?

Also, the much-discussed identity:

x  = y	<-->	x = x   y

is difficult to enforce statically in practice. I think some types would
want to define both to achieve good efficiency. It would be hard for the
compiler to render one unnecessary or to prove that the two are equivalent.

Andrei
```
Oct 14 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Wed, 14 Oct 2009 10:31:06 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Don wrote:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear
algebra it isn't). So why should a * x be interchangeable with x * a?

Also, the much-discussed identity:

x  = y	<-->	x = x   y

is difficult to enforce statically in practice. I think some types would
want to define both to achieve good efficiency. It would be hard for the
compiler to render one unnecessary or to prove that the two are
equivalent.

Andrei

When a is a scaler, a * x <=> x * a generally holds. It's only when
something isn't a scaler, i.e. x1 * x2 != x2 * x1, that community(?)
doesn't hold.
```
Oct 14 2009
"Denis Koroskin" <2korden gmail.com> writes:
```On Wed, 14 Oct 2009 18:39:27 +0400, Robert Jacques <sandford jhu.edu>
wrote:

On Wed, 14 Oct 2009 10:31:06 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Don wrote:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear
algebra it isn't). So why should a * x be interchangeable with x * a?

Also, the much-discussed identity:

x  = y	<-->	x = x   y

is difficult to enforce statically in practice. I think some types
would want to define both to achieve good efficiency. It would be hard
for the compiler to render one unnecessary or to prove that the two are
equivalent.

Andrei

When a is a scaler, a * x <=> x * a generally holds. It's only when
something isn't a scaler, i.e. x1 * x2 != x2 * x1, that community(?)
doesn't hold.

It's commutativity (http://en.wikipedia.org/wiki/Commutativity)
```
Oct 14 2009
Don <nospam nospam.com> writes:
```Andrei Alexandrescu wrote:
Don wrote:
http://www.prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP7
I concentrated on getting the use cases established.

I'm not sure multiplication is generally commutative (e.g. in linear
algebra it isn't). So why should a * x be interchangeable with x * a?

It only applies a is an int or real. Its purpose is to allow
constant-folding in the compiler front-end (specifically, when a is a
manifest constant).

Also, the much-discussed identity:

x  = y    <-->    x = x   y

is difficult to enforce statically in practice. I think some types would
want to define both to achieve good efficiency. It would be hard for the
compiler to render one unnecessary or to prove that the two are equivalent.

Yes, it could not be enforced. But note that there would be no ambiguity
as to which should be used in any given expression.
I would propose that the opXXXAssign() variants should exist *only* for
performance optimisation, and be completely divorced from the "+="
syntax (effectively, += would be discarded after the parsing step).
My ancient Bugzilla proposal actually included opSubAssign() and
opSubAssign_r() for  x = x - y; and x = y - x;
If   the x  = y    <-->    x = x   y transformations became legal, this
would allow unnecessary temporaries to be completely eliminated.

The suggested transformation would be that x = x + y would be
transformed into x.opAddAssign(y) whenever it exists, and x = y + x
The transformations would therefore be entirely predictable.

It would make Numpy-style arithmetic impossible (where z=x; x+=y;
modifies z, but z = x; x = x+y; does not modify z (under this proposal,
the second would be transformed into the first)).

Tightly defined semantics improve performance and reduce the potential
for abuse. But, there are existing libraries/techniques which depend on
able to sacrifice them?
```
Oct 14 2009
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
```On Thu, 15 Oct 2009 02:58:51 -0400, Don <nospam nospam.com> wrote:

Andrei Alexandrescu wrote:
Also, the much-discussed identity:
x  = y    <-->    x = x   y
is difficult to enforce statically in practice. I think some types
would want to define both to achieve good efficiency. It would be hard
for the compiler to render one unnecessary or to prove that the two are
equivalent.

Yes, it could not be enforced. But note that there would be no ambiguity
as to which should be used in any given expression.
I would propose that the opXXXAssign() variants should exist *only* for
performance optimisation, and be completely divorced from the "+="
syntax (effectively, += would be discarded after the parsing step).
My ancient Bugzilla proposal actually included opSubAssign() and
opSubAssign_r() for  x = x - y; and x = y - x;
If   the x  = y    <-->    x = x   y transformations became legal, this
would allow unnecessary temporaries to be completely eliminated.

The suggested transformation would be that x = x + y would be
transformed into x.opAddAssign(y) whenever it exists, and x = y + x
The transformations would therefore be entirely predictable.

Oh, I didn't realize that's what you meant.  I thought that opXxxAssign
was to be eliminated and x += y was to be transformed into
x.opAssign(x.opXxx(y).  I like this proposal better -- opXxxAssign can
exist for optimization reasons, and enforcing the relationship between  =
and =   by parsing one into the other.

By parsing x += y into x = x + y, and allowing overloading of a chain of
operations, you may even get more mileage out of something like x += y + z
+ w;

Someone earlier suggested opXxx(a1, a2, ...) could be interpreted as an
operator for dealing with chained operations.  You could also maybe have
an opChain or something that takes as arguments the operands and the
operators to maybe perform some optimization (i.e. like reordering matrix
operations).

You should update your DIP to specify that opXxxAssign should be allowed
for optimization purposes (BTW, classes could benefit from this, because
then x += y *would* be the same as x = x + y).

-Steve
```
Oct 15 2009
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
```On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

I'm guessing you meant opAssign here, or meant to write +=?

What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex
itself.

The question then becomes, what if you wanted to overload this?

a[b][c] += d;

You can do a[b] returns a ref.  But then you now allow a[b] op x, thereby
possibly exposing a private piece of info.  This may or may not be
important.

I like the way your idea is going.

-Steve
```
Oct 13 2009
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Steven Schveighoffer wrote:
On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

I'm guessing you meant opAssign here, or meant to write +=?

Oh, sorry. I meant to write +=.

What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex
itself.

The question then becomes, what if you wanted to overload this?

a[b][c] += d;

You can do a[b] returns a ref.  But then you now allow a[b] op x,
thereby possibly exposing a private piece of info.  This may or may not
be important.

I like the way your idea is going.

Great. Indeed the proposed solution leaves a[b][c] += d problematic, and
also prevents this potential development:

a += b + c + d;

to be rewritten as:

Andrei
```
Oct 13 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Tue, 13 Oct 2009 12:21:20 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Steven Schveighoffer wrote:
On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

I'm guessing you meant opAssign here, or meant to write +=?

Oh, sorry. I meant to write +=.

What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex
itself.
The question then becomes, what if you wanted to overload this?
a[b][c] += d;
You can do a[b] returns a ref.  But then you now allow a[b] op x,
thereby possibly exposing a private piece of info.  This may or may not
be important.
I like the way your idea is going.

Great. Indeed the proposed solution leaves a[b][c] += d problematic, and
also prevents this potential development:

a += b + c + d;

to be rewritten as:

Andrei

Well, that last case I'd prefer handled by something more generic, like an
opExpression(Expr, T...)(T params); (i.e. a way of doing expression
```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 9:08 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
On Tue, 13 Oct 2009 11:16:01 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign don't
seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading with
the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are different.
Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

I'm guessing you meant opAssign here, or meant to write +=?

What do you think? I may be missing some important cases or threats.

It's simple, and gets rid of all opIndex operators except for opIndex
itself.

Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations.  That just seems confusing to me.   I think the

--bb
```
Oct 13 2009
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Bill Baxter wrote:
Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations.  That just seems confusing to me.   I think the

--bb

That's a good point. But something is inherently problematic about name
explosion (In the proposed solution there is still an explosion in the
count of functions that need to be written.)

Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we
ascribe a distinct type to a .. b.

Andrei
```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 10:08 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
Bill Baxter wrote:
Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations. =A0That just seems confusing to me. =A0 I think the

--bb

That's a good point. But something is inherently problematic about name
explosion (In the proposed solution there is still an explosion in the co=

unt
of functions that need to be written.)

Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we
ascribe a distinct type to a .. b.

Yeh, the name explosion is just a symptom of the real problem, which
is function count explosion.  That's what needs fixing, if anything.
But I don't really think having a lot of functions is the issue, it's
implementers having to *write* a lot of boring repetitive functions
that is the problem.  So if the drudgery can be automated somehow (in
the cases where the pattern is regular), then that would solve the
problem in my mind.  Even if it was still a function explosion under
the hood.

--bb
```
Oct 13 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Tue, 13 Oct 2009 13:08:59 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Bill Baxter wrote:
Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations.  That just seems confusing to me.   I think the
--bb

That's a good point. But something is inherently problematic about name
explosion (In the proposed solution there is still an explosion in the
count of functions that need to be written.)

Now I realize there's also a need for opSliceXxxAssign, bleh. Unless we
ascribe a distinct type to a .. b.

Andrei

A distinct type for a..b is needed to support the mixed slicing and index
that occurs in Nd-array/Matrixes: i.e.
auto row0 = myMatrix[0,0..\$];
```
Oct 13 2009
"Denis Koroskin" <2korden gmail.com> writes:
```On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

a[b1..b2] = c;

?

I could be solved if b1..b2 would return some built-in range type, defined
in object.d, though.
```
Oct 13 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Tue, 13 Oct 2009 12:28:05 -0400, Denis Koroskin <2korden gmail.com>
wrote:

On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

a[b1..b2] = c;

?

I could be solved if b1..b2 would return some built-in range type,
defined in object.d, though.

int opSliceAssign(int v, size_t x, size_t y);
a[3..4] = v;	// same as a.opSliceAssign(v,3,4);
```
Oct 13 2009
"Denis Koroskin" <2korden gmail.com> writes:
```On Tue, 13 Oct 2009 20:34:06 +0400, Robert Jacques <sandford jhu.edu>
wrote:

On Tue, 13 Oct 2009 12:28:05 -0400, Denis Koroskin <2korden gmail.com>
wrote:

On Tue, 13 Oct 2009 19:16:01 +0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

a[b1..b2] = c;

?

I could be solved if b1..b2 would return some built-in range type,
defined in object.d, though.

int opSliceAssign(int v, size_t x, size_t y);
a[3..4] = v;	// same as a.opSliceAssign(v,3,4);

I meant:

a[b1..b2] += c;

translates into "opAddAssign" and doesn't mention "index" while "a[b] =
c;" does ("opIndexAssign").
```
Oct 13 2009
"Steven Schveighoffer" <schveiguy yahoo.com> writes:
```On Tue, 13 Oct 2009 12:44:21 -0400, Denis Koroskin <2korden gmail.com>
wrote:

translates into "opAddAssign" and doesn't mention "index" while "a[b] =
c;" does ("opIndexAssign").

I think the optimization translates to opAssign as well:

a[b] = c; => a.opAssign(b, c);

On Tue, 13 Oct 2009 12:37:50 -0400, Bill Baxter <wbaxter gmail.com> wrote:

Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations.  That just seems confusing to me.   I think the

The point is to avoid having operator function names multiply out of
control.  Re-examining it, I agree with you -- It makes little sense to
have an operator that involves an indexing lack the term Index.

If only there was a way to make the indexing orthogonal to the other
operation.  For example something like:

struct X
{
private int[] arr;
opIndex(int idx) // declares a new "namespace" where idx is an
implicitly passed argument
{
int opAssign(int x)
{
arr[idx] = x;
return x;
}
}
}

I know this probably doesn't parse well, should opIndex be a keyword? or
an attribute?

-Steve
```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 10:00 AM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
On Tue, 13 Oct 2009 12:44:21 -0400, Denis Koroskin <2korden gmail.com>
wrote:

translates into "opAddAssign" and doesn't mention "index" while "a[b] =

=3D c;"
does ("opIndexAssign").

I think the optimization translates to opAssign as well:

a[b] =3D c; =3D> a.opAssign(b, c);

On Tue, 13 Oct 2009 12:37:50 -0400, Bill Baxter <wbaxter gmail.com> wrote=

:
Huh? It didn't sound to me like it would get rid of anything, except
for the use of the word "index" in many methods that have to do with
index operations. =A0That just seems confusing to me. =A0 I think the

The point is to avoid having operator function names multiply out of
control. =A0Re-examining it, I agree with you -- It makes little sense to=

have
an operator that involves an indexing lack the term Index.

If only there was a way to make the indexing orthogonal to the other
operation. =A0For example something like:

struct X
{
=A0 private int[] arr;
=A0 opIndex(int idx) // declares a new "namespace" where idx is an implic=

itly
passed argument
=A0 {
=A0 =A0 =A0int opAssign(int x)
=A0 =A0 =A0{
=A0 =A0 =A0 =A0 arr[idx] =3D x;
=A0 =A0 =A0 =A0 return x;
=A0 =A0 =A0}
=A0 }
}

I know this probably doesn't parse well, should opIndex be a keyword? or =

an
attribute?

I don't think the number of /names/ required is the problem.  It's
just the sheer number of functions themselves that's the issue.  I
think a lot of that could mostly be fixed by some smart macros.   And
until they exist, mixins can help.

struct Vec
{
float x,y,z;
mixin(implementOperators("+ - / * +=3D -=3D /=3D *=3D",
q{
a.x op=3D b.x;
a.y op=3D b.y;
a.z op=3D b.z;
});
}

The code gives a list of operators to implement and one prototypical
op=3D body.  With a smart enough CTFE string function that's all you
need to generate all the listed operators.  Not sure how to work index
operators into that.

--bb
```
Oct 13 2009
Michel Fortin <michel.fortin michelf.com> writes:
```On 2009-10-13 11:16:01 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> said:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

I'd rewrite it as opIndexAddAssign(b, c); That way you can also rewrite:

a[b..c] = d;

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

That looks like a good idea, although I'd be a little tempted to put
the variable-length part at the end so you can easily choose to use

Also noteworthy: none of this work if you want to mix index and slices:

a[b, c..d] = f;

What do you think? I may be missing some important cases or threats.

have to redefine it for every primitive operator? I seem to recall you
arguing for a way to overload all the operators at the same time.
Where's that going?

--
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
```
Oct 13 2009
JC <jcrapuchettes gmail.com> writes:
```This idea along with a slice overload would save me a lot of pain and
performance while working with matrices in my production code. Any ideas when
this could be implemented in D1?
Jonathan

Andrei Alexandrescu wrote:
Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

Last night this idea occurred to me: we could simply use overloading
with the existing operator names. Consider:

a += b

gets rewritten as

a[b] += c

as

There's no chance of ambiguity because the parameter counts are
different. Moreover, this scales to multiple indexes:

a[b1, b2, ..., bn] = c

gets rewritten as

What do you think? I may be missing some important cases or threats.

Andrei

```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 10:22 AM, JC <jcrapuchettes gmail.com> wrote:
This idea along with a slice overload would save me a lot of pain and
performance while working with matrices in my production code. Any ideas
when this could be implemented in D1?
Jonathan

It won't be implemented in D1.  Stability -- it's the beauty and the
beast of D1.

--bb
```
Oct 13 2009
```Forgotten already?

This is the same problem as property lvalue-ness and it has the same
solution.  When property rewriting is done correctly, the opIndexAssign

Just treat opIndex expressions as properties, and when they are the
subject of a side-effect then make sure the write property (AKA
opIndexAssign) gets called.
```
Oct 13 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Tue, Oct 13, 2009 at 10:39 AM, Chad J

Apparently, yes!

This is the same problem as property lvalue-ness and it has the same
solution. =A0When property rewriting is done correctly, the opIndexAssign

Just treat opIndex expressions as properties, and when they are the
subject of a side-effect then make sure the write property (AKA
opIndexAssign) gets called.

Good call.

--bb
```
Oct 13 2009
bearophile <bearophileHUGS lycos.com> writes:
```Don:

BTW we need to deal with slices as well as indexes. I think the way to
do this is to make a slice into a type of index.

Such slice also needs a way to specify the end of the enclosing interval, the \$
syntax.
Slice may enjoy a lot a third optional argument (default = 1), that represents
the stride.

(In the Chapel language all this is generalized into the concept of Domain,
that I think is a good language feature to introduce in D).

Bye,
bearophile
```
Oct 14 2009
Kagamin <spam here.lot> writes:
```Robert Jacques Wrote:

Also needed is an extension of the opDollar to return different values
based on the index:
opDollar(size_t index);

Dollar is just a synonym for length, isn't it?
```
Oct 14 2009
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
```Kagamin wrote:
Robert Jacques Wrote:

Also needed is an extension of the opDollar to return different values
based on the index:
opDollar(size_t index);

Dollar is just a synonym for length, isn't it?

Yes, but if opIndex and opSlice take multiple indices (like in a matrix)
opDollar needs a way to distinguish between the different dimensions.

-Lars
```
Oct 14 2009
Kagamin <spam here.lot> writes:
```Lars T. Kyllingstad Wrote:

Yes, but if opIndex and opSlice take multiple indices (like in a matrix)
opDollar needs a way to distinguish between the different dimensions.

size_t length(size_t idx);
```
Oct 14 2009
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Lars T. Kyllingstad wrote:
Kagamin wrote:
Robert Jacques Wrote:

Also needed is an extension of the opDollar to return different
values  based on the index:
opDollar(size_t index);

Dollar is just a synonym for length, isn't it?

Yes, but if opIndex and opSlice take multiple indices (like in a matrix)
opDollar needs a way to distinguish between the different dimensions.

-Lars

I think the compiler should rewrite \$ to __currentarray.length in unary
index expressions, and to __currentarray.length!(n) in multiple index
expressions.

Andrei
```
Oct 14 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Wed, 14 Oct 2009 06:11:22 -0400, Kagamin <spam here.lot> wrote:

Robert Jacques Wrote:

Also needed is an extension of the opDollar to return different values
based on the index:
opDollar(size_t index);

Dollar is just a synonym for length, isn't it?

User types can also override Dollar, though I don't remember off the top
```
Oct 14 2009
Jason House <jason.james.house gmail.com> writes:
```Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled efficiently with
one function written by the programmer. As I see it, there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

it's possible to use opIndex for #1 and opIndexAssign for #3, but that's not
efficient. #1 and #3 should be part of the same function, but I think #2
shouldnot be. What about defining an opIndexOpOpAssign that accepts a delegate
for #2 and then use compiler magic to specialize/inline it?
```
Oct 14 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] +=3D c;

with reasonable expressiveness and efficiency.

I would hope that *=3D +=3D /=3D and friends could all be handled efficie=

ntly with one function written by the programmer. As I see it, there are 3 =
basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b +=3D c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) +=3D c could benefit from the
same thing (a.length(index)+=3D3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but that's =

not efficient. #1 and #3 should be part of the same function, but I think #=
2 shouldnot be. What about defining an opIndexOpOpAssign that accepts a del=
egate for #2 and then use compiler magic to specialize/inline it?

It could also be done using a template thing to inject the "mutate the
value" operation:

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Op)(Thang c, Thing idx) {
ref v =3D <lookup [idx] however you like>
mixin("v "~Op~" c;");
<store to v to [idx] however you like>
}

or make it an alias function argument and use Op(v, b).

Sparse matrices are a good case to look at for issues.  a[b] is
defined for every [b], but if the value is zero nothing is actually
stored.  So there may or may not be something you can return a
reference to.   In C++ things like std::map just declare that if you
try to access a value that isn't there, it gets created.  That way
operator[] can always return a reference.   It would be great if we
could make a[b] not force a ref return in cases where there is no
lvalue that corresponds to the index (or property) being accessed.
Gracefully degrade to the slow path in those cases.

A good thing about a template is you can pretty easily specify which
cases to allow using template constraints:

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Op)(Thang c, Thing b)
if (Op in "+=3D -=3D")
{
...
}

(+ 1 small dream there about 'in' being defined to mean substring
search for string arguments -- that doesn't currently work does it?)

If the template can't be instantiated for the particular operation,
then the compiler would try to revert to the less efficient standby:
auto tmp =3D a[b];
tmp op=3D c;
a[b] =3D tmp;

The whole thing can generalize to all accessors too.  Instead of just
passing the Op, the compiler could pass the accessor string, and args
for that accessor.  Here an accessor means ".opIndex(b)",  ".foo", or
even a general ".memfun(b)"

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Member, string
Op)(Thang c, Thing b)
if (Member in ".foo() .bar() .opIndex()")
{
string call =3D ctReplace(Member, "()", "(b)");  // Member looks
like ".memfun()"  this turns it into ".memfun(b)"
ref v =3D mixin("this" ~ call ~ ";");
< any extra stuff you want to do on accesses to v >
mixin("v "~Op~" c;");
< store v back to member >
}

It's ugly and perhaps too low-level, but that can be worked on if the
general principle is sound.   Utility functions can be defined to do
whatever it is that turns out to be a recurring pattern.  Lack of
being virtual could be a problem for classes.

--bb
```
Oct 14 2009
Jason House <jason.james.house gmail.com> writes:
```Bill Baxter Wrote:

On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled efficiently with
one function written by the programmer. As I see it, there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b += c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) += c could benefit from the
same thing (a.length(index)+=3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but that's not
efficient. #1 and #3 should be part of the same function, but I think #2
shouldnot be. What about defining an opIndexOpOpAssign that accepts a delegate
for #2 and then use compiler magic to specialize/inline it?

It could also be done using a template thing to inject the "mutate the
value" operation:

The only issue with templates is that they're never virtual
```
Oct 14 2009
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
```Jason House wrote:
Bill Baxter Wrote:

On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled efficiently with
one function written by the programmer. As I see it, there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b += c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) += c could benefit from the
same thing (a.length(index)+=3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but that's not
efficient. #1 and #3 should be part of the same function, but I think #2
shouldnot be. What about defining an opIndexOpOpAssign that accepts a delegate
for #2 and then use compiler magic to specialize/inline it?

It could also be done using a template thing to inject the "mutate the
value" operation:

The only issue with templates is that they're never virtual

You can make virtuals out of templates, but not templates out of
virtuals. I think Walter is now inclined to look at a template-based
solution for operator overloading. That would save a mighty lot of code
without preventing classes that prefer virtual dispatch from doing so.

Andrei
```
Oct 14 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Wed, 14 Oct 2009 16:49:28 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Jason House wrote:
Bill Baxter Wrote:

On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled
efficiently with one function written by the programmer. As I see it,
there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b += c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) += c could benefit from the
same thing (a.length(index)+=3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but
that's not efficient. #1 and #3 should be part of the same function,
but I think #2 shouldnot be. What about defining an opIndexOpOpAssign
that accepts a delegate for #2 and then use compiler magic to
specialize/inline it?

It could also be done using a template thing to inject the "mutate the
value" operation:

The only issue with templates is that they're never virtual

You can make virtuals out of templates, but not templates out of
virtuals. I think Walter is now inclined to look at a template-based
solution for operator overloading. That would save a mighty lot of code
without preventing classes that prefer virtual dispatch from doing so.

Andrei

I've done something similar for a SmallVec struct. Most of the operator
overloads are actually aliases of templated functions (one each for
uni-ops, bi-ops, bi-op_r and opassign)
```
Oct 14 2009
Fawzi Mohamed <fmohamed mac.com> writes:
```On 2009-10-14 23:09:26 +0200, "Robert Jacques" <sandford jhu.edu> said:

On Wed, 14 Oct 2009 16:49:28 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Jason House wrote:
Bill Baxter Wrote:

On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] += c;

with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled
efficiently with one function written by the programmer. As I see it,
there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b += c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) += c could benefit from the
same thing (a.length(index)+=3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but
that's not efficient. #1 and #3 should be part of the same function,
but I think #2 shouldnot be. What about defining an opIndexOpOpAssign
that accepts a delegate for #2 and then use compiler magic to
specialize/inline it?

It could also be done using a template thing to inject the "mutate the
value" operation:

The only issue with templates is that they're never virtual

You can make virtuals out of templates, but not templates out of
virtuals. I think Walter is now inclined to look at a template-based
solution for operator overloading. That would save a mighty lot of code
without preventing classes that prefer virtual dispatch from doing so.

Andrei

I've done something similar for a SmallVec struct. Most of the operator
overloads are actually aliases of templated functions (one each for
uni-ops, bi-ops, bi-op_r and opassign)

I would really like a solution to all the overloading ops, as I missed
them in NArray, I think that some small rewriting is ok, but it must be
*small*, no magic as already said by other numerics can be tricky.
Also Andrei proposal seem workable, but there is also another solution:

Note that a ref return for opIndex, could work in most situations.
As Bill correctly pointed out sparse matrix offer the most challenging
example, there one wants to have two different functions: opIndex and
opIndexLhs, the second being called when the index is on the left hand
side of an assignment, so that reading a 0 entry in a matrix returns 0,
whereas assigning it allocates place for it.
This makes it slightly more complex to control what is being assigned
(as you need to return a structure overloading opXAssign, but I think
it would be ok in most cases.

Fawzi
```
Oct 15 2009
"Robert Jacques" <sandford jhu.edu> writes:
```On Thu, 15 Oct 2009 04:48:57 -0400, Fawzi Mohamed <fmohamed mac.com> wrote:

On 2009-10-14 23:09:26 +0200, "Robert Jacques" <sandford jhu.edu> said:

On Wed, 14 Oct 2009 16:49:28 -0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

Jason House wrote:
Bill Baxter Wrote:

On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and
opIndexAssign
don't seem to be up to snuff because they don't catch operations
like
a[b] += c;
with reasonable expressiveness and efficiency.

I would hope that *= += /= and friends could all be handled
efficiently with one function written by the programmer. As I see
it,  there are 3 basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like  a.b += c.
It's just a matter of  accessing  .b  vs .opIndex(b).   And really
same goes for any function  a.memfun(b) += c could benefit from the
same thing (a.length(index)+=3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but
that's not efficient. #1 and #3 should be part of the same
function,  but I think #2 shouldnot be. What about defining an
opIndexOpOpAssign  that accepts a delegate for #2 and then use
compiler magic to  specialize/inline it?

It could also be done using a template thing to inject the "mutate
the
value" operation:

The only issue with templates is that they're never virtual

You can make virtuals out of templates, but not templates out of
virtuals. I think Walter is now inclined to look at a template-based
code  without preventing classes that prefer virtual dispatch from
doing so.
Andrei

I've done something similar for a SmallVec struct. Most of the
operator  overloads are actually aliases of templated functions (one
each for  uni-ops, bi-ops, bi-op_r and opassign)

I would really like a solution to all the overloading ops, as I missed
them in NArray, I think that some small rewriting is ok, but it must be
*small*, no magic as already said by other numerics can be tricky.
Also Andrei proposal seem workable, but there is also another solution:

Note that a ref return for opIndex, could work in most situations.
As Bill correctly pointed out sparse matrix offer the most challenging
example, there one wants to have two different functions: opIndex and
opIndexLhs, the second being called when the index is on the left hand
side of an assignment, so that reading a 0 entry in a matrix returns 0,
whereas assigning it allocates place for it.
This makes it slightly more complex to control what is being assigned
(as you need to return a structure overloading opXAssign, but I think it
would be ok in most cases.

Fawzi

Would you like some example code?
```
Oct 15 2009
Fawzi Mohamed <fmohamed mac.com> writes:
```On 2009-10-15 17:51:56 +0200, "Robert Jacques" <sandford jhu.edu> said:

On Thu, 15 Oct 2009 04:48:57 -0400, Fawzi Mohamed <fmohamed mac.com> wrote:

[...]
Note that a ref return for opIndex, could work in most situations.
As Bill correctly pointed out sparse matrix offer the most challenging
example, there one wants to have two different functions: opIndex and
opIndexLhs, the second being called when the index is on the left hand
side of an assignment, so that reading a 0 entry in a matrix returns 0,
whereas assigning it allocates place for it.
This makes it slightly more complex to control what is being assigned
(as you need to return a structure overloading opXAssign, but I think
it  would be ok in most cases.

Fawzi

Would you like some example code?

I suppose you would like it ;)

// example 1
class Matrix(T){
T opIndex(size_t i,size_t j){
if (has_(i,j)){
return data[index(i,j)];
} else {
return cast(T)0;
}
}
ref T opIndexLhs(size_t i,size_t j){
if (has_(i,j)){
return &data[index(i,j)];
} else {
//alloc new place and set things so that index(i,j) returns it
return &data[index(i,j)];
}
}
}

then
m[3,4]+=4.0;
would be converted in
m.opIndexLhs(3,4)+=4.0;

typically with just one method (opIndexLhs) all += -=,... are covered

if one needs more control

class AbsMatrix(T){
T opIndex(size_t i,size_t j){
if (has_(i,j)){
return data[index(i,j)];
} else {
return cast(T)0;
}
}
struct Setter{
T* pos;
*pos+=abs(val);
}
}
Setter opIndexLhs(size_t i,size_t j){
Setter pos;
if (has_(i,j)){
res.pos=&data[index(i,j)];
} else {
//alloc new place and set things so that index(i,j) returns it
res.pos=&data[index(i,j)];
}
return res;
}
}

if one does not allow ref T as return type then one can return a pointer and do
static if(is(typeof(*m.opIndexLhs(3,4))))
*m.opIndexLhs(3,4)+=4.0;
else
m.opIndexLhs(3,4)+=4.0;

so that the trick with the struct is still possible.
```
Oct 15 2009
Fawzi Mohamed <fmohamed mac.com> writes:
```On 2009-10-15 22:55:02 +0200, Fawzi Mohamed <fmohamed mac.com> said:

On 2009-10-15 17:51:56 +0200, "Robert Jacques" <sandford jhu.edu> said:

On Thu, 15 Oct 2009 04:48:57 -0400, Fawzi Mohamed <fmohamed mac.com> wrote:

[...]
Note that a ref return for opIndex, could work in most situations.
As Bill correctly pointed out sparse matrix offer the most challenging
example, there one wants to have two different functions: opIndex and
opIndexLhs, the second being called when the index is on the left hand
side of an assignment, so that reading a 0 entry in a matrix returns 0,
whereas assigning it allocates place for it.
This makes it slightly more complex to control what is being assigned
(as you need to return a structure overloading opXAssign, but I think
it  would be ok in most cases.

Fawzi

Would you like some example code?

I suppose you would like it ;)

// example 1
class Matrix(T){
T opIndex(size_t i,size_t j){
if (has_(i,j)){
return data[index(i,j)];
} else {
return cast(T)0;
}
}
ref T opIndexLhs(size_t i,size_t j){
if (has_(i,j)){
return &data[index(i,j)];
} else {
//alloc new place and set things so that index(i,j) returns it
return &data[index(i,j)];
}
}
}

mmmh I mixed up a bit the ref returning and pointer returning case...
clearly there should be no &...

then
m[3,4]+=4.0;
would be converted in
m.opIndexLhs(3,4)+=4.0;

typically with just one method (opIndexLhs) all += -=,... are covered

if one needs more control

class AbsMatrix(T){
T opIndex(size_t i,size_t j){
if (has_(i,j)){
return data[index(i,j)];
} else {
return cast(T)0;
}
}
struct Setter{
T* pos;
*pos+=abs(val);
}
}
Setter opIndexLhs(size_t i,size_t j){
Setter pos;
if (has_(i,j)){
res.pos=&data[index(i,j)];
} else {
//alloc new place and set things so that index(i,j) returns it
res.pos=&data[index(i,j)];
}
return res;
}
}

if one does not allow ref T as return type then one can return a pointer and do
static if(is(typeof(*m.opIndexLhs(3,4))))
*m.opIndexLhs(3,4)+=4.0;
else
m.opIndexLhs(3,4)+=4.0;

so that the trick with the struct is still possible.

```
Oct 15 2009
Bill Baxter <wbaxter gmail.com> writes:
```On Wed, Oct 14, 2009 at 9:34 AM, Bill Baxter <wbaxter gmail.com> wrote:
On Wed, Oct 14, 2009 at 7:42 AM, Jason House
<jason.james.house gmail.com> wrote:
Andrei Alexandrescu Wrote:

Right now we're in trouble with operators: opIndex and opIndexAssign
don't seem to be up to snuff because they don't catch operations like

a[b] +=3D c;

with reasonable expressiveness and efficiency.

I would hope that *=3D +=3D /=3D and friends could all be handled effici=

ently with one function written by the programmer. As I see it, there are 3=
basic steps:
1. Look up a value by index
2. Mutate the value
3. Store the result

And as Chad J reminds us, same goes for in-place property mutations
like =A0a.b +=3D c.
It's just a matter of =A0accessing =A0.b =A0vs .opIndex(b). =A0 And reall=

y
same goes for any function =A0a.memfun(b) +=3D c could benefit from the
same thing (a.length(index)+=3D3 anyone?)

it's possible to use opIndex for #1 and opIndexAssign for #3, but that's=

not efficient. #1 and #3 should be part of the same function, but I think =
#2 shouldnot be. What about defining an opIndexOpOpAssign that accepts a de=
legate for #2 and then use compiler magic to specialize/inline it?
It could also be done using a template thing to inject the "mutate the
value" operation:

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Op)(Thang c, Thing idx)=

{
=A0 =A0 ref v =3D <lookup [idx] however you like>
=A0 =A0 mixin("v "~Op~" c;");
=A0 =A0 <store to v to [idx] however you like>
}

or make it an alias function argument and use Op(v, b).

Sparse matrices are a good case to look at for issues. =A0a[b] is
defined for every [b], but if the value is zero nothing is actually
stored. =A0So there may or may not be something you can return a
reference to. =A0 In C++ things like std::map just declare that if you
try to access a value that isn't there, it gets created. =A0That way
operator[] can always return a reference. =A0 It would be great if we
could make a[b] not force a ref return in cases where there is no
lvalue that corresponds to the index (or property) being accessed.
Gracefully degrade to the slow path in those cases.

A good thing about a template is you can pretty easily specify which
cases to allow using template constraints:

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Op)(Thang c, Thing b)
=A0 =A0 =A0 if (Op in "+=3D -=3D")
{
=A0 ...
}

(+ 1 small dream there about 'in' being defined to mean substring
search for string arguments -- that doesn't currently work does it?)

If the template can't be instantiated for the particular operation,
then the compiler would try to revert to the less efficient standby:
auto tmp =3D a[b];
tmp op=3D c;
a[b] =3D tmp;

The whole thing can generalize to all accessors too. =A0Instead of just
passing the Op, the compiler could pass the accessor string, and args
for that accessor. =A0Here an accessor means ".opIndex(b)", =A0".foo", or
even a general ".memfun(b)"

void opIndexOpOpAssignOpSpamOpSpamSpamSpam(string Member, string
Op)(Thang c, Thing b)
=A0 if (Member in ".foo() .bar() .opIndex()")
{
=A0 =A0 string call =3D ctReplace(Member, "()", "(b)"); =A0// Member look=

s
like ".memfun()" =A0this turns it into ".memfun(b)"
=A0 =A0 ref v =3D mixin("this" ~ call ~ ";");
=A0 =A0 < any extra stuff you want to do on accesses to v >
=A0 =A0 mixin("v "~Op~" c;");
=A0 =A0 < store v back to member >
}

It's ugly and perhaps too low-level, but that can be worked on if the
general principle is sound. =A0 Utility functions can be defined to do
whatever it is that turns out to be a recurring pattern. =A0Lack of
being virtual could be a problem for classes.

After mulling over it some more, basically what I'm describing is
simply a function that gives the user a chance to rewrite the AST of
these kinds of ".memfun(args) op=3D " type operations.

When the compiler sees     "obj.memfun(b) +=3D c"
It gives that bit of the syntax tree to the AST manipulator function
(if obj defines one) and the function can then alter it however
desired.

This is made somewhat clunky by the fact that our only representation
for ASTs is strings.

Actually this could just be a CTFE function.   It doesn't need to be a temp=
late.

Just imagine there's a compile-time struct passed in that could do
things like this:

string opWhateverAssign(AST syntax)
{
// First some examples:
// Assume obj.memfun(b0,b1) +=3D c   is what appeared in source code.
enum s0 =3D syntax.args;  // yields "b0, b1" -- compiler knows args
to this fn are called "b0" and "b1"
enum s1 =3D syntax.args[0]; // yields "b0"
enum s2 =3D syntax.rvalue; // yields "c"
enum s3 =3D syntax.member;  // yields  "memfun"
enum s4 =3D syntax.formatCallString("v =3D \$syntax.member( x,y )");
// yields "v=3Dmemfun(x,y)"
enum s5 =3D syntax.defaultImpl; // yields "auto v=3Dmemfun(b0,b1);
v+=3Dc; memfun(b0,b1)=3Dv;"

// ok now I'll actually do something
static if (syntax.member =3D=3D "opIndex") {
// say this is a sparse matrix class
return ctFormat(q{
if (!this.matrix_contains(\$syntax.args)) {
this.create_entry(\$syntax.args);
}
auto v =3D &this.matrix_storage[\$syntax.args];
*v  \$syntax.op  \$syntax.rvalue;
});
}
else {
return syntax.defaultImpl;
}
}

This assumes we can have CTFE functions inside structs/classes.  It
assumes a function called ctFormat that can format a string at
compile-time and do perl-like variable interpolation.   It assumes we
can pass structs to CTFE functions and use them there.

Really it doesn't have to be just the opAssign type calls either.  You
could allow such interceptors for any method call or member access.

This is really close to a nemerle-like macro, actually.  Just modify 4
lines and it is one.

macro opWhateverAssign(AST syntax)
{
// First some examples:
// ok now I'll actually do something
static if (syntax.member =3D=3D "opIndex") {
// say this is a sparse matrix class
<[
if (!this.matrix_contains(\$syntax.args)) {
this.create_entry(\$syntax.args);
}
auto v =3D &this.matrix_storage[\$syntax.args];
*v  \$syntax.op  \$syntax.rvalue;
]>
}
else {
<[ \$syntax.defaultImpl; ]>
}
}

And this really makes me think it's silly to put off macro syntax till
D3.  Everything needed is basically there.  In contrast to a new