www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Expression evaluation order

reply "David Nadlinger" <code klickverbot.at> writes:
There is a test in the DMD testsuite that verifies that the 
evaluation order in the following statement

  a()[] = b()[] + c()[];

is b, c, a.

I'm trying to figure out why this regressed with the 2.063 merge 
in LDC, but (where) is this specified in the first place?

David
Jun 13 2013
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, June 14, 2013 01:09:18 David Nadlinger wrote:

 There is a test in the DMD testsuite that verifies that the
 evaluation order in the following statement
 
 a()[] = b()[] + c()[];
 
 is b, c, a.
 
 I'm trying to figure out why this regressed with the 2.063 merge
 in LDC, but (where) is this specified in the first place?


Walter has expressed a desire in the past to make it so that D requires that 
the evaluation order of arguments be left-to-right in order to avoid bugs, and 
while these aren't exactly function arguments, they're basically function 
arguments to a built-in function called +, so I could definitely see Walter 
wanting to require that the evaluation order be left-to-right. However, while 
Walter has expressed an interest in making it a requirement, AFAIK, it has 
never officially become one and the spec says nothing on the matter.

- Jonathan M Davis
Jun 13 2013
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 Walter has expressed a desire in the past to make it so that D 
 requires that the evaluation order of arguments be left-to-right
 in order to avoid bugs,


Considering how much important this feature is, I think it should 
have priority over several other things.

Bye,
bearophile
Jun 13 2013
prev sibling next sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:

 Walter has expressed a desire in the past to make it so that D 
 requires that
 the evaluation order of arguments be left-to-right in order to 
 avoid bugs, and
 while these aren't exactly function arguments, …


Actually, behind the scenes all three are arguments to *one* 
function call: the druntime array op implementation.

Anyway, my question was mainly about the assignment operator, 
which is obviously not expected to behave lexically left-to-right 
by that test case.

David
Jun 13 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, June 14, 2013 02:02:23 David Nadlinger wrote:

 On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:

 Walter has expressed a desire in the past to make it so that D
 requires that
 the evaluation order of arguments be left-to-right in order to
 avoid bugs, and
 while these aren't exactly function arguments, …


 
 Actually, behind the scenes all three are arguments to *one*
 function call: the druntime array op implementation.
 
 Anyway, my question was mainly about the assignment operator,
 which is obviously not expected to behave lexically left-to-right
 by that test case.


I don't see how the assignment operator could possibly work left-to-right, 
though I could see how there might be a question with regards to whether 
what's on the left of the assignment operator would be evaluated before what's 
on the right when what's on the left is an expression rather than a variable. 
Personally, I would expect it to fully evaluate the right-hand side of an 
assignment expression before evaluating anything on the left, and I'd expect 
the same of any operator which is right-associative, but I'm not aware of 
anything official on the matter. All the discussions on the matter that I'm 
aware of have referred specifically to function arguments being left-to-right 
and said nothing about operators.

- Jonathan M Davis
Jun 13 2013
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 14 June 2013 at 00:13:37 UTC, Jonathan M Davis wrote:

 I don't see how the assignment operator could possibly work 
 left-to-right,
 though I could see how there might be a question with regards 
 to whether
 what's on the left of the assignment operator would be 
 evaluated before what's
 on the right when what's on the left is an expression rather 
 than a variable.


SDC compute the address of the reciever, then the value on the 
right, and finally store it to the address. I don't see any major 
issue here.
Jun 13 2013
prev sibling next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 June 2013 at 00:13:37 UTC, Jonathan M Davis wrote:

 On Friday, June 14, 2013 02:02:23 David Nadlinger wrote:

 On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis 
 wrote:

 Walter has expressed a desire in the past to make it so that 
 D
 requires that
 the evaluation order of arguments be left-to-right in order 
 to
 avoid bugs, and
 while these aren't exactly function arguments, …


 
 Actually, behind the scenes all three are arguments to *one*
 function call: the druntime array op implementation.
 
 Anyway, my question was mainly about the assignment operator,
 which is obviously not expected to behave lexically 
 left-to-right
 by that test case.


 I don't see how the assignment operator could possibly work 
 left-to-right


What do you mean? assignment requires two arguments: the 
*reference* of lhs, and value/reference or rhs. Regardless of if 
lhs is a value or expression, the compiler still needs to know 
*where* to assign to.

I don't see any reasons why the compiler couldn't choose lhs 
first instead.

*as* a matter of fact, that's how gcc does it for C++:

//----
int main()
{
     int i = 0;
     ++i = ++i;
     assert(i == 2); //passes
}
//----

Left to right
First, evaluate lhs: i has the value 1.
then evaluate rhs, which will now have the value 2.
assign 2 to i.
* i == 2 *

right to left:
First, evalutate rhs: i has the value 1, an rhs 1.
Evaluate lhs: i is now 2.
But then assign rhs (1) to lhs.
* i == 1 *

So yeah, left to right evaluation is completely coherent, and a 
valid scheme.

Heck, you'll notice this *also* passes with DMD, so dmd *also* 
evaluates opEqual left to right.

So I guess this "surprising behavior" is the evaluation order we 
want to enforce?

---------
I remember the discussion:
http://forum.dlang.org/thread/bniaxycuguviwfdtojzf forum.dlang.org

For the record, I don't remember there being a consensus... at 
all...
There were 3.5 positions:
1) left to right evaluation
2) right to left evaluation
3) specifically unspecified
   3.5) error when compiler sees ambiguity

I'm personally in favor of 3, with some 3.5 as a warning. 
Evaluation order, even when defined, remains obscure once 
associative operations come into play, and still bite you in the 
ass anyways, so I don't see anything good out of forcing it one 
way or the other anyways.
Jun 13 2013
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
monarch_dodra:


 *as* a matter of fact, that's how gcc does it for C++:

 //----
 int main()
 {
     int i = 0;
     ++i = ++i;
     assert(i == 2); //passes
 }
 //----

 Left to right
 First, evaluate lhs: i has the value 1.
 then evaluate rhs, which will now have the value 2.
 assign 2 to i.
 * i == 2 *

 right to left:
 First, evalutate rhs: i has the value 1, an rhs 1.
 Evaluate lhs: i is now 2.
 But then assign rhs (1) to lhs.
 * i == 1 *

 So yeah, left to right evaluation is completely coherent, and a 
 valid scheme.


I think the results of your code is not defined by the C++ 
standard, so in practice writing that code is a programmer's 
mistake. And it's a mistake to design a modern programming 
language that accepts code that gives undefined results.
To be considered a grown up language D needs define only one 
semantics, or forbid code like that.



 Evaluation order, even when defined, remains obscure once 
 associative operations come into play, and still bite you in 
 the ass anyways,


Even when the programmer isn't able to know what the result will 
be, it's essential for the D code to give the same result on all 
CPUs and on all compilers.

Bye,
bearophile
Jun 14 2013
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/13/2013 11:14 PM, monarch_dodra wrote:

 3) specifically unspecified
    3.5) error when compiler sees ambiguity

 I'm personally in favor of 3, with some 3.5 as a warning.


Having a warning for that is "boy who cried wolf", as most functions have side 
effects and any expression like:

    f() + g()

would generate such a warning.
Jun 15 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 15 June 2013 at 17:50:39 UTC, Walter Bright wrote:

 On 6/13/2013 11:14 PM, monarch_dodra wrote:

 3) specifically unspecified
   3.5) error when compiler sees ambiguity

 I'm personally in favor of 3, with some 3.5 as a warning.


 Having a warning for that is "boy who cried wolf", as most 
 functions have side effects and any expression like:

    f() + g()

 would generate such a warning.


Well, why would that throw a warning?

I didn't mean expressions that *could* raise an ambiguity, but 
when the compiler *catches* that there *will* be ambiguity.
Jun 15 2013
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/14/2013 02:13 AM, Jonathan M Davis wrote:

 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd expect
 the same of any operator which is right-associative, ...


Why? Associativity is unrelated to evaluation order.
Jun 14 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Saturday, June 15, 2013 01:46:49 Timon Gehr wrote:

 On 06/14/2013 02:13 AM, Jonathan M Davis wrote:

 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd
 expect the same of any operator which is right-associative, ...


 
 Why? Associativity is unrelated to evaluation order.


It seems natural to me that the stuff on the associated side would be evaluated 
before the stuff on the other, but that doesn't mean that it's the best way to 
go. It's just what I would have expected. So, if there's a good reason to do 
it differently, then I don't necessarily have a problem with that.

- Jonathan M Davis
Jun 14 2013
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/15/2013 04:21 AM, Jonathan M Davis wrote:

 On Saturday, June 15, 2013 01:46:49 Timon Gehr wrote:

 On 06/14/2013 02:13 AM, Jonathan M Davis wrote:

 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd
 expect the same of any operator which is right-associative, ...


 Why? Associativity is unrelated to evaluation order.


 It seems natural to me that the stuff on the associated side would be evaluated
 before the stuff on the other, but that doesn't mean that it's the best way to
 go. It's just what I would have expected. So, if there's a good reason to do
 it differently, then I don't necessarily have a problem with that.

 - Jonathan M Davis


Well, I can't think of a _good_ reason now. :)

Since Walter once stated that evaluation is supposed to be LTR, this is 
what I have implemented for CTFE. The stack-based bytecode interpreter 
already relies on having the address-related information pushed before 
the value, and changing the evaluation order would therefore imply a 
little work.
Jun 15 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/15/2013 07:44 PM, Timon Gehr wrote:

 ...

 It seems natural to me that the stuff on the associated side would be
 evaluated
 before the stuff on the other, but that doesn't mean that it's the
 best way to
 go. It's just what I would have expected. So, if there's a good reason
 to do
 it differently, then I don't necessarily have a problem with that.

 - Jonathan M Davis


 Well, I can't think of a _good_ reason now. :)
 ...


Correction: The conditional operator is right-associative, but 
evaluation necessarily proceeds LTR.
Jun 15 2013
prev sibling parent "w0rp" <devw0rp gmail.com> writes:
On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:

 Walter has expressed a desire in the past to make it so that D 
 requires that
 the evaluation order of arguments be left-to-right in order to 
 avoid bugs, and
 while these aren't exactly function arguments, they're 
 basically function
 arguments to a built-in function called +, so I could 
 definitely see Walter
 wanting to require that the evaluation order be left-to-right. 
 However, while
 Walter has expressed an interest in making it a requirement, 
 AFAIK, it has
 never officially become one and the spec says nothing on the 
 matter.

 - Jonathan M Davis


I would like to cast my vote for this feature. It is very nice to 
have, and it would prevent a few bugs from popping up.
Jun 14 2013
prev sibling next sibling parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
On 14 June 2013 00:09, David Nadlinger <code klickverbot.at> wrote:

 There is a test in the DMD testsuite that verifies that the evaluation order
 in the following statement

  a()[] = b()[] + c()[];

 is b, c, a.

 I'm trying to figure out why this regressed with the 2.063 merge in LDC, but
 (where) is this specified in the first place?

 David


Interesting, that certainly doesn't flag up as a regression in the
2.063 merge in GDC... :o)

However yes, the behaviour relies on the order the sucky X86 ABI
pushes arguments onto the stack (which for this array op is from right
to left).  Whereas on *all* other architectures it will execute the
parameters in left to right order, which would be a, c, b in this
case.

--
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 14 2013
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Iain Buclaw:


 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is 
 from right
 to left).  Whereas on *all* other architectures it will execute 
 the
 parameters in left to right order, which would be a, c, b in 
 this case.


D needs the same standard order of evaluations for all 
expressions on all compilers, regardless of the CPUs. The only 
other acceptable alternative is to statically forbid code that 
risks having variable results.

(And I think the right order for that is b, c, a).

Bye,
bearophile
Jun 14 2013
next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Friday, 14 June 2013 at 11:04:04 UTC, bearophile wrote:

 (And I think the right order for that is b, c, a).


Could you derive a set of rules from/compatible with that 
thought? ;)

David
Jun 14 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On 14 June 2013 12:04, bearophile <bearophileHUGS lycos.com> wrote:

 Iain Buclaw:



 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is from right
 to left).  Whereas on *all* other architectures it will execute the
 parameters in left to right order, which would be a, c, b in this case.



 D needs the same standard order of evaluations for all expressions on all
 compilers, regardless of the CPUs. The only other acceptable alternative is
 to statically forbid code that risks having variable results.

 (And I think the right order for that is b, c, a).


Right... and the way to do that would be to evaluate the argument
before calling the array op.

eg:
ref _tmp1 = b()[];
ref _tmp2 = c()[];
ref _tmp3 = a()[];

_tmp3 = _arrayOp(_tmp3, _tmp2, _tmp1);

--
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 14 2013
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/14/2013 01:04 PM, bearophile wrote:

 ...
 (And I think the right order for that is b, c, a).
 ...


Why?
Jun 14 2013
prev sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Friday, 14 June 2013 at 09:59:07 UTC, Iain Buclaw wrote:

 Interesting, that certainly doesn't flag up as a regression in 
 the
 2.063 merge in GDC... :o)

 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is 
 from right
 to left).  Whereas on *all* other architectures it will execute 
 the
 parameters in left to right order, which would be a, c, b in 
 this
 case.


In LDC, the actual evaluation order is completely separate from 
the target ABI (you simply pass the SSA vales to a function call 
in LLVM, and you can decide the order in which to create them 
yourself).

Turns out that for array operations, we actually need to reverse 
the order compared to our normal one (left to right), as their 
signatures have specifically been chosen for the evaluation order 
in DMD that is intertwined with the backend ABI/function call 
implementation details.

David
Jun 15 2013
parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On 15 June 2013 18:16, David Nadlinger <code klickverbot.at> wrote:

 On Friday, 14 June 2013 at 09:59:07 UTC, Iain Buclaw wrote:

 Interesting, that certainly doesn't flag up as a regression in the
 2.063 merge in GDC... :o)

 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is from right
 to left).  Whereas on *all* other architectures it will execute the
 parameters in left to right order, which would be a, c, b in this
 case.



 In LDC, the actual evaluation order is completely separate from the target
 ABI (you simply pass the SSA vales to a function call in LLVM, and you can
 decide the order in which to create them yourself).

 Turns out that for array operations, we actually need to reverse the order
 compared to our normal one (left to right), as their signatures have
 specifically been chosen for the evaluation order in DMD that is intertwined
 with the backend ABI/function call implementation details.



We don't have that luxury, but as I said in the last thread brought up
on this some time ago, for extern(D) we just make temporaries and
compound them all together  (_a = a(), _b = b(), _c = c(),
_arrayop(_a, _b, _c));   There's no reason why this can't be done for
extern (C) - other than it will break a load of code that relies on
X86 ABI behaviour  (such as what array operations apparently do...)

--
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 15 2013
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 14 Jun 2013 01:09:18 +0200
schrieb "David Nadlinger" <code klickverbot.at>:


 There is a test in the DMD testsuite that verifies that the 
 evaluation order in the following statement
 
   a()[] = b()[] + c()[];
 
 is b, c, a.
 
 I'm trying to figure out why this regressed with the 2.063 merge 
 in LDC, but (where) is this specified in the first place?
 
 David


Related old discussions:
http://bugzilla.gdcproject.org/show_bug.cgi?id=8

http://forum.dlang.org/thread/bniaxycuguviwfdtojzf forum.dlang.org
(I knew we discussed this before ;-)
Jun 14 2013