www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - D float types operations vs C++ ones

reply drug <drug2004 bk.ru> writes:
I have two implementation of the same algorithm - D and C++ (that is 
port of D version). I assume that running these implementations on the 
same data should give the same results from both. But with some data the 
results differ (5th decimal digit after point). For my purpose it isn't 
important and I can ignore this difference. But I'd like to have the 
same results totally (I mean bit to bit) to easy maintanence of the 
implementations. Question is - can I rely that D and C++ float types 
operations should give the same result because D and C are similar or no 
and I should use more portable way and compare floats with some epsilon?
Dec 17 2015
next sibling parent reply Andrea Fontana <nospam example.com> writes:
On Thursday, 17 December 2015 at 11:50:02 UTC, drug wrote:

 I have two implementation of the same algorithm - D and C++ 
 (that is port of D version). I assume that running these 
 implementations on the same data should give the same results 
 from both. But with some data the results differ (5th decimal 
 digit after point). For my purpose it isn't important and I can 
 ignore this difference. But I'd like to have the same results 
 totally (I mean bit to bit) to easy maintanence of the 
 implementations. Question is - can I rely that D and C++ float 
 types operations should give the same result because D and C 
 are similar or no and I should use more portable way and 
 compare floats with some epsilon?


You should publish some code to check...
Dec 17 2015
parent reply drug <drug2004 bk.ru> writes:
On 17.12.2015 14:52, Andrea Fontana wrote:

 You should publish some code to check...


Too much code to public - operations are simple, but there are many 
branches and reducing may take much time . In fact I asked to understand 
_in general_ if it worth diving into code to find the source of the 
difference or take it easy and go on...
If D and C++ use the same things to process float types then it worth to 
dive, but if there is some issues, then it doesn't.
Dec 17 2015
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 17 December 2015 at 11:58:35 UTC, drug wrote:

 On 17.12.2015 14:52, Andrea Fontana wrote:

 You should publish some code to check...


 Too much code to public - operations are simple, but there are 
 many branches and reducing may take much time . In fact I asked 
 to understand _in general_ if it worth diving into code to find 
 the source of the difference or take it easy and go on...
 If D and C++ use the same things to process float types then it 
 worth to dive, but if there is some issues, then it doesn't.


Yes the float types are the same. floats doubles are identical 
long double == real ( at least for x86)

The only difference is that float are default initialised to NaN 
in D.
The sources of difference are likely to occur from
- const folding (varying by compiler e.g. gcc uses libgmp for 
high precision floating point for constant folding where as other 
compliers may not)
- type promotions and truncations
I.e.
double a = ... ,b = ... ;
float c = a+b;
float d = cast(float)a + cast(float)b;

c is unlikely to be bit for bit equal to d.

floating point operations are NOT commutative nor are they 
distributive. Order of operations does matter.
i.e. a+(b+c) !=(a+b) +c and a*(b+c) != a*b + a*c (!= means not 
necessarily equal)

you should always compare floats for equality with an epsilon 
(unless 0.0 +-inf )
Dec 17 2015
parent reply drug <drug2004 bk.ru> writes:
On 17.12.2015 16:09, Nicholas Wilson wrote:

 Yes the float types are the same. floats doubles are identical long
 double == real ( at least for x86)

 The only difference is that float are default initialised to NaN in D.
 The sources of difference are likely to occur from
 - const folding (varying by compiler e.g. gcc uses libgmp for high
 precision floating point for constant folding where as other compliers
 may not)
 - type promotions and truncations
 I.e.
 double a = ... ,b = ... ;
 float c = a+b;
 float d = cast(float)a + cast(float)b;

 c is unlikely to be bit for bit equal to d.

 floating point operations are NOT commutative nor are they distributive.
 Order of operations does matter.
 i.e. a+(b+c) !=(a+b) +c and a*(b+c) != a*b + a*c (!= means not
 necessarily equal)

 you should always compare floats for equality with an epsilon (unless
 0.0 +-inf )


Thanks for answer. My C++ version is tracing D version so commutativity 
and distributivity aren't requred because order of operations is the 
same (I guess so at least), so I hoped for bit to bit equality (it 
simplifies comparing serialized data). But I found that error between D 
and C++ versions grows if more data processed so I should investigate it 
anyway.
Dec 17 2015
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 17 December 2015 at 13:30:11 UTC, drug wrote:

 On 17.12.2015 16:09, Nicholas Wilson wrote:

[...]


 Thanks for answer. My C++ version is tracing D version so 
 commutativity and distributivity aren't requred because order 
 of operations is the same (I guess so at least), so I hoped for 
 bit to bit equality (it simplifies comparing serialized data). 
 But I found that error between D and C++ versions grows if more 
 data processed so I should investigate it anyway.


What I mean about order of operations is that if you go
  a = b*a+c*c + e;
the compiler is free to rewrite that as
float __tmp0 = a*b;
float __tmp1 = c*c;
and then do either of
float __tmp2 = __tmp0+__tmp1;
a = __tmp2 + e;
OR
float __tmp2 = __tmp0+e;
a = __tmp2+__tmp1;
Dec 17 2015
parent reply drug <drug2004 bk.ru> writes:
On 18.12.2015 05:58, Nicholas Wilson wrote:

 On Thursday, 17 December 2015 at 13:30:11 UTC, drug wrote:

 On 17.12.2015 16:09, Nicholas Wilson wrote:

 [...]


 Thanks for answer. My C++ version is tracing D version so
 commutativity and distributivity aren't requred because order of
 operations is the same (I guess so at least), so I hoped for bit to
 bit equality (it simplifies comparing serialized data). But I found
 that error between D and C++ versions grows if more data processed so
 I should investigate it anyway.


 What I mean about order of operations is that if you go
   a = b*a+c*c + e;
 the compiler is free to rewrite that as
 float __tmp0 = a*b;
 float __tmp1 = c*c;
 and then do either of
 float __tmp2 = __tmp0+__tmp1;
 a = __tmp2 + e;
 OR
 float __tmp2 = __tmp0+e;
 a = __tmp2+__tmp1;


I see, thanks to all!
Dec 17 2015
parent reply Ola Fosheim Gr <ola.fosheim.grostad+dlang gmail.com> writes:
On Friday, 18 December 2015 at 07:30:52 UTC, drug wrote:

 What I mean about order of operations is that if you go
   a = b*a+c*c + e;
 the compiler is free to rewrite that as
 float __tmp0 = a*b;
 float __tmp1 = c*c;
 and then do either of
 float __tmp2 = __tmp0+__tmp1;
 a = __tmp2 + e;
 OR
 float __tmp2 = __tmp0+e;
 a = __tmp2+__tmp1;


 I see, thanks to all!


I don't think this can be right, unless you use some kind of 
fast-math optimizer.

But:

Modern C++ compilers try to support ieee754-2008, which is needed 
to get reproducible results. D is based on the older 1985 
version, and there is no announced effort to make it modern.
Dec 18 2015
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/18/2015 12:19 AM, Ola Fosheim Gr wrote:

 On Friday, 18 December 2015 at 07:30:52 UTC, drug wrote:

 What I mean about order of operations is that if you go
   a = b*a+c*c + e;
 the compiler is free to rewrite that as
 float __tmp0 = a*b;
 float __tmp1 = c*c;
 and then do either of
 float __tmp2 = __tmp0+__tmp1;
 a = __tmp2 + e;
 OR
 float __tmp2 = __tmp0+e;
 a = __tmp2+__tmp1;


 I see, thanks to all!


 I don't think this can be right, unless you use some kind of fast-math
 optimizer.

 But:

 Modern C++ compilers try to support ieee754-2008, which is needed to get
 reproducible results. D is based on the older 1985 version, and there is
 no announced effort to make it modern.


Thank you for the reference. If the Wikipedia article is precise in 
language, it is just a recommendation:

"Clause 10: Expression evaluation

This clause is new; it recommends how language standards should specify 
the semantics of sequences of operations, and points out the subtleties 
of literal meanings and optimizations that change the value of a result."

 
https://en.wikipedia.org/wiki/IEEE_754_revision#Clause_10:_Expression_evaluation

Ali
Dec 18 2015
prev sibling next sibling parent anonymous <anonymous example.com> writes:
On 17.12.2015 12:50, drug wrote:

 I have two implementation of the same algorithm - D and C++ (that is
 port of D version). I assume that running these implementations on the
 same data should give the same results from both. But with some data the
 results differ (5th decimal digit after point). For my purpose it isn't
 important and I can ignore this difference. But I'd like to have the
 same results totally (I mean bit to bit) to easy maintanence of the
 implementations. Question is - can I rely that D and C++ float types
 operations should give the same result because D and C are similar or no
 and I should use more portable way and compare floats with some epsilon?


D explicitly allows compilers to use greater precision in calculations 
than the involved types have [1]. For example, an expression involving 
`float`s may be evaluated at double or real precision.

That means, you cannot rely on getting the same results even when 
looking only at D. Compiler A may produce higher precision code than 
compiler B; or machine X offers a larger maximum precision than machine 
Y (and the compiler makes use of it).


[1] http://dlang.org/spec/float.html
Dec 17 2015
prev sibling next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/17/2015 03:50 AM, drug wrote:


 D and C++ [...] But with some data the results differ


You may have similar results between two C and two C++ compilers, even 
between two different versions of the same compiler.

In addition to possible reasons that has already been mentioned, note 
that function arguments can be evaluated in any order by compilers of C 
and C++ (D uses left-to-right). For example, for the following 
expression, gcc evaluates bar() first but dmd (and other D compilers) 
evaluate foo() first:

     h(foo(), bar())

If foo() and bar() have side effects on floating values, those values 
can be observed differently.

Ali
Dec 17 2015
prev sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 17 December 2015 at 11:50:02 UTC, drug wrote:

 I have two implementation of the same algorithm - D and C++ 
 (that is port of D version). I assume that running these 
 implementations on the same data should give the same results 
 from both. But with some data the results differ (5th decimal 
 digit after point). For my purpose it isn't important and I can 
 ignore this difference. But I'd like to have the same results 
 totally (I mean bit to bit) to easy maintanence of the 
 implementations. Question is - can I rely that D and C++ float 
 types operations should give the same result because D and C 
 are similar or no and I should use more portable way and 
 compare floats with some epsilon?


If you aim for float reproducibility, you will get trouble in 
32-bit vs 64-bit mode, depending on whether the FPU is used or 
not. The FPU promotes to 80-bits code that uses float or double, 
and may get you more precision thus changing the results.
Dec 17 2015