www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - wrong rounding

reply "Oleg B" <code.viator gmail.com> writes:
Hello. I found unexpected (for me) behavior of rounding double 
values at casting to ulong:

$ cat roundtest.d
import std.stdio;

void main()
{
     double a = 10, b = 0.01;

     writeln( "int:   ", cast(int)(a/b) );
     writeln( "uint:  ", cast(uint)(a/b) );
     writeln( "long:  ", cast(long)(a/b) );
     writeln( "ulong: ", cast(ulong)(a/b) );
}

$ rdmd roundtest.d
int:   1000
uint:  1000
long:  1000
ulong: 999   <----- WTF?? -------

$ dmd --version
DMD64 D Compiler v2.067.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright

$ ldc2 roundtest.d && ./roundtest
int:   1000
uint:  1000
long:  1000
ulong: 1000  <------- Ok --------

$ ldc2 --version
LDC - the LLVM D compiler (0.15.2-beta1):
   based on DMD v2.066.1 and LLVM 3.6.0
   Default target: x86_64-unknown-linux-gnu
   Host CPU: core-avx-i
   http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     x86    - 32-bit X86: Pentium-Pro and above
     x86-64 - 64-bit X86: EM64T and AMD64

$ uname -a

2015 x86_64 x86_64 x86_64 GNU/Linux

have it behavior explain? can I use casting in dmd for rounding 
values or it not safe?
Jun 01 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 5:29 PM, Oleg B wrote:

 Hello. I found unexpected (for me) behavior of rounding double values at
 casting to ulong:

 $ cat roundtest.d
 import std.stdio;

 void main()
 {
      double a = 10, b = 0.01;

      writeln( "int:   ", cast(int)(a/b) );
      writeln( "uint:  ", cast(uint)(a/b) );
      writeln( "long:  ", cast(long)(a/b) );
      writeln( "ulong: ", cast(ulong)(a/b) );
 }

 $ rdmd roundtest.d
 int:   1000
 uint:  1000
 long:  1000
 ulong: 999   <----- WTF?? -------


These are NOT roundings. They are truncations.

Note that for floating point 0.01 is not representable exactly. This is 
the reason you get the error. On other systems, you may not get errors 
for this one case, but you could get errors for other. Please read about 
floating point error: 
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

-Steve
Jun 01 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 1 June 2015 at 21:44:57 UTC, Steven Schveighoffer 
wrote:

 On 6/1/15 5:29 PM, Oleg B wrote:

 Hello. I found unexpected (for me) behavior of rounding double 
 values at
 casting to ulong:

 $ cat roundtest.d
 import std.stdio;

 void main()
 {
     double a = 10, b = 0.01;

     writeln( "int:   ", cast(int)(a/b) );
     writeln( "uint:  ", cast(uint)(a/b) );
     writeln( "long:  ", cast(long)(a/b) );
     writeln( "ulong: ", cast(ulong)(a/b) );
 }

 $ rdmd roundtest.d
 int:   1000
 uint:  1000
 long:  1000
 ulong: 999   <----- WTF?? -------


 These are NOT roundings. They are truncations.

 Note that for floating point 0.01 is not representable exactly. 
 This is the reason you get the error. On other systems, you may 
 not get errors for this one case, but you could get errors for 
 other. Please read about floating point error: 
 http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

 -Steve


Nonetheless, surely in this case the values should be the same 
regardless of the integer target type, no?
Jun 01 2015
next sibling parent Manfred Nowak <svv1999 hotmail.com> writes:
John Colvin wrote:


 regardless of the integer target type


on win64 dmd 2.067.1 gives correct results

int:   999
uint:  999
long:  999
ulong: 999

-manfred
Jun 01 2015
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 6:24 PM, John Colvin wrote:

 Nonetheless, surely in this case the values should be the same
 regardless of the integer target type, no?


Hm... yeah probably :) I didn't grok the example thoroughly.

Makes me curious actually, and that does seem like a bug.

Doing some testing, if you assign a/b to a double and cast, it works. If 
you use a function to truncate taking real as a parameter, it returns 
999 in both ulong and long cases.

The assembly is too much over my head to figure out where the difference is.

I think you should file a codegen bug.

-Steve
Jun 01 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:

 Nonetheless, surely in this case the values should be the same 
 regardless of the integer target type, no?


Not sure... the spec says that the compiler is allowed to do 
intermediate computations at higher precision, but it doesn't say 
it is required to. So, strictly speaking, its at the compiler's 
discretion to use higher precision when rounding to `int` than 
when rounding to `ulong`...
Jun 02 2015
next sibling parent "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
Interestingly dmd computes the a/b with SSE instructions if the 
result is cast to int, uint or long, but uses x87 instructions 
for the division if the result is cast to ulong.
Jun 02 2015
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/2/15 5:53 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:

 On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:

 Nonetheless, surely in this case the values should be the same
 regardless of the integer target type, no?


 Not sure... the spec says that the compiler is allowed to do
 intermediate computations at higher precision, but it doesn't say it is
 required to. So, strictly speaking, its at the compiler's discretion to
 use higher precision when rounding to `int` than when rounding to
 `ulong`...


Right, but there is a difference between ulong and long. These should be 
the same exact code.

Something definitely isn't right.

-Steve
Jun 02 2015