www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - double trouble

reply "John C" <johnch_atms hotmail.com> writes:
I'm writing routines for numeric formatting and this involves round-tripping 
of floating-point numbers. I'm storing the integral and fractional parts in 
a string buffer, and in a separate variable storing the decimal point 
position, and these are converted back into a double when needed. 
Unfortunately the algorithm I'm using is too lossy and as a result a number 
like 2.78 (stored in the buffer as "278", with the decimal pos being 1) 
becomes 2.7799999999998.

I believe the problem is with my scaling code. Any ideas to increase the 
accuracy?

  // v is a numeric representation of the digits, eg 278.
  // exponent is either the exponent for scientific numbers, or the number 
of fractional digits, eg 2.
  double p10 = 10.0;
  int n = exponent;
  if (n < 0)
    n = -n;
  while (n) {
    if (n & 1) {
      if (exponent < 0)
        v /= p10;
      else
        v *= p10;
    }
    n >>= 1;
    p10 *= p10;
  }
Feb 25 2006
next sibling parent "John C" <johnch_atms hotmail.com> writes:
 I believe the problem is with my scaling code. Any ideas to increase the 
 accuracy?


Scratch that. It must be the way I'm converting the digits to a double that 
introduces the inaccuracy.

    // pDigits is a pointer into a zero-terminated buffer (with the digits 
"278").
    double v = 0.0;
    while (*pDigits) {
        v = v * 10 + (*pDigits - '0');
        pDigits++;
    }

v is now 277999999999999.999990. Then it's scaled using the code previously 
posted to get 2.7799999999999998. How would I round this up to the original 
2.78?
Feb 25 2006
prev sibling next sibling parent reply "Lionello Lunesu" <lio remove.lunesu.com> writes:
I would get rid of the while, or at least the /= and *= therein, these will 
only add to the inaccuracy. Better keep track of integers (++n, --n) and use 
pow when you need the 10^n or so. Remember that the 10.0 is already 
inaccurate (imagine it's 9.999 or so), using it multiple times will 
definately add to the inaccuracy.

L.
Mar 01 2006
parent xs0 <xs0 xs0.com> writes:
Lionello Lunesu wrote:

 I would get rid of the while, or at least the /= and *= therein, these will 
 only add to the inaccuracy. Better keep track of integers (++n, --n) and use 
 pow when you need the 10^n or so. Remember that the 10.0 is already 
 inaccurate (imagine it's 9.999 or so), using it multiple times will 
 definately add to the inaccuracy.
 
 L. 
 
 


Actually, 10.0 is accurately representable, but 0.1 isn't..
Perhaps the most accurate way to calculate decimal digits would be

digit/pow(10.0, -exp)

instead of digit*pow(0.1, exp), avoiding exponentiating an inaccurate 
number..


Well, I went ahead and tested it (in Java, but the results should match 
D's double). Using digit*constant (0.000...0001) and digit/pow(10, exp) 
produce about the same results, while repeatedly multiplying with 0.1 
and exponentiation of 0.1 also produce about the same results, except 
the latter two are slightly more off.

http://www.xs0.com/prec/


xs0
Mar 01 2006
prev sibling parent reply Don Clugston <dac nospam.com.au> writes:
John C wrote:

 I'm writing routines for numeric formatting and this involves round-tripping 
 of floating-point numbers. I'm storing the integral and fractional parts in 
 a string buffer, and in a separate variable storing the decimal point 
 position, and these are converted back into a double when needed. 
 Unfortunately the algorithm I'm using is too lossy and as a result a number 
 like 2.78 (stored in the buffer as "278", with the decimal pos being 1) 
 becomes 2.7799999999998.


The trick to maximal efficiency in these conversions is to make sure 
that you only do ONE division (because that's the point at which the 
rounding error occurs). Don't divide by 10 every time, and definitely 
don't use pow. Instead, keep track of a denominator, and every time 
you'd do a v/=10, do denominator*=10 instead. Then, right at the end, 
divide v by denominator.
The reason this works is that integers can be exactly represented in 
reals, so the multiplies aren't introducing any error. But every time 
you divide by something that isn't a multiple of 2, a tiny roundoff 
error creeps in.
You'll also reduce the error if you use
real v=0.0;
instead of double. Even if you ultimately want a double.


 
 I believe the problem is with my scaling code. Any ideas to increase the 
 accuracy?
 
   // v is a numeric representation of the digits, eg 278.
   // exponent is either the exponent for scientific numbers, or the number 
 of fractional digits, eg 2.
   double p10 = 10.0;
   int n = exponent;
   if (n < 0)
     n = -n;
   while (n) {
     if (n & 1) {
       if (exponent < 0)
         v /= p10;
       else
         v *= p10;
     }
     n >>= 1;
     p10 *= p10;
   }
Mar 01 2006
parent "John C" <johnch_atms hotmail.com> writes:
"Don Clugston" <dac nospam.com.au> wrote in message 
news:du48kq$i5i$1 digitaldaemon.com...

 The trick to maximal efficiency in these conversions is to make sure that 
 you only do ONE division (because that's the point at which the rounding 
 error occurs). Don't divide by 10 every time, and definitely don't use 
 pow. Instead, keep track of a denominator, and every time you'd do a 
 v/=10, do denominator*=10 instead. Then, right at the end, divide v by 
 denominator.
 The reason this works is that integers can be exactly represented in 
 reals, so the multiplies aren't introducing any error. But every time you 
 divide by something that isn't a multiple of 2, a tiny roundoff error 
 creeps in.
 You'll also reduce the error if you use
 real v=0.0;
 instead of double. Even if you ultimately want a double.


Thanks. Working on the raw bits (actually a 64-bit integer) eventually 
proved easier (relatively speaking).
Mar 01 2006