digitalmars.D.learn - accuracy of floating point calculations: d vs cpp
- drug (10/10) Jul 22 2019 I have almost identical (I believe it at least) implementation (D and
- rikki cattermole (2/13) Jul 22 2019 https://godbolt.org/z/EtZLG0
- drug (2/4) Jul 22 2019 hmm, in short - this is my local problem?
- rikki cattermole (4/9) Jul 22 2019 That is not how I would describe it.
- Guillaume Piolat (5/17) Jul 22 2019 Typical floating point operations in single-precision like a
- Guillaume Piolat (7/26) Jul 22 2019 What I would recommend is compute the mean relative error, in
- Dennis (16/19) Jul 22 2019 This likely has little to do with the language, and more with the
- Timon Gehr (6/17) Jul 22 2019 This is probably not your problem, but it may be good to know anyway: D
- =?UTF-8?Q?Ali_=c3=87ehreli?= (9/14) Jul 23 2019 For completeness, at least C++ gives the same freedom to the compiler,
I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.
Jul 22 2019
On 23/07/2019 12:49 AM, drug wrote:I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.https://godbolt.org/z/EtZLG0
Jul 22 2019
22.07.2019 15:53, rikki cattermole пишет:https://godbolt.org/z/EtZLG0hmm, in short - this is my local problem?
Jul 22 2019
rikki cattermole <rikki cattermole.co.nz> On 23/07/2019 12:58 AM, drug wrote:22.07.2019 15:53, rikki cattermole пишет:That is not how I would describe it. I would describe it as IEEE-754 doing what IEEE-754 is good at. But my point is, you can get the results to match up, if you care about it.https://godbolt.org/z/EtZLG0hmm, in short - this is my local problem?
Jul 22 2019
On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.
Jul 22 2019
On Monday, 22 July 2019 at 13:23:26 UTC, Guillaume Piolat wrote:On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.
Jul 22 2019
22.07.2019 16:26, Guillaume Piolat пишет:Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it.Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.
Jul 22 2019
22.07.2019 17:19, drug пишет:22.07.2019 16:26, Guillaume Piolat пишет:There was a typo in C++ implementation. I did simple-n-dirt Python version and after the typo fixed all three implementations show the same result if one filter update occurs. But if several updates happen a subtle difference exists nevertheless, error accumulates somewhere else - time for numerical methods using.Unfortunately error has been turned out to be much bigger than I guessed before. So obviously there is a problem either on D side or on C++ side. Error is too huge to ignore it.Typical floating point operations in single-precision like a simple (a * b) + c will provide a -140dB difference if order is changed. It's likely the order of operations is not the same in your program, so the least significant digit should be different.What I would recommend is compute the mean relative error, in double, and if it's below -200 dB, not bother. This is an incredibly low relative error of 0.00000001%. You will have no difficulty making your D program deterministic, but knowing exactly where the C++ and D differ will be long and serve no purpose.
Jul 22 2019
On Monday, 22 July 2019 at 12:49:24 UTC, drug wrote:Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known?This likely has little to do with the language, and more with the implementation. Basic floating point operations at the same precision should give the same results. There can be differences in float printing (see [1]) and math functions (sqrt, cos, pow etc.) however. Tips for getting consistent results between C/C++ and D: - Use the same backend, so compare DMD with DMC, LDC with CLANG and GDC with GCC. - Use the same C runtime library. On Unix glibc will likely be the default, on Windows you likely use snn.lib, libcmt.lib or msvcrt.dll. - On the D side, use core.stdc.math instead of std.math - Use the same optimizations. (Don't use -ffast-math for C) [1] https://forum.dlang.org/post/fndyoiawueefqoeobdur forum.dlang.org
Jul 22 2019
On 22.07.19 14:49, drug wrote:I have almost identical (I believe it at least) implementation (D and C++) of the same algorithm that uses Kalman filtering. These implementations though show different results (least significant digits). Before I start investigating I would like to ask if this issue (different results of floating points calculation for D and C++) is well known? May be I can read something about that in web? Does D implementation of floating point types is different than the one of C++? Most of all I'm interesting in equal results to ease comparing outputs of both implementations between each other. The accuracy itself is enough in my case, but this difference is annoying in some cases.This is probably not your problem, but it may be good to know anyway: D allows compilers to perform arbitrary "enhancement" of floating-point precision for parts of the computation, including those performed at compile time. I think this is stupid, but I haven't been able to convince Walter.
Jul 22 2019
On 07/22/2019 08:48 PM, Timon Gehr wrote:This is probably not your problem, but it may be good to know anyway: D allows compilers to perform arbitrary "enhancement" of floating-point precision for parts of the computation, including those performed at compile time. I think this is stupid, but I haven't been able to convince Walter.For completeness, at least C++ gives the same freedom to the compiler, right? And if I'm not mistaken, an additional potential problem with floating point is the state of floating point flags: they are used for all floating point computations. If a function sets them for its use and then fails to reset them to the previous state, all further computations will be affected. Ali
Jul 23 2019