digitalmars.D.bugs - / operator for float values
- Thomas Kuehne (9/9) Oct 13 2004 code:
- Burton Radons (15/26) Oct 13 2004 This is because the first comparison uses float while the second uses
- Thomas Kuehne (3/21) Oct 13 2004 Thanks for the fix - guess I have to read a little about IEEE :)
- Lynn Allan (4/7) Oct 13 2004 Thanks for the time and effort the two of you are putting into making
code: The first assertion passes but the second fails. test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d
Oct 13 2004
Thomas Kuehne wrote:code: The first assertion passes but the second fails. test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.dThis is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion. The real bug is that this doesn't work properly: assert(f == cast(float) (a/b)); This should cause the value to be truncated, but it doesn't. You can cause your code to succeed if you eliminate the rounding error. Divide by a power of 2 instead: float f = 0.01171875; float a = 12.0; float b = 1024.0; This will then work. That could probably be considered a correct test of the language, as D depends on IEEE being present.
Oct 13 2004
Burton Radons schrieb:Thanks for the fix - guess I have to read a little about IEEE :) Thomascode: The first assertion passes but the second fails. test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.dThis is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion. The real bug is that this doesn't work properly: assert(f == cast(float) (a/b)); This should cause the value to be truncated, but it doesn't.
Oct 13 2004
Thanks for the time and effort the two of you are putting into making 'D' an even better tool. "Burton Radons" <burton-radons smocky.com> wrote in message news:ckj8ps$2te8$1 digitaldaemon.com...Thomas Kuehne wrote:code:
Oct 13 2004