digitalmars.D.announce - Phobos unit testing uncovers a CPU bug
- Don (31/31) Nov 26 2010 The code below compiles to a single machine instruction, yet the results...
- %u (7/38) Nov 26 2010 Must have made you smile ;)
- bearophile (11/13) Nov 26 2010 This doesn't work, but it's supposed to work. Add this to bugzilla if it...
- bearophile (3/4) Nov 26 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5280
- so (5/18) Nov 29 2010 Thank you.
- bearophile (4/10) Nov 29 2010 I have a certain stylistic rule regarding how function calls are written...
- Jonathan M Davis (14/28) Nov 29 2010 LOL. I couldn't figure out what was different about those lines (the ext...
- so so.do (5/28) Nov 29 2010 IMHO this is a good example why inconsistencies are sometimes necessary.
- Jonathan M Davis (6/34) Nov 29 2010 If you say so. I don't see any problem with it. I can see why you'd want...
- so (3/7) Nov 29 2010 I am sorry if it seems that way, wasn't my intention.
- Walter Bright (2/3) Nov 26 2010 Hex float literals are supported by D.
- bearophile (4/8) Nov 26 2010 "hex float string" != "Hex float literal".
- Simen kjaeraas (6/20) Nov 26 2010 Great job!
- KennyTM~ (6/28) Nov 27 2010 Intel is correct.
- Walter Bright (6/8) Nov 26 2010 This is awesome work, Don. Kudos to you, David and Dmitry.
- Don (13/23) Nov 26 2010 I believe that would have once been possible, I doubt it's true any more...
- Kagamin (2/6) Nov 27 2010 The same question goes to you. Why do you call this bug?
- Don (3/11) Nov 27 2010 The Intel CPU gives the correct answer, but AMD's is wrong. They should
- Kagamin (3/5) Nov 28 2010 Really? I think, the answer is neither correct nor wrong. It's approxima...
- Mike James (4/14) Nov 28 2010 If it happens once its a bug, if its repeatable its a feature ;-)
- Walter Bright (4/10) Nov 28 2010 The rules for rounding the mathematical value to the representation are ...
- Kagamin (2/6) Nov 29 2010 Well, maybe, but the result fits well in machine precision (which is equ...
- Walter Bright (4/15) Nov 29 2010 How do you decide how many bits should be enough for any algorithm?
- Kagamin (2/6) Nov 30 2010 It's not me, it's the programmer. He was disgusted that his algorithm pr...
- Andrew Wiley (4/15) Nov 30 2010 What programmer? What algorithm? As far as I can tell, this was found wh...
- Kagamin (2/5) Dec 02 2010 Hmm... Really... I've messed this.
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (5/11) Dec 01 2010 This thread is about a bug in a CPU's floating point implementation.
- so (4/5) Nov 29 2010 It is approximate, but approximation is not an "undefined behavior".
- Bruno Medeiros (7/10) Nov 29 2010 And because of that comment, I've once again checked
- Dmitry Olshansky (16/48) Nov 27 2010 Glad to help!
- Lionello Lunesu (4/59) Nov 27 2010 http://forums.amd.com/forum/messageview.cfm?catid=29&threadid=135771
The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. ---- import std.math; void main() { assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2) == 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD } ---- The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are: Intel: 0x1.bba4a9f774f49d0ap+4L AMD: 0x1.bba4a9f774f49d0cp+4L The least significant bit is different. This corresponds only to a fraction of a bit (that is, it's hardly important for accuracy. For comparison, sin and cos on x86 lose nearly sixty bits of accuracy in some cases!). Its importance is only that it is an undocumented difference between manufacturers. The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing. Thanks to David Simcha and Dmitry Olshansky for help in tracking this down.
Nov 26 2010
== Quote from Don (nospam nospam.com)'s articleThe code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. ---- import std.math; void main() { assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2) == 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD } ---- The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are: Intel: 0x1.bba4a9f774f49d0ap+4L AMD: 0x1.bba4a9f774f49d0cp+4L The least significant bit is different. This corresponds only to a fraction of a bit (that is, it's hardly important for accuracy. For comparison, sin and cos on x86 lose nearly sixty bits of accuracy in some cases!). Its importance is only that it is an undocumented difference between manufacturers. The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing. Thanks to David Simcha and Dmitry Olshansky for help in tracking this down.Must have made you smile ;) Slightly related, do you have some code to convert a hex float string to float? I think the hex format is a nice compromise between size and readability. Regarding unit tests, I should really use them :( I use std2 in my D1 project and a few of std2's unit tests fail, so I run my tests() manually..
Nov 26 2010
%u:Slightly related, do you have some code to convert a hex float string to float?This doesn't work, but it's supposed to work. Add this to bugzilla if it's not already present: import std.conv: to; void main() { auto r = to!real("0x1.0076fc5cc7933866p+40L"); auto d = to!double("0x1.0076fc5cc7933866p+40L"); auto f = to!float("0x1.0076fc5cc7933866p+40L"); }Regarding unit tests, I should really use them :(Yep, and DbC too, and compile your D code with -w. Bye, bearophile
Nov 26 2010
This doesn't work, but it's supposed to work. Add this to bugzilla if it's not already present:http://d.puremagic.com/issues/show_bug.cgi?id=5280 Bye, bearophile
Nov 26 2010
import std.conv: to; void main() { auto r = to!real("0x1.0076fc5cc7933866p+40L"); auto d = to!double("0x1.0076fc5cc7933866p+40L"); auto f = to!float("0x1.0076fc5cc7933866p+40L"); }I have an unrelated question, this is not a criticism but an honest one. Why don't you write these 3 lines like:Regarding unit tests, I should really use them :(Yep, and DbC too, and compile your D code with -w. Bye, bearophileauto r = to!real ("0x1.0076fc5cc7933866p+40L"); auto d = to!double("0x1.0076fc5cc7933866p+40L"); auto f = to!float ("0x1.0076fc5cc7933866p+40L");Thank you. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 29 2010
so:I have an unrelated question, this is not a criticism but an honest one. Why don't you write these 3 lines like:I have a certain stylistic rule regarding how function calls are written in my code, and that rule is more important than such alignment. Adding spaces where I don't expect them to be slows down my visual parsing of the code. Generally in my code I've found that such kind of alignment is a waste of my time, if I change something I may need to waste time realigning things. Bye, bearophileauto r = to!real ("0x1.0076fc5cc7933866p+40L"); auto d = to!double("0x1.0076fc5cc7933866p+40L"); auto f = to!float ("0x1.0076fc5cc7933866p+40L");
Nov 29 2010
On Monday, November 29, 2010 12:36:00 bearophile wrote:so:LOL. I couldn't figure out what was different about those lines (the extra space apparently; the fact that the text isn't monospaced makes it harder to see though). But no one's over going to get people to agree on spacing any more than they'll get them to agree on braces. Personally, I generally don't have extraneous spaces but will periodically line up variable declarations with regards to names (which would require no work here because they're all auto and line up anyway). Other folks I know insist on always putting extra spaces after keywords like if and/or putting extra spaces before or after parens. Trying to convince anyone about spacing is a waste of time, and I'd say that suggesting that spacing be altered in code is generally a waste of time unless there's something abnormally bad about it (like 10 spaces after every paren or something ridiculous like that). - Jonathan M DavisI have an unrelated question, this is not a criticism but an honest one. Why don't you write these 3 lines like:I have a certain stylistic rule regarding how function calls are written in my code, and that rule is more important than such alignment. Adding spaces where I don't expect them to be slows down my visual parsing of the code. Generally in my code I've found that such kind of alignment is a waste of my time, if I change something I may need to waste time realigning things.auto r = to!real ("0x1.0076fc5cc7933866p+40L"); auto d = to!double("0x1.0076fc5cc7933866p+40L"); auto f = to!float ("0x1.0076fc5cc7933866p+40L");
Nov 29 2010
LOL. I couldn't figure out what was different about those lines (the extra space apparently; the fact that the text isn't monospaced makes it harder to see though). But no one's over going to get people to agree on spacing any more than they'll get them to agree on braces. Personally, I generally don't have extraneous spaces but will periodically line up variable declarations with regards to names (which would require no work here because they're all auto and line up anyway). Other folks I know insist on always putting extra spaces after keywords like if and/or putting extra spaces before or after parens. Trying to convince anyone about spacing is a waste of time, and I'd say that suggesting that spacing be altered in code is generally a waste of time unless there's something abnormally bad about it (like 10 spaces after every paren or something ridiculous like that). - Jonathan M DavisIMHO this is a good example why inconsistencies are sometimes necessary. I don't put spaces into random places either but this example is crying for attention :) -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 29 2010
On Monday 29 November 2010 21:30:31 so so.do wrote:If you say so. I don't see any problem with it. I can see why you'd want to add the extra spaces (at least, once I put it in an editor with a monospaced font), but I wouldn't have thought that it would be worth calling someone on it. It just seems nitpicky, honestly. - Jonathan M DavisLOL. I couldn't figure out what was different about those lines (the extra space apparently; the fact that the text isn't monospaced makes it harder to see though). But no one's over going to get people to agree on spacing any more than they'll get them to agree on braces. Personally, I generally don't have extraneous spaces but will periodically line up variable declarations with regards to names (which would require no work here because they're all auto and line up anyway). Other folks I know insist on always putting extra spaces after keywords like if and/or putting extra spaces before or after parens. Trying to convince anyone about spacing is a waste of time, and I'd say that suggesting that spacing be altered in code is generally a waste of time unless there's something abnormally bad about it (like 10 spaces after every paren or something ridiculous like that). - Jonathan M DavisIMHO this is a good example why inconsistencies are sometimes necessary. I don't put spaces into random places either but this example is crying for attention :)
Nov 29 2010
but I wouldn't have thought that it would be worth calling someone on it. It just seems nitpicky, honestly. - Jonathan M DavisI am sorry if it seems that way, wasn't my intention. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 29 2010
%u wrote:Slightly related, do you have some code to convert a hex float string to float?Hex float literals are supported by D.
Nov 26 2010
Walter:%u wrote:"hex float string" != "Hex float literal". Bye, bearophileSlightly related, do you have some code to convert a hex float string to float?Hex float literals are supported by D.
Nov 26 2010
Don <nospam nospam.com> wrote:The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing.This sounds like a great sales argument. Gives us some bragging rights. :pThanks to David Simcha and Dmitry Olshansky for help in tracking this down.Great job! Now, which of the results is correct, and has AMD and Intel been informed? -- Simen
Nov 26 2010
On Nov 27, 10 05:25, Simen kjaeraas wrote:Don <nospam nospam.com> wrote:Intel is correct. yl2x(0x1.0076fc5cc7933866p+40L, LN2) == log(9240117798188457011/8388608) == 0x1.bba4a9f774f49d0a64ac5666c969fd8ca8e...p+4 ^The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing.This sounds like a great sales argument. Gives us some bragging rights. :pThanks to David Simcha and Dmitry Olshansky for help in tracking this down.Great job! Now, which of the results is correct, and has AMD and Intel been informed?
Nov 27 2010
Don wrote:The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent.This is awesome work, Don. Kudos to you, David and Dmitry. BTW, I've read that fine-grained CPU detection can be done, beyond what CPUID gives, by examining slight differences in FPU results. I expect that *, +, -, / should all give exactly the same answers. But the transcendentals, and obviously yl2x, vary.
Nov 26 2010
Walter Bright wrote:Don wrote:I believe that would have once been possible, I doubt it's true any more. Basic arithmetic and sqrt all give correctly rounded results, so they're identical on all processors. The 387 gives greatly improved accuracy, compared to the 287. But AFAIK there have not been intentional changes since then. The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions. I haven't seen any examples of values which are calculated differently between the processors. I only found one vague reference in a paper from CERN.The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent.This is awesome work, Don. Kudos to you, David and Dmitry. BTW, I've read that fine-grained CPU detection can be done, beyond what CPUID gives, by examining slight differences in FPU results. I expect that *, +, -, / should all give exactly the same answers. But the transcendentals, and obviously yl2x, vary.
Nov 26 2010
Don Wrote:The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions.The same question goes to you. Why do you call this bug?
Nov 27 2010
Kagamin wrote:Don Wrote:The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result.The great tragedy was that an early AMD processor gave much accurate sin and cos than the 387. But, people complained that it was different from Intel! So, their next processor duplicated Intel's hopelessly wrong trig functions.The same question goes to you. Why do you call this bug?
Nov 27 2010
Don Wrote:The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result.Really? I think, the answer is neither correct nor wrong. It's approximate. If your friend's program operates on ~0x1p+40 values and critically depends on on the value of the last bit, then double precision doesn't suit his needs (on both Intel and AMD), he should take a couple of classes on computational theory and rewrite his algorithm, or use arithmetic with higher precision.
Nov 28 2010
If it happens once its a bug, if its repeatable its a feature ;-) -=mike=- "Kagamin" <spam here.lot> wrote in message news:icth6h$1nqe$1 digitalmars.com...Don Wrote:The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result.Really? I think, the answer is neither correct nor wrong. It's approximate. If your friend's program operates on ~0x1p+40 values and critically depends on on the value of the last bit, then double precision doesn't suit his needs (on both Intel and AMD), he should take a couple of classes on computational theory and rewrite his algorithm, or use arithmetic with higher precision.
Nov 28 2010
Kagamin wrote:Don Wrote:The rules for rounding the mathematical value to the representation are precise, and so there is such a thing as the correctly rounded result and the wrong result. An FPU should strive to always produce the correctly rounded result.The Intel CPU gives the correct answer, but AMD's is wrong. They should both give the correct result.Really? I think, the answer is neither correct nor wrong. It's approximate.
Nov 28 2010
Walter Bright Wrote:Well, maybe, but the result fits well in machine precision (which is equal to the last bit). And if this precision is not enough for the algorithm, then the algorithm is incorrect on any x87 FPU, and its output is garbage.Really? I think, the answer is neither correct nor wrong. It's approximate.The rules for rounding the mathematical value to the representation are precise, and so there is such a thing as the correctly rounded result and the wrong result.
Nov 29 2010
Kagamin wrote:Walter Bright Wrote:How do you decide how many bits should be enough for any algorithm? The thing is, the FPU has 53 bits of precision and so ought to be correct to the last bit.Well, maybe, but the result fits well in machine precision (which is equal to the last bit). And if this precision is not enough for the algorithm, then the algorithm is incorrect on any x87 FPU, and its output is garbage.Really? I think, the answer is neither correct nor wrong. It's approximate.The rules for rounding the mathematical value to the representation are precise, and so there is such a thing as the correctly rounded result and the wrong result.
Nov 29 2010
Walter Bright Wrote:How do you decide how many bits should be enough for any algorithm? The thing is, the FPU has 53 bits of precision and so ought to be correct to the last bit.It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision.
Nov 30 2010
On Tue, Nov 30, 2010 at 1:43 PM, Kagamin <spam here.lot> wrote:Walter Bright Wrote:What programmer? What algorithm? As far as I can tell, this was found when testing a library explicitly for accuracy, not in an application, so your argument doesn't apply.How do you decide how many bits should be enough for any algorithm? The thing is, the FPU has 53 bits of precision and so ought to be correctto thelast bit.It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision.
Nov 30 2010
Andrew Wiley Wrote:What programmer? What algorithm? As far as I can tell, this was found when testing a library explicitly for accuracy, not in an application, so your argument doesn't apply.Hmm... Really... I've messed this.
Dec 02 2010
Kagamin wrote:It's not me, it's the programmer. He was disgusted that his algorithm produced garbage, which means, the error was unacceptable. Mat be it was 1%, may be 80%, I don't, that was his decision, that the result was unacceptable. The bug description assumes the problem was in the last bit, which means, he wanted precision higher than the machine precision.This thread is about a bug in a CPU's floating point implementation. Regardless of which bit of the floating point representation is affected, it's a bug. Ali
Dec 01 2010
The same question goes to you. Why do you call this bug?It is approximate, but approximation is not an "undefined behavior". It is same as "2 + 1 = 4". -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 29 2010
On 27/11/2010 06:26, Don wrote:I haven't seen any examples of values which are calculated differently between the processors. I only found one vague reference in a paper from CERN.And because of that comment, I've once again checked http://hasthelargehadroncolliderdestroyedtheworldyet.com/ , just to make sure... :P CERN better be aware of that stuff! :D -- Bruno Medeiros - Software Engineer
Nov 29 2010
On 26.11.2010 23:02, Don wrote:The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. ---- import std.math; void main() { assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2) == 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD } ---- The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are: Intel: 0x1.bba4a9f774f49d0ap+4L AMD: 0x1.bba4a9f774f49d0cp+4L The least significant bit is different. This corresponds only to a fraction of a bit (that is, it's hardly important for accuracy. For comparison, sin and cos on x86 lose nearly sixty bits of accuracy in some cases!). Its importance is only that it is an undocumented difference between manufacturers. The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing. Thanks to David Simcha and Dmitry Olshansky for help in tracking this down.Glad to help! I was genuinely intrigued because not more then a few weeks ago I discussed with a friend of mine a possibility of differences in FP calculations of AMD vs Intel. You see, his scientific app yielded different results when working at home/at work, which is a frustrating experience. Since that's exactly same binary, written in Delphi (no C run-time involved and so on) and environment is pretty much the same... I suggested to check CPU vendors just in case... of course, different. In the meantime, I sort of "ported" the test case to M$ c++ inline asm and posted it on AMD forums, let's see what they have to say. http://forums.amd.com/forum/messageview.cfm?catid=319&threadid= 42893&enterthread=y <http://forums.amd.com/forum/messageview.cfm?catid=319&threadid=142893&enterthread=y> -- Dmitry Olshansky
Nov 27 2010
On 28-11-2010 5:49, Dmitry Olshansky wrote:On 26.11.2010 23:02, Don wrote:http://forums.amd.com/forum/messageview.cfm?catid=29&threadid=135771 This post also talks about a fyl2x bug. Wonder if it's the same bug. L.The code below compiles to a single machine instruction, yet the results are CPU manufacturer-dependent. ---- import std.math; void main() { assert( yl2x(0x1.0076fc5cc7933866p+40L, LN2) == 0x1.bba4a9f774f49d0ap+4L); // Pass on Intel, fails on AMD } ---- The results for yl2x(0x1.0076fc5cc7933866p+40L, LN2) are: Intel: 0x1.bba4a9f774f49d0ap+4L AMD: 0x1.bba4a9f774f49d0cp+4L The least significant bit is different. This corresponds only to a fraction of a bit (that is, it's hardly important for accuracy. For comparison, sin and cos on x86 lose nearly sixty bits of accuracy in some cases!). Its importance is only that it is an undocumented difference between manufacturers. The difference was discovered through the unit tests for the mathematical Special Functions which will be included in the next compiler release. Discovery of the discrepancy happened only because of several features of D: - built-in unit tests (encourages tests to be run on many machines) - built-in code coverage (the tests include extreme cases, simply because I was trying to increase the code coverage to high values) - D supports the hex format for floats. Without this feature, the discrepancy would have been blamed on differences in the floating-point conversion functions in the C standard library. This experience reinforces my belief that D is an excellent language for scientific computing. Thanks to David Simcha and Dmitry Olshansky for help in tracking this down.Glad to help! I was genuinely intrigued because not more then a few weeks ago I discussed with a friend of mine a possibility of differences in FP calculations of AMD vs Intel. You see, his scientific app yielded different results when working at home/at work, which is a frustrating experience. Since that's exactly same binary, written in Delphi (no C run-time involved and so on) and environment is pretty much the same... I suggested to check CPU vendors just in case... of course, different. In the meantime, I sort of "ported" the test case to M$ c++ inline asm and posted it on AMD forums, let's see what they have to say. http://forums.amd.com/forum/messageview.cfm?catid=319&threadid=142893&enterthread=y <http://forums.amd.com/forum/messageview.cfm?catid=319&threadid=142893&enterthread=y>
Nov 27 2010