digitalmars.D.learn - cast(int) getting an unexpected number
- Joel Christensen (10/10) Nov 04 2009 Input:
- Steven Schveighoffer (15/25) Nov 04 2009 hehe. .01 is not exactly represented by float, because it's stored as a...
- rmcguire (17/52) Nov 04 2009 why is this not a compiler bug?
- Michal Minich (8/25) Nov 04 2009 I think this may be case of:
- Lars T. Kyllingstad (11/34) Nov 04 2009 Yes, if you do this:
- Charles Hixson (4/38) Nov 04 2009 Can that *really* be the explanation?? I know that float doesn't have
- Lars T. Kyllingstad (21/66) Nov 04 2009 Yes it does, but that's not what matters.
- rmcguire (6/80) Nov 04 2009 Hmm, thanks.
- Lars T. Kyllingstad (32/110) Nov 04 2009 You don't, really. You just stay away from casts. :) IMHO, casts are
- rmcguire (4/128) Nov 05 2009 err, right.
- Ali Cehreli (6/21) Nov 04 2009 f*100f is represented to be less than 0.
- Ali Cehreli (7/22) Nov 04 2009 Just an information: 0.01 is double, there is type conversion there.
- Joel Christensen (5/10) Nov 04 2009 Thanks every one for the replies.
- Lars T. Kyllingstad (10/22) Nov 04 2009 If the values you are working with are monetary, I don't think it makes
- Steven Schveighoffer (26/35) Nov 05 2009 Might I suggest using integers instead of floating point.
- Charles Hixson (21/31) Nov 05 2009 If you're working with an accounting function, you should use bucket
- Joel Christensen (15/15) Nov 05 2009 Seems random. I've got it working though, changed real to float. I seem
Input: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); } Output: 0.01->0 Comment: What!?
Nov 04 2009
On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz gmail.com> wrote:Input: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); } Output: 0.01->0 Comment: What!?hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal. If you imagined that the computer stored things in decimal, see how the 1/3 example would work a = 1.0/3; // truncated to 0.3333333 a *= 3; // 0.9999999 auto i = cast(int)a; // 0 This is analagous to what you are asking the compiler to do. To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -Steve
Nov 04 2009
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz gmail.com> wrote:why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryInput: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); } Output: 0.01->0 Comment: What!?hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal. If you imagined that the computer stored things in decimal, see how the 1/3 example would work a = 1.0/3; // truncated to 0.3333333 a *= 3; // 0.9999999 auto i = cast(int)a; // 0 This is analagous to what you are asking the compiler to do. To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -Steve
Nov 04 2009
Hello rmcguire,why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time. http://www.digitalmars.com/d/2.0/function.html#interpretation It seems that .01*100f is computed at compile time. try declarting float f as const or invariant and see what happens (I'm not sure..)
Nov 04 2009
Michal Minich wrote:Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 04 2009
Lars T. Kyllingstad wrote:Michal Minich wrote:Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 04 2009
Charles Hixson wrote:Lars T. Kyllingstad wrote:Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -LarsMichal Minich wrote:Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 04 2009
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:Charles Hixson wrote:Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator. -RoryLars T. Kyllingstad wrote:Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -LarsMichal Minich wrote:Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 04 2009
rmcguire wrote:"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:You don't, really. You just stay away from casts. :) IMHO, casts are only good for hammering a value of one type into a different type, no worries about the result. I (almost) never use cast(int) to round floating-point numbers. There are two reasons for this: 1. It always rounds towards zero, which is usually not what I want. 2. It doesn't perform any checks, and can easily overflow. If the rounding mode is not important, or if for some reason I want the same result as cast(int), I use std.conv.to(). This function checks that the converted number fits in the target type and then performs an ordinary cast. (Note: D2 only!) import std.conv; void main() { int i = to!int(1.23); // OK assert (i == 1); int j = to!int(real.max); // ConvOverflowError } If "natural" rounding, i.e. towards nearest integer, is what I want, then I use std.math.lround() or std.math.round(). There are a lot of rounding functions in std.math that do different things and that are worth checking out. import std.math; void main() { long i = lround(1.23); assert (i == 1); long j = lround(0.5); assert (j == 1); } -LarsCharles Hixson wrote:Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator.Lars T. Kyllingstad wrote:Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -LarsMichal Minich wrote:Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 04 2009
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:rmcguire wrote:err, right. good point, hehe."Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:You don't, really. You just stay away from casts. :) IMHO, casts are only good for hammering a value of one type into a different type, no worries about the result. I (almost) never use cast(int) to round floating-point numbers. There are two reasons for this: 1. It always rounds towards zero, which is usually not what I want. 2. It doesn't perform any checks, and can easily overflow. If the rounding mode is not important, or if for some reason I want the same result as cast(int), I use std.conv.to(). This function checks that the converted number fits in the target type and then performs an ordinary cast. (Note: D2 only!) import std.conv; void main() { int i = to!int(1.23); // OK assert (i == 1); int j = to!int(real.max); // ConvOverflowError } If "natural" rounding, i.e. towards nearest integer, is what I want, then I use std.math.lround() or std.math.round(). There are a lot of rounding functions in std.math that do different things and that are worth checking out. import std.math; void main() { long i = lround(1.23); assert (i == 1); long j = lround(0.5); assert (j == 1); } -LarsCharles Hixson wrote:Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator.Lars T. Kyllingstad wrote:Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -LarsMichal Minich wrote:Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.Hello rmcguire,Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Larswhy is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -RoryI think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Nov 05 2009
rmcguire Wrote:why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01;results in: 0.01->00.01 is double, there is type conversion there.writefln("%0.2f->%d",f,cast(int)(f*100f));results in: 0.01->1f*100f is represented to be less than 0.writefln("%0.2f->%d",f,cast(int)(.01*100f));.01 * 100f is double and apparently represented as something more than 1.writefln("%0.2f->%f",f,(f*100f));results in: 0.01->1.000000Someting that is less than 0 is being "printed" as 1 at that precision. Printing with a higher precision should print something less than 0. Ali
Nov 04 2009
Ok, I messed up my previous comment while moving rmcguire's lines around. Trying again... Also, I am not sure about the last bit now. :) rmcguire Wrote:why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01;Just an information: 0.01 is double, there is type conversion there.writefln("%0.2f->%d",f,cast(int)(f*100f)); results in: 0.01->0f*100f is represented to be less than 0.writefln("%0.2f->%d",f,cast(int)(.01*100f)); results in: 0.01->1.01 * 100f is double and apparently represented as something more than 1.writefln("%0.2f->%f",f,(f*100f)); results in: 0.01->1.000000If it's like in C, floating point values must be passed as double arguments to functions. So there is float-to-double conversion before the function call, effectively representing the argument as double. Ali
Nov 04 2009
To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -SteveThanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Nov 04 2009
Joel Christensen wrote:If the values you are working with are monetary, I don't think it makes sense to convert to int by casting, since casting always rounds towards zero. Instead you should use the std.math.lround() function, which rounds in a more natural way (towards the nearest integer, away from zero if the fractional part is exactly 0.5). To answer your question, this is how you would get and use the epsilon value in your example: cast(int)(f*100f + float.epsilon); -LarsTo be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -SteveThanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Nov 04 2009
On Wed, 04 Nov 2009 19:38:45 -0500, Joel Christensen <joelcnz gmail.com> wrote:Might I suggest using integers instead of floating point. My experience with floating point is limited, but I've been burned enough to avoid using it wherever possible. To be fair, there are certainly many cases where floating point is useful, like in statistics, but when I can use integers, I do. Translating money into integers is as easy as using the lowest denomination as 1. For example, if you always store your money in cents, then $1 is simply stored as 100. If you want to add tax (say 5%), you do x * 5 / 100. If you want to do rounding, then it's a little trickier, but not too bad: (x * 5 + 50) / 100. The trick is to add half the value of the denominator before dividing to do exact rounding. I've written lots of code that uses decimal numbers without ever using floating point. It's also fun to experiment with fraction types, which are more exact when you are dealing with rational numbers. I once had a very comprehensive C++ fraction type that used a templated base type for the numerator/denominator. I even ported it to Java using BigInt as the numerator/denominator types. It could do very exact calculations with pretty useable speed. This was only in algorithmic coding competitions, I've never had a real world use for them (though I know they exist). the work for you. It would be a nice addition to D. Hope this helps. -SteveTo be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -SteveThanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Nov 05 2009
Joel Christensen wrote:If you're working with an accounting function, you should use bucket rounding. I don't think there's any standard routine for it, but basically first you round, and then you accumulate the error created by rounding in a separate variable. Then when you've processed the "column" or "batch" of numbers, you take the accumulated error and redistribute it over the numbers in a way that makes sense to you. (Perhaps proportional to the size of the number, or some such.) The exact procedure for redistributing the error is case specific, which is probably why there aren't any standard functions to handle this, but uniform distribution rarely works, because the goal is that the totals precisely equal the value. (Note that for this you don't want to use floating point values, but rather fixed point values...which can be emulated with integers that are converted to floats when it's time to export them.) Remember, accounting routines aren't allowed to lose or generate any extra pennies. If you want to say "That's a silly way to thing about numbers", I'll agree with you. But the accountants who wrote the rules back in the 1800's (or earlier?) didn't use computers. They used abacuses, and pen and ink. And preventing cheating in various forms was one of their primary goals.To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -SteveThanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Nov 05 2009
Seems random. I've got it working though, changed real to float. I seem to get different results with my little test program though. Here's from my full program. (I think there's a function in phobos to do this any way): char[] cashToString(dub money,char[] paddingForCents="".dup) { char[] amount; if ( money<1f ) { float cents=money*100f; amount=format("%s%d",paddingForCents,cast(int)cents); } else amount=format("%0.2f",money); return format("%s%s%s", money>=1f ? "$" : "", amount, money<1f ? "c" : ""); }
Nov 05 2009