www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Strange calculation problem

reply JPF <no spam.whatever> writes:
I've hit a strange problem somewhere in my code and I narowed it down to
the following testcase:
----------------------
module test;
import tango.io.Stdout;

const ulong SIZE_IN_B = (1024 * 1024 * 1024 * 2); /*should be 2147483648*/
void main()
{
	Stdout.formatln("{0}", SIZE_IN_B); /*but is 18446744071562067968*/
}
----------------------

It happens with or without the parenthesis and without the const as well
. Am I doing something wrong, or did I just hit a bug?
Aug 25 2009
next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
JPF wrote:

 I've hit a strange problem somewhere in my code and I narowed it down to
 the following testcase:
 ----------------------
 module test;
 import tango.io.Stdout;
 
 const ulong SIZE_IN_B = (1024 * 1024 * 1024 * 2); /*should be 2147483648*/
 void main()
 {
 	Stdout.formatln("{0}", SIZE_IN_B); /*but is 18446744071562067968*/
 }
 ----------------------
 
 It happens with or without the parenthesis and without the const as well
 . Am I doing something wrong, or did I just hit a bug?


I can reproduce this also with D2:

   enum ulong SIZE_IN_B = (1024 * 1024 * 1024 * 2);
   void main() { writeln(SIZE_IN_B); }

I think what happens is that since integer literals (unless they have a 
type-specific suffix) have the int type. An int will overflow on that 
calculation, and give nonsensical results when it is finally converted 
to ulong.

I think the compiler should be smart enough to figure this out for 
itself, but until that happens you can work around it by suffixing at 
least one of the integer literals with LU, so the compiler interprets 
the entire expression as an ulong:

   const ulong SIZE_IN_B = (1024LU * 1024 * 1024 * 2);

-Lars
Aug 25 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Lars T. Kyllingstad:

 I think the compiler should be smart enough to figure this out for 
 itself, but until that happens you can work around it by suffixing at 
 least one of the integer literals with LU, so the compiler interprets 
 the entire expression as an ulong:


To make this fix happen Walter has to know this problem exists...

Bye,
bearophile
Aug 25 2009
next sibling parent grauzone <none example.net> writes:
bearophile wrote:

 Lars T. Kyllingstad:

 I think the compiler should be smart enough to figure this out for 
 itself, but until that happens you can work around it by suffixing at 
 least one of the integer literals with LU, so the compiler interprets 
 the entire expression as an ulong:


 
 To make this fix happen Walter has to know this problem exists...


And this is why bug reports were invented...


 Bye,
 bearophile
Aug 25 2009
prev sibling parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
bearophile wrote:

 Lars T. Kyllingstad:

 I think the compiler should be smart enough to figure this out for 
 itself, but until that happens you can work around it by suffixing at 
 least one of the integer literals with LU, so the compiler interprets 
 the entire expression as an ulong:


 
 To make this fix happen Walter has to know this problem exists...
 
 Bye,
 bearophile


Ahahahahahahhahahha hahahaha. haha. ha. xD
Thanks, you made my day.


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Aug 25 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 25 Aug 2009 05:35:31 -0400, JPF <no spam.whatever> wrote:


 I've hit a strange problem somewhere in my code and I narowed it down to
 the following testcase:
 ----------------------
 module test;
 import tango.io.Stdout;

 const ulong SIZE_IN_B = (1024 * 1024 * 1024 * 2); /*should be  
 2147483648*/
 void main()
 {
 	Stdout.formatln("{0}", SIZE_IN_B); /*but is 18446744071562067968*/
 }
 ----------------------

 It happens with or without the parenthesis and without the const as well
 . Am I doing something wrong, or did I just hit a bug?



This is expected and well-defined behavior.  It's due to the integer  
promotion rules.

What happens is this:

1024 * 1024 * 1024 * 2 => 0x8000_0000, as an int, that's -2147483648 (note  
the sign bit).

When promoting an int to a ulong, the compiler goes through the following  
steps:

int => long => ulong.

So when going from int to long, it sign-extends the value:

0xffff_ffff_8000_0000 is -2147483648 as a long.

And then changes to ulong, so you get that result.

This is not a bug, and will probably not be changed (it's been this way  
since C).  It is impossible for the compiler to know whether you wanted  
sign extension or not, as some code depends on sign extension.

What you do is clarify to the compiler what you want, as Lars suggested,  
define one of the terms as unsigned.  Promoting from unsigned int to  
unsigned long does not go through a sign extension, so I believe it is  
enough to do:

1024U * 1024 * 1024 * 2.

However, since your intent is to assign to a ulong, the best solution is  
to specify the literal as Lars indicated, this protects against a possible  
future modification which requires a long to store your literal.

-Steve
Aug 25 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 This is not a bug, and will probably not be changed (it's been this way  
 since C).  It is impossible for the compiler to know whether you wanted  
 sign extension or not, as some code depends on sign extension.


I'd like overflow errors bot at compile and run-time, to avoid such traps too.

Bye,
bearophile
Aug 26 2009
prev sibling parent Pelle Månsson <pelle mansson.gmail.com> writes:
You find it well defined and expected that the compiler translate
1024*1024*1024*2
to -1*1024*1024*1024*2?

Why would you not want it to actually become, you know, what you write?

Since it is a ulong that it should fit in, why would you expect the compiler to
follow signed integer overflow rules? I mean, is there any time you actually
want
it to become 18446744071562067968 instead of the expected value?
Oct 20 2009
prev sibling parent JPF <no spam.whatever> writes:
Thanks for the answers. (I think I don't have to say specifying the
literal did work ;-))
Actually I should have known that, but somehow I just didn't think of it.
Aug 25 2009