www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is this reasonable?

reply "Steve Teale" <steve.teale britseyeview.com> writes:
Here I feel like a beginner, but it seems very unfriendly:

import std.stdio;

struct ABC
{
    double a;
    int b;
    bool c;
}

ABC[20] aabc;

void foo(int n)
{
    writefln("n: %d, aabc.length: %d", n, aabc.length);
    if (n < aabc.length)
       writeln("A");
    else
       writeln("B");
}

void main(string[] args)
{
    int n = -1;
    foo(n);
}

This comes back with "B".

If I change the test to (n < cast(int) aabc.length), then all is 
well.

Is this unavoidable, or could the compiler safely make the 
conversion implicitly?
Dec 05 2013
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 05, 2013 at 06:15:37PM +0100, Steve Teale wrote:
 Here I feel like a beginner, but it seems very unfriendly:
 
 import std.stdio;
 
 struct ABC
 {
    double a;
    int b;
    bool c;
 }
 
 ABC[20] aabc;
 
 void foo(int n)
 {
    writefln("n: %d, aabc.length: %d", n, aabc.length);
    if (n < aabc.length)
       writeln("A");
    else
       writeln("B");
 }
 
 void main(string[] args)
 {
    int n = -1;
    foo(n);
 }
 
 This comes back with "B".
 
 If I change the test to (n < cast(int) aabc.length), then all is
 well.
 
 Is this unavoidable, or could the compiler safely make the
 conversion implicitly?
Comparing a signed value to an unsigned value is a risky operation. You should always compare values of like signedness, otherwise you'll run into problems like this. You can't compare -1 to an unsigned value because if that unsigned value happens to be uint.max, then there is no machine instruction that will give the correct result (the compiler would have to substitute the code with something like: uint y; if (x < 0 || cast(uint)x < y) { ... } which will probably introduce undesirable overhead. The compiler also can't automatically convert aabc.length to int, because if the length is greater than int.max (which is half of uint.max), the conversion would produce a wrong negative value instead, and the comparison will fail. So, comparing a signed value to an unsigned value is a dangerous, error-prone operation. Sadly, dmd doesn't warn about such risky operations; it just silently casts the values. Bearophile has often complained about this, and I'm starting to agree. This is one of the places in D where subtle mistakes can creep in and the compiler fails to warn the programmer of it. T -- Real Programmers use "cat > a.out".
Dec 05 2013
parent "eles" <eles eles.com> writes:
On Thursday, 5 December 2013 at 17:44:18 UTC, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 06:15:37PM +0100, Steve Teale wrote:
 Here I feel like a beginner, but it seems very unfriendly:
 
 import std.stdio;
 
 struct ABC
 {
    double a;
    int b;
    bool c;
 }
 
 ABC[20] aabc;
 
 void foo(int n)
 {
    writefln("n: %d, aabc.length: %d", n, aabc.length);
    if (n < aabc.length)
       writeln("A");
    else
       writeln("B");
 }
 
 void main(string[] args)
 {
    int n = -1;
    foo(n);
 }
 
 This comes back with "B".
 
 If I change the test to (n < cast(int) aabc.length), then all 
 is
 well.
 
 Is this unavoidable, or could the compiler safely make the
 conversion implicitly?
Comparing a signed value to an unsigned value is a risky operation. You should always compare values of like signedness, otherwise you'll run into problems like this. You can't compare -1 to an unsigned value because if that unsigned value happens to be uint.max, then there is no machine instruction that will give the correct result (the compiler would have to substitute the code with something like: uint y; if (x < 0 || cast(uint)x < y) { ... } which will probably introduce undesirable overhead. The compiler also can't automatically convert aabc.length to int, because if the length is greater than int.max (which is half of uint.max), the conversion would produce a wrong negative value instead, and the comparison will fail. So, comparing a signed value to an unsigned value is a dangerous, error-prone operation. Sadly, dmd doesn't warn about such risky operations; it just silently casts the values. Bearophile has often complained about this, and I'm starting to agree. This is one
me too, me too, me too
Dec 05 2013
prev sibling next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 5 December 2013 at 17:15:39 UTC, Steve Teale wrote:
 Is this unavoidable, or could the compiler safely make the 
 conversion implicitly?
It is example of notorious phenomemena called "integer promotions" and "usual arithmetic conversions". It is unavoidable given Walter's decision to keep this C stuff in D.
Dec 05 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, December 05, 2013 19:16:29 Maxim Fomin wrote:
 On Thursday, 5 December 2013 at 17:15:39 UTC, Steve Teale wrote:
 Is this unavoidable, or could the compiler safely make the
 conversion implicitly?
It is example of notorious phenomemena called "integer promotions" and "usual arithmetic conversions". It is unavoidable given Walter's decision to keep this C stuff in D.
To be fair, you can't solve the problem automatically. It's fundamentally wrong to compare signed and unsigned values, and doing either the conversion to unsigned or to signed could be wrong (or both could be wrong), depending on the values. The best that could be done would be to warn about the comparison or to make it an error. http://d.puremagic.com/issues/show_bug.cgi?id=259 - Jonathan M Davis
Dec 05 2013
next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 5 December 2013 at 18:26:48 UTC, Jonathan M Davis 
wrote:
 On Thursday, December 05, 2013 19:16:29 Maxim Fomin wrote:
 On Thursday, 5 December 2013 at 17:15:39 UTC, Steve Teale 
 wrote:
 Is this unavoidable, or could the compiler safely make the
 conversion implicitly?
It is example of notorious phenomemena called "integer promotions" and "usual arithmetic conversions". It is unavoidable given Walter's decision to keep this C stuff in D.
To be fair, you can't solve the problem automatically. It's fundamentally wrong to compare signed and unsigned values, and doing either the conversion to unsigned or to signed could be wrong (or both could be wrong), depending on the values. The best that could be done would be to warn about the comparison or to make it an error. http://d.puremagic.com/issues/show_bug.cgi?id=259 - Jonathan M Davis
Warning would be an option, but AFAIK Walter does not like warnings, so it is unlikely to be implemented.
Dec 05 2013
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/05/2013 07:26 PM, Jonathan M Davis wrote:
 The best that could be done would be to warn about the comparison
 or to make it an error.

 http://d.puremagic.com/issues/show_bug.cgi?id=259
...
The best that could be done would arguably be to simply do the comparison the 'right' way. E.g. static assert(-1 < 0u).
Dec 05 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 05, 2013 at 10:06:55PM +0100, Timon Gehr wrote:
 On 12/05/2013 07:26 PM, Jonathan M Davis wrote:
The best that could be done would be to warn about the comparison
or to make it an error.

http://d.puremagic.com/issues/show_bug.cgi?id=259
...
The best that could be done would arguably be to simply do the comparison the 'right' way. E.g. static assert(-1 < 0u).
What's the 'right' way? assert(-1 < uint.max) will always fail because no matter whether you convert to int or uint, the comparison simply cannot be carried out at the machine code level. The only way you can get a sane answer out of this is if the compiler translates it into: if (intVar < 0 || cast(uint)intVar < uintVar) { ... } T -- BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL
Dec 05 2013
next sibling parent "Dominikus Dittes Scherkl" writes:
 The best that could be done would arguably be to simply do the
 comparison the 'right' way. E.g. static assert(-1 < 0u).
What's the 'right' way? assert(-1 < uint.max) will always fail because no matter whether you convert to int or uint, the comparison simply cannot be carried out at the machine code level.
But the compiler should be able to do such things at compile time. If you compare an unsigned variable with a negative literal, "-1 < x" or "x > -1": the whole expression can be rewritten to "true", "-1 == x" or "-1 > x" or "-1 >= x" etc. can be rewritten to "false". This is not only no performance problem but instead extremely efficient!
Dec 06 2013
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/06/2013 02:06 AM, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 10:06:55PM +0100, Timon Gehr wrote:
 On 12/05/2013 07:26 PM, Jonathan M Davis wrote:
 The best that could be done would be to warn about the comparison
 or to make it an error.

 http://d.puremagic.com/issues/show_bug.cgi?id=259
 ...
The best that could be done would arguably be to simply do the comparison the 'right' way. E.g. static assert(-1 < 0u).
What's the 'right' way? assert(-1 < uint.max) will always fail because no matter whether you convert to int or uint, the comparison simply cannot be carried out at the machine code level. The only way you can get a sane answer out of this is if the compiler translates it into: if (intVar < 0 || cast(uint)intVar < uintVar) { ... } T
Yup, that's the 'right' way.
Dec 06 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Thursday, 5 December 2013 at 18:26:48 UTC, Jonathan M Davis
wrote:
 On Thursday, December 05, 2013 19:16:29 Maxim Fomin wrote:
 On Thursday, 5 December 2013 at 17:15:39 UTC, Steve Teale 
 wrote:
 the values. The best that could be done would be to warn about 
 the comparison
 or to make it an error.

 http://d.puremagic.com/issues/show_bug.cgi?id=259
If Walter agrees...
Dec 05 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 5 December 2013 at 18:26:48 UTC, Jonathan M Davis 
wrote:
 To be fair, you can't solve the problem automatically. It's 
 fundamentally
 wrong to compare signed and unsigned values, and doing either 
 the conversion
 to unsigned or to signed could be wrong (or both could be 
 wrong), depending on
 the values.
*Actually*... that's not exactly true. std.algorithm.max and std.algorithm.min actually do it perfectly well. It is completely feaseable to imagine having an: std.functional.less(T, U)(T t, U u){} Which could work for *any* (compatible) type, including those with mixed signed-ness. The only "issue" is that if the signs *are* mixed, the comparison is a *bit* more costly. It involved *two* comparisons, and a boolean condition: https://github.com/D-Programming-Language/phobos/blob/ab34fb92addca61755474df04a0d0d6e0f1148a2/std/algorithm.d#L6698 Truth be told, "a < b" could work correctly 100% of the time, no question asked, if the *language* wanted to. The (2) problems would be: 1) Breaks backward compatibility with C and C++. 2) Higher (and hidden) comparison costs. Try to convince Walter that these are worth it XD !
Dec 06 2013
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, December 06, 2013 18:28:09 monarch_dodra wrote:
 On Thursday, 5 December 2013 at 18:26:48 UTC, Jonathan M Davis
 
 wrote:
 To be fair, you can't solve the problem automatically. It's
 fundamentally
 wrong to compare signed and unsigned values, and doing either
 the conversion
 to unsigned or to signed could be wrong (or both could be
 wrong), depending on
 the values.
*Actually*... that's not exactly true. std.algorithm.max and std.algorithm.min actually do it perfectly well. It is completely feaseable to imagine having an: std.functional.less(T, U)(T t, U u){} Which could work for *any* (compatible) type, including those with mixed signed-ness. The only "issue" is that if the signs *are* mixed, the comparison is a *bit* more costly. It involved *two* comparisons, and a boolean condition: https://github.com/D-Programming-Language/phobos/blob/ab34fb92addca61755474d f04a0d0d6e0f1148a2/std/algorithm.d#L6698 Truth be told, "a < b" could work correctly 100% of the time, no question asked, if the *language* wanted to. The (2) problems would be: 1) Breaks backward compatibility with C and C++. 2) Higher (and hidden) comparison costs. Try to convince Walter that these are worth it XD !
Ah, good point. I was assuming that you would just doing a straight up comparison, in which case, you have to pick either signed or unsigned, whereas if you can convert both ways and do both comparisons, that does change things a bit. But still, I wouldn't expect Walter to ever agree to something like that, and honestly, I think that making it an error and requiring the programmer to deal with it is still a better option anyway. They can then use a function to do the comparison that you're describing when they actually want that and thus only incur the cost when that's what they really want. The key is just to make sure that you don't silently do the single conversion and comparison and end up with a bug. How to best do the comparison depends on what you're trying to do. - Jonathan M Davis
Dec 06 2013
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 12/5/13 2:15 PM, Steve Teale wrote:
 Here I feel like a beginner, but it seems very unfriendly:

 import std.stdio;

 struct ABC
 {
     double a;
     int b;
     bool c;
 }

 ABC[20] aabc;

 void foo(int n)
 {
     writefln("n: %d, aabc.length: %d", n, aabc.length);
     if (n < aabc.length)
        writeln("A");
     else
        writeln("B");
 }

 void main(string[] args)
 {
     int n = -1;
     foo(n);
 }

 This comes back with "B".

 If I change the test to (n < cast(int) aabc.length), then all is well.

 Is this unavoidable, or could the compiler safely make the conversion
 implicitly?
Cough, cough, make array length be an int. Do you really need arrays that big? :-S (I'm talking to Mr. D Compiler here)
Dec 05 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig wrote:
[...]
 Cough, cough, make array length be an int.
 
 Do you really need arrays that big? :-S
 
 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense. Plus, being a systems language, D should be able to represent an entire 64-bit address space as an array of ubytes (even if this is rarely done!). If one were to write a kernel in D, it would be laughable to use signed array lengths. T -- Stop staring at me like that! It's offens... no, you'll hurt your eyes!
Dec 05 2013
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 12/5/13 4:35 PM, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig wrote:
 [...]
 Cough, cough, make array length be an int.

 Do you really need arrays that big? :-S

 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense.
Of course not. And it will never be negative. But make it signed and all the problems everyone is having several times every month will be gone forever.
 Plus, being a systems language, D should be able to represent an entire
 64-bit address space as an array of ubytes (even if this is rarely
 done!). If one were to write a kernel in D, it would be laughable to use
 signed array lengths.
You can use raw pointers for that if you are going to write a kernel in D. Or you can make a struct that wraps a pointer and length, make it uint and that's it. But to make array.length uint by default and have these surprises all of the time just because "a negative length doesn't make sense"... I don't know, I feel it's not the right way to do it.
Dec 05 2013
next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Thursday, 5 December 2013 at 19:51:52 UTC, Ary Borenszweig 
wrote:
 But to make array.length uint by default and have these 
 surprises all of the time just because "a negative length 
 doesn't make sense"... I don't know, I feel it's not the right 
 way to do it.
Length of array type is not uint by default. This is second issue in array.length.
Dec 05 2013
parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 12/5/13 4:59 PM, Maxim Fomin wrote:
 t and have these surprises all of the time just because "a negative
 length doesn't make sense"... I don't know, I feel it's not the right
 way to do it.
ulong, it's the same thing.
Dec 05 2013
prev sibling parent "eles" <eles eles.com> writes:
On Thursday, 5 December 2013 at 19:51:52 UTC, Ary Borenszweig 
wrote:
 On 12/5/13 4:35 PM, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig 
 wrote:
 [...]
 Cough, cough, make array length be an int.

 Do you really need arrays that big? :-S

 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense.
Of course not. And it will never be negative. But make it signed and all the problems everyone is having several times every month will be gone forever.
I defended once the approach took by FreePascal: they declared a signed integer that covered the larges available range on a machine (now it occurred to me that the compiler could even cover *twice* that range with a simple trick) and then, all other integer types, signed or unsigned/positive were sub-ranges of the first. Comparisons were made at this largest level. I dunno how complicated and how much overhead for this.
Dec 05 2013
prev sibling next sibling parent "eles" <eles eles.com> writes:
On Thursday, 5 December 2013 at 19:36:46 UTC, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig wrote:
 [...]
 Cough, cough, make array length be an int.
 
 Do you really need arrays that big? :-S
 
 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense.
Does really "unsigned" means "positive"? Or only "a value without sign"? I remember Walter defending the latter point of view and not accepting "unsigned" == "positive".
Dec 05 2013
prev sibling parent reply "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 5 December 2013 at 19:36:46 UTC, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig wrote:
 [...]
 Cough, cough, make array length be an int.
 
 Do you really need arrays that big? :-S
 
 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense. Plus, being a systems language, D should be able to represent an entire 64-bit address space as an array of ubytes (even if this is rarely done!). If one were to write a kernel in D, it would be laughable to use signed array lengths. T
Chandler Carruth of LLVM fame (also on the C++ ISO Committee) said during Going Native (it was either his talk or the panel, I can't remember which unfortunately) that C++'s decision to make size_t unsigned was a big mistake and you should almost always use signed integers unless you need two's complement arithmetic for some weird reason. I can't remember the entire rationale for this though. http://channel9.msdn.com/Events/GoingNative/2013/The-Care-and-Feeding-of-C-s-Dragons http://channel9.msdn.com/Events/GoingNative/2013/Interactive-Panel-Ask-Us-Anything
Dec 05 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, December 05, 2013 23:44:33 Brad Anderson wrote:
 On Thursday, 5 December 2013 at 19:36:46 UTC, H. S. Teoh wrote:
 On Thu, Dec 05, 2013 at 03:47:27PM -0300, Ary Borenszweig wrote:
 [...]
 
 Cough, cough, make array length be an int.
 
 Do you really need arrays that big? :-S
 
 (I'm talking to Mr. D Compiler here)
A negative length array makes no sense. Plus, being a systems language, D should be able to represent an entire 64-bit address space as an array of ubytes (even if this is rarely done!). If one were to write a kernel in D, it would be laughable to use signed array lengths. T
Chandler Carruth of LLVM fame (also on the C++ ISO Committee) said during Going Native (it was either his talk or the panel, I can't remember which unfortunately) that C++'s decision to make size_t unsigned was a big mistake and you should almost always use signed integers unless you need two's complement arithmetic for some weird reason. I can't remember the entire rationale for this though. http://channel9.msdn.com/Events/GoingNative/2013/The-Care-and-Feeding-of-C-s -Dragons http://channel9.msdn.com/Events/GoingNative/2013/Interactive-Panel-Ask-Us-A nything
Don loves to argue the same thing. If all we had to deal with was 64-bit and larger, then it would probably be fine, but it's much more debatable with 32- bit and smaller, as with those architectures, you actually can need the full address range for large data sets. 64-bit is large enough that that's not going to be an issue - you couldn't even have that much memory to address at this point. Maybe someday it could matter but probably not except with something like mmap on systems with a ton of hard drives mapped as a single drive or somesuch. So, if we only supported 64-bit and larger, then I might support the idea of size_t being signed, but as long as we have to worry about 32-bit, I don't think that that's really an option. Regardless, we're pretty much stuck at this point. Changing it would silently break lots of code. Making the signed/unsigned comparison an error is probably our only real option at this point, and that would have the advantage of fixing it in all cases where you're mixing signed and unsigned numbers, not just in cases where you're dealing with length. - Jonathan M Davis
Dec 05 2013
parent reply "Fra" <Fra b.it> writes:
On Friday, 6 December 2013 at 02:11:20 UTC, Jonathan M Davis 
wrote:
 Regardless, we're pretty much stuck at this point. Changing it 
 would silently
 break lots of code.
I still wonder what was the reasoning behind C's warning about comparison between signed and unsigned integers instead of giving an error. And as much as I do understand the "no warnings policy", I really think that if you keep features from C, you should also keep C warnings. (BTW: I was quite sure dmd warned the user, maybe it was GDC or LDC instead?)
Dec 06 2013
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, December 06, 2013 12:02:29 Fra wrote:
 On Friday, 6 December 2013 at 02:11:20 UTC, Jonathan M Davis
 
 wrote:
 Regardless, we're pretty much stuck at this point. Changing it
 would silently
 break lots of code.
I still wonder what was the reasoning behind C's warning about comparison between signed and unsigned integers instead of giving an error. And as much as I do understand the "no warnings policy", I really think that if you keep features from C, you should also keep C warnings. (BTW: I was quite sure dmd warned the user, maybe it was GDC or LDC instead?)
An error is something defined by the language, whereas warnings are completely compiler-specific. As warning about comparing signed and unsigned integers is a warning, it's not standard, and the decision to warn about it or not is a matter of what the devs of a particular compiler decided to do. There's no such thing as standard warnins in C. What gets warned about depends entirely on the compiler and what it's settings are. For instance, gcc doesn't warn about comparing signed and unsigned values (at least not by default), whereas Visual Studio does. So, trying to follow what C does with regards to warnings doesn't make sense, because it varies from compiler to compiler. Each case should be decided on its own merits and not because of what a particular C compiler happens to do. - Jonathan M Davis
Dec 06 2013
parent "Fra" <Fra b.it> writes:
On Friday, 6 December 2013 at 11:56:35 UTC, Jonathan M Davis 
wrote:
 There's no such thing as standard warnins in C. What gets 
 warned about depends
 entirely on the compiler and what it's settings are.
You are 100% right, I was speaking out of my mind. Anyway, I still don't understand the rationale behind allowing those comparisons in C in the first place, but I guess I will never know :P
Dec 06 2013
prev sibling parent "Steve Teale" <steve.teale britseyeview.com> writes:
On Thursday, 5 December 2013 at 17:15:39 UTC, Steve Teale wrote:
 Here I feel like a beginner, but it seems very unfriendly:

 import std.stdio;

 struct ABC
 {
    double a;
    int b;
    bool c;
 }

 ABC[20] aabc;

 void foo(int n)
 {
    writefln("n: %d, aabc.length: %d", n, aabc.length);
    if (n < aabc.length)
       writeln("A");
    else
       writeln("B");
 }

 void main(string[] args)
 {
    int n = -1;
    foo(n);
 }
Oh my, what a hornet's nest! I'd settle for an error that said that it's not permissible to compare a negative integer with a size_t. But almost any way you go is ugly. Thank you all. I feel less like a beginner now ;=)
Dec 06 2013