www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Comparing floating point: == vs is

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
Under what circumstances would a comparison of floating point values return 
true for is but not ==? If I understand corcectly, is does a bit-by-bit 
comparison. So, I would expect that if is returns true, == would return true. 
However, that does not seem to always be the case. Does anyone know why that 
would be?

In particular, I'm trying to write a function for swapping the endianness of 
floating point values, and while it seems to be generally be returning true for 
is, == keeps failing. So, it sounds like I have two floating point values which 
have the exact same bits (since is succeeds) but which aren't equal according 
to ==. I don't understand what's going on at all. Does anyone have a clue what 
might be happening?

- Jonathan M Davis
Jul 16 2011
next sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1687.1310805329.14074.digitalmars-d-learn puremagic.com...

 Under what circumstances would a comparison of floating point values 
 return
 true for is but not ==? If I understand corcectly, is does a bit-by-bit
 comparison. So, I would expect that if is returns true, == would return 
 true.
 However, that does not seem to always be the case. Does anyone know why 
 that
 would be?

 In particular, I'm trying to write a function for swapping the endianness 
 of
 floating point values, and while it seems to be generally be returning 
 true for
 is, == keeps failing. So, it sounds like I have two floating point values 
 which
 have the exact same bits (since is succeeds) but which aren't equal 
 according
 to ==. I don't understand what's going on at all. Does anyone have a clue 
 what
 might be happening?

 - Jonathan M Davis


As of the last release, 'is' for floating point types is broken. (before it 
just did the same as ==)
At compile time, a is b will be true if a and b have the same bit pattern, 
or if a and b are both nans.
At run time, a is b will return true if they have the same bit pattern only, 
but may sometimes return false on platforms other than windows as it 
includes the padding bytes in the comparison.
Jul 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 19:03:51 Daniel Murphy wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1687.1310805329.14074.digitalmars-d-learn puremagic.com...
 

 Under what circumstances would a comparison of floating point values
 return
 true for is but not ==? If I understand corcectly, is does a bit-by-bit
 comparison. So, I would expect that if is returns true, == would return
 true.
 However, that does not seem to always be the case. Does anyone know why
 that
 would be?
 
 In particular, I'm trying to write a function for swapping the
 endianness
 of
 floating point values, and while it seems to be generally be returning
 true for
 is, == keeps failing. So, it sounds like I have two floating point
 values
 which
 have the exact same bits (since is succeeds) but which aren't equal
 according
 to ==. I don't understand what's going on at all. Does anyone have a
 clue
 what
 might be happening?
 
 - Jonathan M Davis


 
 As of the last release, 'is' for floating point types is broken. (before it
 just did the same as ==)
 At compile time, a is b will be true if a and b have the same bit pattern,
 or if a and b are both nans.
 At run time, a is b will return true if they have the same bit pattern only,
 but may sometimes return false on platforms other than windows as it
 includes the padding bytes in the comparison.


Hmm. Good to know. But assuming that two floating point values have the same 
bit pattern, shouldn't == return true for them? So, if == is failing, then two 
floating point values aren't identical, correct?

- Jonathan M Davis
Jul 16 2011
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1689.1310808023.14074.digitalmars-d-learn puremagic.com...

 Hmm. Good to know. But assuming that two floating point values have the 
 same
 bit pattern, shouldn't == return true for them? So, if == is failing, then 
 two
 floating point values aren't identical, correct?

 - Jonathan M Davis


Unless one or both of the values are nans, yes.  If I had to guess I'd say 
you've been swapping the padding bytes too.
Jul 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 19:23:12 Daniel Murphy wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1689.1310808023.14074.digitalmars-d-learn puremagic.com...
 

 Hmm. Good to know. But assuming that two floating point values have the
 same
 bit pattern, shouldn't == return true for them? So, if == is failing,
 then two
 floating point values aren't identical, correct?
 
 - Jonathan M Davis


 
 Unless one or both of the values are nans, yes.  If I had to guess I'd say
 you've been swapping the padding bytes too.


I tried both casting to an integer of the appropriate size and swapping that 
and doing this:

private T swapEndianImpl(T)(T val)
    if(isFloatingPoint!T)
{
    import std.algorithm;

    union Union
    {
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    }

    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);

    return u._floating;
}

I was testing the function by reversing the values twice, and casting seemed 
to fry NaNs, whereas the union/array trick seems to usually result in the 
correct values when I print them (at least for the ones I've looked at) and is 
was returning true for most of them, but == has been failing.

I don't know anything about padding bytes in floating point though, so maybe 
that's part of the problem. 
http://stackoverflow.com/questions/2782725/converting-float-values-from-big-

swap the bytes using the union/array trick, but I'm not understanding 
something here and/or something is off with either my implementation or the 
compiler.

- Jonathan M Davis
Jul 16 2011
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1690.1310809801.14074.digitalmars-d-learn puremagic.com...

 I tried both casting to an integer of the appropriate size and swapping 
 that
 and doing this:

 private T swapEndianImpl(T)(T val)
    if(isFloatingPoint!T)
 {
    import std.algorithm;

    union Union
    {
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    }

    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);

    return u._floating;
 }

 I was testing the function by reversing the values twice, and casting 
 seemed
 to fry NaNs, whereas the union/array trick seems to usually result in the
 correct values when I print them (at least for the ones I've looked at) 
 and is
 was returning true for most of them, but == has been failing.

 I don't know anything about padding bytes in floating point though, so 
 maybe
 that's part of the problem.
 http://stackoverflow.com/questions/2782725/converting-float-values-from-big-

 just
 swap the bytes using the union/array trick, but I'm not understanding
 something here and/or something is off with either my implementation or 
 the
 compiler.

 - Jonathan M Davis


The padding issue only applies to reals.  On x86 they're 10 bytes with 0 
bytes padding (windows) or 2 bytes padding (linux) or 6 bytes padding (osx).

I don't know why it would be failing for floats or doubles though.  Do you 
have some tests that fail?
Jul 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 21:19:27 Daniel Murphy wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1690.1310809801.14074.digitalmars-d-learn puremagic.com...
 

 I tried both casting to an integer of the appropriate size and swapping
 that
 and doing this:
 
 private T swapEndianImpl(T)(T val)
 
    if(isFloatingPoint!T)
 
 {
 
    import std.algorithm;
    
    union Union
    {
    
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    
    }
    
    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);
    
    return u._floating;
 
 }
 
 I was testing the function by reversing the values twice, and casting
 seemed
 to fry NaNs, whereas the union/array trick seems to usually result in
 the
 correct values when I print them (at least for the ones I've looked at)
 and is
 was returning true for most of them, but == has been failing.
 
 I don't know anything about padding bytes in floating point though, so
 maybe
 that's part of the problem.
 http://stackoverflow.com/questions/2782725/converting-float-values-from-

 could just
 swap the bytes using the union/array trick, but I'm not understanding
 something here and/or something is off with either my implementation or
 the
 compiler.
 
 - Jonathan M Davis


 
 The padding issue only applies to reals.  On x86 they're 10 bytes with 0
 bytes padding (windows) or 2 bytes padding (linux) or 6 bytes padding (osx).
 
 I don't know why it would be failing for floats or doubles though.  Do you
 have some tests that fail?


This is also being discussed on the main list in the "Byte Order Swapping" 
thread, so you may want to discuss it there, but this is what I have at the 
moment:

import core.bitop;

/++
    Swaps the endianness of the given value. Any integral value,
    character, or floating point value is accepted.
  +/
T swapEndian(T)(T val)
    if(isNumeric!T || isSomeChar!T)
{
    static if(val.sizeof == 1)
        return val;
    else static if(isUnsigned!T || isFloatingPoint!T)
        return swapEndianImpl(val);
    else static if(isIntegral!T)
        return swapEndianImpl(cast(Unsigned!T) val);
    else static if(is(Unqual!T == wchar))
        return cast(T)swapEndian(cast(ushort)val);
    else static if(is(Unqual!T == dchar))
        return cast(T)swapEndian(cast(uint)val);
    else
        static assert(0, T.stringof ~ " unsupported by swapEndian.");
}

private T swapEndianImpl(T)(T val)
    if(is(Unqual!T == ushort))
{
    return ((val & 0xff00U) >> 8) |
           ((val & 0x00ffU) << 8);
}

private T swapEndianImpl(T)(T val)
    if(is(Unqual!T == uint))
{
    return bswap(val);
}

private T swapEndianImpl(T)(T val)
    if(is(Unqual!T == ulong))
{
    return ((val & 0xff00000000000000UL) >> 56) |
           ((val & 0x00ff000000000000UL) >> 40) |
           ((val & 0x0000ff0000000000UL) >> 24) |
           ((val & 0x000000ff00000000UL) >> 8) |
           ((val & 0x00000000ff000000UL) << 8) |
           ((val & 0x0000000000ff0000UL) << 24) |
           ((val & 0x000000000000ff00UL) << 40) |
           ((val & 0x00000000000000ffUL) << 56);
}

private T swapEndianImpl(T)(T val)
    if(isFloatingPoint!T)
{
    import std.algorithm;

    union Union
    {
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    }

    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);

    return u._floating;
}

unittest
{
    import std.stdio;
    import std.typetuple;

    foreach(T; TypeTuple!(byte, ubyte, short, ushort, int, uint, long, ulong,
                          char, wchar, dchar, float, double, real))
    {
        scope(failure) writefln("Failed type: %s", T.stringof);
        T val;
        const T cval;
        immutable T ival;

        assert(swapEndian(swapEndian(val)) == val);
        assert(swapEndian(swapEndian(cval)) == cval);
        assert(swapEndian(swapEndian(ival)) == ival);
        assert(swapEndian(swapEndian(T.min)) == T.min);
        assert(swapEndian(swapEndian(T.max)) == T.max);
        static if(isSigned!T) assert(swapEndian(swapEndian(cast(T)0)) == 0);
    }
}

I suspect that the problem is a combination of issues with NaN and real. If I 
change the first three tests (which test init which is NaN) to is, then they 
pass, but the test for real.max fails regardless of whether is or == is used. 
The other tests seem to pass regardless of whether is or == is used. So, I may 
need to do something special for NaN, and I may have to do something to deal 
with the padding in real. I don't know.

- Jonathan M Davis
Jul 16 2011
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1705.1310853321.14074.digitalmars-d-learn puremagic.com...

 I suspect that the problem is a combination of issues with NaN and real. 
 If I
 change the first three tests (which test init which is NaN) to is, then 
 they
 pass, but the test for real.max fails regardless of whether is or == is 
 used.
 The other tests seem to pass regardless of whether is or == is used. So, I 
 may
 need to do something special for NaN, and I may have to do something to 
 deal
 with the padding in real. I don't know.

 - Jonathan M Davis


They all pass for me when using 'is' on win32, so assuming you're on linux 
or osx the padding issue is what you're running into.  The workaround is 
probably using isIdentical from std.math for floating point types instead of 
'is' and only swapping the first 10 bytes in endianswap (for reals only, the 
other types should work).  The 'is' problem will be fixed, but swapping the 
full 12 or 16 bytes of real will always result in the wrong value.
Jul 16 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 17 July 2011 14:19:12 Daniel Murphy wrote:

 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1705.1310853321.14074.digitalmars-d-learn puremagic.com...
 

 I suspect that the problem is a combination of issues with NaN and real.
 If I
 change the first three tests (which test init which is NaN) to is, then
 they
 pass, but the test for real.max fails regardless of whether is or == is
 used.
 The other tests seem to pass regardless of whether is or == is used. So,
 I may
 need to do something special for NaN, and I may have to do something to
 deal
 with the padding in real. I don't know.
 
 - Jonathan M Davis


 
 They all pass for me when using 'is' on win32, so assuming you're on linux
 or osx the padding issue is what you're running into.  The workaround is
 probably using isIdentical from std.math for floating point types instead of
 'is' and only swapping the first 10 bytes in endianswap (for reals only,
 the other types should work).  The 'is' problem will be fixed, but swapping
 the full 12 or 16 bytes of real will always result in the wrong value.


I'm on Linux.

I need to do whatever will result in the correct value with == if that is at 
all possible. The question the is twofold:

1. How to make it so that swapping the endianness twice doesn't butcher NaNs.

2. How to properly swap reals if they have padding.

It sounds like the solution for reals is to only swap part of it. So, I assume 
that the first 10 bytes are a different ten bytes on each platform? If so, are 
they the ones on the left in little endian or the ones on the right (making 
big endian the other)? i.e. value[0 .. 10] or value[$ - 10 .. $]?

And that still leaves the question of NaN. Why would reversing the bytes twice 
make NaN's not equal with == anymore?

- Jonathan M Davis
Jul 16 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1708.1310882400.14074.digitalmars-d-learn puremagic.com...

 I'm on Linux.

 I need to do whatever will result in the correct value with == if that is 
 at
 all possible. The question the is twofold:

 1. How to make it so that swapping the endianness twice doesn't butcher 
 NaNs.

 2. How to properly swap reals if they have padding.

 It sounds like the solution for reals is to only swap part of it. So, I 
 assume
 that the first 10 bytes are a different ten bytes on each platform? If so, 
 are
 they the ones on the left in little endian or the ones on the right 
 (making
 big endian the other)? i.e. value[0 .. 10] or value[$ - 10 .. $]?


They're just padding bytes, not included in the structure, so they always 
come afterwards.
eg.
union { real f; ubyte[10] b; }
should give you them in little or big endian.


 And that still leaves the question of NaN. Why would reversing the bytes 
 twice
 make NaN's not equal with == anymore?


NaNs would never be equal with ==.  Using 'is', it works as expected for me.
If you were reversing the full 12 bytes, then comparing, some of the data 
would have ended up in the padding bytes and could have got lost when moving 
to/from the stack or the floating point registers.  Don't forget that is 
will do a bitwise comparison including the padding bytes on linux, so 
isIdentical will likely give you the results you want.
Jul 17 2011
prev sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
I'd recommend using Don's isIdentical (or something like that) in std.math 
until it's fixed.
Jul 16 2011