www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Always false float comparisons

reply Walter Bright <newshound2 digitalmars.com> writes:
Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

will be true.

So, should the compiler emit a warning for the former case?
May 09 2016
next sibling parent reply qznc <qznc web.de> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


What is the actual reason for the mismatch?

Does f lose precision as a float, while the 1.30 literal is a 
more precise double/real? Comparing float and double might be 
worth a warning.

Does it encode the two literals differently? If so, why?
May 09 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 2:25 AM, qznc wrote:

 What is the actual reason for the mismatch?


floats cannot represent 1.30 exactly, and promoting it to a double gives a 
different result than 1.30 as a double.
May 09 2016
prev sibling next sibling parent Robert burner Schadek <rburners gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 So, should the compiler emit a warning for the former case?


I'm not for a compiler change. IMO a library called 
std.sanity_float with a equal and a notequal function would be 
better.
May 09 2016
prev sibling next sibling parent reply Jens Mueller via Digitalmars-d <digitalmars-d puremagic.com> writes:
Walter Bright via Digitalmars-d wrote:

 Don Clugston pointed out in his DConf 2016 talk that:
 
     float f = 1.30;
     assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a float. However,
 
     float f = 1.30;
     assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?


Since

assert(f == 1.30f);

passes I find the root cause lies in the implicit type conversion from
float to double. Warning for those comparisons should be fine. Shouldn't
mix them anyway.
I wonder what's the difference between 1.30f and cast(float)1.30.

Jens
May 09 2016
next sibling parent Chris <wendlec tcd.ie> writes:
On Monday, 9 May 2016 at 10:16:54 UTC, Jens Mueller wrote:

 Walter Bright via Digitalmars-d wrote:

 Don Clugston pointed out in his DConf 2016 talk that:
 
     float f = 1.30;
     assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a 
 float. However,
 
     float f = 1.30;
     assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?


 Since

 assert(f == 1.30f);

 passes I find the root cause lies in the implicit type 
 conversion from
 float to double. Warning for those comparisons should be fine. 
 Shouldn't
 mix them anyway.
 I wonder what's the difference between 1.30f and 
 cast(float)1.30.

 Jens


+1
May 09 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:

 passes I find the root cause lies in the implicit type conversion from
 float to double.


That isn't going to change.


 Warning for those comparisons should be fine. Shouldn't mix them anyway.


Too onerous.


 I wonder what's the difference between 1.30f and cast(float)1.30.


There isn't one.
May 09 2016
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:

 On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:

 Warning for those comparisons should be fine. Shouldn't mix 
 them anyway.


 Too onerous.


Surely not too onerous if we're only talking about == ? Mixing 
floating point types on either side of == seems like a pretty 
solidly bad idea.
May 09 2016
parent qznc <qznc web.de> writes:
On Monday, 9 May 2016 at 12:24:05 UTC, John Colvin wrote:

 On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:

 On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:

 Warning for those comparisons should be fine. Shouldn't mix 
 them anyway.


 Too onerous.


 Surely not too onerous if we're only talking about == ? Mixing 
 floating point types on either side of == seems like a pretty 
 solidly bad idea.


Why only == ? The example also applies to opCmp.

     float f = 1.30;
     assert(f >= 1.30);
     assert(f <= 1.30);
May 09 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/9/16 7:26 AM, Walter Bright wrote:

 I wonder what's the difference between 1.30f and cast(float)1.30.


 There isn't one.


I know this is a bit band-aid-ish, but if one is comparing literals to a 
float, why not treat the literal as the type being compared against? In 
other words, imply the 1.3f. This isn't integer-land where promotions do 
not change the outcome.

What I see here is that double(1.3) cannot be represented as a float. So 
right there, the compiler can tell you, no, this is never going to be 
true. Something stinks when you can write an always-false expression as 
an if conditional by accident.

-Steve
May 09 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 6:46 AM, Steven Schveighoffer wrote:

 I know this is a bit band-aid-ish, but if one is comparing literals to a float,
 why not treat the literal as the type being compared against? In other words,
 imply the 1.3f. This isn't integer-land where promotions do not change the
outcome.


Because it's yet another special case, and we know where those lead. For 
example, what if the 1.30 was the result of CTFE?
May 09 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/9/16 4:22 PM, Walter Bright wrote:

 On 5/9/2016 6:46 AM, Steven Schveighoffer wrote:

 I know this is a bit band-aid-ish, but if one is comparing literals to
 a float,
 why not treat the literal as the type being compared against? In other
 words,
 imply the 1.3f. This isn't integer-land where promotions do not change
 the outcome.


 Because it's yet another special case, and we know where those lead. For
 example, what if the 1.30 was the result of CTFE?


This is true, it's a contrived example.

-Steve
May 09 2016
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 9 May 2016 04:26:55 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 I wonder what's the difference between 1.30f and cast(float)1.30. =20


=20
 There isn't one.


=20
Oh yes, there is! Don't you love floating-point...

cast(float)1.30 rounds twice, first from a base-10
representation to a base-2 double value and then again to a
float. 1.30f directly converts to float. In some cases this
does not yield the same value as converting the base-10
literal directly to a float!

Imagine this example (with mantissa bit count reduced for
illustration):

Original base-10 rational number converted to base-2:
111110|10000000|011010111011...
       =E2=86=96        =E2=86=96
        float &  double mantissa precision limits

The 1st segment is the mantissa width of a float.
The 2nd segment is the mantissa width of a double.
The 3rd segment is the fraction used for rounding
 the base-10 literal to a double.

Conversion to double rounds down, since fraction <0.5:
111110|10000000 (double mantissa)

Proposed cast to float rounds down, too, because of
round-to-even rule for 0.5:
111110 (float mantissa)

But the conversion of base-10 directly to float, rounds UP
since the fraction is then >0.5
111110|10000000_011010111011...
111111

It happens when the 29 bits difference between double and
float mantissa are 100=E2=80=A6000 and the bit in front and after are
0. The error would occur to ~0.000000047% of numbers.

--=20
Marco
May 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 4:06 PM, Marco Leise wrote:

 Am Mon, 9 May 2016 04:26:55 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:


 I wonder what's the difference between 1.30f and cast(float)1.30.


 There isn't one.


 Oh yes, there is! Don't you love floating-point...

 cast(float)1.30 rounds twice, first from a base-10
 representation to a base-2 double value and then again to a
 float. 1.30f directly converts to float.


This is one reason why the compiler carries everything internally to 80 bit 
precision, even if they are typed as some other precision. It avoids the double 
rounding.
May 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.05.2016 21:25, Walter Bright wrote:

 On 5/12/2016 4:06 PM, Marco Leise wrote:

 Am Mon, 9 May 2016 04:26:55 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:


 I wonder what's the difference between 1.30f and cast(float)1.30.


 There isn't one.


 Oh yes, there is! Don't you love floating-point...

 cast(float)1.30 rounds twice, first from a base-10
 representation to a base-2 double value and then again to a
 float. 1.30f directly converts to float.


 This is one reason why the compiler carries everything internally to 80
 bit precision, even if they are typed as some other precision. It avoids
 the double rounding.


IMO the compiler should never be allowed to use a precision different 
from the one specified.
May 13 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 12:48 PM, Timon Gehr wrote:

 IMO the compiler should never be allowed to use a precision different from the
 one specified.


I take it you've never been bitten by accumulated errors :-)

Reduced precision is only useful for storage formats and increasing speed. If a 
less accurate result is desired, your algorithm is wrong.
May 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.05.2016 23:35, Walter Bright wrote:

 On 5/13/2016 12:48 PM, Timon Gehr wrote:

 IMO the compiler should never be allowed to use a precision different
 from the one specified.


 I take it you've never been bitten by accumulated errors :-)
 ...


If that was the case it would be because I explicitly ask for high 
precision if I need it.

If the compiler using or not using a higher precision magically fixes an 
actual issue with accumulated errors, that means the correctness of the 
code is dependent on something hidden, that you are not aware of, and 
that could break any time, for example at a time when you really don't 
have time to track it down.


 Reduced precision is only useful for storage formats and increasing
 speed.  If a less accurate result is desired, your algorithm is wrong.


Nonsense. That might be true for your use cases. Others might actually 
depend on IEE 754 semantics in non-trivial ways. Higher precision for 
temporaries does not imply higher accuracy for the overall computation.

E.g., correctness of double-double arithmetic is crucially dependent on 
correct rounding semantics for double:
https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic

Also, it seems to me that for e.g. 
https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
the result can actually be made less precise by adding casts to higher 
precision and truncations back to lower precision at appropriate places 
in the code.

And even if higher precision helps, what good is a "precision-boost" 
that e.g. disappears on 64-bit builds and then creates inconsistent results?

Sometimes reproducibility/predictability is more important than maybe 
making fewer rounding errors sometimes. This includes reproducibility 
between CTFE and runtime.

Just actually comply to the IEEE floating point standard when using 
their terminology. There are algorithms that are designed for it and 
that might stop working if the language does not comply.

Then maybe add additional built-in types with a given storage size that 
additionally /guarantee/ a certain amount of additional scratch space 
when used for function-local computations.
May 13 2016
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.05.2016 02:49, Timon Gehr wrote:

 IEE


IEEE.
May 13 2016
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.05.2016 02:49, Timon Gehr wrote:

 result can actually be made less precise


less accurate. I need to go to sleep.
May 13 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 5:49 PM, Timon Gehr wrote:

 Nonsense. That might be true for your use cases. Others might actually depend
on
 IEE 754 semantics in non-trivial ways. Higher precision for temporaries does
not
 imply higher accuracy for the overall computation.


Of course it implies it.

An anecdote: a colleague of mine was once doing a chained calculation. At every 
step, he rounded to 2 digits of precision after the decimal point, because 2 
digits of precision was enough for anybody. I carried out the same calculation 
to the max precision of the calculator (10 digits). He simply could not 
understand why his result was off by a factor of 2, which was a couple hundred 
times his individual roundoff error.



 E.g., correctness of double-double arithmetic is crucially dependent on correct
 rounding semantics for double:
 https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic


Double-double has its own peculiar issues, and is not relevant to this
discussion.



 Also, it seems to me that for e.g.
 https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
 the result can actually be made less precise by adding casts to higher
precision
 and truncations back to lower precision at appropriate places in the code.


I don't see any support for your claim there.



 And even if higher precision helps, what good is a "precision-boost" that e.g.
 disappears on 64-bit builds and then creates inconsistent results?


That's why I was thinking of putting in 128 bit floats for the compiler
internals.



 Sometimes reproducibility/predictability is more important than maybe making
 fewer rounding errors sometimes. This includes reproducibility between CTFE and
 runtime.


A more accurate answer should never cause your algorithm to fail. It's like 
putting better parts in your car causing the car to fail.



 Just actually comply to the IEEE floating point standard when using their
 terminology. There are algorithms that are designed for it and that might stop
 working if the language does not comply.


Conjecture. I've written FP algorithms (from Cody+Waite, for example), and none 
of them degraded when using more precision.


Consider that the 8087 has been operating at 80 bits precision by default for
30 
years. I've NEVER heard of anyone getting actual bad results from this. They 
have complained about their test suites that tested for less accurate results 
broke. They have complained about the speed of x87. And Intel has been trying
to 
get rid of the x87 forever. Sometimes I wonder if there's a disinformation 
campaign about more accuracy being bad, because it smacks of nonsense.

BTW, I once asked Prof Kahan about this. He flat out told me that the only 
reason to downgrade precision was if storage was tight or you needed it to run 
faster. I am not making this up.
May 13 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:

 BTW, I once asked Prof Kahan about this. He flat out told me 
 that the only reason to downgrade precision was if storage was 
 tight or you needed it to run faster. I am not making this up.


He should have been aware of reproducibility since people use 
fixed point to achieve it, if he wasn't then shame on him.

In Java all compile time constants are done using strict settings 
and it provides a keyword «strictfp» to get strict behaviour for 
a particular class/function.

In C++ template parameters cannot be floating point, you use 
std::ratio so you get exact rational number instead. This is to 
avoid inaccuracy problems in the type system.

In interval-arithmetics you need to round up and down correctly 
on the bounds-computations to get correct results. (It is ok for 
the interval to be larger than the real result, but the opposite 
is a disaster).

With reproducible arithmetics you can do advanced accurate static 
analysis of programs using floating point code.

With reproducible arithmetics you can sync nodes in a cluster 
based on "time" alone, saving exchanges of data in simulations.

There are lots of reasons to default to well defined floating 
point arithmetics.
May 13 2016
next sibling parent reply QAston <qaston gmail.com> writes:
On Saturday, 14 May 2016 at 05:46:38 UTC, Ola Fosheim Grøstad 
wrote:

 In Java all compile time constants are done using strict 
 settings and it provides a keyword «strictfp» to get strict 
 behaviour for a particular class/function.


In java everything used to be strictfp (and there was no 
keyword), but it was changed to do non-strict arithmetic by 
default after a backlash from numeric programmers.
May 14 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 09:11:49 UTC, QAston wrote:

 On Saturday, 14 May 2016 at 05:46:38 UTC, Ola Fosheim Grøstad 
 wrote:

 In Java all compile time constants are done using strict 
 settings and it provides a keyword «strictfp» to get strict 
 behaviour for a particular class/function.


 In java everything used to be strictfp (and there was no 
 keyword), but it was changed to do non-strict arithmetic by 
 default after a backlash from numeric programmers.


Java had a healthy default, but switched in order to not look so 
bad in comparison to C on current day hardware. However, they 
retained the ability to get stricter floating point.

At the end of the day there is literally no end to how far you 
can move down the line of implementation defined floating point. 
Take a look at the ARM instruction set, it makes x86 look high 
level. You can even choose how many iterations you want for 
complex instructions (i.e. choose the approximation level for 
faster execution).

However, these days IEEE754-2008 is becoming available in 
hardware and therefore one is better off choosing the most 
well-defined semantics for the regular case. It means less 
optimization opportunities unless you specify relaxed semantics, 
but that isn't such a bad trade off as long as specifying relaxed 
semantics is easy.
May 14 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 10:46 PM, Ola Fosheim Grøstad wrote:

 On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:

 BTW, I once asked Prof Kahan about this. He flat out told me that the only
 reason to downgrade precision was if storage was tight or you needed it to run
 faster. I am not making this up.


 He should have been aware of reproducibility since people use fixed point to
 achieve it, if he wasn't then shame on him.


Kahan designed the x87 and wrote the IEEE 754 standard, so I'd do my homework 
before telling him he is wrong about basic floating point stuff.



 In Java all compile time constants are done using strict settings and it
 provides a keyword «strictfp» to get strict behaviour for a particular
 class/function.


What happened with Java was interesting. The original spec required double 
arithmetic to be done with double precision. This wound up failing all over the 
place on x86 machines, which (as I explained) does temporaries to 80 bits. 
Forcing the x87 to use doubles for intermediate values caused Java to run much 
slower, and Sun was forced to back off on that requirement.
May 14 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 18:58:35 UTC, Walter Bright wrote:

 Kahan designed the x87 and wrote the IEEE 754 standard, so I'd 
 do my homework before telling him he is wrong about basic 
 floating point stuff.


You don't have to tell me who Kahan is. I don't see the 
relevance. You are trying to appeal to authority. Stick to facts. 
:-)
May 14 2016
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:

 An anecdote: a colleague of mine was once doing a chained 
 calculation. At every step, he rounded to 2 digits of precision 
 after the decimal point, because 2 digits of precision was 
 enough for anybody. I carried out the same calculation to the 
 max precision of the calculator (10 digits). He simply could 
 not understand why his result was off by a factor of 2, which 
 was a couple hundred times his individual roundoff error.


I'm sympathetic to this. Some of my work deals with statistics 
and you see people try to use formula that are faster but less 
accurate and it can really get you in to trouble. Var(X) = E(X^2) 
- E(X)^2 is only true for real numbers, not floating point 
arithmetic. It can also lead to weird results when dealing with 
matrix inverses.

I like the idea of a float type that is effectively the largest 
precision on your machine (the D real type). However, I could be 
convinced by the argument that you should have to opt-in for this 
and that internal calculations should not implicitly use it. 
Mainly because I'm sympathetic to the people who would prefer 
speed to precision. Not everybody needs all the precision all the 
time.
May 13 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 10:52 PM, jmh530 wrote:

 I like the idea of a float type that is effectively the largest precision on
 your machine (the D real type). However, I could be convinced by the argument
 that you should have to opt-in for this and that internal calculations should
 not implicitly use it. Mainly because I'm sympathetic to the people who would
 prefer speed to precision. Not everybody needs all the precision all the time.


Speed matters on the generated target program, not in the compiler internal 
floating point calculations, simply because the compiler does very, very few of 
them.
May 14 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:

 Sometimes reproducibility/predictability is more important 
 than maybe making
 fewer rounding errors sometimes. This includes reproducibility 
 between CTFE and
 runtime.


 A more accurate answer should never cause your algorithm to 
 fail. It's like putting better parts in your car causing the 
 car to fail.


This is all quite discouraging from a scientific programmers 
point of view. Precision is important, more precision is good, 
but reproducibility and predictability are critical.

Tables of constants that change value if I put a `static` in 
front of them?

Floating point code that produces different results after a 
compiler upgrade / with different non-fp-related switches?

Ewwwww.
May 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/14/2016 3:16 AM, John Colvin wrote:

 This is all quite discouraging from a scientific programmers point of view.
 Precision is important, more precision is good, but reproducibility and
 predictability are critical.


I used to design and build digital electronics out of TTL chips. Over time, TTL 
chips got faster and faster. The rule was to design the circuit with a minimum 
signal propagation delay, but never a maximum. Therefore, putting in faster 
parts will never break the circuit.

Engineering is full of things like this. It's sound engineering practice. I've 
never ever heard of a circuit requiring a resistor with 20% tolerance that
would 
fail if a 10% tolerance one was put in, for another example.



 Tables of constants that change value if I put a `static` in front of them?

 Floating point code that produces different results after a compiler upgrade /
 with different non-fp-related switches?

 Ewwwww.


Floating point is not exact calculation. It just isn't. Designing an algorithm 
that relies on worse answers is absurd to my ears.

Results should be tested to have a minimum number of correct bits in the
answer, 
not a maximum number. This is, in fact, how std.math checks the result of the 
algorithms implemented in it, and how it should be done.


This is not some weird crazy idea of mine, as I said, the x87 FPU in every x86 
chip has been doing this for several decades.
May 14 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/14/2016 11:46 AM, Walter Bright wrote:

 I used to design and build digital electronics out of TTL chips. Over time, TTL
 chips got faster and faster. The rule was to design the circuit with a minimum
 signal propagation delay, but never a maximum. Therefore, putting in faster
 parts will never break the circuit.


Eh, I got the min and max backwards.
May 14 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 14 May 2016 at 20:38:54 UTC, Walter Bright wrote:

 On 5/14/2016 11:46 AM, Walter Bright wrote:

 I used to design and build digital electronics out of TTL 
 chips. Over time, TTL
 chips got faster and faster. The rule was to design the 
 circuit with a minimum
 signal propagation delay, but never a maximum. Therefore, 
 putting in faster
 parts will never break the circuit.


 Eh, I got the min and max backwards.


Heh, I read that and thought, "wtf is he talking about, never a 
max?" :D

Regarding floating-point, I'll go farther than you and say that 
if an algorithm depends on lower-precision floating-point to be 
accurate, it's a bad algorithm.  Now, people can always make 
mistakes in their implementation and unwittingly depend on lower 
precision somehow, but that _should_ fail.

None of this is controversial to me: you shouldn't be comparing 
floating-point numbers with anything other than approxEqual, 
increasing precision should never bother your algorithm, and a 
higher-precision, common soft-float for CTFE will help 
cross-compiling and you'll never notice the speed hit.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 19:38:10 UTC, Joakim wrote:

 Regarding floating-point, I'll go farther than you and say that 
 if an algorithm depends on lower-precision floating-point to be 
 accurate, it's a bad algorithm.


No. In system level programming a good algorithm is an effective 
and efficient algorithm on the hardware at hand.

A good system level programming language give you control at the 
hardware level. Including the floating point unit, in the 
language itself.

There are lots of algorithms that will break if you randomly 
switch precision of different expressions.

Heck, nearly all of the SIMD optimizations on differentials that 
I use will break if one lane is computed with a different 
precision than the other lanes. I don't give a rats ass about 
increased precision. I WANT THE DAMN LANES TO BE IN THE SAME 
PHASE (or close to it)!! Phase-locking is much more important 
than value accuracy.

Or to put it differently: It does not matter if all clocks are 
too slow, as long as they are running at the same speed. It is a 
lot worse if some clocks are too slow and others are too fast. 
That would lead to some serious noise in a time series.

Of course, no hardware vendor is crazy enough to randomly switch 
precision on their ALU. Hardware vendors do understand that this 
_WILL_ lead disaster. Sadly many in the D community don't. 
Presumably because they don't actually try to write performant 
floating point code, they are not system level programmers.

(Btw, many error correcting strategies break too if you randomly 
switch precision.)


 Now, people can always make mistakes in their implementation 
 and unwittingly depend on lower precision somehow, but that 
 _should_ fail.


People WILL make mistakes, but if you cannot control precision 
then you cannot:

1. create a reference implementation to compare with

2. unit test floating point code in a reliable way

3. test for convergence/divergence in feedback loops (which can 
have _disastrous_ results and could literally ruin your 
speakers/hearing in the case of audio).


 None of this is controversial to me: you shouldn't be comparing 
 floating-point numbers with anything other than approxEqual,


I don't agree.

1. Comparing typed constants for equality should be 
unproblematic. In D that is broken.

2. Testing for zero is a necessity when doing division.

3. Comparing for equality is the same as subtraction followed by 
testing for zero.

So, the rule is: you shouldn't compare at all unless you know the 
error bounds, but that depends on WHY you are comparing.

However, with constants/sentinels and some methods you do know... 
Also, with some input you do know that the algorithm WILL fail 
for certain values at a _GIVEN_ precision. Testing for equality 
for those values makes a lot of sense, until some a**hole decides 
to randomly "improve" precision where it was typed to something 
specific and known.

Take this:

f(x) = 1/(2-x)

Should I not be able to test for the exact value "2" here? I 
don't see why "1.3" typed to a given precision should be 
different. You want to force me to a more than 3x more expensive 
test just to satisfy some useless FP semantic that does not 
provide any real world benefits whatsoever?



 increasing precision should never bother your algorithm, and a 
 higher-precision, common soft-float for CTFE will help 
 cross-compiling and you'll never notice the speed hit.


Randomly increasing precision is never a good idea. Yes, having 
different precision in different code paths can ruin the quality 
of both rendering, data analysis and break algorithms.

Having infinite precision untyped real that may downgrade to say 
64 bits mantissa is acceptable. Or in the case of Go, a 256 bit 
mantissa. That's a different story.

Having single precision floats that randomly are turned into 
arbitrary precision floats is not acceptable. Not at all.
May 17 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 17 May 2016 at 14:59:45 UTC, Ola Fosheim Grøstad 
wrote:

 There are lots of algorithms that will break if you randomly 
 switch precision of different expressions.


There is nothing "random" about increasing precision till the 
end, it follows a well-defined rule.


 Heck, nearly all of the SIMD optimizations on differentials 
 that I use will break if one lane is computed with a different 
 precision than the other lanes. I don't give a rats ass about 
 increased precision. I WANT THE DAMN LANES TO BE IN THE SAME 
 PHASE (or close to it)!! Phase-locking is much more important 
 than value accuracy.


So you're planning on running phase-locking code partially in 
CTFE and runtime and it's somehow very sensitive to precision?  
If your "phase-locking" depends on producing bit-exact results 
with floating-point, you're doing it wrong.


 Now, people can always make mistakes in their implementation 
 and unwittingly depend on lower precision somehow, but that 
 _should_ fail.


 People WILL make mistakes, but if you cannot control precision 
 then you cannot:

 1. create a reference implementation to compare with

 2. unit test floating point code in a reliable way

 3. test for convergence/divergence in feedback loops (which can 
 have _disastrous_ results and could literally ruin your 
 speakers/hearing in the case of audio).


If any of this depends on comparing bit-exact floating-point 
results, you're doing it wrong.


 None of this is controversial to me: you shouldn't be 
 comparing floating-point numbers with anything other than 
 approxEqual,


 I don't agree.

 1. Comparing typed constants for equality should be 
 unproblematic. In D that is broken.


If the constant is calculated rather than a literal, you should 
be checking using approxEqual.


 2. Testing for zero is a necessity when doing division.


If the variable being tested is calculated, you should be using 
approxEqual.


 3. Comparing for equality is the same as subtraction followed 
 by testing for zero.


So what?  You should be using approxEqual.


 So, the rule is: you shouldn't compare at all unless you know 
 the error bounds, but that depends on WHY you are comparing.


No, you should always use error bounds.  Sometimes you can get 
away with checking bit-exact equality, say for constants that you 
defined yourself with no calculation, but it's never a good 
practice.


 However, with constants/sentinels and some methods you do 
 know... Also, with some input you do know that the algorithm 
 WILL fail for certain values at a _GIVEN_ precision. Testing 
 for equality for those values makes a lot of sense, until some 
 a**hole decides to randomly "improve" precision where it was 
 typed to something specific and known.

 Take this:

 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?


It would make more sense to figure out what the max value of f(x) 
is you're trying to avoid, say 1e6, and then check for 
approxEqual(x, 2, 2e-6).  That would make much more sense than 
only avoiding 2, when an x that is arbitrarily close to 2 can 
also blow up f(x).


 I don't see why "1.3" typed to a given precision should be 
 different. You want to force me to a more than 3x more 
 expensive test just to satisfy some useless FP semantic that 
 does not provide any real world benefits whatsoever?


Oh, it's real world alright, you should be avoiding more than 
just 2 in your example above.


 increasing precision should never bother your algorithm, and a 
 higher-precision, common soft-float for CTFE will help 
 cross-compiling and you'll never notice the speed hit.


 Randomly increasing precision is never a good idea. Yes, having 
 different precision in different code paths can ruin the 
 quality of both rendering, data analysis and break algorithms.

 Having infinite precision untyped real that may downgrade to 
 say 64 bits mantissa is acceptable. Or in the case of Go, a 256 
 bit mantissa. That's a different story.

 Having single precision floats that randomly are turned into 
 arbitrary precision floats is not acceptable. Not at all.


Simply repeating the word "random" over and over again does not 
make it so.
May 17 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:

 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.


Can you please quote that well-defined rule?

It is indeed random, or arbitrary (which is the same thing):

if(x<0){
   // DMD choose 64 bit mantissa
   const float y = ...
   ...

} else {
   // DMD choose 24 bit mantissa
   float y = ...
   ...
}

How is this not arbitrary?

If x is the amplitude then a flaw like this can cause a DC offset 
to accumulate and you end up with reduced precision, not improved 
precision. A DC filter at the end does not help on this precision 
loss.


 So you're planning on running phase-locking code partially in 
 CTFE and runtime and it's somehow very sensitive to precision?  
 If your "phase-locking" depends on producing bit-exact results 
 with floating-point, you're doing it wrong.


I am not doing anything wrong. D is doing it wrong. If you add 
different deltas then you will get drift. So, no improved 
precision in calculating some deltas is not improving the 
accuracy. It makes it worse.


 If any of this depends on comparing bit-exact floating-point 
 results, you're doing it wrong.


It depends on the unit tests running with the exact same 
precision as the production code.

Fast floating point code depends on the specifics of the 
hardware. A system level language should not introduce a 
different kind of bias that isn't present in the hardware!

D is doing it wrong because it makes it is thereby forcing 
programmers to use algorithms that are 10-100x slower to get 
reliable results.

That is _wrong_.


 If the constant is calculated rather than a literal, you should 
 be checking using approxEqual.


No. 1+2+3+4 is exact on all floating point units I know of.


 3. Comparing for equality is the same as subtraction followed 
 by testing for zero.


 So what?  You should be using approxEqual.


No.


 So, the rule is: you shouldn't compare at all unless you know 
 the error bounds, but that depends on WHY you are comparing.


 No, you should always use error bounds.  Sometimes you can get 
 away with checking bit-exact equality, say for constants that 
 you defined yourself with no calculation, but it's never a good 
 practice.


It is if you know WHY you are doing a test.

Btw, have you ever tried to prove error bounds for an iterative 
method?
You actually think most people prove them to be correct?

Or perhaps almost all of them just pick a number out of thin air 
which they think will work out and rely on testing their code?

Well, the latter is no better than checking for exact equality. 
And I can assure you that the vast majority of programmers do not 
prove error bounds with the level of rigour it takes to get it 
correct.

The reality is that it is common practice to write code that 
seems to work. But that does not make it correct. However, making 
it correct is way too time consuming and often not worth the 
trouble. So people rely on testing. Floating point code is no 
different.

But with D semantics you cannot rely on testing. That's bad, 
because most people write incorrect code. Whether they are 
experts or not. (it is only matter of difference in frequency)


 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?


 It would make more sense to figure out what the max value of 
 f(x) is you're trying to avoid, say 1e6, and then check for 
 approxEqual(x, 2, 2e-6).  That would make much more sense than 
 only avoiding 2, when an x that is arbitrarily close to 2 can 
 also blow up f(x).


I am trying to avoid an exception, not a value.


 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.


Which number would that be?


 Simply repeating the word "random" over and over again does not 
 make it so.


That's right. It is DMD that makes it so, not my words.

However, in order to reject what other say, you have to make an 
argument. And in this case we have:

1. A system level programming language that claims to excel at 
floating point.
2. Hardware platforms with specific behaviour.

Unfortunately 2 is true, but not 1.

D is not matching up to the minimum requirements for people 
wanting to write fast and reliable floating point code for a 
given platform.

According to the D spec, the compiler could schedule typed single 
precision floating point calculations to two completely different 
floating point units (and yes, there are platforms that provide 
multiple incompatible floating point units with widely differing 
characteristics).

That is random.
And so is "float" behaving differently than "const float".
May 17 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:

 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.


 Can you please quote that well-defined rule?


It appears to be "the compiler carries everything internally to 
80 bit precision, even if they are typed as some other precision."
http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com


 It is indeed random, or arbitrary (which is the same thing):


No, they're not the same thing: rules can be arbitrarily set yet 
consistent over time, whereas random usually means both arbitrary 
and inconsistent over time.


 if(x<0){
   // DMD choose 64 bit mantissa
   const float y = ...
   ...

 } else {
   // DMD choose 24 bit mantissa
   float y = ...
   ...
 }

 How is this not arbitrary?


I believe that means any calculation used to compute y at 
compile-time will be done in 80-bit or larger reals, then rounded 
to a const float for run-time, so your code comments would be 
wrong.


 If x is the amplitude then a flaw like this can cause a DC 
 offset to accumulate and you end up with reduced precision, not 
 improved precision. A DC filter at the end does not help on 
 this precision loss.


I don't understand why you're using const for one block and not 
the other, seems like a contrived example.  If the precision of 
such constants matters so much, I'd be careful to use the same 
const float everywhere.


 So you're planning on running phase-locking code partially in 
 CTFE and runtime and it's somehow very sensitive to precision?
  If your "phase-locking" depends on producing bit-exact 
 results with floating-point, you're doing it wrong.


 I am not doing anything wrong. D is doing it wrong. If you add 
 different deltas then you will get drift. So, no improved 
 precision in calculating some deltas is not improving the 
 accuracy. It makes it worse.


If matching such small deltas matters so much, I wouldn't be 
using floating-point in the first place.


 If any of this depends on comparing bit-exact floating-point 
 results, you're doing it wrong.


 It depends on the unit tests running with the exact same 
 precision as the production code.


What makes you think they don't?


 Fast floating point code depends on the specifics of the 
 hardware. A system level language should not introduce a 
 different kind of bias that isn't present in the hardware!


He's doing this to take advantage of the hardware, not the 
opposite!


 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.


If programmers want to run their code 10-100x slower to get 
reliably inaccurate results, that is their problem.


 If the constant is calculated rather than a literal, you 
 should be checking using approxEqual.


 No. 1+2+3+4 is exact on all floating point units I know of.


If you're so convinced it's exact for a few cases, then check 
exact equality there.  For most calculation, you should be using 
approxEqual.


 Btw, have you ever tried to prove error bounds for an iterative 
 method?
 You actually think most people prove them to be correct?

 Or perhaps almost all of them just pick a number out of thin 
 air which they think will work out and rely on testing their 
 code?


No, I have never done so, and I'm well aware that most just pull 
the error bound they use out of thin air.


 Well, the latter is no better than checking for exact equality. 
 And I can assure you that the vast majority of programmers do 
 not prove error bounds with the level of rigour it takes to get 
 it correct.


Even an unproven error bound is better than "exact" equality, 
which technically is really assuming that the error bound is 
smaller than the highest precision you can specify the number.  
Since the real error bound is always larger than that, almost any 
error bound you pick will tend to be closer to the real error 
bound, or at least usually bigger and therefore more realistic, 
than checking for exact equality.


 The reality is that it is common practice to write code that 
 seems to work. But that does not make it correct. However, 
 making it correct is way too time consuming and often not worth 
 the trouble. So people rely on testing. Floating point code is 
 no different.

 But with D semantics you cannot rely on testing. That's bad, 
 because most people write incorrect code. Whether they are 
 experts or not. (it is only matter of difference in frequency)


You can still test with approxEqual, so I don't understand why 
you think that's not testing.  std.math uses feqrel, approxEqual, 
and equalsDigit and other such lower-precision checks extensively 
in its tests.


 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?


 It would make more sense to figure out what the max value of 
 f(x) is you're trying to avoid, say 1e6, and then check for 
 approxEqual(x, 2, 2e-6).  That would make much more sense than 
 only avoiding 2, when an x that is arbitrarily close to 2 can 
 also blow up f(x).


 I am trying to avoid an exception, not a value.


That is the problem.  In the real world, all such formulas are 
approximations that only apply over a certain range.  If you were 
cranking that out by hand and some other calculation gave you an 
x really close to 2, say 2.0000035, you'd go back and check your 
math, as f(x) would blow up and give you unrealistically large 
numbers for the rest of your calculations.

The computer doesn't know that, so it will just plug that x in 
and keep cranking, till you get nonsense data out the end, if you 
don't tell it to check that x isn't too close to 2 and not just 2.

You have a wrong mental model that the math formulas are the 
"real world," and that the computer is mucking it up.  The truth 
is that the computer, with its finite maximums and bounded 
precision, better models _the measurements we make to estimate 
the real world_ than any math ever written.


 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.


 Which number would that be?


I told you, any numbers too close to 2.


 Simply repeating the word "random" over and over again does 
 not make it so.


 That's right. It is DMD that makes it so, not my words.

 However, in order to reject what other say, you have to make an 
 argument. And in this case we have:

 1. A system level programming language that claims to excel at 
 floating point.
 2. Hardware platforms with specific behaviour.

 Unfortunately 2 is true, but not 1.

 D is not matching up to the minimum requirements for people 
 wanting to write fast and reliable floating point code for a 
 given platform.


On the contrary, it is done because 80-bit is faster and more 
precise, whereas your notion of reliable depends on an incorrect 
notion that repeated bit-exact results are better.


 According to the D spec, the compiler could schedule typed 
 single precision floating point calculations to two completely 
 different floating point units (and yes, there are platforms 
 that provide multiple incompatible floating point units with 
 widely differing characteristics).


You noted that you don't care that the C++ spec says similar 
things, so I don't see why you care so much about the D spec now. 
  As for that scenario, nobody has suggested it.


 That is random.


It may be arbitrary, but it is not random unless it's 
inconsistently done.


 And so is "float" behaving differently than "const float".


I don't believe it does.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:

 On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
 wrote:

 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:

 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.


 Can you please quote that well-defined rule?


 It appears to be "the compiler carries everything internally to 
 80 bit precision, even if they are typed as some other 
 precision."
 http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com


"The compiler" means: implementation defined. That is the same as 
not being well-defined. :-)


 It is indeed random, or arbitrary (which is the same thing):


 No, they're not the same thing: rules can be arbitrarily set 
 yet consistent over time, whereas random usually means both 
 arbitrary and inconsistent over time.


In this case it is the same thing then. I have no guarantee that 
my unit tests and production code will behave the same.


 I believe that means any calculation used to compute y at 
 compile-time will be done in 80-bit or larger reals, then 
 rounded to a const float for run-time, so your code comments 
 would be wrong.


No. The "const float y" will not be coerced to 32 bit, but the 
"float y" will be coerced to 32 bit. So you get two different y 
values. (On a specific compiler, i.e. DMD.)


 I don't understand why you're using const for one block and not 
 the other, seems like a contrived example.  If the precision of 
 such constants matters so much, I'd be careful to use the same 
 const float everywhere.


Now, that is a contrived defense for brittle language semantics! 
:-)


 If matching such small deltas matters so much, I wouldn't be 
 using floating-point in the first place.


Why not? The hardware gives the same delta. It only goes wrong if 
the compiler decides to "improve".


 It depends on the unit tests running with the exact same 
 precision as the production code.


 What makes you think they don't?


Because the language says that I cannot rely on it and the 
compiler implementation proves that to be correct.


 Fast floating point code depends on the specifics of the 
 hardware. A system level language should not introduce a 
 different kind of bias that isn't present in the hardware!


 He's doing this to take advantage of the hardware, not the 
 opposite!


I don't understand what you mean.  He is not taking advantage of 
the hardware?



 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.


 If programmers want to run their code 10-100x slower to get 
 reliably inaccurate results, that is their problem.


Huh?

What I said is that D is doing it wrong because the 
"improvements" is forcing me to write code that is 10-100x slower 
to get the same level of reliability and required accuracy as I 
would get without the "improvements".



 If you're so convinced it's exact for a few cases, then check 
 exact equality there.  For most calculation, you should be 
 using approxEqual.


I am sorry, but this is not a normative rule at all. The rule is 
that you check for the bounds required. If it is exact, it just 
means the bounds are the same value (e.g. tight).

It does not help to say that people should use "approxEqual", 
because it does not improve on correctness. Saying such things 
just means that non-expert programmers assume that guessing the 
bounds will be sufficient. Well, it isn't sufficient.



 Since the real error bound is always larger than that, almost 
 any error bound you pick will tend to be closer to the real 
 error bound, or at least usually bigger and therefore more 
 realistic, than checking for exact equality.


I disagree. It is much better to get extremely wrong results 
frequently and therefore detect the error in testing.

What you are saying is that is better to get extremely wrong 
results infrequently which usually leads to error passing testing 
and enter production.

In order to test well you also need to understand for input makes 
the algorithm unstable/fragile.



 You can still test with approxEqual, so I don't understand why 
 you think that's not testing.


It is not testing anything if the compiler can change the 
semantics when you use a different context.



 The computer doesn't know that, so it will just plug that x in 
 and keep cranking, till you get nonsense data out the end, if 
 you don't tell it to check that x isn't too close to 2 and not 
 just 2.


Huh? I am not getting nonsense data. I am getting what I am 
asking for, I only want to avoid dividing by zero because it will 
make the given hardware 100x slower than the test.



 You have a wrong mental model that the math formulas are the 
 "real world," and that the computer is mucking it up.


Nothing wrong with my mental model. My mental model is the 
hardware specification + the specifics of the programming 
platform. That is the _only_ model that matters.

What D prevents me from getting is the specifics of the 
programming platform by making the specifics hidden.



 The truth is that the computer, with its finite maximums and 
 bounded precision, better models _the measurements we make to 
 estimate the real world_ than any math ever written.


I am not estimating anything. I am synthesising artificial 
worlds. My code is the model, the world is my code running at 
specific hardware.

It is self contained. I don't want the compiler to change my 
model because that will generate the wrong world. ;-)



 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.


 Which number would that be?


 I told you, any numbers too close to 2.


All numbers close to 2 in the same precision will work out ok.



 On the contrary, it is done because 80-bit is faster and more 
 precise, whereas your notion of reliable depends on an 
 incorrect notion that repeated bit-exact results are better.


80 bit is much slower. 80 bit mul takes 3 micro ops, 64 bit takes 
1. Without SIMD 64 bit is at least twice as fast. With SIMD 
multiply-add is maybe 10x faster in 64bit.


And it is neither more precise or more accurate when you don't 
get consistent precision.

In the real world you can get very good performance for the 
desired accuracy by using unstable algorithms by adding a stage 
that compensate for the instability. That does not mean that it 
is acceptable to have differences in the bias as that can lead to 
accumulating an offset that brings the result away from zero 
(thus a loss of precision).



 You noted that you don't care that the C++ spec says similar 
 things, so I don't see why you care so much about the D spec 
 now.
  As for that scenario, nobody has suggested it.


I care about what the C++ spec. I care about how the platform 
interprets the spec. I never rely on _ONLY_ the C++ spec for 
production code.

You have said previously that you know the ARM platform. On Apple 
CPUs you have 3 different floating point units: 32 bit NEON, 64 
bit NEON and 64 bit IEEE.

It supports 1x64bit IEEE, 2x64bit NEON and 4x32 bit NEON.

You have to know the language, the compiler and the hardware to 
make this work out.



 And so is "float" behaving differently than "const float".


 I don't believe it does.


I have proven that it does, and posted it in this thread.
May 18 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:

 No. The "const float y" will not be coerced to 32 bit, but the 
 "float y" will be coerced to 32 bit. So you get two different y 
 values. (On a specific compiler, i.e. DMD.)


I'm not sure that the `const float` vs `float` is the difference 
per se.  The difference is that in the examples you've given, the 
`const float` is being determined (and used) at compile time.

But a `const float` won't _always_ be determined or used at 
compile time, depending on the context and manner in which the 
value is set.

Let's be clear about the problem -- compile time vs. runtime, 
rather than `const` vs non-`const`.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 11:12:16 UTC, Joseph Rushton 
Wakeling wrote:

 I'm not sure that the `const float` vs `float` is the 
 difference per se.  The difference is that in the examples 
 you've given, the `const float` is being determined (and used) 
 at compile time.


They both have to be determined at compile time... as there is 
only a cast involved. The real issue is that the const float 
binding is treated textually.


 But a `const float` won't _always_ be determined or used at 
 compile time, depending on the context and manner in which the 
 value is set.


Which makes the problem worse, not better.
May 18 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:

 On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
 wrote:

 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:

 There is nothing "random" about increasing precision till 
 the end, it follows a well-defined rule.


 Can you please quote that well-defined rule?


 It appears to be "the compiler carries everything internally 
 to 80 bit precision, even if they are typed as some other 
 precision."
 http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com


 "The compiler" means: implementation defined. That is the same 
 as not being well-defined. :-)


Welcome to the wonderful world of C++! :D

More seriously, it is well-defined for that implementation, you 
did not raise the issue of the spec till now.  In fact, you 
seemed not to care what the specs say.


 I don't understand why you're using const for one block and 
 not the other, seems like a contrived example.  If the 
 precision of such constants matters so much, I'd be careful to 
 use the same const float everywhere.


 Now, that is a contrived defense for brittle language 
 semantics! :-)


No, it has nothing to do with language semantics and everything 
to do with bad numerical programming.


 If matching such small deltas matters so much, I wouldn't be 
 using floating-point in the first place.


 Why not? The hardware gives the same delta. It only goes wrong 
 if the compiler decides to "improve".


Because floating-point is itself fuzzy, in so many different 
ways.  You are depending on exactly repeatable results with a 
numerical type that wasn't meant for it.


 It depends on the unit tests running with the exact same 
 precision as the production code.


 What makes you think they don't?


 Because the language says that I cannot rely on it and the 
 compiler implementation proves that to be correct.


You keep saying this: where did anyone mention unit tests not 
running with the same precision till you just brought it up out 
of nowhere?  The only prior mention was that compile-time 
calculation of constants that are then checked for bit-exact 
equality in the tests might have problems, but that's certainly 
not all tests and I've repeatedly pointed out you should never be 
checking for bit-exact equality.


 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.


 If programmers want to run their code 10-100x slower to get 
 reliably inaccurate results, that is their problem.


 Huh?


The point is that what you consider reliable will be less 
accurate, sometimes much less.


 If you're so convinced it's exact for a few cases, then check 
 exact equality there.  For most calculation, you should be 
 using approxEqual.


 I am sorry, but this is not a normative rule at all. The rule 
 is that you check for the bounds required. If it is exact, it 
 just means the bounds are the same value (e.g. tight).

 It does not help to say that people should use "approxEqual", 
 because it does not improve on correctness. Saying such things 
 just means that non-expert programmers assume that guessing the 
 bounds will be sufficient. Well, it isn't sufficient.


The point is that there are _always_ bounds, so you can never 
check for the same value.  Almost any guessed bounds will be 
better than incorrectly checking for the bit-exact value.


 Since the real error bound is always larger than that, almost 
 any error bound you pick will tend to be closer to the real 
 error bound, or at least usually bigger and therefore more 
 realistic, than checking for exact equality.


 I disagree. It is much better to get extremely wrong results 
 frequently and therefore detect the error in testing.

 What you are saying is that is better to get extremely wrong 
 results infrequently which usually leads to error passing 
 testing and enter production.

 In order to test well you also need to understand for input 
 makes the algorithm unstable/fragile.


Nobody is talking about the general principle of how often you 
get wrong results or unit testing.  We were talking about a very 
specific situation: how should compile-time constants be checked 
and variables compared to constants, compile-time or not, to 
avoid exceptional situations.  My point is that both should 
always be thought about.  In the latter case, ie your f(x) 
example, it has nothing to do with error bounds, but that your 
f(x) is not only invalid at 2, but in a range around 2.

Now, both will lead to less "wrong results," but those are wrong 
results you _should_ be trying to avoid as early as possible.


 The computer doesn't know that, so it will just plug that x in 
 and keep cranking, till you get nonsense data out the end, if 
 you don't tell it to check that x isn't too close to 2 and not 
 just 2.


 Huh? I am not getting nonsense data. I am getting what I am 
 asking for, I only want to avoid dividing by zero because it 
 will make the given hardware 100x slower than the test.


Zero is not the only number that screws up that calculation.


 You have a wrong mental model that the math formulas are the 
 "real world," and that the computer is mucking it up.


 Nothing wrong with my mental model. My mental model is the 
 hardware specification + the specifics of the programming 
 platform. That is the _only_ model that matters.

 What D prevents me from getting is the specifics of the 
 programming platform by making the specifics hidden.


Your mental model determines what you think is valid input to 
f(x) and what isn't, that has nothing to do with D.  You want D 
to provide you a way to only check for 0.0, whereas my point is 
that there are many numbers in the neighborhood of 0.0 which will 
screw up your calculation, so really you should be using 
approxEqual.


 The truth is that the computer, with its finite maximums and 
 bounded precision, better models _the measurements we make to 
 estimate the real world_ than any math ever written.


 I am not estimating anything. I am synthesising artificial 
 worlds. My code is the model, the world is my code running at 
 specific hardware.

 It is self contained. I don't want the compiler to change my 
 model because that will generate the wrong world. ;-)


It isn't changing your model, you can always use a very small 
threshold in approxEqual. Yes, a few more values would be 
disallowed as input and output than if you were to compare 
exactly to 0.0, but your model is almost certainly undefined 
there too.

If your point is that you're modeling artificial worlds that have 
nothing to do with reality, you can always change your threshold 
around 0.0 to be much smaller, and who cares if it can't go all 
the way to zero, it's all artificial, right? :) If you're 
modeling the real world, any function that blows up and gives you 
bad data, blows up over a range, never a single point, because 
that's how measurement works.


 Oh, it's real world alright, you should be avoiding more 
 than just 2 in your example above.


 Which number would that be?


 I told you, any numbers too close to 2.


 All numbers close to 2 in the same precision will work out ok.


They will give you large numbers that can be represented in the 
computer, but do not work out to describe the real world, because 
such formulas are really invalid in a neighborhood of 2, not just 
at 2.


 On the contrary, it is done because 80-bit is faster and more 
 precise, whereas your notion of reliable depends on an 
 incorrect notion that repeated bit-exact results are better.


 80 bit is much slower. 80 bit mul takes 3 micro ops, 64 bit 
 takes 1. Without SIMD 64 bit is at least twice as fast. With 
 SIMD multiply-add is maybe 10x faster in 64bit.


I have not measured this speed myself so I can't say.


 And it is neither more precise or more accurate when you don't 
 get consistent precision.

 In the real world you can get very good performance for the 
 desired accuracy by using unstable algorithms by adding a stage 
 that compensate for the instability. That does not mean that it 
 is acceptable to have differences in the bias as that can lead 
 to accumulating an offset that brings the result away from zero 
 (thus a loss of precision).


A lot of hand-waving about how more precision is worse, with no 
real example, which is what Walter keeps asking for.


 You noted that you don't care that the C++ spec says similar 
 things, so I don't see why you care so much about the D spec 
 now.
  As for that scenario, nobody has suggested it.


 I care about what the C++ spec. I care about how the platform 
 interprets the spec. I never rely on _ONLY_ the C++ spec for 
 production code.


Then you must be perfectly comfortable with a D spec that says 
similar things. ;)


 You have said previously that you know the ARM platform. On 
 Apple CPUs you have 3 different floating point units: 32 bit 
 NEON, 64 bit NEON and 64 bit IEEE.

 It supports 1x64bit IEEE, 2x64bit NEON and 4x32 bit NEON.

 You have to know the language, the compiler and the hardware to 
 make this work out.


Sure, but nobody has suggested interchanging the three randomly.


 And so is "float" behaving differently than "const float".


 I don't believe it does.


 I have proven that it does, and posted it in this thread.


I don't think that example has much to do with what we're talking 
about.  It appears to be some sort of constant folding in the 
assert that produces the different results, as Joe says, which 
goes away if you use approxEqual.  If you look at the actual 
const float initially, it is very much a float, contrary to your 
assertions.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:

 Welcome to the wonderful world of C++! :D

 More seriously, it is well-defined for that implementation, you 
 did not raise the issue of the spec till now.  In fact, you 
 seemed not to care what the specs say.


Eh? All C/C++ compilers I have ever used coerce to single 
precision upon binding.

I care about what the spec says, why it says it and how it is 
used when targetting the specific hardware.

Or more generally, when writing libraries I target the language 
spec, when writing applications I target the platform (language + 
compiler + hardware).


 No, it has nothing to do with language semantics and everything 
 to do with bad numerical programming.


No. You can rant all you want about bad numerical programming. 
But by your definition all DSP programmers are bad at numerics. 
Which _obviously_ is not true.

That is grasping for straws.

There is NO excuse for preventing programmers from writing 
reliable performant code that solves the problem at hand to the 
accuracy that is required by the application.


 Because floating-point is itself fuzzy, in so many different 
 ways.  You are depending on exactly repeatable results with a 
 numerical type that wasn't meant for it.


Floating point is not fuzzy. It is basically a random sample on 
an interval of potential solutions with a range that increase 
with each computation. Unfortunately, you have to use interval 
arithmetics to get that interval, as in regular floating point 
you loose the information about the interval. It is an 
approximation. IEEE 754-2008 makes it possible to get that 
interval, btw.

Fuzzy is different. Fuzzy means that the range itself is the 
value. It does not represent an approximation. ;-)



 You keep saying this: where did anyone mention unit tests not 
 running with the same precision till you just brought it up out 
 of nowhere?  The only prior mention was that compile-time 
 calculation of constants that are then checked for bit-exact 
 equality in the tests might have problems, but that's certainly 
 not all tests and I've repeatedly pointed out you should never 
 be checking for bit-exact equality.


I have stated time and time again that it is completely 
unacceptable if there is even a remote chance for anything in the 
unit test to evaluate at a higher precision than in the 
production code.

That is not a unit test, it is a sham.


 The point is that what you consider reliable will be less 
 accurate, sometimes much less.


Define accurate. Accurate has no meaning without a desired 
outcome to reference.

I care about 4 oscillators having the same phase. THAT MEANS: 
they all have to use the exact same constants.

If not, they _will_ drift and phase cancellation _will_ occur.

I care about adding and removing the same constant. If not, 
(more) DC offsets will build up.

It is all about getting below the right tolerance threshold while 
staying real time. You can compensate by increasing the control 
rate (reduce the number of interpolated values).



 The point is that there are _always_ bounds, so you can never 
 check for the same value.  Almost any guessed bounds will be 
 better than incorrectly checking for the bit-exact value.


No. Guessed bounds are not better. I have never said that anyone 
should _incorrectly_ do anything. I have said that they should 
_correctly_ understand what they are doing and that guessing 
bounds just leads to a worse problem:

Programs that intermittently fail in spectacular ways that are 
very difficult to debug.

You cannot even compute the bounds if the compiler can use higher 
precision. IEEE754-2008 makes it possible to accurately compute 
bounds.

Not supporting that is _very_ bad.



 should always be thought about.  In the latter case, ie your 
 f(x) example, it has nothing to do with error bounds, but that 
 your f(x) is not only invalid at 2, but in a range around 2.


It isn't. For what typed number besides 2 is it invalid?


 Zero is not the only number that screws up that calculation.


It is. Not only can it screw up the calculation. It can screw up 
the real time properties of the algorithm on a specific FPU. 
Which is even worse.


 f(x) and what isn't, that has nothing to do with D.  You want D 
 to provide you a way to only check for 0.0, whereas my point is 
 that there are many numbers in the neighborhood of 0.0 which 
 will screw up your calculation, so really you should be using 
 approxEqual.


What IEEE32 number besides 2 can cause problems?


 It isn't changing your model, you can always use a very small


But it is!


 If your point is that you're modeling artificial worlds that 
 have nothing to do with reality, you can always change your 
 threshold around 0.0 to be much smaller, and who cares if it 
 can't go all the way to zero, it's all artificial, right? :)


Why would I have a threshold around 2, when only 2 is causing 
problems at the hardware level?



 If you're modeling the real world, any function that blows up 
 and gives you bad data, blows up over a range, never a single 
 point, because that's how measurement works.


If I am analyzing real world data I _absolutely_ want all code 
paths to use the specified precision. I absolutely don't want to 
wonder whether some computations have a different pattern than 
others because of the compiler.

Now, providing 64, 128 or 256 bits mantissa and software 
emulation is _perfectly_ sound. Computing 10 and 24 bits 
mantissas as if they are 256 bits without the programmer 
specifying it is bad.

And yes, half-precision is only 10 bits.


 They will give you large numbers that can be represented in the 
 computer, but do not work out to describe the real world, 
 because such formulas are really invalid in a neighborhood of 
 2, not just at 2.


I don't understand what you mean. Even Inf as an outcome tells me 
something. Or in the case of simulation Inf might be completely 
valid input for the next stage.


 I have not measured this speed myself so I can't say.


www.agner.org

It is of course even worse for 32 bit floats. Then we are at 
10x-20x faster than 80bit.


 A lot of hand-waving about how more precision is worse, with no 
 real example, which is what Walter keeps asking for.


I have provided plenty of examples.

You guys just don't want to listen. Because it is against the 
sects religious beliefs that D falls short of expectations both 
on integers and floating point.

This keeps D at the hobby level as a language. It is a very 
deep-rooted cultural problem.

But that is ok. Just don't complain about people saying that D is 
not fit for production. Because they have several good reasons to 
say that.
May 18 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

 And yes, half-precision is only 10 bits.


Actually, it turns out that the mantissa is 11 bits. So it 
clearly plays louder than other floats. ;-)
May 18 2016
parent reply Matthias Bentrup <matthias.bentrup googlemail.com> writes:
On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
 wrote:

 And yes, half-precision is only 10 bits.


 Actually, it turns out that the mantissa is 11 bits. So it 
 clearly plays louder than other floats. ;-)


The mantissa is 10 bits, but it has 11 bit precision, just as the 
float type has a 23 bit mantissa and 24 bit precision. AFAIK the 
only float format that stores the always-one-bit of the mantissa 
is the x87 80 bit format.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 15:30:42 UTC, Matthias Bentrup wrote:

 On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
 wrote:

 On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
 wrote:

 And yes, half-precision is only 10 bits.


 Actually, it turns out that the mantissa is 11 bits. So it 
 clearly plays louder than other floats. ;-)


 The mantissa is 10 bits, but it has 11 bit precision, just as 
 the float type has a 23 bit mantissa and 24 bit precision. 
 AFAIK the only float format that stores the always-one-bit of 
 the mantissa is the x87 80 bit format.


It turns out both 10bits and 11bits are "right" and that one 
should qualify it with «with/without the hidden bit», except that 
it shouldn't be called the «mantissa», but the «significand»:

https://en.wikipedia.org/wiki/Significand

Thanks, always fun with learning  ;-)
May 18 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:

 Welcome to the wonderful world of C++! :D

 More seriously, it is well-defined for that implementation, 
 you did not raise the issue of the spec till now.  In fact, 
 you seemed not to care what the specs say.


 Eh? All C/C++ compilers I have ever used coerce to single 
 precision upon binding.

 I care about what the spec says, why it says it and how it is 
 used when targetting the specific hardware.

 Or more generally, when writing libraries I target the language 
 spec, when writing applications I target the platform (language 
 + compiler + hardware).


I see, so the fact that both the C++ and D specs say the same 
thing doesn't matter, and the fact that D also has the const 
float in your example as single-precision at runtime, contrary to 
your claims, none of that matters.


 No, it has nothing to do with language semantics and 
 everything to do with bad numerical programming.


 No. You can rant all you want about bad numerical programming. 
 But by your definition all DSP programmers are bad at numerics. 
 Which _obviously_ is not true.

 That is grasping for straws.

 There is NO excuse for preventing programmers from writing 
 reliable performant code that solves the problem at hand to the 
 accuracy that is required by the application.


No sane DSP programmer would write that like you did.


 Because floating-point is itself fuzzy, in so many different 
 ways.  You are depending on exactly repeatable results with a 
 numerical type that wasn't meant for it.


 Floating point is not fuzzy. It is basically a random sample on 
 an interval of potential solutions with a range that increase 
 with each computation. Unfortunately, you have to use interval 
 arithmetics to get that interval, as in regular floating point 
 you loose the information about the interval. It is an 
 approximation. IEEE 754-2008 makes it possible to get that 
 interval, btw.


Or in the vernacular, fuzzy. :) Of course, this is true of all 
non-integer calculation.


 Fuzzy is different. Fuzzy means that the range itself is the 
 value. It does not represent an approximation. ;-)


It's all approximations.


 You keep saying this: where did anyone mention unit tests not 
 running with the same precision till you just brought it up 
 out of nowhere?  The only prior mention was that compile-time 
 calculation of constants that are then checked for bit-exact 
 equality in the tests might have problems, but that's 
 certainly not all tests and I've repeatedly pointed out you 
 should never be checking for bit-exact equality.


 I have stated time and time again that it is completely 
 unacceptable if there is even a remote chance for anything in 
 the unit test to evaluate at a higher precision than in the 
 production code.

 That is not a unit test, it is a sham.


Since the vast majority of tests will never use such compile-test 
constants, your opinion is not only wrong but irrelevant.


 The point is that what you consider reliable will be less 
 accurate, sometimes much less.


 Define accurate. Accurate has no meaning without a desired 
 outcome to reference.

 I care about 4 oscillators having the same phase. THAT MEANS: 
 they all have to use the exact same constants.

 If not, they _will_ drift and phase cancellation _will_ occur.

 I care about adding and removing the same constant. If not, 
 (more) DC offsets will build up.

 It is all about getting below the right tolerance threshold 
 while staying real time. You can compensate by increasing the 
 control rate (reduce the number of interpolated values).


Then don't use differently defined constants in different places 
and don't use floating point, simple.


 The point is that there are _always_ bounds, so you can never 
 check for the same value.  Almost any guessed bounds will be 
 better than incorrectly checking for the bit-exact value.


 No. Guessed bounds are not better. I have never said that 
 anyone should _incorrectly_ do anything. I have said that they 
 should _correctly_ understand what they are doing and that 
 guessing bounds just leads to a worse problem:

 Programs that intermittently fail in spectacular ways that are 
 very difficult to debug.

 You cannot even compute the bounds if the compiler can use 
 higher precision. IEEE754-2008 makes it possible to accurately 
 compute bounds.

 Not supporting that is _very_ bad.


If you're comparing bit-exact equality, you're not using _any_ 
error bounds: that's the worst possible choice.


 should always be thought about.  In the latter case, ie your 
 f(x) example, it has nothing to do with error bounds, but that 
 your f(x) is not only invalid at 2, but in a range around 2.


 It isn't. For what typed number besides 2 is it invalid?


 Zero is not the only number that screws up that calculation.


 It is. Not only can it screw up the calculation. It can screw 
 up the real time properties of the algorithm on a specific FPU. 
 Which is even worse.


 f(x) and what isn't, that has nothing to do with D.  You want 
 D to provide you a way to only check for 0.0, whereas my point 
 is that there are many numbers in the neighborhood of 0.0 
 which will screw up your calculation, so really you should be 
 using approxEqual.


 What IEEE32 number besides 2 can cause problems?


Sigh, any formula f(x) like the one you present is an 
approximation for the real world.  The fact that it blows up at 2 
means they couldn't measure any values there, so they just stuck 
that singularity in the formula.  That means it's not merely 
undefined at 2, but that they couldn't take measurements _around_ 
2, which is why any number close to 2 will give you unreasonably 
large output.

Simply blindly plugging that formula in and claiming that it's 
only invalid at 2 and nowhere else means you don't really 
understand the math, nor the science that underlies it.


 It isn't changing your model, you can always use a very small


 But it is!


Not in any meaningful way.


 If your point is that you're modeling artificial worlds that 
 have nothing to do with reality, you can always change your 
 threshold around 0.0 to be much smaller, and who cares if it 
 can't go all the way to zero, it's all artificial, right? :)


 Why would I have a threshold around 2, when only 2 is causing 
 problems at the hardware level?


Because it's not only the hardware that matters, the validity of 
your formula in the neighborhood of 2 also matters.


 If you're modeling the real world, any function that blows up 
 and gives you bad data, blows up over a range, never a single 
 point, because that's how measurement works.


 If I am analyzing real world data I _absolutely_ want all code 
 paths to use the specified precision. I absolutely don't want 
 to wonder whether some computations have a different pattern 
 than others because of the compiler.

 Now, providing 64, 128 or 256 bits mantissa and software 
 emulation is _perfectly_ sound. Computing 10 and 24 bits 
 mantissas as if they are 256 bits without the programmer 
 specifying it is bad.

 And yes, half-precision is only 10 bits.


None of this has anything to do with the point you responded to.


 They will give you large numbers that can be represented in 
 the computer, but do not work out to describe the real world, 
 because such formulas are really invalid in a neighborhood of 
 2, not just at 2.


 I don't understand what you mean. Even Inf as an outcome tells 
 me something. Or in the case of simulation Inf might be 
 completely valid input for the next stage.


Then why are you checking for it?  This entire example was 
predicated on your desire to avoid Inf at precisely 2.  My point 
is that it's not only Inf that has to be avoided, other output 
can be finite and still worthless, because the formula doesn't 
really apply there. So you should really be using approxEqual, as 
anywhere else with floating point math, and your example is wrong.


 A lot of hand-waving about how more precision is worse, with 
 no real example, which is what Walter keeps asking for.


 I have provided plenty of examples.


No, you only provided one, the one about const float, that had 
little relevance.  The rest was a lot of hand-waving about vague 
scenarios, with no actual example.


 You guys just don't want to listen. Because it is against the 
 sects religious beliefs that D falls short of expectations both 
 on integers and floating point.

 This keeps D at the hobby level as a language. It is a very 
 deep-rooted cultural problem.

 But that is ok. Just don't complain about people saying that D 
 is not fit for production. Because they have several good 
 reasons to say that.


Funny, considering this entire thread has had many making 
countervailing claims about the choices Walter has made for D.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 15:42:56 UTC, Joakim wrote:

 I see, so the fact that both the C++ and D specs say the same 
 thing doesn't matter, and the fact that D also has the const 
 float in your example as single-precision at runtime, contrary 
 to your claims, none of that matters.


D doesn't even have a spec, so how can they possibly say the same 
thing?

However, quoting the Wikipedia page on IEEE floats:

«The IEEE 754-1985 allowed many variations in implementations 
(such as the encoding of some values and the detection of certain 
exceptions). IEEE 754-2008 has strengthened up many of these, but 
a few variations still remain (especially for binary formats). 
The reproducibility clause recommends that language standards 
should provide a means to write reproducible programs (i.e., 
programs that will produce the same result in all implementations 
of a language), and describes what needs to be done to achieve 
reproducible results.»

That's the map people who care about floating point follow.



 No sane DSP programmer would write that like you did.


What kind of insult is that?  I've read lots of DSP code written 
by others.  I know what kind of programming it entails.

In fact, what I have described here are techniques picked up from 
state-of-the-art DSP code written by top-the-of-line DSP 
programmers.



 Since the vast majority of tests will never use such 
 compile-test constants, your opinion is not only wrong but 
 irrelevant.


Oh... Not only am I wrong, but my opinion is irrelevant. Well, 
with this attitude D will remain irrelevant as well.

For good reasons.



 Then don't use differently defined constants in different places


I don't, and I didn't. DMD did it.
May 18 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 18 May 2016 at 07:49, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:

 There is nothing "random" about increasing precision till the end, it
 follows a well-defined rule.



 Can you please quote that well-defined rule?

 It is indeed random, or arbitrary (which is the same thing):

 if(x<0){
   // DMD choose 64 bit mantissa
   const float y = ...
   ...

 } else {
   // DMD choose 24 bit mantissa
   float y = ...
   ...
 }

 How is this not arbitrary?


Can you back that up statistically?  Try running this same operation
600 million times plot a graph for the result from each run for it so
we can get an idea of just how random or arbitrary it really is.

If you get the same result back each time, maybe it isn't as arbitrary
or random as you would have some believe.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:

 Can you back that up statistically?  Try running this same 
 operation 600 million times plot a graph for the result from 
 each run for it so we can get an idea of just how random or 
 arbitrary it really is.


Huh? This isn't about statistics. It is about math. The magnitude 
of the difference depends on what you do with the constant. It 
can be exponentially boosted.
May 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 3:46 AM, Ola Fosheim Grøstad wrote:

 On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:

 Can you back that up statistically?  Try running this same operation 600
 million times plot a graph for the result from each run for it so we can get
 an idea of just how random or arbitrary it really is.


 Huh? This isn't about statistics.


It is when you say 'random'. D's fp math is not random, it is completely 
deterministic, and I've told you so before. You've been pushing the 'random' 
notion here, and other people have picked it up and inferred they'd get random 
results every time they ran a D compiled program.

Please use correct meanings of words.
May 18 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 14 May 2016 at 18:46:50 UTC, Walter Bright wrote:

 On 5/14/2016 3:16 AM, John Colvin wrote:

 This is all quite discouraging from a scientific programmers 
 point of view.
 Precision is important, more precision is good, but 
 reproducibility and
 predictability are critical.


 I used to design and build digital electronics out of TTL 
 chips. Over time, TTL chips got faster and faster. The rule was 
 to design the circuit with a minimum signal propagation delay, 
 but never a maximum. Therefore, putting in faster parts will 
 never break the circuit.

 Engineering is full of things like this. It's sound engineering 
 practice. I've never ever heard of a circuit requiring a 
 resistor with 20% tolerance that would fail if a 10% tolerance 
 one was put in, for another example.


Should scientific software be written to not break if the 
floating-point precision is enhanced, and to allow greater 
precision to be used when the hardware supports it?  Sure.

However, that's not the same as saying that the choice of 
precision should be in the hands of the hardware, rather than the 
person building + running the program.  I for one would not like 
to have to spend time working out why my program was producing 
different results, just because I (say) switched from a machine 
supporting maximum 80-bit float to one supporting 128-bit.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 6:49 AM, Joseph Rushton Wakeling wrote:

 However, that's not the same as saying that the choice of precision should be
in
 the hands of the hardware, rather than the person building + running the
 program.




 I for one would not like to have to spend time working out why my
 program was producing different results, just because I (say) switched from a
 machine supporting maximum 80-bit float to one supporting 128-bit.


If you wrote it "to not break if the floating-point precision is enhanced, and 
to allow greater precision to be used when the hardware supports it" then
what's 
the problem?

Can you provide an example of a legitimate algorithm that produces degraded 
results if the precision is increased?
May 15 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 15 May 2016 at 18:30:20 UTC, Walter Bright wrote:

 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?


I've just been observing this argument (over and over again) and 
I think the two sides are talking about different things.

Walter, you keep saying degraded results.

The scientific folks keep saying *consistent* results.


Think about a key goal in scientific experiments: to measure 
changes across repeated experiments, to reproduce and confirm or 
falsify results. They want to keep as much equal as they can.

I suppose people figure if they use the same compiler, same build 
options, same source code and feed the same data into it, they 
expect to get the *same* results. It is a deterministic machine, 
right?

You might argue they should add "same hardware" to that list, but 
apparently it isn't that easy, or I doubt people would be talking 
about this.
May 15 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:41:57 UTC, Adam D. Ruppe wrote:

 I suppose people figure if they use the same compiler, same 
 build options, same source code and feed the same data into it, 
 they expect to get the *same* results. It is a deterministic 
 machine, right?


In this case it is worse, you get this:

float f = some_pure_function();
const float cf = some_pure_function();

assert(cf==f); // SHUTDOWN!!!

But Walter doesn't think this is an issue, because correctness is 
just a high school feature.
May 15 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:59:51 UTC, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 18:41:57 UTC, Adam D. Ruppe wrote:

 I suppose people figure if they use the same compiler, same 
 build options, same source code and feed the same data into 
 it, they expect to get the *same* results. It is a 
 deterministic machine, right?


 In this case it is worse, you get this:

 float f = some_pure_function();
 const float cf = some_pure_function();

 assert(cf==f); // SHUTDOWN!!!


That was too quick, but you get the idea:

assert((cf==N) == (f==N)); //SHUTDOWN!!
May 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:41 AM, Adam D. Ruppe wrote:

 Walter, you keep saying degraded results.

 The scientific folks keep saying *consistent* results.


Programs will not produce inconsistent results. You run the same program, you 
get the same results.

Scientists know that all measurements contain error. The idea is to identify 
what the maximum error is. I've never heard of a scientific experiment that 
required a minimum error.



 You might argue they should add "same hardware" to that list, but apparently it
 isn't that easy, or I doubt people would be talking about this.


You're arguing for floating point portability, which is something else again. 
Java tried that (discussed earlier) and it was a near complete failure.
May 15 2016
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 15 May 2016 at 18:30:20 UTC, Walter Bright wrote:

 If you wrote it "to not break if the floating-point precision 
 is enhanced, and to allow greater precision to be used when the 
 hardware supports it" then what's the problem?

 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?


No, but I think that's not really relevant to the point I was 
making.  Results don't have to be degraded to be inconsistent.
May 15 2016
prev sibling parent reply poliklosio <poliklosio happypizza.com> writes:
 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?


The real problem here is the butterfly effect (the chaos theory 
thing). Imagine programming a multiplayer game. Ideally you only 
need to synchronize user events, like key presses etc. Other 
computation can be duplicated on all machines participating in a 
session. Now imagine that some logic other than display (e.g. 
player-bullet collision detection) is using floating point. If 
those computations are not reproducible, a higher precision on 
one player's machine can lead to huge inconsistencies in game 
states between the machines (e.g. my character is dead on your 
machine but alive on mine)!
If the game developer cannot achieve reproducibility or it takes  
too much work, the workarounds can be wery costly. He can, for 
example, convert implementation to soft float or increase amount 
of synchronization over the network.

Also I think Adam is making a very good point about generl 
reproducibility here. If a researcher gets a little bit different 
results, he has to investigate why, because he needs to rule out 
all the serious mistakes that could be the cause of the 
difference. If he finds out that the source was an innocuous 
refactoring of some D code, he will be rightly frustrated that D 
has caused so much unnecessary churn.

I think the same problem can occur in mission-critical software 
which undergoes strict certification.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 1:49 PM, poliklosio wrote:

 Can you provide an example of a legitimate algorithm that produces degraded
 results if the precision is increased?


 The real problem here is the butterfly effect (the chaos theory thing). Imagine
 programming a multiplayer game. Ideally you only need to synchronize user
 events, like key presses etc. Other computation can be duplicated on all
 machines participating in a session. Now imagine that some logic other than
 display (e.g. player-bullet collision detection) is using floating point. If
 those computations are not reproducible, a higher precision on one player's
 machine can lead to huge inconsistencies in game states between the machines
 (e.g. my character is dead on your machine but alive on mine)!
 If the game developer cannot achieve reproducibility or it takes  too much
work,
 the workarounds can be wery costly. He can, for example, convert implementation
 to soft float or increase amount of synchronization over the network.


If you use the same program binary, you *will* get the same results.



 Also I think Adam is making a very good point about generl reproducibility
here.
 If a researcher gets a little bit different results, he has to investigate why,
 because he needs to rule out all the serious mistakes that could be the cause
of
 the difference. If he finds out that the source was an innocuous refactoring of
 some D code, he will be rightly frustrated that D has caused so much
unnecessary
 churn.

 I think the same problem can occur in mission-critical software which undergoes
 strict certification.



Frankly, I think you are setting unreasonable expectations. Today, if you take
a 
standard compliant C program, and compile it with different switch settings, or 
run it on a machine with a different CPU, you can very well get different 
answers. If you reorganize the code expressions, you can very well get
different 
answers.
May 15 2016
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 08:05, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 1:49 PM, poliklosio wrote:


 Also I think Adam is making a very good point about generl reproducibility
 here.
 If a researcher gets a little bit different results, he has to investigate
 why,
 because he needs to rule out all the serious mistakes that could be the
 cause of
 the difference. If he finds out that the source was an innocuous
 refactoring of
 some D code, he will be rightly frustrated that D has caused so much
 unnecessary
 churn.

 I think the same problem can occur in mission-critical software which
 undergoes
 strict certification.




 Frankly, I think you are setting unreasonable expectations. Today, if you
 take a standard compliant C program, and compile it with different switch
 settings, or run it on a machine with a different CPU, you can very well get
 different answers. If you reorganize the code expressions, you can very well
 get different answers.


The argument you used against me earlier was that it was unacceptable
for some C code pasted in D to behave differently than in C... but
here you've just destroyed your own argument by noting that C behaves
differently than itself.
The rest of us would never get away with this ;)


For reference:


 I think it's the only reasonable solution.





 It may be, but it is unusual and therefore surprising behavior.



 What is the problem with this behaviour I suggest?





 Code will do one thing in C, and the same code will do something unexpectedly
different in D.


So let's reopen the discussion from my first post that you dismissed?
If the situation is that C compilers produce no predictable behaviour
(as you claim above, and I agree), what is the harm to applying a
behaviour that actually works?
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 6:59 PM, Manu via Digitalmars-d wrote:

 The argument you used against me earlier was that it was unacceptable
 for some C code pasted in D to behave differently than in C... but
 here you've just destroyed your own argument by noting that C behaves
 differently than itself.


D nails down some C behavior that is specified as "implementation defined".
This 
is not being incompatible.
May 15 2016
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:

 On 5/13/2016 5:49 PM, Timon Gehr wrote:

 And even if higher precision helps, what good is a 
 "precision-boost" that e.g.
 disappears on 64-bit builds and then creates inconsistent 
 results?


 That's why I was thinking of putting in 128 bit floats for the 
 compiler internals.


  I almost wonder if the cent/ucent will be avaliable soon as 
types rather than reserved keywords.


  Anyways my two cents. I remember having issues doing a direct 
compare on an identical value when it was immutable which proved 
false, and this made no sense and still doesn't. Regardless I've 
tried to exhaustively do the compares and found some unexpected 
results.

     float f = 1.30;
     const float cf = 1.30;
     immutable If = 1.30;

     writeln(f == 1.30);     //false
     writeln(cf == 1.30);
     writeln(If == 1.30);

     writeln(f == 1.30f);
     writeln(cf == 1.30f);
     writeln(If == 1.30f);

     writeln(f == cf);
     writeln(f == If);       //false
     writeln(If == cf);

  The real reason it's false is because you're comparing against 
an immutable (constant), or my results lead me to believe that; 
Otherwise why could cf and If work?
May 14 2016
next sibling parent reply Binarydepth <binarydepth gmail.com> writes:
On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:

     writeln(f == 1.30f);
     writeln(cf == 1.30f);
     writeln(If == 1.30f);



I guess this returned True ?
May 14 2016
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 14 May 2016 at 20:54:44 UTC, Binarydepth wrote:

 On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:

     writeln(f == 1.30f);
     writeln(cf == 1.30f);
     writeln(If == 1.30f);



 I guess this returned True ?


I only noted the ones that returned false, since those are the 
interesting cases.
May 14 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:

  The real reason it's false is because you're comparing against 
 an immutable (constant), or my results lead me to believe that; 
 Otherwise why could cf and If work?


The "immutable" is of type real... Unfortunately the "cf" is also 
computing as real whereas "f" is implicitly converted to real 
from float. It makes comparing floating point types for equality 
implementation defined in D, i.e. unreliable.

The assignment to "cf" ought to have brought cf to 32 bit float.

So the "false" results are the correct ones, and the "true" are 
the unexpected ones where you compare different types.

import numpy
f=numpy.float32(1.30)
r=numpy.float64(1.30)
May 14 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 06:49:19 UTC, Ola Fosheim Grøstad wrote:

 On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:

  The real reason it's false is because you're comparing 
 against an immutable (constant), or my results lead me to 
 believe that; Otherwise why could cf and If work?


 The "immutable" is of type real... Unfortunately the "cf" is 
 also computing as real whereas "f" is implicitly converted to 
 real from float. It makes comparing floating point types for 
 equality implementation defined in D, I.e. unreliable.


  But the const and immutable still have to fit within the 
confines of the size provided, so the fact they were calculated 
as real doesn't matter when they are stored as floats. If all the 
results are inverted, I don't see how that works...

Although it does make sense that a float is being promoted up, 
and the lost precision vs the real is the problem... to which 
point it should issue a warning for different types.

Does this mean that instead for compares with a float/constant 
should be demoted down a level if it can? Although I'm sure using 
the f will force it to be correct.


But then why does f==If fail? It doesn't answer that critical 
question, since they should be identical.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 07:10:49 UTC, Era Scarecrow wrote:

  But the const and immutable still have to fit within the 
 confines of the size provided, so the fact they were calculated 
 as real doesn't matter when they are stored as floats. If all 
 the results are inverted, I don't see how that works...


The problem is that they aren't stored... It is implementation 
defined...

What is really bad about this is that you have:

	writeln(f==cf);  // true
	writeln(f==1.30);  // false
	writeln(cf==1.30); // true

hence the following fallacy:

((f==cf) && (cf==1.30)) implies (f != 1.30)

That is _very_ bad.


 Although it does make sense that a float is being promoted up, 
 and the lost precision vs the real is the problem...


The problem is that D "optimizes" out casts to floats so that 
"cast(real)cast(float)somereal" is replaced with "somereal":

writeln(1.30==cast(real)cast(float)1.30);  // true



 But then why does f==If fail? It doesn't answer that critical 
 question, since they should be identical.


Because D does not cancel out "cast(real)cast(float)1.30)" for 
"f", but does cancel it out for cf.

Or basically: it uses textual substitution (macro 
expansion/inlining) for "cf", but not for "f".

?
May 15 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:

 That is _very_ bad.


Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) in floating
point 
math? FP math is simply not the math you learned in high school. You'll have 
nothing but grief with it if you insist it must be the same.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:35:15 UTC, Walter Bright wrote:

 On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:

 That is _very_ bad.


 Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) 
 in floating point math? FP math is simply not the math you 
 learned in high school. You'll have nothing but grief with it 
 if you insist it must be the same.


Err... these kind of problems only applies to D.

Please leave your high school experiences out of it. :-)
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:45 AM, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 18:35:15 UTC, Walter Bright wrote:

 On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:

 That is _very_ bad.


 Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) in floating
 point math? FP math is simply not the math you learned in high school. You'll
 have nothing but grief with it if you insist it must be the same.


 Err... these kind of problems only applies to D.


Nope. They occur with every floating point implementation in every programming 
language. FP math does not adhere to associative identities.

   http://www.walkingrandomly.com/?p=5380

   http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Ironically, the identity is more likely to hold with D's extended precision for 
intermediate values than with other languages.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 21:01:14 UTC, Walter Bright wrote:

 Err... these kind of problems only applies to D.


 Nope. They occur with every floating point implementation in 
 every programming language. FP math does not adhere to 
 associative identities.


No. ONLY D give different results for the same pure function call 
because you bind the result to a "const float" rather than a 
"float".

Yes. Algorithms can break because of it.


 Ironically, the identity is more likely to hold with D's 
 extended precision for intermediate values than with other 
 languages.


No, it is not more likely to hold with D's hazard game. I don't 
know of a single language that doesn't heed a request to 
truncate/round the mantissa if it provides the means to do it.

I care about algorithms working they way I designed them to work 
and what I have tested them for. If I request rounding to a 24 
bit mantissa then I _expect_ the rounding to take place. And yes, 
it can break algorithms if you don't.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 2:36 PM, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 21:01:14 UTC, Walter Bright wrote:

 Err... these kind of problems only applies to D.


 Nope. They occur with every floating point implementation in every programming
 language. FP math does not adhere to associative identities.


 No. ONLY D give different results for the same pure function call because you
 bind the result to a "const float" rather than a "float".


It's a fact that with floating point arithmetic, (x+y)+z is not equal to 
x+(y+x), on every programming language, whether or not the result is bound to a 
const float or a float.



 Yes. Algorithms can break because of it.


So far, nobody has posted a legitimate one (i.e. not contrived).



 Ironically, the identity is more likely to hold with D's extended precision
 for intermediate values than with other languages.


 No, it is not more likely to hold with D's hazard game. I don't know of a
single
 language that doesn't heed a request to truncate/round the mantissa if it
 provides the means to do it.


Standard C provides no such hooks for constant folding at compile time. Neither 
does Standard C++. In fact I know of no language that does. Perhaps you can 
provide a link?



 I care about algorithms working they way I designed them to work and what I
have
 tested them for. If I request rounding to a 24 bit mantissa then I _expect_ the
 rounding to take place.


Example, please, of how you 'request' rounding/truncation.



 And yes, it can break algorithms if you don't.


Example, please.

----

Something you should know from the C++ Standard:

"The values of the floating operands and the results of floating expressions
may 
be represented in greater precision and range than that required by the type; 
the types are not changed thereby"  -- 5.0.11

D is clearly making FP arithmetic *more* predictable, not less. Since you care 
about FP results, I seriously suggest studying 
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html and what the C++ 
Standard actually says.
May 15 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 22:34:24 UTC, Walter Bright wrote:

 So far, nobody has posted a legitimate one (i.e. not contrived).


Oh, so comparing the exact same calculation, using the exact same 
binary executable function, is  not a legitimate algorithm. It is 
the most trivial thing to do and rather common. It should be 
_very_ convincing.

But hey, here is another one:

const real x = f();
assert(0<=x && x<1);
x += 1;

const float f32 = cast(float)(x);
const real residue = x - cast(real)f32; // ZERO!!!!

output(dither(f32, residue)); // DITHERING IS FUBAR!!!
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:14 PM, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 22:34:24 UTC, Walter Bright wrote:

 So far, nobody has posted a legitimate one (i.e. not contrived).


 Oh, so comparing the exact same calculation, using the exact same binary
 executable function, is  not a legitimate algorithm.


What algorithm relies on such? And as numerous links posted here show, it 
doesn't work reliably on just about any C/C++ compiler, either. If you're 
writing C++ code that relies on such, you're in for some tragic late night 
debugging sessions as Manu relates.



 It is the most trivial
 thing to do and rather common. It should be _very_ convincing.


Post a link to some algorithm that does that.



 But hey, here is another one:

 const real x = f();
 assert(0<=x && x<1);
 x += 1;

 const float f32 = cast(float)(x);
 const real residue = x - cast(real)f32; // ZERO!!!!

 output(dither(f32, residue)); // DITHERING IS FUBAR!!!


Why would anyone dither based on the difference in precision of floats and 
doubles? Dithering is usually done based on different pixel densities.

I'm sure you can (and did) write code contrived to show a difference. I asked 
for something different, though.
May 16 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 16 May 2016 00:52:41 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 On 5/15/2016 11:14 PM, Ola Fosheim Gr=C3=B8stad wrote:

 But hey, here is another one:

 const real x =3D f();
 assert(0<=3Dx && x<1);
 x +=3D 1;

 const float f32 =3D cast(float)(x);
 const real residue =3D x - cast(real)f32; // ZERO!!!!

 output(dither(f32, residue)); // DITHERING IS FUBAR!!! =20


=20
 Why would anyone dither based on the difference in precision of floats an=


d=20

 doubles? Dithering is usually done based on different pixel densities.


To be fair, several image editors have various dithering
algorithms that seek to reduce the apparent banding when
converting an image from a higher to a lower bit depth.
3dfx also performed dithering in hardware when going from
internal 24-bit RGB calculations to 16-bit output.
Ola's example could be some X-ray imaging format. Although
it is still a vague example, this "cast doesn't do anything
to floating point constants" semantic surprised me too, I
have to say. I don't want to dive too deep into the matter
though.

--=20
Marco
May 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 09:00:49 UTC, Marco Leise wrote:

 Ola's example could be some X-ray imaging format. Although
 it is still a vague example,


I don't think quantization is a particularly vague example! I 
think that is rather common.

Doing something like floor(x*(1<<24))*(1.0/(1<<24)) to quantize 
to 24 bit is just silly and inefficient.

Error diffusion is useful in scenarios where you want to reduce 
the accumulation-error or avoid banding. It can be used in audio 
in many other settings.

Anyway, irregular higher precision is almost always worse than 
regular lower precision when computing time series, comparing 
results or doing unit testing.
May 16 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 00:34, Walter Bright wrote:

 Yes. Algorithms can break because of it.


 So far, nobody has posted a legitimate one (i.e. not contrived).


My examples were not contrived. Manu's even less so.

What you call "illegitimate" are really just legitimate examples that 
you dismiss because they do not match your own specific experience. I 
think that's a little disrespectful.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 1:43 AM, Timon Gehr wrote:

 My examples were not contrived.


I don't see where you posted examples in this thread. You did link to two 
wikipedia articles, which seemed to be more conjecture than actual examples.



 Manu's even less so.


Manu has said he's had trouble with it. I believe him. But I haven't seen his 
code to see just what he was doing.



 What you call "illegitimate" are really just legitimate examples that you
 dismiss because they do not match your own specific experience.


Of course, legitimate is a matter of opinion. Can code be written to rely on 
lower precision? Of course. Is it portable to any standard conforming C/C++ 
compiler? Nope. Can algorithms be coded to not require reduced precision? I'm 
pretty sure they can be, and should be.

The links I've posted here make it clear that there is no "obvious" right way 
for compilers to implement FP. If you're going to write sensitive FP
algorithms, 
it only makes sense to take the arbitrariness of FP implementations into
account.

My proposal for D means less is implementation defined than C/C++, the
semantics 
fall within the "implementation defined" part of the C/C++ standards, and the 
implementation is completely IEEE compliant.

I don't believe it is a service to programmers to have the panopoly of FP 
behavior switches sported by typical compilers that apparently very few people 
understand. Worse, writing code that is invisibly, subtly, and critically 
dependent on specific compiler switches and versions is not something I can 
recommend as a best practice.



 I think that's a little disrespectful.


Perhaps, but disrespecting code is not an ad homenim argument.
May 16 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 09:56:07 UTC, Walter Bright wrote:

 What you call "illegitimate" are really just legitimate 
 examples that you
 dismiss because they do not match your own specific experience.


 Of course, legitimate is a matter of opinion. Can code be 
 written to rely on lower precision? Of course. Is it portable 
 to any standard conforming C/C++ compiler? Nope. Can algorithms 
 be coded to not require reduced precision? I'm pretty sure they 
 can be, and should be.


If I've understood people's arguments right, the point of concern 
is that there are use cases where the programmer wants to be able 
to guarantee _a specific precision of their choice_.

That strikes me as a legitimate use-case that it would be worth 
trying to support.
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:26 AM, Joseph Rushton Wakeling wrote:

 If I've understood people's arguments right, the point of concern is that there
 are use cases where the programmer wants to be able to guarantee _a specific
 precision of their choice_.

 That strikes me as a legitimate use-case that it would be worth trying to
support.


Yes, and I've proposed roundToFloat() and roundToDouble() intrinsics in this 
thread and the last two or three times this came up.

I also strongly feel that use of those intrinsics is a red flag that there's a 
problem with the algorithm. At least the use of them will make it obvious in
the 
code that there's a bug :-) in the algorithm.

Floats are chosen for speed and storage. Insisting that lower accuracy is 
desired as a default consequence just baffles me. It's like saying people
choose 
sugary snacks because they want dental cavities.
May 16 2016
prev sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 08:33:44 UTC, Ola Fosheim Grøstad wrote:

 Because D does not cancel out "cast(real)cast(float)1.30)" for 
 "f", but does cancel it out for cf.


  Hmmm...

  I noticed my code example fails to specify float for the 
immutable, fixing that only the first line with f == 1.30 fail 
while all others succeed (for some reason?), which is good news. 
Does that mean the error that I had noted a few years ago 
actually was fixed?
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 20:34:19 UTC, Era Scarecrow wrote:

  I noticed my code example fails to specify float for the 
 immutable, fixing that only the first line with f == 1.30 fail 
 while all others succeed (for some reason?), which is good news.


Well, it isn't actually good news. Both "const float" and 
"immutable float" should fail as you have requested coercion to 
32 bit floats which ought to change the value of "1.30", but 
unfortunately D does not heed the coercion.

The net result is that adding const/immutable to a type can 
change the semantics of the program entirely at the whim of the 
compiler implementor.

In comparison C++:

   float f = 1.30;
   const float c = 1.30;
   constexpr float i = 1.30;

   std::cout << (f==1.30) << std::endl;  // false
   std::cout << (c==1.30) << std::endl;  // false
   std::cout << (i==1.30) << std::endl;  // false
   std::cout << (1.30==(float)1.30) << std::endl;  // false
May 15 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 21:06:22 UTC, Ola Fosheim Grøstad wrote:

   std::cout << (f==1.30) << std::endl;  // false
   std::cout << (c==1.30) << std::endl;  // false
   std::cout << (i==1.30) << std::endl;  // false
   std::cout << (1.30==(float)1.30) << std::endl;  // false


If we want equality then we should compare to the representation 
for a 32 bit float:

  std::cout << (f == 1.2999999523162841796875) << std::endl; // 
true
May 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can change the
semantics
 of the program entirely at the whim of the compiler implementor.


C++ Standard allows the same increased precision, at the whim of the compiler 
implementor, as quoted to you earlier.

What your particular C++ compiler does is not relevant, as its behavior is not 
required by the Standard.

My proposal removes the "whim" by requiring 128 bit precision for CTFE.
May 15 2016
next sibling parent Seb <seb wilzba.ch> writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can 
 change the semantics
 of the program entirely at the whim of the compiler 
 implementor.


 C++ Standard allows the same increased precision, at the whim 
 of the compiler implementor, as quoted to you earlier.

 What your particular C++ compiler does is not relevant, as its 
 behavior is not required by the Standard.


I am watching this conversation for quite a while now and it's 
interesting to see how we went from the problem

float f = 1.30;
assert(f == 1.30); // NO
assert(f == cast(float)1.30); //OK

to this (endless) discussion.


 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.


 Walter: I think this is a great idea and increased precision is 
always better!
As you pointed out, yes the only reason to decrease precision if 
a trade-off for speed. Please keep doing/pushing this!
May 15 2016
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.


  Is there an option to use a reproducible fraction type that 
doesn't have the issues floating point has?

  Consider for most things perhaps this format would work better?

  struct Ftype { //Ftype a placeholder name
    long whole;
    long fraction;
    long divider;
  }

  Now 1.30 can perfectly be represented as Ftype(1,3,10) (or 
13/10) and would never fail, represent incredibly small and 
incredibly large things, just not nearly to the degree 
floating/doubles could for scientific calculations, while at the 
same time avoiding the imprecision (or requirement) of the FPU 
entirely.
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 5:08 PM, Era Scarecrow wrote:

  Is there an option to use a reproducible fraction type that doesn't have the
 issues floating point has?


D has excellent facilities for creating your own types with the semantics you
wish.
May 15 2016
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can 
 change the semantics
 of the program entirely at the whim of the compiler 
 implementor.


 C++ Standard allows the same increased precision, at the whim 
 of the compiler implementor, as quoted to you earlier.

 What your particular C++ compiler does is not relevant, as its 
 behavior is not required by the Standard.


This is a crazy attitude to take. C++ provides means to detect 
that IEEE floats are being used in the standard library. C/C++ 
supports non-standard floating point because some platforms only 
provide non-standard floating point. They don't do it because it 
is _desirable_ in general.

You might as well say that you are not required to drive on the 
right side on the road, because you occasionally have to drive on 
the left. So therefore it is ok to always drive on left.


 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.


No, D's take on floating point is FUBAR.

const float value = 1.30;
float  copy = value;
assert(value*0.5 ==  copy*0.5); // FAILS! => shutdown
May 15 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 08:00, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can change the
 semantics
 of the program entirely at the whim of the compiler implementor.



 C++ Standard allows the same increased precision, at the whim of the
 compiler implementor, as quoted to you earlier.

 What your particular C++ compiler does is not relevant, as its behavior is
 not required by the Standard.



 This is a crazy attitude to take. C++ provides means to detect that IEEE
 floats are being used in the standard library. C/C++ supports non-standard
 floating point because some platforms only provide non-standard floating
 point. They don't do it because it is _desirable_ in general.

 You might as well say that you are not required to drive on the right side
 on the road, because you occasionally have to drive on the left. So
 therefore it is ok to always drive on left.


 My proposal removes the "whim" by requiring 128 bit precision for CTFE.



 No, D's take on floating point is FUBAR.

 const float value = 1.30;
 float  copy = value;
 assert(value*0.5 ==  copy*0.5); // FAILS! => shutdown


This says more about promoting float operations to double than
anything else, and has nothing to do with CTFE.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 This says more about promoting float operations to double than 
 anything else, and has nothing to do with CTFE.


No, promoting to double is ok. NOT coercing the value to 32 bits 
representation when it is requested is what is FUBAR and that is 
the same issue as CTFE. But worse.

This alone is a good reason to avoid D in production. Debugging 
signal processing/3D code is hard enough as it is. Not having a 
reliable way to cast to float32 representation is really really 
really bad.

Random precision is not a good thing, in any way.

1. It does not improve accuracy in a predictable manner. If 
anything it adds noise.
2. It makes harnessing library code by extensive testing near 
impossible.
3. It makes debugging of complex system level floating point code 
very hard.

This ought to be obvious.
May 16 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 This says more about promoting float operations to double than anything
 else, and has nothing to do with CTFE.



 No, promoting to double is ok.


Your own example:

const float value = 1.30;
float  copy = value;
assert(value*0.5 ==  copy*0.5);

Versus how you'd expect it to work.

const float value = 1.30;
float  copy = value;
assert(cast(float)(value*0.5) ==  cast(float)(copy*0.5)); // Compare
as float, not double.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 07:54:32 UTC, Iain Buclaw wrote:

 On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 No, promoting to double is ok.


 Your own example:

 const float value = 1.30;
 float  copy = value;
 assert(value*0.5 ==  copy*0.5);

 Versus how you'd expect it to work.

 const float value = 1.30;
 float  copy = value;
 assert(cast(float)(value*0.5) ==  cast(float)(copy*0.5)); // 
 Compare
 as float, not double.


Promoting to double is sound. That is not the issue. Just replace 
the assert with a function like "check(real x, real y)".  The 
programmer shouldn't have to look it up. Using "check(real x, 
real y)" should _not_ give a different result than using 
"check(float x, float y)". If you have implemented the former, 
you shouldn't have to implement the latter to get reasonable 
results.

Binding to "const float" or "float" should not change the outcome.

The problem is not the promotion, the problem is that you don't 
get the coercion to 32 bit floats where you requested it. This 
violates the principle of "least surprise".
May 16 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 10:25, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 07:54:32 UTC, Iain Buclaw wrote:

 On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 No, promoting to double is ok.



 Your own example:

 const float value = 1.30;
 float  copy = value;
 assert(value*0.5 ==  copy*0.5);

 Versus how you'd expect it to work.

 const float value = 1.30;
 float  copy = value;
 assert(cast(float)(value*0.5) ==  cast(float)(copy*0.5)); // Compare
 as float, not double.



 Promoting to double is sound. That is not the issue. Just replace the assert
 with a function like "check(real x, real y)".  The programmer shouldn't have
 to look it up. Using "check(real x, real y)" should _not_ give a different
 result than using "check(float x, float y)". If you have implemented the
 former, you shouldn't have to implement the latter to get reasonable
 results.

 Binding to "const float" or "float" should not change the outcome.

 The problem is not the promotion, the problem is that you don't get the
 coercion to 32 bit floats where you requested it. This violates the
 principle of "least surprise".


But you *didn't* request coercion to 32 bit floats.  Otherwise you
would have used 1.30f.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:

 But you *didn't* request coercion to 32 bit floats.  Otherwise 
 you would have used 1.30f.


	const float f = 1.3f;
	float c = f;
	assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
May 16 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:

 But you *didn't* request coercion to 32 bit floats.  Otherwise you would
 have used 1.30f.



         const float f = 1.3f;
         float c = f;
         assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!


Your still using doubles.  Are you intentionally missing the point?
May 16 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:

 On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:

 But you *didn't* request coercion to 32 bit floats.  
 Otherwise you would have used 1.30f.



         const float f = 1.3f;
         float c = f;
         assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!


 Your still using doubles.  Are you intentionally missing the 
 point?


I think you're missing what Ola means when he's talking about 
32-bit floats.  Of course when you multiply a float by a double 
(here, 1.0) you promote it to a double; but you'd expect the 
result to reflect the data available in the 32-bit float.

Whereas in Ola's example, the fact that `const float f` is known 
at compile-time means that the apparent 32-bit precision is in 
fact entirely ignored when doing the * 1.0 calculation.

In other words, Ola's expecting

     float * double == float * double

but is getting something more like

     float * double == double * double

or maybe even

     float * double == real * real

... because of the way FP constants are treated at compile time.
May 16 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:

 Your still using doubles.  Are you intentionally missing the 
 point?


You'll never know, he is Poe's law in action. At the end, it 
doesn't matter if he a moron or doing it intentionally, the 
result is the same.
May 16 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 7:28 AM, deadalnix wrote:

 On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:

 Your still using doubles.  Are you intentionally missing the point?


 You'll never know, he is Poe's law in action. At the end, it doesn't
 matter if he a moron or doing it intentionally, the result is the same.


To all: please do not engage trolls. The best way to keep their 
incompetent platitudes away is to get them bored. -- Andrei
May 16 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:

 On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:

 But you *didn't* request coercion to 32 bit floats.  
 Otherwise you would have used 1.30f.



         const float f = 1.3f;
         float c = f;
         assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!


 Your still using doubles.  Are you intentionally missing the 
 point?


What is your point? My point is that no other language I have 
ever used has overruled me requesting a coercion to single 
precision floats. And yes, binding it to a single precision float 
does qualify as a coercion on all platforms I have ever used.

I should not have to implement a function with float parameters 
if I have a working function with real parameters, just to get 
reasonable behaviour:

void assert_equality(real x, real y){ assert(x==y);}

void main(){
   const float f = 1.3f;
   float c = f;
   assert_equality(f,c); // Fails!
}


Stuff like this makes the codebase brittle.

Not being able to control precision in unit tests make unit tests 
potentially succeed when they should fail. That makes testing 
floating point code for correctness virtually impossible in D.

I don't use 32 bit float scalars to save space. I use it to get 
higher performance and in order to be able to turn the code into 
pure SIMD code at a later stage. So I require SIMD like semantics 
for float scalars. Anything less is unacceptable.

If I want high precision at compile time, then I use rational 
numbers like std::ratio in C++ which gives me _exact_ values. If 
I want something more advanced then I use Maxima and literally 
copy in the results.

C++17 is getting hex literals for floating point for a reason: 
accurate bit level representation.
May 16 2016
parent reply Max Samukha <maxsamukha gmail.com> writes:
On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad wrote:


 C++17 is getting hex literals for floating point for a reason: 
 accurate bit level representation.


D has had hex FP literals for ages.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 7:47 AM, Max Samukha wrote:

 On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad wrote:


 C++17 is getting hex literals for floating point for a reason: accurate bit
 level representation.


 D has had hex FP literals for ages.



Since the first version, if I recall correctly. Of course, C++ had the idea
first!

(Actually, the idea came from the NCEG (Numerical C Extensions Group) work in 
the early 90's, which was largely abandoned. C++ has been very slow to adapt to 
the needs of numerics programmers.)
May 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 17 May 2016 at 20:50:06 UTC, Walter Bright wrote:

 On 5/16/2016 7:47 AM, Max Samukha wrote:

 On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad 
 wrote:


 C++17 is getting hex literals for floating point for a 
 reason: accurate bit
 level representation.


 D has had hex FP literals for ages.



 Since the first version, if I recall correctly. Of course, C++ 
 had the idea first!

 (Actually, the idea came from the NCEG (Numerical C Extensions 
 Group) work in the early 90's, which was largely abandoned. C++ 
 has been very slow to adapt to the needs of numerics 
 programmers.)


The argument is not that C++ did anything first. The argument is 
that bit level precision for float is becoming the norm. Both in 
standards, languages and hardware. And that this makes it easier 
to write faster and more accurate code.

Please note that I don't complain about D providing 80 bit 
floats. Just don't apply it when I request 32 bit floats.

Some file formats have fields that are in the 80 bit intel 
format, so having access to it is a good thing.
May 18 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 08:52:16 UTC, Ola Fosheim Grøstad wrote:

 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:

 But you *didn't* request coercion to 32 bit floats.  Otherwise 
 you would have used 1.30f.


 	const float f = 1.3f;
 	float c = f;
 	assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!


IIRC, there are circumstances where you can get an equality 
failure even for non-compile-time calculations that superficially 
look identical: e.g. if one of the sides of the comparison is 
still stored in one of the 80-bit registers used for calculation. 
  I ran into this when doing some work in std.complex ... :-(

Reworking Ola's example:

     import std.stdio : writefln, writeln;

     const float constVar = 1.3f;
     float var = constVar;

     writefln("%.64g", constVar);
     writefln("%.64g", var);
     writefln("%.64g", constVar * 1.0);
     writefln("%.64g", var * 1.0);
     writefln("%.64g", 1.30 * 1.0);
     writefln("%.64g", 1.30f * 1.0);

... produces:

1.2999999523162841796875
1.2999999523162841796875
1.3000000000000000444089209850062616169452667236328125
1.2999999523162841796875
1.3000000000000000444089209850062616169452667236328125
1.3000000000000000444089209850062616169452667236328125

... which is unintuitive, to say the least; the `f` qualifier on 
the manifest constant being assigned to `const float constVar` is 
being completely ignored for the purposes of the multiplication 
by 1.0.

In other words, the specified precision of floating-point values 
is being ignored for calculations performed at compile-time, 
regardless of how much you "qualify" what you want.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:14 AM, Joseph Rushton Wakeling wrote:

 1.2999999523162841796875
 1.3000000000000000444089209850062616169452667236328125


Note the increase in correctness of the result by 10 digits.



 ... which is unintuitive, to say the least;


It isn't any less intuitive than:

    f + f + 1.3f

being calculated in 64 or 80 bit precision, which is commonplace, or for that 
matter:

    ubyte b = 200;
    ubyte c = 100;
    writeln(b + c);

giving an answer of 300 (instead of 44), which every C/C++/D compiler does.
May 16 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 10:57:00 UTC, Walter Bright wrote:

 On 5/16/2016 3:14 AM, Joseph Rushton Wakeling wrote:

 1.2999999523162841796875
 1.3000000000000000444089209850062616169452667236328125


 Note the increase in correctness of the result by 10 digits.


As Adam mentioned, you keep saying "correctness" or "accuracy", 
when people are consistently talking to you about "consistency" 
... :-)

I can always request more precision if I need or want it.  
Getting different results for a superficially identical float * 
double calculation, because one was performed at compile time and 
another at runtime, is an inconsistency that it might be nicer to 
avoid.


 ... which is unintuitive, to say the least;


 It isn't any less intuitive than:

    f + f + 1.3f

 being calculated in 64 or 80 bit precision


It is less intuitive.  If someFloat + 1.3f is calculated in 64 or 
80 bit precision at runtime, it's still constrained by the fact 
that someFloat only provides 32 bits of floating-point input to 
the calculation.

If someFloat + 1.3f is calculated instead at compile time, the 
reasonable assumption is that someFloat _still_ only brings 32 
bits of floating-point input.  But as we've seen above, it 
doesn't.


 or for that matter:

    ubyte b = 200;
    ubyte c = 100;
    writeln(b + c);

 giving an answer of 300 (instead of 44), which every C/C++/D 
 compiler does.


The latter result, at least (AIUI) is consistent depending on 
whether the calculation is done at compile time or runtime.
May 16 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 11:18:45 UTC, Joseph Rushton Wakeling 
wrote:

 As Adam mentioned, you keep saying "correctness" or "accuracy"


... meant to say '"correctness" or "precision"'.
May 16 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 4:18 AM, Joseph Rushton Wakeling wrote:

 you keep saying "correctness" or "accuracy", when people are
 consistently talking to you about "consistency" ... :-)


I see consistently wrong answers as being as virtuous as getting tires for my 
car that are not round :-)



 I can always request more precision if I need or want it.  Getting different
 results for a superficially identical float * double calculation, because one
 was performed at compile time and another at runtime, is an inconsistency that
 it might be nicer to avoid.


As the links I posted explain in great detail, such a design produces other 
problems. Secondly, as I think this thread amply demonstrates, few programmers 
are particularly aware of the issues of cumulative roundoff error. It's very 
easy to miss it and just assume the result of the calculation is precise to 7 
digits, when it might be off by factors of 2.



 The latter result, at least (AIUI) is consistent depending on whether the
 calculation is done at compile time or runtime.


True, but I was talking about intuitiveness. Integral promotion rules are not 
intuitive, they are a surprise to every C/C++/D newbie. Except for me, since I 
came from a PDP-11 assembler background, and that's how the PDP-11 CPU worked, 
and C's semantics are based on the 11.
May 16 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:00 PM, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can change the
semantics
 of the program entirely at the whim of the compiler implementor.


 C++ Standard allows the same increased precision, at the whim of the compiler
 implementor, as quoted to you earlier.

 What your particular C++ compiler does is not relevant, as its behavior is not
 required by the Standard.


 This is a crazy attitude to take. C++ provides means to detect that IEEE floats
 are being used in the standard library.


IEEE floats do not specify precision of intermediate results. A C/C++ compiler 
can be fully IEEE compliant and yet legitimately have increased precision for 
intermediate results.

I posted several links here pointing out this behavior in VC++ and g++. If your 
C++ numerics code didn't have a problem with it, it's likely you wrote the code 
in such a way that more accurate answers were not wrong.

FP behavior has complex trade-offs with speed, accuracy, compatibility, and 
size. There are no easy, obvious answers.
May 16 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 08:10:02 UTC, Walter Bright wrote:

 IEEE floats do not specify precision of intermediate results. A 
 C/C++ compiler can be fully IEEE compliant and yet legitimately 
 have increased precision for intermediate results.


IEEE 754-2008 provide language designers with features that 
enables predictable bit-accurate computations using floats for 
ordinary floating point operations. Only a subset of functions 
are implementation defined.

This also has the advantage that you can do bit-level 
optimizations... including proving asserts to hold at compile 
time and "assume assert" optimizations that you are fond of ;-).

But all the C/C++ compilers I have used support reliable coercion 
to 32 bit floats.


 I posted several links here pointing out this behavior in VC++ 
 and g++. If your C++ numerics code didn't have a problem with 
 it, it's likely you wrote the code in such a way that more 
 accurate answers were not wrong.


I use clang++ only for production, and I don't really care how I 
wrote my code. What I do know is that in performance optimized 
code for 32 bit floats and simd I do rely upon unit-testing with 
guaranteed 32 bit floats. I absolutely do not want unit tests to 
execute with higher precision. I want it to break if 32 bit 
floats fails. If I cannot be sure of this I risk having libraries 
that enter infinite loops in production code.

Keep in mind that even "simple" algorithms can get complex when 
you write for high performance. E.g. sound processing. So having 
predictable outcome is very much desirable. Such code also don't 
gain much from compiler optimizations...



 FP behavior has complex trade-offs with speed, accuracy, 
 compatibility, and size. There are no easy, obvious answers.


Well, but randomly increasing precision is always a bad idea when 
you deal with related computations, like time series. It is 
better to have consistent noise/bias than random noise/bias. 
Also, in the case of audio processing it is not unheard of to 
exploit the 24 bit mantissa and the specifics of the IEEE 32 bit 
floating point format.

Now, I don't object to having a "real" type that works the way 
you want. What I object to is having float and double act that 
way. Or rather, not having strict ieee32 and ieee64 types.
May 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 4:10 AM, Walter Bright wrote:

 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.


That's a fair statement. My understanding is also that 80-bit math is on 
the wrong side of the tradeoff simply because it's disproportionately 
slow (again I cite 
http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All modern 
ALUs I looked at have 32- and 64-bit FP units only. I'm trying to figure 
why. -- Andrei
May 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:33 AM, Andrei Alexandrescu wrote:

 On 5/16/16 4:10 AM, Walter Bright wrote:

 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.


 That's a fair statement. My understanding is also that 80-bit math is on the
 wrong side of the tradeoff simply because it's disproportionately slow (again I
 cite http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All modern
 ALUs I looked at have 32- and 64-bit FP units only. I'm trying to figure why.
--



I think it is slow because no effort has been put into speeding it up. All the 
effort went into SIMD. The x87 FPU is a library module that is just plopped
onto 
the chip for compatibility.

The x87 register stack also sux because it's hard to generate good code for. 
That didn't help.
May 16 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 7:53 AM, Walter Bright wrote:

 On 5/16/2016 3:33 AM, Andrei Alexandrescu wrote:

 On 5/16/16 4:10 AM, Walter Bright wrote:

 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.


 That's a fair statement. My understanding is also that 80-bit math is
 on the
 wrong side of the tradeoff simply because it's disproportionately slow
 (again I
 cite http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All
 modern
 ALUs I looked at have 32- and 64-bit FP units only. I'm trying to
 figure why. --



 I think it is slow because no effort has been put into speeding it up.
 All the effort went into SIMD. The x87 FPU is a library module that is
 just plopped onto the chip for compatibility.


That makes sense.


 The x87 register stack also sux because it's hard to generate good code
 for. That didn't help.


It may indeed be a bummer but it's the way it is. So we should act 
accordingly. -- Andrei
May 16 2016
prev sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 16 May 2016 at 10:33:58 UTC, Andrei Alexandrescu wrote:

 My understanding is also that 80-bit math is on the wrong side 
 of the tradeoff simply because it's disproportionately slow.


  Speed in theory shouldn't be that big of a problem. As I recall 
the FPU *was* a separate processor; Sending the instructions took 
like 3 cycles. Following that you could do other stuff before 
returning for the result(s), but that assumes you aren't *only* 
doing FPU work. The issue would then come up when you are waiting 
for the result after the fact (and that's only for really slow 
operations, most operations are very fast, but my 
knowledge/experience is probably more than a decade out of date).

  Today I'm sure the FPU is built directly into the CPU.

  Not to mention there's a whole slew of instructions for the x86 
that haven't improved much because they aren't used at all and 
instead are there for historical reasons or for compatibility. 
jcxz, xlat, rep, and probably another dozen I am not sure just 
off the top of my head.

  Perhaps more processors and in general should move to a RISC 
style instructions.
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 8:15 PM, Era Scarecrow wrote:

  Speed in theory shouldn't be that big of a problem. As I recall the FPU *was*
a
 separate processor; Sending the instructions took like 3 cycles. Following that
 you could do other stuff before returning for the result(s), but that assumes
 you aren't *only* doing FPU work. The issue would then come up when you are
 waiting for the result after the fact (and that's only for really slow
 operations, most operations are very fast, but my knowledge/experience is
 probably more than a decade out of date).


With some clever programming, you could execute other instructions in parallel 
with the x87.
May 17 2016
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 18:10, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 11:00 PM, Ola Fosheim Grøstad wrote:

 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:

 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:

 The net result is that adding const/immutable to a type can change the
 semantics
 of the program entirely at the whim of the compiler implementor.



 C++ Standard allows the same increased precision, at the whim of the
 compiler
 implementor, as quoted to you earlier.

 What your particular C++ compiler does is not relevant, as its behavior
 is not
 required by the Standard.



 This is a crazy attitude to take. C++ provides means to detect that IEEE
 floats
 are being used in the standard library.



 IEEE floats do not specify precision of intermediate results. A C/C++
 compiler can be fully IEEE compliant and yet legitimately have increased
 precision for intermediate results.

 I posted several links here pointing out this behavior in VC++ and g++. If
 your C++ numerics code didn't have a problem with it, it's likely you wrote
 the code in such a way that more accurate answers were not wrong.

 FP behavior has complex trade-offs with speed, accuracy, compatibility, and
 size. There are no easy, obvious answers.


I think I might have misunderstood Iain's initial post on the matter.
I understand the links you've posted, but the point I was objecting to
was the moment an assignment is made, truncation must occur.
If you do:
float f()
{
  float x = some_float_expression;
  float y = some_expression_involving_x;
  return some_expression_involving_y;
}

If I run that in CTFE, will assignment to x, y, and the return
statement all truncate intermediate results to float at those moments?
If the answer is yes, I apologise for the confusion.
May 16 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sunday, May 15, 2016 15:49:27 Walter Bright via Digitalmars-d wrote:

 My proposal removes the "whim" by requiring 128 bit precision for CTFE.


Based on some of the recent discussions, it sounds like having soft floating
point in the compiler would also help with cross-compilation. So, completely
aside from the precision chosen, it sounds like having a soft floating point
implementation in CTFE would definitely help - though maybe I misunderstood.
My understanding of the floating point stuff is pretty bad, unfortunately.
And Don's talk and this discussion don't exactly make me feel any better
about it - since of course what I'd really like is math math and not
magician's math, but that's not what we get, and it will never be what we
get. I really should study up on FP in detail in the near future. My typical
approach has been to avoid it as much as possible and then use the highest
precision possible when I'm stuck using it in an attempt to minimize the
error that creeps in.

- Jonathan M Davis
May 16 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 16 May 2016 at 18:44:57 UTC, Jonathan M Davis wrote:

 On Sunday, May 15, 2016 15:49:27 Walter Bright via 
 Digitalmars-d wrote:

 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.


 Based on some of the recent discussions, it sounds like having 
 soft floating point in the compiler would also help with 
 cross-compilation. So, completely aside from the precision 
 chosen, it sounds like having a soft floating point 
 implementation in CTFE would definitely help - though maybe I 
 misunderstood. My understanding of the floating point stuff is 
 pretty bad, unfortunately.


  My understanding of floating point is bad too. I understand 
fixed floating point (a number of bits is considered the 
fraction) but as I recall while trying to break down and answer 
questions while referring to as much of the information as I 
could, the power/significand portion I got stuck when looking at 
the raw bits and proper answers failed to come out.


  As for soft floating point, I'd hopefully see an option to 
control how much precision you can raise it to (so it would 
probably be a template struct), as well as making it a library we 
can use. Same with fixed integers which we could then incorporate 
cent and ucent until such a time that the types are commonly 
avaliable. (Afterall 128bit computers will come sooner or later, 
maybe in 20 years? I doubt the memory model would need to move up 
from 64bit though)

  Although for implementation it could use BCD math instead (which 
is easier to print off as decimal); Just specify the number of 
digits for precision (40+), how many digits it can shift 
larger/smaller; The floating point type would be related to the 
8-bit computers of old but on steroids! Actually basic floating 
point like that wouldn't be too hard to implement over a weekend.
May 16 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 17 May 2016 04:12:09 +0000
schrieb Era Scarecrow <rtcvb32 yahoo.com>:


 I understand fixed floating point


Quote of the day :D

-- 
Marco
May 17 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
I had written and sent this message three days ago, but it seemingly 
never showed up on the newsgroup. I'm sorry if it seemed that I didn't 
explain myself, I was operating under the assumption that this message 
had been made available to you.


On 14.05.2016 03:26, Walter Bright wrote:

 On 5/13/2016 5:49 PM, Timon Gehr wrote:

 Nonsense. That might be true for your use cases. Others might actually
 depend on
 IEE 754 semantics in non-trivial ways. Higher precision for
 temporaries does not
 imply higher accuracy for the overall computation.


 Of course it implies it.
 ...


No, see below.



 An anecdote: a colleague of mine was once doing a chained calculation.
 At every step, he rounded to 2 digits of precision after the decimal
 point, because 2 digits of precision was enough for anybody. I carried
 out the same calculation to the max precision of the calculator (10
 digits). He simply could not understand why his result was off by a
 factor of 2, which was a couple hundred times his individual roundoff
 error.
 ...


Now assume that colleague of your was doing that chained calculation, 
and his calculator magically added the additional digits behind his back 
(it can do this by caching the last full-precision value for each number 
prefix). He wouldn't even notice that his rounding strategy does not 
work. Sometime later he might then use a calculator that does not do the 
magical enhancing.


 E.g., correctness of double-double arithmetic is crucially dependent
 on correct
 rounding semantics for double:
 




https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic


 Double-double has its own peculiar issues, and is not relevant to this
 discussion.
 ...


It is relevant to this discussion insofar that it can occur in 
algorithms that use double-precision floating-point arithmetic. It 
illustrates a potential issue with implicit enhancement of precision. 
For double-double, there are two values of type double that together 
represent a higher-precision value. (One of them has a shifted exponent, 
such that their mantissa bits do not overlap.)

You have mantissas like:

|------double1------| |------double2--------|


Now assume that the compiler instead uses extended precision, what you 
get is something we might call extended-extended of the form:

|---------extended1---------| |---------extended2-----------|

Now those values are written back into 64-bit double storage, now 
observe which part of the double-double mantissa is lost:


|---------extended1---xxxxxx| |---------extended2-----xxxxxx|

|
v

|------double1------| |------double2--------|


The middle part of the mantissa is thrown away, and we are left with 
single double-precision plus some noise. Implicitly using extended 
precision for some parts of the computation approximately cuts the 
number of accurate mantissa bits in half. I don't want to have to deal 
with this. Just give me what I ask for.



 Also, it seems to me that for e.g.
 https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
 the result can actually be made less precise by adding casts to higher
 precision
 and truncations back to lower precision at appropriate places in the
 code.


 I don't see any support for your claim there.
 ....


It's using the same trick that double-double does. The above reasoning 
should apply.


 And even if higher precision helps, what good is a "precision-boost"
 that e.g.
 disappears on 64-bit builds and then creates inconsistent results?


 That's why I was thinking of putting in 128 bit floats for the compiler
 internals.
 ...


Runtime should do the same as CTFE. Are you suggesting we use 128-bit 
soft-floats at run time for all float types?



 Sometimes reproducibility/predictability is more important than maybe
 making
 fewer rounding errors sometimes. This includes reproducibility between
 CTFE and
 runtime.


 A more accurate answer should never cause your algorithm to fail.


It's not more accurate, just more precise, and it is only for some 
temporary computations, and you don't necessarily know which. The way 
the new roundoff errors propagate is chaotic, and might not be what the 
code anticipated.


 It's like putting better parts in your car causing the car to fail.
 ...


It's like (possibly repeatedly) interchanging "better" parts and "worse" 
parts while the engine is still running.

Anyway, it should be obvious that this kind of reasoning by analogy does 
not lead anywhere.



 Just actually comply to the IEEE floating point standard when using 




their

 terminology. There are algorithms that are designed for it and that
 might stop
 working if the language does not comply.


 Conjecture.


I have given a concrete example.


 I've written FP algorithms (from Cody+Waite, for example),
 and none of them degraded when using more precision.
 ...


For the entire computation or some random temporaries?


 Consider that the 8087 has been operating at 80 bits precision by
 default for 30 years. I've NEVER heard of anyone getting actual bad
 results from this.


Fine, so you haven't.


 They have complained about their test suites that
 tested for less accurate results broke.


What happened is that the test suites broke.



 They have complained about the
 speed of x87. And Intel has been trying to get rid of the x87 forever.


It's nice to have 80-bit precision. I just want to explicitly ask for it.



 Sometimes I wonder if there's a disinformation campaign about more
 accuracy being bad, because it smacks of nonsense.

 BTW, I once asked Prof Kahan about this. He flat out told me that the
 only reason to downgrade precision was if storage was tight or you
 needed it to run faster. I am not making this up.


Obviously, but I think his comment was about enhancing precision for the 
entire computation front-to-back, not just some parts of it. I can do 
that on my own. I don't need the compiler to second-guess me.



 May 18 2016



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 14.05.2016 02:49, Timon Gehr wrote:

 On 13.05.2016 23:35, Walter Bright wrote:

 On 5/13/2016 12:48 PM, Timon Gehr wrote:

 IMO the compiler should never be allowed to use a precision different
 from the one specified.


 I take it you've never been bitten by accumulated errors :-)
 ...


 
 If that was the case it would be because I explicitly ask for high 
 precision if I need it.
 
 If the compiler using or not using a higher precision magically fixes an 
 actual issue with accumulated errors, that means the correctness of the 
 code is dependent on something hidden, that you are not aware of, and 
 that could break any time, for example at a time when you really don't 
 have time to track it down.
 

 Reduced precision is only useful for storage formats and increasing
 speed.  If a less accurate result is desired, your algorithm is wrong.


 
 Nonsense. That might be true for your use cases. Others might actually 
 depend on IEE 754 semantics in non-trivial ways. Higher precision for 
 temporaries does not imply higher accuracy for the overall computation.
 
 E.g., correctness of double-double arithmetic is crucially dependent on 
 correct rounding semantics for double:
 https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic


We finally have someone on D.learn who is running into this exact problem:

http://forum.dlang.org/post/vimvfarzqkcmbvtnznrf forum.dlang.org



 Jun 28 2017



prev sibling  parent  Binarydepth <binarydepth gmail.com>  writes:


On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:

 I wonder what's the difference between 1.30f and 
 cast(float)1.30.


 There isn't one.


I always saw type casting happening   run-time and "literals" 
(pardon if that's not the correct term) to happen during 
compilation. I know that there's no difference in this topic 
since it is a comparison of values.



 May 14 2016



prev sibling next sibling parent reply ZombineDev <petar.p.kirov gmail.com>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


Yes, I think it is a good idea, just like emitting a warning for 
mismatched signed/unsigned comparison.



 May 09 2016



 next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 4:16 AM, ZombineDev wrote:

 just like emitting a warning for mismatched signed/unsigned comparison.


Not at all the same, such are not always false.



 May 09 2016



prev sibling  parent reply Ecstatic Coder <ecstatic.coder gmail.com>  writes:


On Monday, 9 May 2016 at 11:16:53 UTC, ZombineDev wrote:

 On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


 Yes, I think it is a good idea, just like emitting a warning 
 for mismatched signed/unsigned comparison.


I agree for the float but not for the long/ulong comparisons.

I often do code like "x < array.length" where x needs to be a 
long to be able to handle negative values.

I want my code to compile without warning, and therefore I'm 
against requiring "x < array.length.to!long()" to remove that 
warning.



 Jun 29 2017



  parent reply ag0aep6g <anonymous example.com>  writes:


On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:

 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.


`x < array.length` and `x < array.length.to!long` have different 
results when x is negative. That's why a warning/error is in 
order.



 Jun 29 2017



 next sibling parent  Ecstatic Coder <ecstatic.coder gmail.com>  writes:


On Thursday, 29 June 2017 at 19:12:24 UTC, ag0aep6g wrote:

 On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:

 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.


 `x < array.length` and `x < array.length.to!long` have 
 different results when x is negative. That's why a 
 warning/error is in order.


I often have array indices that go up and down (x++/x--) 
depending on the logic.

I find convenient to be able to test them (x >= 0, x < a.length) 
without having to manage the fact that the array stores its 
length as an unsigned integer to double its potential size, since 
anyway I never use arrays with 2^63 items.



 Jun 29 2017



prev sibling  parent reply Ecstatic Coder <ecstatic.coder gmail.com>  writes:


On Thursday, 29 June 2017 at 19:12:24 UTC, ag0aep6g wrote:

 On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:

 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.


 `x < array.length` and `x < array.length.to!long` have 
 different results when x is negative. That's why a 
 warning/error is in order.


I can perfectly understand that others may want to check their 
code in a "strict" mode.

So actually I'm not against a warning for signed/unsigned 
implicit conversions.

I'm just against putting it on by default, so that the current 
behavior is kept, because I don't see where is the language 
improvement in having to put these ugly manual conversion 
everywhere just because the string/array length was made unsigned.

I always use signed integers for string/array indices because 
unsigned indices become dangerous as soon as your algorithm 
starts to decrement it...

And as I said, I compile my D code with the 64-bit compiler, and 
2^63 characters/items is much more than needed for my personal 
use cases.

So until the day I will have a computer which can store a string 
of 16 million terabytes, at the moment I prefer to use long 
values for indices, instead of ulong, because negative indices 
can already occur right now with my current algorithms.



 Jun 29 2017



  parent reply ag0aep6g <anonymous example.com>  writes:


On 06/30/2017 07:38 AM, Ecstatic Coder wrote:

 I'm just against putting it on by default, so that the current behavior 
 is kept, because I don't see where is the language improvement in having 
 to put these ugly manual conversion everywhere just because the 
 string/array length was made unsigned.


So what you really want is: signed array lengths. You only have a use 
for the sloppy conversion, because D doesn't have signed array lengths.


 I always use signed integers for string/array indices because unsigned 
 indices become dangerous as soon as your algorithm starts to decrement 
 it...


I guess they're "dangerous" because you get ulong.max instead of (a 
signed) -1? But that's exactly what happens with the implicit 
conversion, too.

At some point you have to think of that. Either when adding/subtracting 
or when comparing signed with unsigned. With a warning/error on the 
implicit conversion, you'd get a nice reminder.

By the way, std.experimental.checkedint may interesting:

----
import std.experimental.checkedint: checked, ProperCompare,
     Saturate;

/* If negative values should not occur, throw an error on wrap
around: */
auto x = checked(size_t(0));
--x; /* error */

/* Or stop at 0: */
auto y = checked!Saturate(size_t(0));
--y; /* does nothing */
assert(y == 0); /* passes */

/* If negative values are valid, you can use proper comparisons
without sloppy implicit conversions or verbose manual ones: */
auto z = checked!ProperCompare(int(-1));
auto array = [1, 2, 3];
assert(z < array.length); /* passes */
----



 Jun 30 2017



  parent  Ecstatic Coder <ecstatic.coder gmail.com>  writes:


 So what you really want is: signed array lengths. You only have 
 a use for the sloppy conversion, because D doesn't have signed 
 array lengths.


Yes and no. Signed safety is better, but the current behavior 
works too.

Honestly I don't care if the bit maniacs want 8 million terabytes 
more for their strings, as long as I can safely ignore that 
implementation detail.



 Jun 30 2017



prev sibling next sibling parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 So, should the compiler emit a warning for the former case?


Yes, please. I would prefer, at least, a warning flag for that.



 May 09 2016



prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 So, should the compiler emit a warning for the former case?


Would that include comparison of variables only aswell?

     float f = 1.3;
     double d = 1.3;
     assert(f == d); // compiler warning



 May 09 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 4:38 AM, Nordlöw wrote:

 Would that include comparison of variables only aswell?

     float f = 1.3;
     double d = 1.3;
     assert(f == d); // compiler warning


No.



 May 09 2016



 next sibling parent reply Temtaime <temtaime gmail.com>  writes:


On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:

 On 5/9/2016 4:38 AM, Nordlöw wrote:

 Would that include comparison of variables only aswell?

     float f = 1.3;
     double d = 1.3;
     assert(f == d); // compiler warning


 No.


Just get rid of the problem : remove == and != from floats.



 May 09 2016



  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 9 May 2016 at 12:29:14 UTC, Temtaime wrote:

 Just get rid of the problem : remove == and != from floats.


Diagnostics Suggestion:

Issue a warning including a direction to approxEqual()

https://dlang.org/phobos/std_math.html#.approxEqual



 May 09 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:

 On 5/9/2016 4:38 AM, Nordlöw wrote:

 Would that include comparison of variables only aswell?


 No.


Why?



 May 09 2016



  parent  Marco Leise <Marco.Leise gmx.de>  writes:


Am Mon, 09 May 2016 15:56:21 +0000
schrieb Nordl=C3=B6w <per.nordlow gmail.com>:


 On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:

 On 5/9/2016 4:38 AM, Nordl=C3=B6w wrote: =20

 Would that include comparison of variables only aswell? =20


 No. =20


=20
 Why?


Because the float would be converted to double without loss of
precision just like uint =3D=3D ulong is a valid comparison which
can yield 'true' in 2^32 cases.
float =3D=3D 1.30 on the other hand is false in any case.
You'd have to warn on _every_ comparison with a widening
conversion to be consistent!

--=20
Marco



 May 09 2016



prev sibling next sibling parent reply Ethan Watson <gooberman gmail.com>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


I'd assume in the first case that the float is being promoted to 
double for the comparison. Is there already a warning for loss of 
precision? We treat warnings as errors in our C++ code, so C4244 
triggers all the time in MSVC with integer operations. I just 
tested that float initialisation in MSVC, initialising a float 
with a double triggers C4305.

So my preference is "Yes please".

https://msdn.microsoft.com/en-us/library/th7a07tz.aspx
https://msdn.microsoft.com/en-us/library/0as1ke3f.aspx



 May 09 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 5:21 AM, Ethan Watson wrote:

 I'd assume in the first case that the float is being promoted to double for the
 comparison. Is there already a warning for loss of precision?


Promoting to double does not lose precision.


 We treat warnings
 as errors in our C++ code, so C4244 triggers all the time in MSVC with integer
 operations. I just tested that float initialisation in MSVC, initialising a
 float with a double triggers C4305.


That's going quite a bit further.



 May 09 2016



  parent  Ethan Watson <gooberman gmail.com>  writes:


On Monday, 9 May 2016 at 12:30:13 UTC, Walter Bright wrote:

 Promoting to double does not lose precision.


Yes, badly worded on my part, I was getting at the original 
assignment from double to float.



 May 09 2016



prev sibling next sibling parent reply Xinok <xinok live.com>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


(1) Yes, emit a warning for this case.
(2) Generalize it to all variables, like Nordlöw suggested.
(3) Generalize it to all comparisons as well, including < and > .
(4) While we're at it, let's also emit a warning when comparing 
signed and unsigned types.
(5) Dare I say it... warn against implicit conversions of double 
to float.
(6) The same applies to "real" as well.

All of these scenarios are capable of producing "incorrect" 
results, are a source of discrete bugs (often corner cases that 
we failed to consider and test), and can be hard to detect. It's 
about time we stopped being stubborn and flagged these things as 
warnings. Even if they require a special compiler flag and are 
disabled by default, that's better than nothing.



 May 09 2016



 next sibling parent reply tsbockman <thomas.bockman gmail.com>  writes:


On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:

 (1) Yes, emit a warning for this case.
 (2) Generalize it to all variables, like Nordlöw suggested.
 (3) Generalize it to all comparisons as well, including < and > 
 .
 (4) While we're at it, let's also emit a warning when comparing 
 signed and unsigned types.
 (5) Dare I say it... warn against implicit conversions of 
 double to float.
 (6) The same applies to "real" as well.

 All of these scenarios are capable of producing "incorrect" 
 results, are a source of discrete bugs (often corner cases that 
 we failed to consider and test), and can be hard to detect. 
 It's about time we stopped being stubborn and flagged these 
 things as warnings. Even if they require a special compiler 
 flag and are disabled by default, that's better than nothing.


(1) is good, because the code in question is always wrong.

(2) is a logical extension, in those cases where constant folding 
and VRP can prove that the code is always wrong.

(3) Makes no sense though; inequalities with mixed floating-point 
types are perfectly safe. (Well, as safe as any floating-point 
code can be, anyway.)

(4) is already planned; it's just taking *a lot* longer than 
anticipated to actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229



 May 09 2016



 next sibling parent reply Xinok <xinok live.com>  writes:


On Monday, 9 May 2016 at 18:51:58 UTC, tsbockman wrote:

 On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:

 ...
 (3) Generalize it to all comparisons as well, including < and 
 ...


 (3) Makes no sense though; inequalities with mixed 
 floating-point types are perfectly safe. (Well, as safe as any 
 floating-point code can be, anyway.)
 ...


Not necessarily. Reusing 1.3 from the original case, the 
following assertion passes:

     float f = 1.3;
     assert(f < 1.3);

And considering we also have <= and >=, we may as well check all 
of the comparison operators. It's a complex issue because there 
isn't necessarily right or wrong behavior, only things that 
*might* be wrong. But if we want to truly detect all possible 
cases of incorrect behavior, then we have to be exhaustive in our 
checks.



 May 09 2016



  parent reply tsbockman <thomas.bockman gmail.com>  writes:


On Monday, 9 May 2016 at 19:15:20 UTC, Xinok wrote:

 It's a complex issue because there isn't necessarily right or 
 wrong behavior, only things that *might* be wrong. But if we 
 want to truly detect all possible cases of incorrect behavior, 
 then we have to be exhaustive in our checks.


We absolutely, emphatically, DO NOT want to "detect all possible 
cases of incorrect behavior". Given the limited information 
available to the compiler and the intractability of the Halting 
Problem, the correct algorithm for doing so is this:

foreach(line; sourceCode)
     warning("This line could not be proven correct.");

Only warnings with a reasonably high signal-to-noise ratio should 
be implemented.


 Not necessarily. Reusing 1.3 from the original case, the 
 following assertion passes:

     float f = 1.3;
     assert(f < 1.3);

 And considering we also have <= and >=, we may as well check 
 all of the comparison operators.


Because of the inevitability of rounding errors, FP code 
generally cannot be allowed to depend upon precise equality 
comparisons to any value other than zero (even implicit ones as 
in your example).

Warning about this makes sense:

     float f = 1.3;
     assert(f == 1.3);

Because the `==` makes it clear that the programmer's intent was 
to test for equality, and that will fail unexpectedly.

On the other hand, with `<`, `>`, `<=`, and `>=`, the compiler 
should generally assume that they are being used correctly, in a 
way that is tolerant of small rounding errors. Doing otherwise 
would cause tons of false positives in competently written FP 
code.

Educating programmers who've never studied how to write correct 
FP code is too complex of a task to implement via compiler 
warnings. The warnings should be limited to cases that are either 
obviously wrong, or where the warning is likely to be a net 
positive even for FP experts.



 May 09 2016



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 12:39 PM, tsbockman wrote:

 Educating programmers who've never studied how to write correct FP code is too
 complex of a task to implement via compiler warnings. The warnings should be
 limited to cases that are either obviously wrong, or where the warning is
likely
 to be a net positive even for FP experts.


I've seen a lot of proposals which try to hide the reality of how FP works. The 
cure is worse than the disease. The same goes for hiding signed/unsigned, and 
the autodecode mistake of pretending that code units aren't there.



 May 09 2016



 next sibling parent  John Colvin <john.loughran.colvin gmail.com>  writes:


On Monday, 9 May 2016 at 20:20:00 UTC, Walter Bright wrote:

 On 5/9/2016 12:39 PM, tsbockman wrote:

 Educating programmers who've never studied how to write 
 correct FP code is too
 complex of a task to implement via compiler warnings. The 
 warnings should be
 limited to cases that are either obviously wrong, or where the 
 warning is likely
 to be a net positive even for FP experts.


 I've seen a lot of proposals which try to hide the reality of 
 how FP works. The cure is worse than the disease. The same goes 
 for hiding signed/unsigned, and the autodecode mistake of 
 pretending that code units aren't there.


I completely agree that complexity that cannot be properly hidden 
should not be hidden. The underlying mechanisms of floating point 
is complexity that we shouldn't paper over. However, the 
peculiarities of language conventions w.r.t. floating point 
expressions doesn't quite fit that category.



 May 09 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 09.05.2016 22:20, Walter Bright wrote:

 On 5/9/2016 12:39 PM, tsbockman wrote:

 Educating programmers who've never studied how to write correct FP
 code is too
 complex of a task to implement via compiler warnings. The warnings
 should be
 limited to cases that are either obviously wrong, or where the warning
 is likely
 to be a net positive even for FP experts.


 I've seen a lot of proposals which try to hide the reality of how FP
 works. The cure is worse than the disease. The same goes for hiding
 signed/unsigned, and the autodecode mistake of pretending that code
 units aren't there.


I feel the same way about automated enhancement of precision for 
intermediate computations behind the back of the programmer. In the best 
case, you are pretending that the algorithm has better numerical 
properties than it actually has, in the worst case you are destroying 
the accuracy of the result.



 May 17 2016



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 05/17/2016 02:13 PM, Timon Gehr wrote:

 On 09.05.2016 22:20, Walter Bright wrote:

 On 5/9/2016 12:39 PM, tsbockman wrote:

 Educating programmers who've never studied how to write correct FP
 code is too
 complex of a task to implement via compiler warnings. The warnings
 should be
 limited to cases that are either obviously wrong, or where the warning
 is likely
 to be a net positive even for FP experts.


 I've seen a lot of proposals which try to hide the reality of how FP
 works. The cure is worse than the disease. The same goes for hiding
 signed/unsigned, and the autodecode mistake of pretending that code
 units aren't there.


 I feel the same way about automated enhancement of precision for
 intermediate computations behind the back of the programmer. In the best
 case, you are pretending that the algorithm has better numerical
 properties than it actually has, in the worst case you are destroying
 the accuracy of the result.


That's an interesting assessment, thanks. -- Andrei



 May 17 2016



prev sibling  parent  Observer <here nowhere.org>  writes:


On Monday, 9 May 2016 at 19:39:52 UTC, tsbockman wrote:

 Educating programmers who've never studied how to write correct 
 FP code is too complex of a task to implement via compiler 
 warnings. The warnings should be limited to cases that are 
 either obviously wrong, or where the warning is likely to be a 
 net positive even for FP experts.


Any warning message for this type of problem should mention the
"What Every Computer Scientist Should Know About Floating-Point
Arithmetic" paper (and perhaps give a standard public URL such as
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
at which the paper can be easily accessed).



 May 10 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 11:51 AM, tsbockman wrote:

 On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:

 (4) While we're at it, let's also emit a warning when comparing signed and
 unsigned types.


 (4) is already planned; it's just taking *a lot* longer than anticipated to
 actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229


I oppose this change. You'd be better off not having unsigned types at all than 
this mess, which was Java's choice. But then there are more problems created.



 May 09 2016



 next sibling parent  tsbockman <thomas.bockman gmail.com>  writes:


On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:

 On 5/9/2016 11:51 AM, tsbockman wrote:

 (4) is already planned; it's just taking *a lot* longer than 
 anticipated to
 actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229


 I oppose this change. You'd be better off not having unsigned 
 types at all than this mess, which was Java's choice. But then 
 there are more problems created.


What mess? The actual fix for issue 259 is simple, elegant, and 
shouldn't require much code in the wild to be changed.

The difficulties and delays have all been associated with the 
necessary improvements to VRP and constant folding, which are 
worthwhile in their own right, since they help the compiler 
generate faster code.



 May 09 2016



prev sibling next sibling parent  tsbockman <thomas.bockman gmail.com>  writes:


On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:

 (4) is already planned; it's just taking *a lot* longer than 
 anticipated to
 actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229


 I oppose this change. You'd be better off not having unsigned 
 types at all than this mess, which was Java's choice. But then 
 there are more problems created.


One other thing - according to the bug report discussion, the 
proposed solution was pre-approved both by Andrei Alexandrescu 
and by *YOU*.

Proposing a solution, letting various people work on implementing 
it for three years, and then suddenly announcing that you "oppose 
this change" and calling the solution a "mess" with no 
explanation is a fantastic way to destroy all motivation for 
outside contributors.



 May 09 2016



prev sibling  parent  Kagamin <spam here.lot>  writes:


On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:

 I oppose this change. You'd be better off not having unsigned 
 types at all than this mess, which was Java's choice.


The language forces usage of unsigned types. Though in my 
experience it's relatively easy to fight back including 
interfacing with C that uses unsigned types exclusively.


 But then there are more problems created.


I've seen no problem from using signed types so far. The last 
prejudice left is usage of ubyte[] for buffers. How often one 
looks into individual bytes in some abstract buffer?



 May 11 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 11:37 AM, Xinok wrote:

 All of these scenarios are capable of producing "incorrect" results, are a
 source of discrete bugs (often corner cases that we failed to consider and
 test), and can be hard to detect. It's about time we stopped being stubborn and
 flagged these things as warnings. Even if they require a special compiler flag
 and are disabled by default, that's better than nothing.


I've used a B+D language that does as you suggest (Wirth Pascal). It was highly 
unpleasant to use, as the code became littered with casts. Casts introduce
their 
own set of bugs.



 May 09 2016



  parent reply Xinok <xinok live.com>  writes:


On Monday, 9 May 2016 at 20:14:36 UTC, Walter Bright wrote:

 On 5/9/2016 11:37 AM, Xinok wrote:

 All of these scenarios are capable of producing "incorrect" 
 results, are a
 source of discrete bugs (often corner cases that we failed to 
 consider and
 test), and can be hard to detect. It's about time we stopped 
 being stubborn and
 flagged these things as warnings. Even if they require a 
 special compiler flag
 and are disabled by default, that's better than nothing.


 I've used a B+D language that does as you suggest (Wirth 
 Pascal). It was highly unpleasant to use, as the code became 
 littered with casts. Casts introduce their own set of bugs.


Maybe it's a bad idea to enable these warnings by default but 
what's wrong with providing a compiler flag to perform these 
checks anyways? For example, GCC has a compiler flag to yield 
warnings for signed+unsigned comparisons but it's not even 
enabled with the -Wall flag, only by specifying -Wextra or 
-Wsign-compare.



 May 09 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 8:00 PM, Xinok wrote:

 Maybe it's a bad idea to enable these warnings by default but what's wrong with
 providing a compiler flag to perform these checks anyways? For example, GCC has
 a compiler flag to yield warnings for signed+unsigned comparisons but it's not
 even enabled with the -Wall flag, only by specifying -Wextra or -Wsign-compare.


Warnings balkanize the language into endless dialects.



 May 10 2016



prev sibling next sibling parent  Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On Monday, May 09, 2016 02:10:19 Walter Bright via Digitalmars-d wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

      float f = 1.30;
      assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

      float f = 1.30;
      assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


It does seem like having implicit conversions with floating point numbers is
problematic in general, though warning about it or making it illegal could
very well be too annoying to be worth it. But at bare minimum, warning about
literals not matching the type that they're being compared against when
there _is_ a literal that would be of the same type is probably worth
warning about - and that could apply to more than just floating point
values. But figuring out when implicit conversions are genuinely useful and
should be allowed and when they're more trouble than they're worth is
surprisingly hard to get right. :(

- Jonathan M Davis



 May 09 2016



prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de>  writes:


Am Mon, 9 May 2016 02:10:19 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 Don Clugston pointed out in his DConf 2016 talk that:
 
      float f = 1.30;
      assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a float. However,
 
      float f = 1.30;
      assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?


I'd say yes, but exclude the case where it can be statically
verified, that the comparison can yield true, because the
constant can be losslessly converted to the type of 'f'.

By example, don't warn for these:
f == 1.0, f == -0.5, f == 3.625, f == 2UL^^60

But do warn for:
f == 1.30, f == 2UL^^60+1

As an extension of the existing "comparison is always
false/true" check it could read "Comparison is always false:
literal 1.30 is not representable as 'float'".

There is a whole bunch in this warning category:
  byte b;
  if (b == 1000) {}
"Comparison is always false: literal 1000 is not representable
as 'byte'"

-- 
Marco



 May 09 2016



 next sibling parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 10 May 2016 at 06:25, Marco Leise via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Am Mon, 9 May 2016 02:10:19 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:


 Don Clugston pointed out in his DConf 2016 talk that:

      float f = 1.30;
      assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

      float f = 1.30;
      assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


 I'd say yes, but exclude the case where it can be statically
 verified, that the comparison can yield true, because the
 constant can be losslessly converted to the type of 'f'.

 By example, don't warn for these:
 f == 1.0, f == -0.5, f == 3.625, f == 2UL^^60

 But do warn for:
 f == 1.30, f == 2UL^^60+1

 As an extension of the existing "comparison is always
 false/true" check it could read "Comparison is always false:
 literal 1.30 is not representable as 'float'".

 There is a whole bunch in this warning category:
   byte b;
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"

 --
 Marco


This.



 May 10 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/9/2016 1:25 PM, Marco Leise wrote:

 There is a whole bunch in this warning category:
   byte b;
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"


You're right, we may be opening a can of worms with this.



 May 10 2016



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2016-05-10 23:44, Walter Bright wrote:

 On 5/9/2016 1:25 PM, Marco Leise wrote:

 There is a whole bunch in this warning category:
   byte b;
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"


 You're right, we may be opening a can of worms with this.


Scala gives a warning/error (don't remember which) for "isInstanceOf" 
where it can prove at compile time it will never be true. That has 
helped me a couple of times.

-- 
/Jacob Carlborg



 May 10 2016



prev sibling  parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Tuesday, 10 May 2016 at 21:44:45 UTC, Walter Bright wrote:

   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"




What's wrong with having this warning?


 You're right, we may be opening a can of worms with this.





 May 12 2016



  parent  =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com>  writes:


On Thursday, 12 May 2016 at 09:22:02 UTC, Nordlöw wrote:

 "Comparison is always false: literal 1000 is not representable
 as 'byte'"




 What's wrong with having this warning?


This is, IMO, just a more informative diagnostic than:

"Statement at `foo()` is not reachable":

in the following code:

     if (b == 1000)
     {
         foo();
     }



 May 12 2016



prev sibling next sibling parent  Joe Duarte <jose.duarte asu.edu>  writes:


On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


I think it really depends on what the warning actually says. I 
think people have different expectations for what that warning 
would be.

When you say 1.30 is not representable as a float, when is the 
"not representable" enforced? Because it looks like the 
programmer just represented it in the assignment of the literal – 
but that's not where the warning would be right? I mean I assume 
so because people need nonrational literals all the time, and 
this is the only way they can do it, which means it's a hole in 
the type system right? There should be a decimal type to cover 
all these cases, like some databases have.

Would the warning say that you can't compare 1.30 to a float 
because 1.30 is not representable as a float? Or would it say 
that f was rounded upon assignment and is no longer 1.30?

Short of a decimal type, I think it would be nice to have a 
"float equality" operator that covered this whole class of cases, 
where floats that started their lives as nonrational literals and 
floats that have been rounded with loss of precision can be 
treated as equal if they're within something like .0000001% of 
each other (well a  percentage that can actually be represented 
as a float...) Basically equality that covers the known 
mutational properties of fp arithmetic.

There's no way to do this right now without ranges right? I know 
that ~ is for concat. I saw ~= is an operator. What does that do? 
The Unicode ≈ would be nice for this.

I assume IEEE 754 or ISO 10967 don't cover this? I was just 
reading the latter (zip here: 
http://standards.iso.org/ittf/PubliclyAvailableStandards/c051317_ISO_IEC_10967-1_2012.zip)



 May 09 2016



prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 9 May 2016 at 19:10, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


Perhaps float comparison should *always* be done at the lower
precision? There's no meaningful way to perform a float/double
comparison where the float is promoted, whereas demoting the double
for the comparison will almost certainly yield the expected result.



 May 10 2016



  parent  Joseph Rushton Wakeling <joseph.wakeling webdrake.net>  writes:


On Tuesday, 10 May 2016 at 07:28:21 UTC, Manu wrote:

 Perhaps float comparison should *always* be done at the lower 
 precision? There's no meaningful way to perform a float/double 
 comparison where the float is promoted, whereas demoting the 
 double for the comparison will almost certainly yield the 
 expected result.


Assuming that's what you want, it's reasonably straightforward to 
use

     feqrel(someFloat, someDouble) >= float.mant_dig

... to compare to the level of precision that matters to you.  
That's probably a better option than adjusting `==` to always 
prefer a lower level of precision (because it's arguably accurate 
to say that 1.3f != 1.3).



 May 16 2016



prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 10 May 2016 at 17:28, Manu <turkeyman gmail.com> wrote:

 On 9 May 2016 at 19:10, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?


 Perhaps float comparison should *always* be done at the lower
 precision? There's no meaningful way to perform a float/double
 comparison where the float is promoted, whereas demoting the double
 for the comparison will almost certainly yield the expected result.


Think of it like this; a float doesn't represent a precise point (it's
an approximation by definition), so see the float as representing the
interval from the absolute value it stores, and that + 1 mantissa bit.
If you see float's that way, then the natural way to compare them is
to demote to the lowest common precision, and it wouldn't be
considered erroneous, or even warning-worthy; just documented
behaviour.



 May 10 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/10/2016 12:31 AM, Manu via Digitalmars-d wrote:

 Think of it like this; a float doesn't represent a precise point (it's
 an approximation by definition), so see the float as representing the
 interval from the absolute value it stores, and that + 1 mantissa bit.
 If you see float's that way, then the natural way to compare them is
 to demote to the lowest common precision, and it wouldn't be
 considered erroneous, or even warning-worthy; just documented
 behaviour.


Floating point behavior is so commonplace, I am wary of inventing new, unusual 
semantics for it.



 May 10 2016



  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 11 May 2016 at 07:47, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/10/2016 12:31 AM, Manu via Digitalmars-d wrote:

 Think of it like this; a float doesn't represent a precise point (it's
 an approximation by definition), so see the float as representing the
 interval from the absolute value it stores, and that + 1 mantissa bit.
 If you see float's that way, then the natural way to compare them is
 to demote to the lowest common precision, and it wouldn't be
 considered erroneous, or even warning-worthy; just documented
 behaviour.



 Floating point behavior is so commonplace, I am wary of inventing new,
 unusual semantics for it.


Is it unusual to demote to the lower common precision? I think it's
the only reasonable solution.
It's never reasonable to promote a float, since it has already
suffered precision loss. It can't meaningfully be compared against
anything higher precision than itself.
What is the problem with this behaviour I suggest?

The reason I'm wary about emitting a warning is because people will
encounter the warning *all the time*, and for a user who doesn't have
comprehensive understanding of floating point (and probably many that
do), the natural/intuitive thing to do would be to place an explicit
cast of the lower precision value to the higher precision type, which
is __exactly the wrong thing to do__.
I don't think the warning improves the problem, it likely just causes
people to emit the same incorrect code explicitly.

Honestly, who would naturally respond to such a warning by demoting
the higher precision type? I don't know that guy, other than those of
us who have just watched Don's talk.



 May 11 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:

 Floating point behavior is so commonplace, I am wary of inventing new,
 unusual semantics for it.


 Is it unusual to demote to the lower common precision?


Yes.



 I think it's the only reasonable solution.


It may be, but it is unusual and therefore surprising behavior.



 What is the problem with this behaviour I suggest?


Code will do one thing in C, and the same code will do something unexpectedly 
different in D.



 The reason I'm wary about emitting a warning is because people will
 encounter the warning *all the time*, and for a user who doesn't have
 comprehensive understanding of floating point (and probably many that
 do), the natural/intuitive thing to do would be to place an explicit
 cast of the lower precision value to the higher precision type, which
 is __exactly the wrong thing to do__.
 I don't think the warning improves the problem, it likely just causes
 people to emit the same incorrect code explicitly.


The warning is intended for people who understand, as then they will figure out 
what they actually wanted and implement that. People who randomly and without 
comprehension insert casts hoping to make the compiler shut up cannot be helped.



 May 12 2016



 next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 12 May 2016 at 17:32, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:


 The reason I'm wary about emitting a warning is because people will
 encounter the warning *all the time*, and for a user who doesn't have
 comprehensive understanding of floating point (and probably many that
 do), the natural/intuitive thing to do would be to place an explicit
 cast of the lower precision value to the higher precision type, which
 is __exactly the wrong thing to do__.
 I don't think the warning improves the problem, it likely just causes
 people to emit the same incorrect code explicitly.



 The warning is intended for people who understand, as then they will figure
 out what they actually wanted and implement that. People who randomly and
 without comprehension insert casts hoping to make the compiler shut up
 cannot be helped.


But they can easily be helped by implementing behaviour that makes sense.
If you're set on a warning, at least make the warning recommend
down-casting the higher precision term to the lower precision?



 May 12 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 3:30 AM, Manu via Digitalmars-d wrote:

 But they can easily be helped by implementing behaviour that makes sense.


One thing that's immutably true about computer floating point is that it does 
not make intuitive sense if your intuition is based on mathematics. It's a 
hopeless cause trying to bash it into 'making sense'. It's something one has to 
simply take the time and learn.

This reminds me of all the discussions around trying to hide the fact that D 
strings are UTF-8 code units. The ultimate outcome of trying to make it "make 
sense" was the utter disaster of autodecoding.



 If you're set on a warning, at least make the warning recommend
 down-casting the higher precision term to the lower precision?


Yes, of course. I believe error messages should suggest corrective action.



 May 12 2016



 next sibling parent reply Guillaume Chatelet <chatelet.guillaume gmail.com>  writes:


On Thursday, 12 May 2016 at 15:55:52 UTC, Walter Bright wrote:

 This reminds me of all the discussions around trying to hide 
 the fact that D strings are UTF-8 code units. The ultimate 
 outcome of trying to make it "make sense" was the utter 
 disaster of autodecoding.


Well maybe it was a disaster because the problem was only half 
solved.
It looks like Perl 6 got it right:
https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/



 May 12 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 9:15 AM, Guillaume Chatelet wrote:

 Well maybe it was a disaster because the problem was only half solved.
 It looks like Perl 6 got it right:
 https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/


Perl isn't a systems programming language. A systems language requires access
to 
code units, invalid encodings, etc. Nor is Perl efficient. There are a lot of 
major efficiency gains by not autodecoding.



 May 12 2016



 next sibling parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Thu, May 12, 2016 at 09:20:05AM -0700, Walter Bright via Digitalmars-d wrote:
[...]

 There are a lot of major efficiency gains by not autodecoding.


Any chance of killing autodecoding in Phobos in the foreseeable future?
It's one of the things that I really liked about D in the beginning, but
over time, I've come to realize more and more that it was a mistake.
It's one of those things that only becomes clear in retrospect.


T

-- 
Ph.D. = Permanent head Damage



 May 12 2016



prev sibling  parent reply Guillaume Chatelet <chatelet.guillaume gmail.com>  writes:


On Thursday, 12 May 2016 at 16:20:05 UTC, Walter Bright wrote:

 On 5/12/2016 9:15 AM, Guillaume Chatelet wrote:

 Well maybe it was a disaster because the problem was only half 
 solved.
 It looks like Perl 6 got it right:
 https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/


 Perl isn't a systems programming language. A systems language 
 requires access to code units, invalid encodings, etc. Nor is 
 Perl efficient. There are a lot of major efficiency gains by 
 not autodecoding.


[Sorry for the OT]

I never claimed Perl was a systems programming language nor that 
it was efficient, just that their design looks more mature than 
ours.

Also I think you missed this part of the article:

"Of course, that’s all just for the default Str type. If you 
don’t want to work at a grapheme level, then you have several 
other string types to choose from: If you’re interested in 
working within a particular normalization, there’s the 
self-explanatory types of NFC, NFD, NFKC, and NFKD. If you just 
want to work with codepoints and not bother with normalization, 
there’s the Uni string type (which may be most appropriate in 
cases where you don’t want the NFC normalization that comes with 
normal Str, and keep text as-is). And if you want to work at the 
binary level, well, there’s always the Blob family of types :)."

We basically have "Uni" in D, no normalized nor grapheme level.



 May 12 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 9:36 AM, Guillaume Chatelet wrote:

 just that their design looks more mature than ours.


I don't think that can be inferred from a brief article.

If you want to access D strings by various means, there's .byChar, .byWchar, 
.byDchar, .byCodeunit, etc. foreach can also pick off characters by various
schemes.



 May 12 2016



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/12/16 6:55 PM, Walter Bright wrote:

 This reminds me of all the discussions around trying to hide the fact
 that D strings are UTF-8 code units. The ultimate outcome of trying to
 make it "make sense" was the utter disaster of autodecoding.


I am as unclear about the problems of autodecoding as I am about the 
necessity to remove curl. Whenever I ask I hear some arguments that work 
well emotionally but are scant on reason and engineering. Maybe it's 
time to rehash them? I just did so about curl, no solid argument seemed 
to come together. I'd be curious of a crisp list of grievances about 
autodecoding. -- Andrei



 May 12 2016



 next sibling parent reply ag0aep6g <anonymous example.com>  writes:


On 05/12/2016 06:29 PM, Andrei Alexandrescu wrote:

 I'd be curious of a crisp list of grievances about
 autodecoding. -- Andrei


It emits code points (dchar) which is an awkward middle point between 
code units (char/wchar) and graphemes.

Without any auto-decoding at all, every array T[] would be a 
random-access range of Ts as well. `.front` would be the same as `[0]`, 
`.length` would be the same as `.walkLength`, etc. That would make 
things less confusing for newbies, and more experienced programmers 
wouldn't accidentally mix the two abstraction levels.

Of course, you'd have to be aware that a (w)char is not a character as 
perceived by humans, but a code unit. But auto-decoding to code points 
only shifts that problem: You have to be aware that a dchar is not a 
character either. Multiple dchars may form one visible character, one 
grapheme. For example, "\u00E4" and "a\u0308" encode the same grapheme: "ä".

If char[], wchar[], dchar[] (and qualified variants) were ranges of 
graphemes, things would make the most sense for people who are not aware 
of delicate details of Unicode. You wouldn't accidentally cut code 
points or graphemes in half, `.walkLength` makes intuitive sense, etc. 
You could still accidentally use `.length` or `[0]`, though. So it still 
has some pitfalls.



 May 12 2016



  parent  ag0aep6g <anonymous example.com>  writes:


On 05/12/2016 07:12 PM, ag0aep6g wrote:
[...]

tl;dr:

This is surprising to newbies, and a possible source of bugs for 
experienced programmers:

----
writeln("ä".length); /* "2" */
writeln("ä"[0]); /* question mark in a diamond */
----

When people understand the above, they might still not understand this:

----
writeln("length of 'a\u0308': ", "a\u0308".walkLength); /* "length of 
'ä': 2" */
writeln("a\u0308".front); /* "a" */
----



 May 12 2016



prev sibling next sibling parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 5/12/16 12:29 PM, Andrei Alexandrescu wrote:

 On 5/12/16 6:55 PM, Walter Bright wrote:

 This reminds me of all the discussions around trying to hide the fact
 that D strings are UTF-8 code units. The ultimate outcome of trying to
 make it "make sense" was the utter disaster of autodecoding.


 I am as unclear about the problems of autodecoding as I am about the
 necessity to remove curl. Whenever I ask I hear some arguments that work
 well emotionally but are scant on reason and engineering. Maybe it's
 time to rehash them? I just did so about curl, no solid argument seemed
 to come together. I'd be curious of a crisp list of grievances about
 autodecoding. -- Andrei


Autodecoding, IMO, is not the problem. The problem is hijacking an array 
to mean something other than an array.

I ran into this the other day. In iopipe, I treat char[] buffers as 
actual buffers of code units. This makes sense, as I'm reading/writing 
text to buffers, and care very little about decoding.

I wanted to test my system's ability to handle random-access ranges, and 
I'm using isRandomAccessRange || isNarrowString to get around this.

As soon as I do chain(a, b) where a and b are narrow strings, this 
doesn't work, and I can't get it back.

See my exception in the unit test: 
https://github.com/schveiguy/iopipe/blob/master/source/iopipe/traits.d#L91

If you want to avoid auto-decoding, you have to be very cautious about 
using Phobos features.

-Steve



 May 12 2016



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 9:29 AM, Andrei Alexandrescu wrote:

 I am as unclear about the problems of autodecoding


Started a new thread on that.



 May 12 2016



prev sibling  parent  Marco Leise <Marco.Leise gmx.de>  writes:


Am Thu, 12 May 2016 08:55:52 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 On 5/12/2016 3:30 AM, Manu via Digitalmars-d wrote:

 If you're set on a warning, at least make the warning recommend
 down-casting the higher precision term to the lower precision?  


 
 Yes, of course. I believe error messages should suggest corrective action.


Because of afore mentioned difference between cast(float)1.30
and 1.30f, the correct action for the original case is to
suffix the literal with 'f'. That gives you the correct number
to compare to.

-- 
Marco



 May 12 2016



prev sibling  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 5/12/16 3:32 AM, Walter Bright wrote:

 On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:

 What is the problem with this behaviour I suggest?


 Code will do one thing in C, and the same code will do something
 unexpectedly different in D.


Not taking one side or another on this, but due to D doing everything 
with reals, this is already the case.

-Steve



 May 12 2016



 next sibling parent reply Ethan Watson <gooberman gmail.com>  writes:


On Thursday, 12 May 2016 at 13:03:58 UTC, Steven Schveighoffer 
wrote:

 Not taking one side or another on this, but due to D doing 
 everything with reals, this is already the case.


Mmmm. I don't want to open up another can of worms right now, but 
our x64 C++ code only emits SSE instructions at compile time (or 
AVX on the Xbox One). The only thing that attempts to use reals 
in our codebase is our D code.



 May 12 2016



  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 5/12/16 10:05 AM, Ethan Watson wrote:

 On Thursday, 12 May 2016 at 13:03:58 UTC, Steven Schveighoffer wrote:

 Not taking one side or another on this, but due to D doing everything
 with reals, this is already the case.


 Mmmm. I don't want to open up another can of worms right now, but our
 x64 C++ code only emits SSE instructions at compile time (or AVX on the
 Xbox One). The only thing that attempts to use reals in our codebase is
 our D code.


There was a question on the forums a while back about equivalent C++ 
code that didn't work in D. The answer turned out to be, you had to 
shoehorn everything into doubles in order to get the same answer.

-Steve



 May 12 2016



  parent  Ethan Watson <gooberman gmail.com>  writes:


On Thursday, 12 May 2016 at 14:29:01 UTC, Steven Schveighoffer 
wrote:

 There was a question on the forums a while back about 
 equivalent C++ code that didn't work in D. The answer turned 
 out to be, you had to shoehorn everything into doubles in order 
 to get the same answer.


I can certainly see that being the case, especially when dealing 
with SSE-based code. floats and doubles in XMM registers don't 
get calculated at 80-bit precision internally, their storage size 
dictates their calculation precision. Which has led MSVC to 
promoting floats to doubles for CRT functions when it thinks it 
can get away with it (and one instance where the compiler forgot 
to convert back to float afterwards and thus the lower 32 bits of 
a double were being treated as a float...)

It's fun comparing assembly too. There's one particular function 
we have here that collapsed to something like 20-30 lines of 
SSE-based code (half after I hand optimised it with branchless 
SSE intrinsics and without a call to fmod). The same function in 
D resulted in a significantly larger amount of x87 code.

I don't miss x87 at all. But this is getting OT.



 May 12 2016



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 6:03 AM, Steven Schveighoffer wrote:

 Not taking one side or another on this, but due to D doing everything with
 reals, this is already the case.


Actually, C allows that behavior if I understood the spec correctly. D just 
makes things more explicit.



 May 12 2016



  parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 5/12/16 12:00 PM, Walter Bright wrote:

 On 5/12/2016 6:03 AM, Steven Schveighoffer wrote:

 Not taking one side or another on this, but due to D doing everything
 with
 reals, this is already the case.


 Actually, C allows that behavior if I understood the spec correctly. D
 just makes things more explicit.


This is the thread I was thinking about: 
https://forum.dlang.org/post/ugxmeiqsbitxxzoyakko forum.dlang.org

Essentially, copy-pasted code from C results in different behavior in D 
because of the different floating point decisions made by the compilers.

Your quote was: "[The problem is] Code will do one thing in C, and the 
same code will do something unexpectedly different in D."

My response is that it already happens, so it may not be a convincing 
argument. I don't know the requirements or allowances of the C spec. All 
I know is the testable reality of the behavior on the same platform.

-Steve



 May 12 2016



prev sibling  parent reply Marco Leise <Marco.Leise gmx.de>  writes:


Am Thu, 12 May 2016 09:03:58 -0400
schrieb Steven Schveighoffer <schveiguy yahoo.com>:


 Not taking one side or another on this, but due to D doing everything 
 with reals, this is already the case.
 
 -Steve


As far as I have understood the situation:
- FPU instructions are inaccurate
- FPU is typically set to highest precision (80-bit)
  to give accurate float/double results
- SSE instructions yield accurately rounded results
- Need for 80 bits greatly reduced with SSE
- FPU deprectated on 64-bit x86
- Results of FPU and SSE math differ
- Compiler writers discordant
   GCC: compiler switch for SSE or FPU, SSE default
   DMD: FPU only
- Unless CTFE uses soft-float implementation, depending on
  compiler and flags used to compile a D compiler, resulting
  executable produces different CTFE floating-point results

-- 
Marco



 May 12 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 4:32 PM, Marco Leise wrote:

 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results


I've actually been thinking of writing a 128 bit float emulator, and then using 
that in the compiler internals to do all FP computation with.

It's not a panacea, as it won't change how things work in the back ends, nor 
will it change what happens at runtime, but it means the front end will give 
portable, consistent results.



 May 12 2016



 next sibling parent reply Jack Stouffer <jack jackstouffer.com>  writes:


On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:

 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.

 It's not a panacea, as it won't change how things work in the 
 back ends, nor will it change what happens at runtime, but it 
 means the front end will give portable, consistent results.


And be 20x slower than hardware floats. Is it really worth it?



 May 12 2016



 next sibling parent  xenon325 <anm programmer.net>  writes:


On Friday, 13 May 2016 at 03:18:05 UTC, Jack Stouffer wrote:

 On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:

 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.
 [...]


 And be 20x slower than hardware floats. Is it really worth it?


Emulator is meant for computation during compilation only, so 
CTFE results are consistent across different compilers and 
compiler host hardware (IIUC).

-- Alexander



 May 12 2016



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 8:18 PM, Jack Stouffer wrote:

 And be 20x slower than hardware floats. Is it really worth it?


I seriously doubt the slowdown would be measurable, as the number of float ops 
the compiler performs is insignificant.



 May 13 2016



prev sibling  parent reply deadalnix <deadalnix gmail.com>  writes:


On Friday, 13 May 2016 at 03:18:05 UTC, Jack Stouffer wrote:

 On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:

 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.

 It's not a panacea, as it won't change how things work in the 
 back ends, nor will it change what happens at runtime, but it 
 means the front end will give portable, consistent results.


 And be 20x slower than hardware floats. Is it really worth it?


Yes. What is the proportion of CTFE floating point operation 
right know in the total compile time ? I'd be highly supersized 
if it was anything close to 1% .



 May 15 2016



  parent  Max Samukha <maxsamukha gmail.com>  writes:


On Sunday, 15 May 2016 at 12:11:58 UTC, deadalnix wrote:


 Yes. What is the proportion of CTFE floating point operation 
 right know in the total compile time ? I'd be highly supersized 
 if it was anything close to 1% .


That was said about CTFE in general before people started to use 
it and discovered it is unusable because of performance issues 
(vibe.d is a prominent example).



 May 15 2016



prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/12/2016 4:32 PM, Marco Leise wrote:

 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results



 I've actually been thinking of writing a 128 bit float emulator, and then
 using that in the compiler internals to do all FP computation with.


No. Do not.
I've worked on systems where the compiler and the runtime don't share
floating point precisions before, and it was a nightmare.
One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit
iirc?) float precision, code compiled for it used 32bits in the
compiler... to make it worse, the CPU also ran 32bits. The result was,
literals/constants, or float data fed from the CPU didn't match data
calculated by the vector unit at runtime (ie, runtime computation of
the same calculation that may have occurred at compile time to produce
some constant didn't match). The result was severe cracking and
visible/shimmering seams between triangles as sub-pixel alignment
broke down.
We struggled with this for years. It was practically impossible to
solve, and mostly involved workarounds.

I really just want D to use double throughout, like all the cpu's that
run code today. This 80bit real thing (only on x86 cpu's though!) is a
never ending pain.



 It's not a panacea, as it won't change how things work in the back ends, nor
 will it change what happens at runtime, but it means the front end will give
 portable, consistent results.


This sounds like designing specifically for my problem from above,
where the frontend is always different than the backend/runtime.
Please have the frontend behave such that it operates on the precise
datatype expressed by the type... the backend probably does this too,
and runtime certainly does; they all match.



 May 12 2016



 next sibling parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Friday, 13 May 2016 at 05:12:14 UTC, Manu wrote:

 No. Do not.
 I've worked on systems where the compiler and the runtime don't 
 share
 floating point precisions before, and it was a nightmare.


Use reproducible cross platform IEEE754-2008 and use exact 
rational numbers. All other representations are just painful. 
Nothing wrong with supporting 16, 32, 64 and 128 bit, but stick 
to the reproducible standard. If people want "non-reproducible 
fast math", then they should specify it.



 May 13 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/12/2016 10:12 PM, Manu via Digitalmars-d wrote:

 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.
 One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit
 iirc?) float precision, code compiled for it used 32bits in the
 compiler... to make it worse, the CPU also ran 32bits. The result was,
 literals/constants, or float data fed from the CPU didn't match data
 calculated by the vector unit at runtime (ie, runtime computation of
 the same calculation that may have occurred at compile time to produce
 some constant didn't match). The result was severe cracking and
 visible/shimmering seams between triangles as sub-pixel alignment
 broke down.
 We struggled with this for years. It was practically impossible to
 solve, and mostly involved workarounds.


I understand there are some cases where this is needed, I've proposed
intrinsics 
for that.



 I really just want D to use double throughout, like all the cpu's that
 run code today. This 80bit real thing (only on x86 cpu's though!) is a
 never ending pain.


It's 128 bits on other CPUs.



 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.


Except this never happens anyway.



 May 13 2016



 next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Friday, 13 May 2016 at 18:16:29 UTC, Walter Bright wrote:

 Please have the frontend behave such that it operates on the 
 precise
 datatype expressed by the type... the backend probably does 
 this too,
 and runtime certainly does; they all match.


 Except this never happens anyway.


It should in C++ with the right strict-settings, which makes the 
compiler use reproducible floating point operations. AFAIK it 
should work out even in modern JavaScript.



 May 13 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:

 It should in C++ with the right strict-settings,


Consider what the C++ Standard says, not what the endless switches to tweak the 
compiler do.



 May 13 2016



  parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Friday, 13 May 2016 at 21:36:52 UTC, Walter Bright wrote:

 On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:

 It should in C++ with the right strict-settings,


 Consider what the C++ Standard says, not what the endless 
 switches to tweak the compiler do.


The C++ standard cannot even require IEEE754. Nobody relies only 
on what the C++ standard says in real projects. They rely on what 
the chosen compiler(s) on concrete platform(s) do.



 May 13 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/13/2016 2:42 PM, Ola Fosheim Grøstad wrote:

 On Friday, 13 May 2016 at 21:36:52 UTC, Walter Bright wrote:

 On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:

 It should in C++ with the right strict-settings,


 Consider what the C++ Standard says, not what the endless switches to tweak
 the compiler do.


 The C++ standard cannot even require IEEE754. Nobody relies only on what the
C++
 standard says in real projects. They rely on what the chosen compiler(s) on
 concrete platform(s) do.



Nevertheless, C++ is what the Standard says it is. If Brand X compiler does 
something else, you should call it "Brand X C++".



 May 13 2016



prev sibling  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 14 May 2016 at 04:16, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/12/2016 10:12 PM, Manu via Digitalmars-d wrote:

 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.
 One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit
 iirc?) float precision, code compiled for it used 32bits in the
 compiler... to make it worse, the CPU also ran 32bits. The result was,
 literals/constants, or float data fed from the CPU didn't match data
 calculated by the vector unit at runtime (ie, runtime computation of
 the same calculation that may have occurred at compile time to produce
 some constant didn't match). The result was severe cracking and
 visible/shimmering seams between triangles as sub-pixel alignment
 broke down.
 We struggled with this for years. It was practically impossible to
 solve, and mostly involved workarounds.



 I understand there are some cases where this is needed, I've proposed
 intrinsics for that.


Intrinsics for... what? Making the compiler use the type specified at
compile time?
Is it true that that's not happening already?

I really don't want to use an intrinsic to have float behave like a
float at CTFE... nobody will EVER do that.



 I really just want D to use double throughout, like all the cpu's that
 run code today. This 80bit real thing (only on x86 cpu's though!) is a
 never ending pain.



 It's 128 bits on other CPUs.


What?



 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.



 Except this never happens anyway.


Huh?


I'm sorry, I didn't follow those points.



 May 15 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 7:04 PM, Manu via Digitalmars-d wrote:

 I understand there are some cases where this is needed, I've proposed
 intrinsics for that.


 Intrinsics for... what?


    float roundToFloat(float f);



 I really don't want to use an intrinsic to have float behave like a
 float at CTFE... nobody will EVER do that.


Floats aren't required to have float precision by the C or C++ Standards. I 
quoted it for Ola :-)



 It's 128 bits on other CPUs.


 What?


Some CPUs have 128 bit floats.



 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.



 Except this never happens anyway.


 Huh? I'm sorry, I didn't follow those points.


The belief that compile time and runtime are exactly the same floating point in 
C/C++ is false.



 May 15 2016



 next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 12:56, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 7:04 PM, Manu via Digitalmars-d wrote:

 I understand there are some cases where this is needed, I've proposed
 intrinsics for that.


 Intrinsics for... what?



    float roundToFloat(float f);



 I really don't want to use an intrinsic to have float behave like a
 float at CTFE... nobody will EVER do that.



 Floats aren't required to have float precision by the C or C++ Standards. I
 quoted it for Ola :-)



 It's 128 bits on other CPUs.


 What?



 Some CPUs have 128 bit floats.


Yes, but you don't accidentally use 128bit floats, you type:

extended x = 1.3;
x + y;

If that code were to CTFE, I expect the CTFE to use extended precision.
My point is, CTFE should surely follow the types and language
semantics as if it were runtime generated code... It's not reasonable
that CTFE has higher precision applied than the same code at runtime.
CTFE must give the exact same result as runtime execution of the function.


 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.




 Except this never happens anyway.



 Huh? I'm sorry, I didn't follow those points.



 The belief that compile time and runtime are exactly the same floating point
 in C/C++ is false.


I'm not interested in C/C++, I gave some anecdotes where it's gone
wrong for me too, but regardless; generally, they do match, and I
can't think of a single modern example where that's not true. If you
*select* fast-math, then you may generate code that doesn't match, but
that's a deliberate selection.

If I want 'real' math (in CTFE or otherwise), I will type "real". It
is completely unreasonable to reinterpret the type that the user
specified. CTFE should execute code the same way runtime would execute
the code (without -ffast-math, and conformant ieee hardware). This is
not a big ask.

Incidentally, I made the mistake of mentioning this thread (due to my
astonishment that CTFE ignores float types) out loud to my
colleagues... and they actually started yelling violently out loud.
One of them has now been on a 10 minute angry rant with elevated tone
and waving his arms around about how he's been shafted by this sort
behaviour so many times before. I wish I recorded it, I'd love to have
submit it as evidence.



 May 15 2016



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 9:02 PM, Manu via Digitalmars-d wrote:

 Yes, but you don't accidentally use 128bit floats, you type:

 extended x = 1.3;
 x + y;


The C/C++ standards allow constant folding at 128 bits, despite floats being 32 
bits.



 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.


It hasn't for decades on x86 machines, and the world hasn't collapsed, in fact, 
few people ever even notice. Not many people prefer less accurate answers.

The initial Java spec worked as you desired, and they were pretty much forced
to 
back off of it.



 The belief that compile time and runtime are exactly the same floating point
 in C/C++ is false.


 I'm not interested in C/C++, I gave some anecdotes where it's gone
 wrong for me too, but regardless; generally, they do match, and I
 can't think of a single modern example where that's not true. If you
 *select* fast-math, then you may generate code that doesn't match, but
 that's a deliberate selection.


They won't match on any code that uses the x87. The standard doesn't require 
float math to use float instructions, they can (and do) use double instructions 
for temporaries.



 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)


Float types are not selected because they are less accurate, they are selected 
because they are smaller and faster.


 out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.


I'm interested to hear how he was "shafted" by this. This apparently also 
contradicts the claim that other languages do as you ask.



 May 15 2016



 next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 14:26, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 9:02 PM, Manu via Digitalmars-d wrote:

 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.



 It hasn't for decades on x86 machines, and the world hasn't collapsed, in
 fact, few people ever even notice. Not many people prefer less accurate
 answers.


That doesn't mean it's not wrong.
Don noticed, he gave a lecture on floating-point gotchas.
I'm still firmly engaged in trying to use D professionally... should I
ever successfully pass that barrier, then it's just a matter of time
until such a piece of code that tend to challenge these things is
written. There are not enough uses of D that we can offer anecdotes,
but we can offer anecdotes from our decades with C++ that we would
desperately like to avoid in the future.

If we don't care about doing what's right, then we just accept that
float remains a highly expert/special-knowledge-centric field, and
talks like the one Don gave should be taken as thought-provoking,
however of no practical relevance for the language, since it's 'fine
for most people, most of the time', and we get on with other things.
This thread is evidence that people would like to do the best we can.

1.3f != 1.3 is not accurate, it's wrong.



 The initial Java spec worked as you desired, and they were pretty much
 forced to back off of it.


Ok, why's that?



 The belief that compile time and runtime are exactly the same floating
 point
 in C/C++ is false.



 I'm not interested in C/C++, I gave some anecdotes where it's gone
 wrong for me too, but regardless; generally, they do match, and I
 can't think of a single modern example where that's not true. If you
 *select* fast-math, then you may generate code that doesn't match, but
 that's a deliberate selection.



 They won't match on any code that uses the x87. The standard doesn't require
 float math to use float instructions, they can (and do) use double
 instructions for temporaries.


If it does, then it is careful to make sure the precision expectations
are maintained. If you don't '-ffast-math', the FPU code produces a
IEEE conformant result on reasonable compilers. We depend on this.



 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)



 Float types are not selected because they are less accurate, they are
 selected because they are smaller and faster.


They are selected because they are smaller and faster with the
understood trade-off that they are less accurate.
They are certainly selected with the _intent_ that they are less accurate.

It's not reasonable that a CTFE function may produce a radically
different result than the same function at runtime.


 out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.



 I'm interested to hear how he was "shafted" by this. This apparently also
 contradicts the claim that other languages do as you ask.


I've explained prior the cases where this has happened are most often
invoked by the hardware having a reduced runtime precision than the
compiler. The only cases I know of where this has happened due to the
compiler internally is CodeWarrior; an old/dead C++ compiler that
always sucked and caused us headaches of all kinds.
The point is, the CTFE behaviour is essentially identical to our
classic case where the hardware runs a different precision than the
compiler, and that's built into the language! It's not just an anomaly
expressed by one particular awkward platform we're required to
support.



 May 15 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 10:13 PM, Manu via Digitalmars-d wrote:

 1.3f != 1.3 is not accurate, it's wrong.


I'm sorry, there is no way to make FP behave like mathematics. It's its own 
beast, with its own rules.



 The initial Java spec worked as you desired, and they were pretty much
 forced to back off of it.


 Ok, why's that?


Because forcing the x87 to work at reduced precision caused a 2x slowdown or 
something like that, making Java uncompetitive (i.e. unusable) for numerics
work.



 They won't match on any code that uses the x87. The standard doesn't require
 float math to use float instructions, they can (and do) use double
 instructions for temporaries.


 If it does, then it is careful to make sure the precision expectations
 are maintained.


Have you tested this?


 If you don't '-ffast-math', the FPU code produces a
 IEEE conformant result on reasonable compilers.


Googling 'fp:fast' yields:

"Creates the fastest code in most cases by relaxing the rules for optimizing 
floating-point operations. This enables the compiler to optimize floating-point 
code for speed at the expense of accuracy and correctness. When /fp:fast is 
specified, the compiler may not round correctly at assignment statements, 
typecasts, or function calls, and may not perform rounding of intermediate 
expressions. The compiler may reorder operations or perform algebraic 
transforms—for example, by following associative and distributive
rules—without 
regard to the effect on finite precision results. The compiler may change 
operations and operands to single-precision instead of following the C++ type 
promotion rules. Floating-point-specific contraction optimizations are always 
enabled (fp_contract is ON). Floating-point exceptions and FPU environment 
access are disabled (/fp:except- is implied and fenv_access is OFF)."

This doesn't line up with what you said it does?


 We depend on this.


I googled 'fp:precise', which is the VC++ default, and found this:

"Using /fp:precise when fenv_access is ON disables optimizations such as 
compile-time evaluations of floating-point expressions."

How about that? No CTFE! Is that really what you wanted? :-)



 They are certainly selected with the _intent_ that they are less accurate.


This astonishes me. What algorithm requires less accuracy?



 It's not reasonable that a CTFE function may produce a radically
 different result than the same function at runtime.


Yeah, since the less accurate version can suffer from a phenomenon called
"total 
loss of precision" where accumulated roundoff errors make the value utter 
garbage. When is this desirable?



 I'm interested to hear how he was "shafted" by this. This apparently also
 contradicts the claim that other languages do as you ask.


 I've explained prior the cases where this has happened are most often
 invoked by the hardware having a reduced runtime precision than the
 compiler. The only cases I know of where this has happened due to the
 compiler internally is CodeWarrior; an old/dead C++ compiler that
 always sucked and caused us headaches of all kinds.
 The point is, the CTFE behaviour is essentially identical to our
 classic case where the hardware runs a different precision than the
 compiler, and that's built into the language! It's not just an anomaly
 expressed by one particular awkward platform we're required to
 support.


You mentioned there was a "shimmer" effect. With the extremely limited ability 
of C++ compilers to fold constants, I'm left wondering how your code suffered 
from this, and why you would calculate the same value at both compile and run
time.



 May 15 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 16.05.2016 06:26, Walter Bright wrote:

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)


 Float types are not selected because they are less accurate,


(AFAIK, accuracy is a property of a value given some additional context. 
Types have precision.)


 they are selected because they are smaller and faster.


Right. Hence, the 80-bit CTFE results have to be converted to the final 
precision at some point in order to commence the runtime computation. 
This means that additional rounding happens, which was not present in 
the original program. The additional total roundoff error this 
introduces can exceed the roundoff error you would have suffered by 
using the lower precision in the first place, sometimes completely 
defeating precision-enhancing improvements to an algorithm.

This might be counter-intuitive, but this is floating point. The 
precision should just stay the specified one during the entire 
computation (even if part of it is evaluated at compile-time).

The claim here is /not/ that lower precision throughout delivers more 
accurate results. The additional rounding is the problem.


There are other reasons why I think that this kind of 
implementation-defined behaviour is a terribly bad idea, eg.:

- it breaks common assumptions about code, especially how it behaves 
under seemingly innocuous refactorings, or with a different set of 
compiler flags.

- it breaks reproducibility, which is sometimes more important that 
being close to the infinite precision result (which you cannot guarantee 
with any finite floating point type anyway).
   (E.g. in a game, it is enough if the result seems plausible, but it 
should be the same for everyone. For some scientific experiments, the 
ideal case is to have 100% reproducibility of the computation, even if 
it is horribly wrong, such that other scientists can easily uncover and 
diagnose the problem, for example.)



 May 17 2016



 next sibling parent reply deadalnix <deadalnix gmail.com>  writes:


On Tuesday, 17 May 2016 at 18:08:47 UTC, Timon Gehr wrote:

 Right. Hence, the 80-bit CTFE results have to be converted to 
 the final precision at some point in order to commence the 
 runtime computation. This means that additional rounding 
 happens, which was not present in the original program. The 
 additional total roundoff error this introduces can exceed the 
 roundoff error you would have suffered by using the lower 
 precision in the first place, sometimes completely defeating 
 precision-enhancing improvements to an algorithm.


WAT ? Is that really possible ?



 May 17 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 17.05.2016 21:31, deadalnix wrote:

 On Tuesday, 17 May 2016 at 18:08:47 UTC, Timon Gehr wrote:

 Right. Hence, the 80-bit CTFE results have to be converted to the
 final precision at some point in order to commence the runtime
 computation. This means that additional rounding happens, which was
 not present in the original program. The additional total roundoff
 error this introduces can exceed the roundoff error you would have
 suffered by using the lower precision in the first place, sometimes
 completely defeating precision-enhancing improvements to an algorithm.


 WAT ? Is that really possible ?


Yes, I'm sorry, but this can and does happen.
Consider http://forum.dlang.org/post/nhi7m4$css$1 digitalmars.com

You can build similar examples involving only CTFE. Refer to 
http://forum.dlang.org/post/nhi9gh$fa4$1 digitalmars.com for an 
explanation of one case where this can happen. (I had actually written 
that post three days ago, and assumed that it had been posted to the 
newsgroup, but something went wrong, apparently.)



 May 18 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/17/2016 11:08 AM, Timon Gehr wrote:

 Right. Hence, the 80-bit CTFE results have to be converted to the final
 precision at some point in order to commence the runtime computation. This
means
 that additional rounding happens, which was not present in the original
program.
 The additional total roundoff error this introduces can exceed the roundoff
 error you would have suffered by using the lower precision in the first place,
 sometimes completely defeating precision-enhancing improvements to an
algorithm.


I'd like to see an example of double rounding "completely defeating" an 
algorithm, and why an unusual case of producing a slightly worse answer trumps 
the usual case of producing better answers.



 There are other reasons why I think that this kind of implementation-defined
 behaviour is a terribly bad idea, eg.:

 - it breaks common assumptions about code, especially how it behaves under
 seemingly innocuous refactorings, or with a different set of compiler flags.


As pointed out, this already happens with just about every language. It happens 
with all C/C++ compilers I'm aware of. It happens as the default behavior of
the 
x86. And as pointed out, refactoring (x+y)+z to x+(y+z) often produces
different 
results, and surprises a lot of people.



 - it breaks reproducibility, which is sometimes more important that being close
 to the infinite precision result (which you cannot guarantee with any finite
 floating point type anyway).
   (E.g. in a game, it is enough if the result seems plausible, but it should be
 the same for everyone. For some scientific experiments, the ideal case is to
 have 100% reproducibility of the computation, even if it is horribly wrong,
such
 that other scientists can easily uncover and diagnose the problem, for
example.)


Nobody is proposing a D feature that does not produce reproducible results with 
the same program on the same inputs. This complaint is a strawman, as I've 
pointed out multiple times.

In fact, the results would be MORE portable than with C/C++, because the FP 
behavior is completely implementation defined, and compilers take advantage of
that.



 May 17 2016



 next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Tue, May 17, 2016 at 02:07:21PM -0700, Walter Bright via Digitalmars-d wrote:

 On 5/17/2016 11:08 AM, Timon Gehr wrote:


[...]

- it breaks reproducibility, which is sometimes more important that
being close to the infinite precision result (which you cannot
guarantee with any finite floating point type anyway).  (E.g. in a
game, it is enough if the result seems plausible, but it should be
the same for everyone. For some scientific experiments, the ideal
case is to have 100% reproducibility of the computation, even if it
is horribly wrong, such that other scientists can easily uncover and
diagnose the problem, for example.)


 
 Nobody is proposing a D feature that does not produce reproducible
 results with the same program on the same inputs. This complaint is a
 strawman, as I've pointed out multiple times.


Wasn't Manu's original complaint that, given a particular piece of FP
code that uses floats, evaluating that code at compile-time may produce
different results than evaluating it at runtime, because (as you're
proposing) the compiler will use higher precision than specified for
intermediate results?  Of course, the compile-time answer is arguably
"more correct" because it has less roundoff error, but the point here is
not how accurate that answer is, but that it *doesn't match the runtime
results*. This mismatch, from what I understand, is what causes the
graphical glitches that Manu was referring to.

According to your prescription, then, the runtime code should be "fixed"
to use higher precision, so that it will also produce the same, "more
correct" answer.  But unfortunately, that's not workable because of the
performance implications. At the end of the day, nobody cares whether a
game draws a polygon with the most precise coordinates, what people *do*
care is that there's a mismatch between the "more correct" and "less
correct" rendering of the polygon (produced, respectively, from CTFE and
from runtime) that causes a visually noticeable glitch. It *looks*
wrong, no matter how much you may argue that it's "more correct". You
are probably right scientifically, but in a game, people are concerned
about what they see, not whether polygon coordinates have less roundoff
error at CTFE vs. at runtime.


T

-- 
We are in class, we are supposed to be learning, we have a teacher... Is it too
much that I expect him to teach me??? -- RL



 May 17 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/17/2016 5:22 PM, H. S. Teoh via Digitalmars-d wrote:

 Wasn't Manu's original complaint that, given a particular piece of FP
 code that uses floats, evaluating that code at compile-time may produce
 different results than evaluating it at runtime, because (as you're
 proposing) the compiler will use higher precision than specified for
 intermediate results?  Of course, the compile-time answer is arguably
 "more correct" because it has less roundoff error, but the point here is
 not how accurate that answer is, but that it *doesn't match the runtime
 results*. This mismatch, from what I understand, is what causes the
 graphical glitches that Manu was referring to.


Except this happens with C/C++ too, and in fact Manu wasn't using D.


 According to your prescription, then, the runtime code should be "fixed"
 to use higher precision, so that it will also produce the same, "more
 correct" answer.  But unfortunately, that's not workable because of the
 performance implications. At the end of the day, nobody cares whether a
 game draws a polygon with the most precise coordinates, what people *do*
 care is that there's a mismatch between the "more correct" and "less
 correct" rendering of the polygon (produced, respectively, from CTFE and
 from runtime) that causes a visually noticeable glitch. It *looks*
 wrong, no matter how much you may argue that it's "more correct". You
 are probably right scientifically, but in a game, people are concerned
 about what they see, not whether polygon coordinates have less roundoff
 error at CTFE vs. at runtime.


That wasn't my prescription. My prescription was either changing the algorithm 
so it was not sensitive to exact bits-in-last-place, or to use roundToFloat() 
and roundToDouble() functions.



 May 17 2016



  parent reply Ethan Watson <gooberman gmail.com>  writes:


On Wednesday, 18 May 2016 at 05:40:57 UTC, Walter Bright wrote:

 That wasn't my prescription. My prescription was either 
 changing the algorithm so it was not sensitive to exact 
 bits-in-last-place, or to use roundToFloat() and 
 roundToDouble() functions.


With Manu's example, that would have been a good old fashioned 
matrix multiply to transform a polygon vertex from local space to 
screen space, with whatever other values were required for the 
render effect. The problem there being that the hardware itself 
only calculated 24 bits of precision while dealing with 32 bit 
values. Such a solution was not an option.

Gaming hardware has gotten a lot less cheap and nasty. But Manu 
brought it up because it is conceptually the same problem as 
32/64 bit run time values vs 80 bit compile time values. Every 
solution offered here either comes down to "rewrite your code" or 
"increase code complexity", neither of which is often an option 
(changing the code in Manu's example required a seven+ hour 
compile time each iteration of the code; and being a very hot 
piece of code, it needed to be as simple as possible to maintain 
speed). Unlike the hardware, game programming has not gotten less 
cheap nor nasty. We will cheat our way to the fastest performing 
code using whatever trick we can find that doesn't cause the 
whole thing to come crashing down. The standard way for creating 
float values at compile time is to calculate them manually at the 
correct precision and put a #define in with that value. Being 
unable to specify/override compile time precision means that the 
solution is to declare enums in the exact same manner, and might 
result in more maintenance work if someone decides they want to 
switch from float to double etc. for their value.



 May 17 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/17/2016 11:15 PM, Ethan Watson wrote:

 With Manu's example, that would have been a good old fashioned matrix multiply
 to transform a polygon vertex from local space to screen space, with whatever
 other values were required for the render effect. The problem there being that
 the hardware itself only calculated 24 bits of precision while dealing with 32
 bit values. Such a solution was not an option.


I don't understand the 24 vs 32 bit value thing. There is no 32 bit mantissa 
floating point type. Floats have 24 bit mantissas, doubles 52.


 Gaming hardware has gotten a lot less cheap and nasty. But Manu brought it up
 because it is conceptually the same problem as 32/64 bit run time values vs 80
 bit compile time values. Every solution offered here either comes down to
 "rewrite your code" or "increase code complexity", neither of which is often an
 option (changing the code in Manu's example required a seven+ hour compile time
 each iteration of the code; and being a very hot piece of code, it needed to be
 as simple as possible to maintain speed). Unlike the hardware, game programming
 has not gotten less cheap nor nasty. We will cheat our way to the fastest
 performing code using whatever trick we can find that doesn't cause the whole
 thing to come crashing down. The standard way for creating float values at
 compile time is to calculate them manually at the correct precision and put a
 #define in with that value. Being unable to specify/override compile time
 precision means that the solution is to declare enums in the exact same manner,
 and might result in more maintenance work if someone decides they want to
switch
 from float to double etc. for their value.


I do not understand why the compile time version cannot use roundToFloat() in 
places where it matters. And if the hardware was using a different precision 
than float/double, which appears to have been the case, the code would have to 
be written to account for that anyway.

In any case, the problem Manu was having was with C++. The precision of 
calculations is implementation defined in C++, and does vary all over the place 
depending on compiler brands, compiler versions, compiler switches, and exactly 
how the code is laid out. There can also be differences in how the FP hardware 
works on the compiler host machine and the target machine.

My proposal would make the behavior more consistent than C++, not less.

Lastly, it is hard to make suggestions on how to deal with the problem without 
seeing the actual offending code. There may very well be something else going 
on, or some simple adjustment that can be made.

One way that *will* make the results exactly the same as on the target hardware 
is to actually run the code on the target hardware, save the results to a file, 
and incorporate that file in the build.



 May 17 2016



  parent reply Ethan Watson <gooberman gmail.com>  writes:


On Wednesday, 18 May 2016 at 06:57:58 UTC, Walter Bright wrote:

 I don't understand the 24 vs 32 bit value thing. There is no 32 
 bit mantissa floating point type. Floats have 24 bit mantissas, 
 doubles 52.


Not in the standards, no. But older gaming hardware was never 
known to be standards-conformant.

As it turns out, the original hardware manuals can be found on 
the internet.

https://www.dropbox.com/s/rsgx6xmpkf2zzz8/VU_Users_Manual.pdf

Relevant info copied from page 27:

Calculation
* A 24-bit calculation including hidden bits is performed, and 
the result is truncated.  The rounding-off operation in IEEE 754 
is performed in the 0 direction, so the values for the least 
significant bit may vary.


 In any case, the problem Manu was having was with C++.


VU code was all assembly, I don't believe there was a C/C++ 
compiler for it.


 My proposal would make the behavior more consistent than C++, 
 not less.


This is why I ask for a compiler option to make it consistent 
with the C++ floating point architecture I select. Making it 
better than C++ is great for use cases where you're not 
inter-opping with C++ extensively.

Although I do believe saying C++ is just clouding things here. 
Language doesn't matter, it's the runtime code using a different 
floating point instruction set/precision to compile time code 
that's the problem. See the SSE vs x87 comparisons posted in this 
thread for a concrete example. Same C++ code, different 
instruction sets and precisions.

Regardless. Making extra build steps with pre-calculated values 
or whatever is of course a workable solution, but it also raises 
the barrier of entry. You can't just, for example, select a 
compiler option in your IDE and have it just work. You have to go 
out of your way to make it work the way you want it to. And if 
there's one thing you can count on, it's end users being lazy.



 May 18 2016



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 12:56 AM, Ethan Watson wrote:

 In any case, the problem Manu was having was with C++.


 VU code was all assembly, I don't believe there was a C/C++ compiler for it.


The constant folding part was where, then?



 This is why I ask for a compiler option to make it consistent with the C++
 floating point architecture I select. Making it better than C++ is great for
use
 cases where you're not inter-opping with C++ extensively.


Trying to make D behave exactly like various C++ compilers do, with all their 
semi-documented behavior and semi-documented switches that affect constant 
folding behavior, is a hopeless task.

I doubt various C++ compilers are this compatible, even if they follow the same
ABI.

You're also asking for a mode where the compiler for one machine is supposed to 
behave like hand-coded assembler for another machine with a different 
instruction set. I doubt any compiler is going to deliver that, unless you 
create a customized compiler specifically for that.



 Although I do believe saying C++ is just clouding things here. Language doesn't
 matter, it's the runtime code using a different floating point instruction
 set/precision to compile time code that's the problem. See the SSE vs x87
 comparisons posted in this thread for a concrete example. Same C++ code,
 different instruction sets and precisions.

 Regardless. Making extra build steps with pre-calculated values or whatever is
 of course a workable solution, but it also raises the barrier of entry. You
 can't just, for example, select a compiler option in your IDE and have it just
 work. You have to go out of your way to make it work the way you want it to.
And
 if there's one thing you can count on, it's end users being lazy.


 From your description, there would have been a problem *regardless* of the 
precision used for constant folding. This is because the target hardware used 
truncation, and constant folding (in all compilers I know of) use rounding, and 
no compiler I know of allows controlling the rounding mode in constant folding.



 May 18 2016



 next sibling parent reply Ethan Watson <gooberman gmail.com>  writes:


On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:

 The constant folding part was where, then?


Probably on the EE, executed with different precision (32-bit) 
and passed over to the VU via one of its registers. Manu spent 
more time with that code than I did and can probably give exact 
details. But pasting the code, given it's proprietary code from a 
15 year old engine, will be difficult at best considering the 
code is likely on a backup tape somewhere.


 You're also asking for a mode where the compiler for one 
 machine is supposed to behave like hand-coded assembler for 
 another machine with a different instruction set.


Actually, I'm asking for something exactly like the arch option 
for MSVC/-mfmath option for GCC/etc, and have it respect that for 
CTFE.



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 1:30 AM, Ethan Watson wrote:

 You're also asking for a mode where the compiler for one machine is supposed
 to behave like hand-coded assembler for another machine with a different
 instruction set.


 Actually, I'm asking for something exactly like the arch option for
MSVC/-mfmath
 option for GCC/etc, and have it respect that for CTFE.



MSVC doesn't appear to have a switch that does what you ask for:

   https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx



 May 18 2016



 next sibling parent reply Ethan Watson <gooberman gmail.com>  writes:


On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

 MSVC doesn't appear to have a switch that does what you ask for


I'm still not entirely sure what the /fp switch does for x64 
builds. The documentation is not clear in the slightest and I 
haven't been able to find any concrete information. As near as I 
can tell it has no effect as the original behaviour was tied to 
how it handles the x87 control words. But it might also be 
possible that the SSE instructions emitted can differ depending 
on what operation you're trying to do. I have not dug deep to see 
exactly how the code gen differs. I can take a guess that 
/fp:precise was responsible for promoting my float to a double to 
call CRT functions, but I have not tested that so that's purely 
theoretical at the moment.

Of course, while this conversation has mostly been for compile 
time constant folding, the example of passing a value from the EE 
and treating it as a constant in the VU is still analagous to 
calculating a value at compile time in D at higher precision than 
the instruction set the runtime code is compiled to work with.

/arch:sse2 is the default with MSVC x64 builds (Xbox One defaults 
to /arch:avx), and it sounds like the DMD has defaulted to sse2 
for a long time. The exception being the compile time behaviour. 
That compile time behaviour conforming to the the runtime 
behaviour is an option I want, with the default being whatever is 
decided in here. Executing code at compile time at a higher 
precision than what SSE dictates is effectively undesired 
behaviour for our use cases.

And in cases where we compile code for another architecture on 
x64 (let's say ARM code with NEON instructions, as it's the most 
common case thanks to iOS development) then it would be forced to 
fallback to the default. Fine for most use cases as well. It 
would be up to the user to compile their ARM code on an ARM 
processor to get the code execution match if they need it.



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 2:54 AM, Ethan Watson wrote:

 On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

 MSVC doesn't appear to have a switch that does what you ask for


 I'm still not entirely sure what the /fp switch does for x64 builds. The
 documentation is not clear in the slightest and I haven't been able to find any
 concrete information. As near as I can tell it has no effect as the original
 behaviour was tied to how it handles the x87 control words. But it might also
be
 possible that the SSE instructions emitted can differ depending on what
 operation you're trying to do. I have not dug deep to see exactly how the code
 gen differs. I can take a guess that /fp:precise was responsible for promoting
 my float to a double to call CRT functions, but I have not tested that so
that's
 purely theoretical at the moment.

 Of course, while this conversation has mostly been for compile time constant
 folding, the example of passing a value from the EE and treating it as a
 constant in the VU is still analagous to calculating a value at compile time in
 D at higher precision than the instruction set the runtime code is compiled to
 work with.

 /arch:sse2 is the default with MSVC x64 builds (Xbox One defaults to
/arch:avx),
 and it sounds like the DMD has defaulted to sse2 for a long time. The exception
 being the compile time behaviour. That compile time behaviour conforming to the
 the runtime behaviour is an option I want, with the default being whatever is
 decided in here. Executing code at compile time at a higher precision than what
 SSE dictates is effectively undesired behaviour for our use cases.

 And in cases where we compile code for another architecture on x64 (let's say
 ARM code with NEON instructions, as it's the most common case thanks to iOS
 development) then it would be forced to fallback to the default. Fine for most
 use cases as well. It would be up to the user to compile their ARM code on an
 ARM processor to get the code execution match if they need it.



Again, even if the precision matches, the rounding will NOT match, and you will 
get different results randomly dependent on the exact operand values.

If those differences matter, then you'll randomly be up all night debugging it. 
If you're willing to try the approach I mentioned, it'll cost you a bit more 
time up front, but may save a lot of agony later.



 May 18 2016



  parent reply Ethan Watson <gooberman gmail.com>  writes:


On Wednesday, 18 May 2016 at 11:17:14 UTC, Walter Bright wrote:

 Again, even if the precision matches, the rounding will NOT 
 match, and you will get different results randomly dependent on 
 the exact operand values.


We've already been burned by middlewares/APIS toggling MMX flags 
on and off and not cleaning up after themselves, and as such we 
strictly control those flags going in to and out of such areas. 
We even have a little class with implementations for x87 
(thoroughly deprecated) and SSE that is used in a RAII manner, 
copying the MMX flag on construction and restoring it on 
destruction.

I appreciate that it sounds like I'm starting to stretch to hold 
to my point, but I imagine we'd also be able to control such 
things with the compiler - or at least know what flags it uses so 
that we can ensure consistent behaviour between compilation and 
runtime.



 May 18 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 4:30 AM, Ethan Watson wrote:

 I appreciate that it sounds like I'm starting to stretch to hold to my point,
 but I imagine we'd also be able to control such things with the compiler - or
at
 least know what flags it uses so that we can ensure consistent behaviour
between
 compilation and runtime.


All compilers I know of use "round to nearest" for constant folding, and that
is 
not adjustable.



 May 19 2016



prev sibling  parent reply ixid <adamsibson hotmail.com>  writes:


On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

 On 5/18/2016 1:30 AM, Ethan Watson wrote:

 You're also asking for a mode where the compiler for one 
 machine is supposed
 to behave like hand-coded assembler for another machine with 
 a different
 instruction set.


 Actually, I'm asking for something exactly like the arch 
 option for MSVC/-mfmath
 option for GCC/etc, and have it respect that for CTFE.



 MSVC doesn't appear to have a switch that does what you ask for:

   https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx


Apologies if this has been addressed in the thread, it's a 
difficult structure to follow for technical discussion. You seem 
positive about software implementations of float. What are your 
thoughts on having the compile time implementation of a given 
type mirror the behaviour of the runtime version?

Fundamentally whatever rules are chosen it would seem better to 
have fewer rules for people  to remember.



 May 18 2016



  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 18 May 2016 at 21:28, ixid via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:

 On 5/18/2016 1:30 AM, Ethan Watson wrote:

 You're also asking for a mode where the compiler for one machine is
 supposed
 to behave like hand-coded assembler for another machine with a different
 instruction set.



 Actually, I'm asking for something exactly like the arch option for
 MSVC/-mfmath
 option for GCC/etc, and have it respect that for CTFE.




 MSVC doesn't appear to have a switch that does what you ask for:

   https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx



 Apologies if this has been addressed in the thread, it's a difficult
 structure to follow for technical discussion. You seem positive about
 software implementations of float. What are your thoughts on having the
 compile time implementation of a given type mirror the behaviour of the
 runtime version?

 Fundamentally whatever rules are chosen it would seem better to have fewer
 rules for people  to remember.


That's precisely the suggestion; that compile time execution of a
given type mirror the runtime, that is, matching precisions in this
case.
...within reason; as Walter has pointed out consistently, it's very
difficult to be PERFECT for all the reasons he's been repeating, but
there's still a massive difference between the runtime executing a
bunch of float code, and the compile time executing it all promoted to
80bits. Results will drift apart very quickly.



 May 18 2016



  parent reply ixid <adamsibson hotmail.com>  writes:


On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:

 That's precisely the suggestion; that compile time execution of 
 a
 given type mirror the runtime, that is, matching precisions in 
 this
 case.
 ...within reason; as Walter has pointed out consistently, it's 
 very
 difficult to be PERFECT for all the reasons he's been 
 repeating, but
 there's still a massive difference between the runtime 
 executing a
 bunch of float code, and the compile time executing it all 
 promoted to
 80bits. Results will drift apart very quickly.


What do you think can be productively done to improve the 
situation? I am beginning to think a Wiki-like structure would be 
better for these discussions where each major debater has their 
thoughts on a specific issue in a relevantly headed section so 
there is more clarity.



 May 18 2016



  parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 18 May 2016 at 21:53, ixid via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:

 That's precisely the suggestion; that compile time execution of a
 given type mirror the runtime, that is, matching precisions in this
 case.
 ...within reason; as Walter has pointed out consistently, it's very
 difficult to be PERFECT for all the reasons he's been repeating, but
 there's still a massive difference between the runtime executing a
 bunch of float code, and the compile time executing it all promoted to
 80bits. Results will drift apart very quickly.



 What do you think can be productively done to improve the situation? I am
 beginning to think a Wiki-like structure would be better for these
 discussions where each major debater has their thoughts on a specific issue
 in a relevantly headed section so there is more clarity.


I've said so many times; I think it would be more useful if CTFE
operated on the *specified type*. That is all. That should produce
results as close as reasonably possible to the runtime.
If I want CTFE to operate at real precision (as is often the case),
float functions just take an 'isFloatingPoint!T' and users can execute
them at whatever precision they like. Most float functions already
have this form.



 May 18 2016



prev sibling next sibling parent reply deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:

 Trying to make D behave exactly like various C++ compilers do, 
 with all their semi-documented behavior and semi-documented 
 switches that affect constant folding behavior, is a hopeless 
 task.

 I doubt various C++ compilers are this compatible, even if they 
 follow the same ABI.


They aren't. For instance, GCC use arbitrary precision FB, and 
LLVM uses 128 bits soft floats in their innards.



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 3:15 AM, deadalnix wrote:

 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:

 Trying to make D behave exactly like various C++ compilers do, with all their
 semi-documented behavior and semi-documented switches that affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if they follow the
 same ABI.


 They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128
 bits soft floats in their innards.


Looks like LLVM had the same idea as myself.

Anyhow, this pretty much destroys the idea that I have proposed some sort of 
cowboy FP that's going to wreck FP programs.

(What is arbitrary precision FB?)



 May 18 2016



 next sibling parent  Johannes Pfau <nospam example.com>  writes:


Am Wed, 18 May 2016 04:11:08 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 On 5/18/2016 3:15 AM, deadalnix wrote:

 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:  

 Trying to make D behave exactly like various C++ compilers do,
 with all their semi-documented behavior and semi-documented
 switches that affect constant folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if they
 follow the same ABI.
  


 They aren't. For instance, GCC use arbitrary precision FB, and LLVM
 uses 128 bits soft floats in their innards.  


 
 Looks like LLVM had the same idea as myself.
 
 Anyhow, this pretty much destroys the idea that I have proposed some
 sort of cowboy FP that's going to wreck FP programs.
 
 (What is arbitrary precision FB?)


A claim from GMP, a library used by GCC:


 GMP is a free library for arbitrary precision arithmetic, operating on
 signed integers, rational numbers, and floating-point numbers. There
 is
 no practical limit to the precision except the ones implied by the
 available memory in the machine GMP runs on.


It's difficult to find reliable information, but I think GCC always uses
the target precision for constant folding:
https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gccint/Floating-Point.html#Floating-Point


 Because different representation systems may offer different amounts
 of
 range and precision, all floating point constants must be represented
 in the target machine's format. Therefore, the cross compiler cannot
 safely use the host machine's floating point arithmetic; it must
 emulate the target's arithmetic. To ensure consistency, GCC always
 uses
 emulation to work with floating point values, even when the host and
 target floating point formats are identical. 





 May 18 2016



prev sibling  parent reply deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:

 On 5/18/2016 3:15 AM, deadalnix wrote:

 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:

 Trying to make D behave exactly like various C++ compilers 
 do, with all their
 semi-documented behavior and semi-documented switches that 
 affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if 
 they follow the
 same ABI.


 They aren't. For instance, GCC use arbitrary precision FB, and 
 LLVM uses 128
 bits soft floats in their innards.


 Looks like LLVM had the same idea as myself.

 Anyhow, this pretty much destroys the idea that I have proposed 
 some sort of cowboy FP that's going to wreck FP programs.

 (What is arbitrary precision FB?)


Typo: arbitrary precision FP. Meaning some soft float that grows 
as big as necessary to not lose precision à la BitInt but for 
floats.



 May 18 2016



 next sibling parent reply Johannes Pfau <nospam example.com>  writes:


Am Wed, 18 May 2016 11:48:49 +0000
schrieb deadalnix <deadalnix gmail.com>:


 On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:

 On 5/18/2016 3:15 AM, deadalnix wrote: =20

 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote: =20

 Trying to make D behave exactly like various C++ compilers=20
 do, with all their
 semi-documented behavior and semi-documented switches that=20
 affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if=20
 they follow the
 same ABI.
 =20


 They aren't. For instance, GCC use arbitrary precision FB, and=20
 LLVM uses 128
 bits soft floats in their innards. =20


 Looks like LLVM had the same idea as myself.

 Anyhow, this pretty much destroys the idea that I have proposed=20
 some sort of cowboy FP that's going to wreck FP programs.

 (What is arbitrary precision FB?) =20


=20
 Typo: arbitrary precision FP. Meaning some soft float that grows=20
 as big as necessary to not lose precision =C3=A0 la BitInt but for=20
 floats.
=20


Do you have a link explaining GCC actually uses such a soft float? For
example
https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20
still says "This file should be rewritten to use an arbitrary
precision..."



 May 18 2016



 next sibling parent  deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:

 Do you have a link explaining GCC actually uses such a soft 
 float? For example 
 https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says
"This file should be rewritten to use an arbitrary precision..."


Alright, it seems that GCC doesn't use arbitrary precision float 
everywhere :)



 May 18 2016



prev sibling  parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:

 Do you have a link explaining GCC actually uses such a soft 
 float?


I'm confused as to why the compiler would be using soft floats 
instead of hard floats.



 May 18 2016



  parent reply deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 18 May 2016 at 19:20:20 UTC, jmh530 wrote:

 On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:

 Do you have a link explaining GCC actually uses such a soft 
 float?


 I'm confused as to why the compiler would be using soft floats 
 instead of hard floats.


Cross compilation.



 May 18 2016



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 18 May 2016 at 19:30:12 UTC, deadalnix wrote:

 I'm confused as to why the compiler would be using soft floats 
 instead of hard floats.


 Cross compilation.


Ah, looking back on the discussion, I see the comments about 
cross compilation and soft floats. Making more sense now...

So if compiling on x86 for x86, you could just use hard floats, 
but if compiling on x86 for some other system, then use soft 
floats to mimic what the result would be as if you had compiled 
on that system. Correct?

But what if you are compiling for a system whose float behavior 
matches the system you're compiling on? So for instance, suppose 
you are only using 32bit floats and not allowing anything fancy 
like 80bit intermediate calculations. And you're compiling for a 
system that treats floats the same way. Then, you could 
theoretically use hard floats in the compiler and the results 
would be the same.



 May 18 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 4:48 AM, deadalnix wrote:

 Typo: arbitrary precision FP. Meaning some soft float that grows as big as
 necessary to not lose precision à la BitInt but for floats.


0.10 is not representable in a binary format regardless of precision.



 May 18 2016



  parent reply deadalnix <deadalnix gmail.com>  writes:


On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:

 On 5/18/2016 4:48 AM, deadalnix wrote:

 Typo: arbitrary precision FP. Meaning some soft float that 
 grows as big as
 necessary to not lose precision à la BitInt but for floats.


 0.10 is not representable in a binary format regardless of 
 precision.


You should ask the gcc guys how they do it, but you can surely 
represent this as a fraction, so I see no major blocker.



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 1:22 PM, deadalnix wrote:

 On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:

 On 5/18/2016 4:48 AM, deadalnix wrote:

 Typo: arbitrary precision FP. Meaning some soft float that grows as big as
 necessary to not lose precision à la BitInt but for floats.


 0.10 is not representable in a binary format regardless of precision.


 You should ask the gcc guys how they do it, but you can surely represent this
as
 a fraction,


Right.


 so I see no major blocker.


Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by 
definition, not representable as a ratio.



 May 18 2016



 next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net>  writes:


On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:

 Now try the square root of 2. Or pi, e, etc. The irrational 
 numbers are, by definition, not representable as a ratio.


Continued fraction? :-)



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:

 On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:

 Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by
 definition, not representable as a ratio.


 Continued fraction? :-)


Somehow I don't think gcc is using Mathematica for constant folding.



 May 18 2016



  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, May 18, 2016 at 04:28:13PM -0700, Walter Bright via Digitalmars-d wrote:

 On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:

 On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:

 Now try the square root of 2. Or pi, e, etc. The irrational
 numbers are, by definition, not representable as a ratio.


 
 Continued fraction? :-)


 
 Somehow I don't think gcc is using Mathematica for constant folding.


http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/drl_ieee01.pdf

Side-note: continued fractions of quadratic irrationals (a + b*sqrt(c))
are periodic, so it's possible to do exact arithmetic with them using
only finite storage.


T

-- 
All problems are easy in retrospect.



 May 18 2016



prev sibling  parent  "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Wed, May 18, 2016 at 04:09:28PM -0700, Walter Bright via Digitalmars-d wrote:
[...]

 Now try the square root of 2. Or pi, e, etc. The irrational numbers
 are, by definition, not representable as a ratio.


This is somewhat tangential, but in certain applications it is perfectly
possible to represent certain irrationals exactly. For example, if
you're working with quadratic fractions of the form a + b*sqrt(r) where
r is fixed, you can exactly represent all such numbers as a pair of
rational coefficients. These numbers are closed under addition,
subtraction, multiplication, and division, so you have a nice closed
system that can handle numbers that contain square roots.

It is possible to have multiple square roots (e.g., a + b*sqrt(r) +
c*sqrt(s) + d*sqrt(r*s)), but the tuple gets exponentially long with
each different root, so it quickly becomes unwieldy (not to mention
slow).

Similar techniques can be used for exactly representing roots of cubic
equations (including cube roots) -- a single cubic root can be
represented as a 3-tuple -- or higher. Again, these are closed under +,
*, -, /, so you can do quite a lot of interesting things with them
without sacrificing exact arithmetic. Though again, things quickly
become unwieldy.  Higher order roots are also possible, though of
questionable value since storage and performance costs quickly become
too high.

Transcendentals like pi or e are out of reach by this method, though.
You'd need a variable-length vector to represent numbers that contain pi
or e as a factor, because their powers never become integral, so you
potentially need to represent an arbitrary power of pi in order to
obtain closure under + and *.

Of course, with increasing sophistication the computation and storage
costs increase accordingly, but the point is that if your application is
known to only need a certain subset of irrationals, it may well be
possible to represent them exactly within reasonable costs. (There's
also RealLib, that can exactly represent *all* computable reals, but may
not be feasible depending on what your application does.)


T

-- 
Let's eat some disquits while we format the biskettes.



 May 18 2016



prev sibling  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 18 May 2016 at 18:21, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/18/2016 12:56 AM, Ethan Watson wrote:

 In any case, the problem Manu was having was with C++.


 VU code was all assembly, I don't believe there was a C/C++ compiler for
 it.



 The constant folding part was where, then?


The comparison was a 24bit fpu doing runtime work but where some
constant input data was calculated with a separate 32bit fpu. The
particulars were not ever intended to be relevant to the conversation,
except the fact that 2 differently precisioned float units were
producing output that then had to reconcile.

The analogy was to CTFE doing all its work at 80bits, and then the
processor doing work with the types explicitly stated by the
programmer; a runtime calculation compared against the same compile
time calculate is likely to be quite radically different. I don't care
about the precision, I just care that they're really different.
Ideally, CTFE would produce a result that is as similar to the runtime
result as reasonably possible, and I expect using the stated types to
do the calculations would get much much closer.
I don't know if a couple of least-significant bits of difference would
have caused problems for us, I suspect not, but I know that doing math
at radically different precisions (ie, 32bits vs 80bits) does lead to
radically different results, not just a couple of bits. That is my
concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit
fpu's.



 May 18 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 4:27 AM, Manu via Digitalmars-d wrote:

 The comparison was a 24bit fpu doing runtime work but where some
 constant input data was calculated with a separate 32bit fpu. The
 particulars were not ever intended to be relevant to the conversation,
 except the fact that 2 differently precisioned float units were
 producing output that then had to reconcile.

 The analogy was to CTFE doing all its work at 80bits, and then the
 processor doing work with the types explicitly stated by the
 programmer; a runtime calculation compared against the same compile
 time calculate is likely to be quite radically different. I don't care
 about the precision, I just care that they're really different.
 Ideally, CTFE would produce a result that is as similar to the runtime
 result as reasonably possible, and I expect using the stated types to
 do the calculations would get much much closer.
 I don't know if a couple of least-significant bits of difference would
 have caused problems for us, I suspect not, but I know that doing math
 at radically different precisions (ie, 32bits vs 80bits) does lead to
 radically different results, not just a couple of bits. That is my
 concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit
 fpu's.


"radically different results" means you were getting catastrophic loss of 
precision in the runtime results, which is just what doing the CTFE at a higher 
precision is attempting to avoid (and apparently successfully).

I get that the game did not care about the value produced, even if it was total 
garbage. I suspect that games are the only category of FP apps where there is
no 
concern for the values computed. I do not understand the tolerance for bad 
results in scientific, engineering, medical, or finance applications.

The only way to *reliably* get the same results at compile time as at runtime, 
regardless of what technology is put into the language/compiler, is to actually 
run the code on the target hardware, save the results, and insert those values 
into the code. It isn't that hard to do, and will save you from endless ghosts 
popping up that waste your debugging time.



 May 18 2016



  parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net>  writes:


On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:

 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.


I don't think anyone has suggested tolerance for bad results in 
any of those applications.

What _has_ been argued for is that in order to _prevent_ bad 
results it's necessary for the programmer to have control and 
clarity over the choice of precision as much as possible.

If I'm writing a numerical simulation or calculation using 
insufficient floating-point precision, I don't _want_ to be saved 
by under-the-hood precision increases -- I would like it to break 
because then I will be forced to improve either the 
floating-point precision or the choice of algorithm (or both).

To be clear: the fact that D makes it a priority to offer me the 
highest possible floating-point precision is awesome.  But 
precision is not the only factor in generating accurate 
scientific results.



 May 18 2016



  parent reply jmh530 <john.michael.hall gmail.com>  writes:


On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
Wakeling wrote:

 On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:

 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.


 I don't think anyone has suggested tolerance for bad results in 
 any of those applications.


I don't think its about tolerance for bad results, so much as the 
ability to make the trade-off between speed and precision when 
you need to.

Just thinking of finance: a market maker has to provide quotes on 
potentially thousands of instruments in real-time. This might 
involve some heavy calculations for options pricing. When dealing 
with real-time tick data (the highest frequency of financial 
data), sometimes you take shortcuts that you wouldn't be willing 
to do if you were working with lower frequency data. It's not 
that you don't care about precision. It's just that sometimes 
it's more important to be fast than accurate.

I'm not a market maker and don't work with high frequency data. I 
usually look at low enough frequency data so that I actually do 
generally care more about accurate results than speed. 
Nevertheless, sometimes with hefty simulations that take several 
hours or days to run, I might be willing to take some short cuts 
to get a general idea of the results. Then, when I implement the 
strategy, I might do something different.



 May 18 2016



  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Wednesday, 18 May 2016 at 22:16:44 UTC, jmh530 wrote:

 On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
 Wakeling wrote:

 On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:

 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.


 I don't think anyone has suggested tolerance for bad results 
 in any of those applications.


 I don't think its about tolerance for bad results, so much as 
 the ability to make the trade-off between speed and precision 
 when you need to.


It isn't only about speed. It is about correctness as well. 
Compilers should not change the outcome (including precision and 
rounding) unless the programmer has explicitly requested it, and 
if you do the effects should be well documented so that the 
effect is clear to the programmer. This is a well known and 
universally accepted principle in compiler design.

Take for instance the documentation page for a professional level 
compiler targeting embedded programming:

http://processors.wiki.ti.com/index.php/Floating_Point_Optimization

It gives the programmer explicit control over what kind of 
deviations the compiler can create.

If you look at Intel's compiler you'll see that they even turn 
off fused-multiply-add in strict mode because it skips a single 
rounding between the multiply and the add.

https://software.intel.com/sites/default/files/article/326703/fp-control-2012-08.pdf

Some languages allow constant folding of literals in 
_expressions_ to use infinite precision, but once you have bound 
it to a variable it should use the same precision, and you always 
have to use the same rounding mode. This means that if you should 
check the rounding mode before using precomputed values.

If you cannot emulate the computation then you can simply run the 
computations prior to entering main and put the "constant 
computations" in global variables.

That said, compilers may have some problematic settings enabled 
by default in order to look good in benchmarks/test suites.

In order to be IEEE compliant you cannot even optimize "0.0 - x" 
to "-x", you also cannot optimize "x - 0.0" to "x". Such issues 
make compiler vendors provide many different floating point 
options as command line flags.



 May 19 2016



prev sibling  parent reply Era Scarecrow <rtcvb32 yahoo.com>  writes:


On Wednesday, 18 May 2016 at 07:56:58 UTC, Ethan Watson wrote:

 Not in the standards, no. But older gaming hardware was never 
 known to be standards-conformant.

 As it turns out, the original hardware manuals can be found on 
 the internet.


  Hmmm... I can't help but look at this and think about x86 
instructions of old, and how they aren't utilized usually fully 
or properly in language specs to take advantage of them, like the 
jump carry, overflow bits/checks, or the nature of how the CPUs 
handle arithmetic. I'm referring more to multiplication where the 
result of 2 16-bit multiplies will result in a 32bit output 
(AX&DX, is the same for 32/64bit instructions), but most likely 
the upper 32bits are just outright ignored rather than making use 
of those possible features.

  Heh, I'd love to see more hardware level abstraction that's 
built into the language. Almost like:

  try {}    // Considers the result of 1 line of basic math to be 
caught by:
  carry     {} //only activates if carry is set
  overflow  {} //if overflowed during some math
  modulus(m){} //get the remainder as m after a division operation
  mult(dx)  {} //get upper 32/64/whatever after a multiply and set 
as dx

  Of course I'd understand if some hardware doesn't offer such 
support, so the else could be thrown in to allow a workaround 
code to detect such an event, or only allow it if it's a 
compliant architecture. Although workaround detection is always 
possible, just not as fast as hardware supplied.

  Although I'm not fully familiar with all the results a FPU 
result could give, or if such a system would be beneficial to the 
current discussion on floats. I would prefer not to inject fixed 
x86 instructions if I can avoid it.



 May 18 2016



  parent reply tsbockman <thomas.bockman gmail.com>  writes:


On Wednesday, 18 May 2016 at 08:38:07 UTC, Era Scarecrow wrote:

  try {}    // Considers the result of 1 line of basic math to 
 be caught by:
  carry     {} //only activates if carry is set
  overflow  {} //if overflowed during some math
  modulus(m){} //get the remainder as m after a division 
 operation
  mult(dx)  {} //get upper 32/64/whatever after a multiply and 
 set as dx

  Of course I'd understand if some hardware doesn't offer such 
 support, so the else could be thrown in to allow a workaround 
 code to detect such an event, or only allow it if it's a 
 compliant architecture. Although workaround detection is always 
 possible, just not as fast as hardware supplied.


https://code.dlang.org/packages/checkedint
https://dlang.org/phobos/core_checkedint.html



 May 18 2016



  parent reply Era Scarecrow <rtcvb32 yahoo.com>  writes:


On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:

 On Wednesday, 18 May 2016 at 08:38:07 UTC, Era Scarecrow wrote:

  try {}    // Considers the result of 1 line of basic math to 
 be caught by:
  carry     {} //only activates if carry is set
  overflow  {} //if overflowed during some math
  modulus(m){} //get the remainder as m after a division 
 operation
  mult(dx)  {} //get upper 32/64/whatever after a multiply and 
 set as dx

 Of course I'd understand if some hardware doesn't offer such 
 support, so the else could be thrown in to allow a workaround 
 code to detect such an event, or only allow it if it's a 
 compliant architecture. Although workaround detection is 
 always possible, just not as fast as hardware supplied.


 https://code.dlang.org/packages/checkedint
 https://dlang.org/phobos/core_checkedint.html


  Glancing at the checkedInt I really don't see it as being the 
same as what I'm talking about. Overflow/carry for add perhaps, 
but unless it breaks down to a single instruction for the 
compiler to determine if it needs to do something, I see it as a 
failure (at best, a workaround).

  That's just my thoughts. CheckedInt simply _doesn't_ cover what 
I was talking about. Obtaining the modulus for 0 
cost/instructions after doing a division which is in the 
hardware's opcode side effects (unless the compiler recognizes 
the pattern and offers it as an optimization), or having the full 
result of a multiply on hand (that exceeds it's built-in size, 
long.max*long.max = 128bit result, which the hardware hands to 
you if you check the register it stores the other half of the 
result in).

  Perhaps what I want is more limited to handling certain tasks 
(making software math libraries) but I'd still like/want to see 
access to the other effects of these opcodes.



 May 18 2016



  parent reply tsbockman <thomas.bockman gmail.com>  writes:


On Wednesday, 18 May 2016 at 11:46:37 UTC, Era Scarecrow wrote:

 On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:

 https://code.dlang.org/packages/checkedint
 https://dlang.org/phobos/core_checkedint.html


  Glancing at the checkedInt I really don't see it as being the 
 same as what I'm talking about. Overflow/carry for add perhaps, 
 but unless it breaks down to a single instruction for the 
 compiler to determine if it needs to do something, I see it as 
 a failure (at best, a workaround).

 That's just my thoughts. CheckedInt simply _doesn't_ cover what 
 I was talking about.


The functions in druntime's `core.checkedint` are intrinsics that 
map directly to the hardware overflow/carry instructions.

The DUB package I linked provides various wrapper functions and 
data structures to make it easier to use the `core.checkedint` 
intrinsics (among other things). The performance cost of the 
wrappers is low with proper inlining, which GDC and LDC are able 
to provide. (DMD is another story...)


 Obtaining the modulus for 0 cost/instructions after doing a 
 division which is in the hardware's opcode side effects (unless 
 the compiler recognizes the pattern and offers it as an 
 optimization), or having the full result of a multiply on hand 
 (that exceeds it's built-in size, long.max*long.max = 128bit 
 result, which the hardware hands to you if you check the 
 register it stores the other half of the result in).


I agree that intrinsics for this would be nice. I doubt that any 
current D platform is actually computing the full 128 bit result 
for every 64 bit multiply though - that would waste both power 
and performance, for most programs.



 May 18 2016



  parent reply Era Scarecrow <rtcvb32 yahoo.com>  writes:


On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:

 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste both 
 power and performance, for most programs.


  Except the 128 result is _already_ there for 0 cost (at least 
for x86 instructions that I'm aware). There's bound to be enough 
cases (say pseudo random number generation, encryption, or 
numerical processing above 64bits) I'd like access to it 
supported by the language and not having to inject instructions 
using the asm command.

  This is also the same with the division where the number could 
be a 128bit number divided by a 64bit divisor. We could get the 
main benefit of the 128bit cent with very few instructions if we 
simply guarantee one of the two arguments smaller than 65 bits 
(although the dividend and remainder both need to be 64 bits or 
smaller)



 May 18 2016



  parent reply tsbockman <thomas.bockman gmail.com>  writes:


On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:

 On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:

 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste 
 both power and performance, for most programs.


  Except the 128 result is _already_ there for 0 cost (at least 
 for x86 instructions that I'm aware).


Can you give me a source for this, or at least the name of the 
relevant op code? (I'm new to x86 assembly.)


 There's bound to be enough cases (say pseudo random number 
 generation, encryption, or numerical processing above 64bits) 
 I'd like access to it supported by the language and not having 
 to inject instructions using the asm command.


Of course it would be useful to have in the language; I wasn't 
disputing that. I'd like to have as much support for 128-bit 
integers in the language as possible. Among other things, this 
would greatly simplify getting 128-bit floating-point working.

I'm just surprised that the CPU would really calculate the upper 
64 bits of a multiply without being explicitly asked to.



 May 18 2016



  parent reply Era Scarecrow <rtcvb32 yahoo.com>  writes:


On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:

 On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:

 On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:

 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste 
 both power and performance, for most programs.


  Except the 128 result is _already_ there for 0 cost (at least 
 for x86 instructions that I'm aware).


 Can you give me a source for this, or at least the name of the 
 relevant op code? (I'm new to x86 assembly.)


http://www.mathemainzel.info/files/x86asmref.html#mul

http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html

  There's div, idiv, mul, and imul which follow this exact 
pattern. Although the instruction mentioned in the following 
pages is meant for 32bit or less, the pattern used is no 
different.

(mathemainzel.info)
Usage
     MUL     src
Modifies flags
     CF OF (AF,PF,SF,ZF undefined)

Unsigned multiply of the accumulator by the source. If "src" is a 
byte value, then AL is used as the other multiplicand and the 
result is placed in AX. If "src" is a word value, then AX is 
multiplied by "src" and DX:AX receives the result. If "src" is a 
double word value, then EAX is multiplied by "src" and EDX:EAX 
receives the result. The 386+ uses an early out algorithm which 
makes multiplying any size value in EAX as fast as in the 8 or 16 
bit registers.


(intel.com)
Downloading the 64 intel manual on opcodes says the same thing, 
only the registers become RDX:RAX with 64bit instructions.

Quadword RAX r/m64 RDX:RAX



 May 18 2016



  parent  tsbockman <thomas.bockman gmail.com>  writes:


On Wednesday, 18 May 2016 at 22:06:43 UTC, Era Scarecrow wrote:

 On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:

 Can you give me a source for this, or at least the name of the 
 relevant op code? (I'm new to x86 assembly.)


 http://www.mathemainzel.info/files/x86asmref.html#mul

 http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html


Thanks.


 [...]
 Downloading the 64 intel manual on opcodes says the same thing, 
 only the registers become RDX:RAX with 64bit instructions.

 Quadword RAX r/m64 RDX:RAX


I will look into adding intrinsics for full-width multiply and 
combined division-modulus.



 May 18 2016



prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 17.05.2016 23:07, Walter Bright wrote:

 On 5/17/2016 11:08 AM, Timon Gehr wrote:

 Right. Hence, the 80-bit CTFE results have to be converted to the final
 precision at some point in order to commence the runtime computation.
 This means
 that additional rounding happens, which was not present in the
 original program.
 The additional total roundoff error this introduces can exceed the
 roundoff
 error you would have suffered by using the lower precision in the
 first place,
 sometimes completely defeating precision-enhancing improvements to an
 algorithm.


 I'd like to see an example of double rounding "completely defeating" an
 algorithm,


I have given this example, and I have explained it.

However, let me provide one of the examples I have given before, in a 
more concrete fashion. Unfortunately, there is no way to use "standard" 
D to illustrate the problem, as there is no way to write an 
implementation that is guaranteed not to be broken, so let us assume 
hypothetically for now that we are using D', where all computations are 
performed at the specified precision.

I'm copying the code from:
https://en.wikipedia.org/wiki/Kahan_summation_algorithm


$ cat kahanDemo.d

module kahanDemo;

double sum(double[] arr){
     double s=0.0;
     foreach(x;arr) s+=x;
     return s;
}

double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;
         double t=sum+y;
         c = (t-sum)-y;
         sum=t;
     }
     return sum;
}

double kahanBroken(double[] arr){
     double sum = 0;
     double c= 0.0;
     foreach(x;arr){
         real y=x-c;
         real t=sum+y;
         c = (t-sum)-y;
         sum=t;
     }
     return sum;
}

void main(){
     double[] data=[1e16,1,-9e15];
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
}

In D, the compiler is in principle allowed to transform the non-broken 
version to the broken version. (And maybe, it will soon be allowed to 
transform the baseline version to the Kahan version. Who knows.)

Now let's see what DMD does:

$ dmd --version
DMD64 D Compiler v2.071.0
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright

$ dmd -m64 -run kahanDemo.d
1000000000000000.000000
1000000000000001.000000
1000000000000000.000000

Nice, this is what I expect.

$ dmd -m64 -O -run kahanDemo.d
1000000000000000.000000
1000000000000001.000000
1000000000000000.000000

Still great.

$ dmd -m32 -run kahanDemo.d
1000000000000000.000000
1000000000000001.000000
1000000000000000.000000

Liking this.

$ dmd -m32 -O -run kahanDemo.d
1000000000000000.000000
1000000000000000.000000
1000000000000000.000000

Screw you, DMD!

And suddenly, I need to compile and test my code with all combinations 
of compiler flags, and even then I am not sure the compiler is not 
intentionally screwing me over. How is this remotely acceptable?


 and why an unusual case of producing a slightly worse


It's not just slightly worse, it can cut the number of useful bits in 
half or more! It is not unusual, I have actually run into those problems 
in the past, and it can break an algorithm that is in Phobos today!


 answer trumps the usual case of producing better answers.
 ...


The 'usual case of producing better answers' /is not actually 
desirable/, because the compiler does not guarantee that it happens all 
the time! I don't want my code to rely on something to happen that might 
not always happen. I want to be sure that my code is correct. I cannot 
conveniently do so if you don't tell me in advance what it does, and/or 
if the behaviour has a lot of abstraction-breaking special cases.


 There are other reasons why I think that this kind of
 implementation-defined
 behaviour is a terribly bad idea, eg.:

 - it breaks common assumptions about code, especially how it behaves
 under
 seemingly innocuous refactorings, or with a different set of compiler
 flags.


 As pointed out, this already happens with just about every language. It
 happens with all C/C++ compilers I'm aware of.


I'm not claiming those languages don't have broken floating point 
semantics. I have sometimes been using inline assembler in C++ to get 
the results I want. It's painful and unnecessary.


 It happens as the default behavior of the x86.


I know. I don't care. It is a stupid idea. See above.


 And as pointed out, refactoring (x+y)+z to x+(y+z)
 often produces different results, and surprises a lot of people.
 ...


As far as I can tell, you are saying: "You think A is bad, but A is 
similar to B, and B is bad, but B is hard to fix, hence A is actually 
good." I disagree.


For floating point, '+' means: "add precisely, then round". It is indeed 
potentially surprising and not very helpful for generic code that 
floating point types and integral types use the same syntax for 
conceptually different things (i.e., the language commits operator 
overloading abuse), but we are stuck with that now. 
Implementation-defined behaviour can actually be eliminated in a 
backward-compatible way.

Refactorings as simple as moving some expression into its own function 
should be possible without surprisingly wreaking havoc. Implementations 
can (and do) choose strange and useless behaviours. The fact that types 
do not actually specify what kind of value will be used at runtime is a 
pointless waste of type safety.


 - it breaks reproducibility, which is sometimes more important that
 being close
 to the infinite precision result (which you cannot guarantee with any
 finite
 floating point type anyway).
   (E.g. in a game, it is enough if the result seems plausible, but it
 should be
 the same for everyone. For some scientific experiments, the ideal case
 is to
 have 100% reproducibility of the computation, even if it is horribly
 wrong, such
 that other scientists can easily uncover and diagnose the problem, for
 example.)


 Nobody is proposing a D feature that does not produce reproducible
 results


Do you disagree with the notion that implementation-defined behaviour is 
almost by definition detrimental to reproducibility? (I'm not trying to 
say that it makes reproducing results impossible. It is a continuum.)


 with the same program on the same inputs.


I'm talking about computations, not just programs, and I ideally want 
consistent behaviour across compilers/compiler versions/compiler flags 
(at least those flags not specifically designed to change language 
semantics in precisely that way). If you can additionally give me 
refactorability (i.e. a lack of unprincipled interplay between language 
features that should be orthogonal [1]), that would be really awesome.

The result should ideally not depend on e.g. whether a computation is 
run at compile-time or run-time, or whatever other irrelevant implicit 
detail that is changed during refactoring or when switching machines. 
(e.g. someone might be running a 32-bit system, and another person (or 
the same person) might be running a 64-bit system, and they get 
different floating-point results. It just adds a lot of friction and 
pointless work.)


 This complaint is a strawman, as I've pointed out multiple times.
 ...


A strawman is an argument one incorrectly claims that the other party 
has made.

Here, I think the underlying problem is that there is a 
misunderstanding. (I.e. you think I'm saying something I didn't intend 
to say, or vice-versa. [2])

This is often the case, hence I try to avoid calling out strawmen by 
name and instead try to put my argument differently. (E.g., when you 
seemingly started to claim that my argument was something like "using 
lower precision for the entire computation throughout should be expected 
to yield more accurate results" and then ridiculed it.)


 In fact, the results would be MORE portable than with C/C++,


I know that it is better than completely broken. I'm sorry, but I have 
higher standards. It should be not broken.


 because the
 FP behavior is completely implementation defined, and compilers take
 advantage of that.


I guess the reason why it is completely implementation defined is the 
desire that there should be completely standard-compliant C/C++ 
compilers for each platform, and the fact that implementers didn't want 
to support IEEE 754 everywhere. One way to implement a specification is 
to weaken the specification.



[1] This is the single most important thing programming languages 
developed in academia often get right.

[2] To clarify: I know that if I compile a program on systems with 
identical states, on an identical compiler with identical flags, I will 
(or should) get identical binaries that then produce identical results 
on conforming hardware. (See how I didn't need to say "identical 
hardware"? I want to be there!) It is not always desirable or practical 
to keep around all that additional state though.



 May 18 2016



 next sibling parent reply Joakim <dlang joakim.fea.st>  writes:


On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of useful 
 bits in half or more! It is not unusual, I have actually run 
 into those problems in the past, and it can break an algorithm 
 that is in Phobos today!


I wouldn't call that broken.  Looking at the hex output by 
replacing %f with %A in writefln, it appears the only differences 
in all those results is the last byte in the significand.  As 
Don's talk pointed out, all floating-point calculations will see 
loss of precision starting there.

In this case, not increasing precision gets the more accurate 
result, but other examples could be constructed that _heavily_ 
favor increasing precision.  In fact, almost any real-world, 
non-toy calculation would favor it.

In any case, nobody should depend on the precision out that far 
being accurate or "reliable."



 May 18 2016



 next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:

 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that _heavily_ 
 favor increasing precision.  In fact, almost any real-world, 
 non-toy calculation would favor it.


Please stop saying this. It is very wrong.

Algorithms that need higher accuracy need error correction 
mechanisms, not unpredictable precision and rounding. 
Unpredictable precision and rounding makes adding error 
correction difficult so it does not improve accuracy, it harms 
accuracy when you need it.



 May 19 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:

 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that _heavily_ 
 favor increasing precision.  In fact, almost any real-world, 
 non-toy calculation would favor it.


 Please stop saying this. It is very wrong.


I will keep saying it because it is _not_ wrong.


 Algorithms that need higher accuracy need error correction 
 mechanisms, not unpredictable precision and rounding. 
 Unpredictable precision and rounding makes adding error 
 correction difficult so it does not improve accuracy, it harms 
 accuracy when you need it.


And that is what _you_ need to stop saying: there's _nothing 
unpredictable_ about what D does.  You may find it unintuitive, 
but that's your problem.  The notion that "error correction" can 
fix the inevitable degradation of accuracy with each 
floating-point calculation is just laughable.



 May 19 2016



  parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:

 On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
 wrote:

 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:

 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that 
 _heavily_ favor increasing precision.  In fact, almost any 
 real-world, non-toy calculation would favor it.


 Please stop saying this. It is very wrong.


 I will keep saying it because it is _not_ wrong.


Can you then please read this paper in it's entirety before 
continuing saying it. Because changing precision breaks 
properties of the semantics of IEEE floating point.

What Every Computer Scientist Should Know About Floating-Point 
Arithmetic

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377

«Conventional wisdom maintains that extended-based systems must 
produce results that are at least as accurate, if not more 
accurate than those delivered on single/double systems, since the 
former always provide at least as much precision and often more 
than the latter. Trivial examples such as the C program above as 
well as more subtle programs based on the examples discussed 
below show that this wisdom is naive at best: some apparently 
portable programs, which are indeed portable across single/double 
systems, deliver incorrect results on extended-based systems 
precisely because the compiler and hardware conspire to 
occasionally provide more precision than the program expects.»


 And that is what _you_ need to stop saying: there's _nothing 
 unpredictable_ about what D does.  You may find it unintuitive, 
 but that's your problem.


No. It is not my problem as I would never use a system with this 
kind of semantics for anything numerical.

It is a problem for D. Not for me.



 The notion that "error correction" can fix the inevitable 
 degradation of accuracy with each floating-point calculation is 
 just laughable.


Well, it is not laughable to computer scientists that accuracy 
depends on knowledge about precision and rounding... And I am a 
computer scientists, in case you have forgotten...



 May 19 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


On Thursday, 19 May 2016 at 11:00:31 UTC, Ola Fosheim Grøstad 
wrote:

 On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:

 On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
 wrote:

 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:

 In this case, not increasing precision gets the more 
 accurate result, but other examples could be constructed 
 that _heavily_ favor increasing precision.  In fact, almost 
 any real-world, non-toy calculation would favor it.


 Please stop saying this. It is very wrong.


 I will keep saying it because it is _not_ wrong.


 Can you then please read this paper in it's entirety before 
 continuing saying it. Because changing precision breaks 
 properties of the semantics of IEEE floating point.

 What Every Computer Scientist Should Know About Floating-Point 
 Arithmetic

 https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377

 «Conventional wisdom maintains that extended-based systems must 
 produce results that are at least as accurate, if not more 
 accurate than those delivered on single/double systems, since 
 the former always provide at least as much precision and often 
 more than the latter. Trivial examples such as the C program 
 above as well as more subtle programs based on the examples 
 discussed below show that this wisdom is naive at best: some 
 apparently portable programs, which are indeed portable across 
 single/double systems, deliver incorrect results on 
 extended-based systems precisely because the compiler and 
 hardware conspire to occasionally provide more precision than 
 the program expects.»


The example he refers to is laughable because it also checks for 
equality.


 The notion that "error correction" can fix the inevitable 
 degradation of accuracy with each floating-point calculation 
 is just laughable.


 Well, it is not laughable to computer scientists that accuracy 
 depends on knowledge about precision and rounding... And I am a 
 computer scientists, in case you have forgotten...


Computer scientists are no good if they don't know any science.



 May 19 2016



 next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net>  writes:


On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:

 The example he refers to is laughable because it also checks 
 for equality.


With good reason, because it's intended to illustrate the point 
that two calculations that _look_ identical in code, that 
intuitively should produce identical results, actually don't, 
because of under-the-hood precision differences.



 May 19 2016



  parent  Joakim <dlang joakim.fea.st>  writes:


On Thursday, 19 May 2016 at 12:00:33 UTC, Joseph Rushton Wakeling 
wrote:

 On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:

 The example he refers to is laughable because it also checks 
 for equality.


 With good reason, because it's intended to illustrate the point 
 that two calculations that _look_ identical in code, that 
 intuitively should produce identical results, actually don't, 
 because of under-the-hood precision differences.


And that's my point too, that floating point will always subvert 
your expectations, particularly such simplistic notions of 
equality, which is why you should always use approxEqual or it's 
analogues.



 May 19 2016



prev sibling  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:

 Computer scientists are no good if they don't know any science.


Even the computer scientists that does not know any science are 
infinitely better than those who refuse to read papers and debate 
on a rational level.

Blind D zealotry at its finest.



 May 19 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 19.05.2016 08:04, Joakim wrote:

 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!


 I wouldn't call that broken.  Looking at the hex output by replacing %f
 with %A in writefln, it appears the only differences in all those
 results is the last byte in the significand.


Argh...

// ...

void main(){
     //double[] data=[1e16,1,-9e15];
     import std.range;
     double[] data=1e16~repeat(1.0,100000000).array~(-9e15);
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
}


dmd -run kahanDemo.d
1000000000000000.000000
1000000100000000.000000
1000000000000000.000000

dmd -m32 -O -run kahanDemo.d
1000000000000000.000000
1000000000000000.000000
1000000000000000.000000


Better?

Obviously there is more structure in the data that I invent manually 
than in a real test case where it would go wrong. The problems carry 
over though.



 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting there.
 ...



This is implicitly assuming a development model where the programmer 
first writes down the computation as it would be correct in the real 
number system and then naively replaces every operation by the rounding 
equivalent and hopes for the best.

It is a useful rule if that is what you're doing. One might be doing 
something else. Consider the following paper for an example where the 
last bit in the significant actually carries useful information for many 
of the values used in the program.

http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


 In this case, not increasing precision gets the more accurate result,
 but other examples could be constructed that _heavily_ favor increasing
 precision.


Sure. In such cases, you should use higher precision. What is the 
problem? This is already supported (the compiler is not allowed to use 
lower precision than requested).


 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that far being
 accurate or "reliable."



IEEE floating point has well-defined behaviour and there is absolutely 
nothing wrong with code that delivers more accurate results just because 
it is actually aware of the actual semantics of the operations being 
carried out.



 May 19 2016



 next sibling parent reply Joakim <dlang joakim.fea.st>  writes:


On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 On 19.05.2016 08:04, Joakim wrote:

 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of useful 
 bits in
 half or more! It is not unusual, I have actually run into 
 those
 problems in the past, and it can break an algorithm that is 
 in Phobos
 today!


 I wouldn't call that broken.  Looking at the hex output by 
 replacing %f
 with %A in writefln, it appears the only differences in all 
 those
 results is the last byte in the significand.


 Argh...

 // ...

 void main(){
     //double[] data=[1e16,1,-9e15];
     import std.range;
     double[] data=1e16~repeat(1.0,100000000).array~(-9e15);
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
 }


 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000


 Better?

 Obviously there is more structure in the data that I invent 
 manually than in a real test case where it would go wrong. The 
 problems carry over though.


I looked over your code a bit.  If I define sum and c as reals in 
"kahanBroken" at runtime, this problem goes away.  Since that's 
what the CTFE rule is actually doing, ie extending all 
floating-point to reals at compile-time, I don't see what you're 
complaining about.  Try it, run even your original naive 
summation algorithm through CTFE and it will produce the result 
you want:

enum double[] ctData=[1e16,1,-9e15];
enum ctSum = sum(ctData);
writefln("%f", ctSum);


 As Don's talk pointed out,
 all floating-point calculations will see loss of precision 
 starting there.
 ...



 This is implicitly assuming a development model where the 
 programmer first writes down the computation as it would be 
 correct in the real number system and then naively replaces 
 every operation by the rounding equivalent and hopes for the 
 best.


No, it is intrinsic to any floating-point calculation.


 It is a useful rule if that is what you're doing. One might be 
 doing something else. Consider the following paper for an 
 example where the last bit in the significant actually carries 
 useful information for many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


Did you link to the wrong paper? ;) I skimmed it and that paper 
explicitly talks about error bounds all over the place.  The only 
mention of "the last bit" is when they say they calculated their 
constants in arbitrary precision before rounding them for runtime 
use, which is ironically similar to what Walter suggested doing 
for D's CTFE also.


 In this case, not increasing precision gets the more accurate 
 result,
 but other examples could be constructed that _heavily_ favor 
 increasing
 precision.


 Sure. In such cases, you should use higher precision. What is 
 the problem? This is already supported (the compiler is not 
 allowed to use lower precision than requested).


I'm not the one with the problem, you're the one complaining.


 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that 
 far being
 accurate or "reliable."



 IEEE floating point has well-defined behaviour and there is 
 absolutely nothing wrong with code that delivers more accurate 
 results just because it is actually aware of the actual 
 semantics of the operations being carried out.


You just made the case for Walter doing what he did. :)



 May 20 2016



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 20.05.2016 11:14, Joakim wrote:

 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 On 19.05.2016 08:04, Joakim wrote:

 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!


 I wouldn't call that broken.  Looking at the hex output by replacing %f
 with %A in writefln, it appears the only differences in all those
 results is the last byte in the significand.


 Argh...

 // ...

 void main(){
     //double[] data=[1e16,1,-9e15];
     import std.range;
     double[] data=1e16~repeat(1.0,100000000).array~(-9e15);
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
 }


 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000


 Better?

 Obviously there is more structure in the data that I invent manually
 than in a real test case where it would go wrong. The problems carry
 over though.


 I looked over your code a bit.  If I define sum and c as reals in
 "kahanBroken" at runtime, this problem goes away.


Yes. That's absolutely obvious, and I have noted it before, but thanks. 
Maybe try to understand why this problem occurs in the first place.


 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point to reals at
 compile-time, I don't see what you're complaining about.  Try it, run
 even your original naive summation algorithm through CTFE and it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...


This example wasn't specifically about CTFE, but just imagine that only 
part of the computation is done at CTFE, all local variables are 
transferred to runtime and the computation is completed there.



 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting
 there.
 ...



 This is implicitly assuming a development model where the programmer
 first writes down the computation as it would be correct in the real
 number system and then naively replaces every operation by the
 rounding equivalent and hopes for the best.


 No, it is intrinsic to any floating-point calculation.
 ...


How do you even define accuracy if you don't specify an infinitely 
precise reference result?


 It is a useful rule if that is what you're doing. One might be doing
 something else. Consider the following paper for an example where the
 last bit in the significant actually carries useful information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


 Did you link to the wrong paper? ;)


No. That paper uses multiple doubles per approximated real value to 
implement arithmetic that is more precise than using just plain doubles. 
If any bit in the first double is off, this is no better than using a 
single double.


 I skimmed it and that paper
 explicitly talks about error bounds all over the place.


It is enough to read the abstract to figure out what the problem is. 
This demonstrates a non-contrived case where CTFE using enhanced 
precision throughout can break your program. Compute something as a 
double-double at compile-time, and when it is transferred to runtime you 
lose all the nice extra precision, because bits in the middle of the 
(conceptual) mantissa are lost.



 The only mention of "the last bit" is


This part is actually funny. Thanks for the laugh. :-)
I was going to say that your text search was too naive, but then I 
double-checked your claim and there are actually two mentions of "the 
last bit", and close by to the other mention, the paper says that "the 
first double a_0 is a double-precision approximation to the number a, 
accurate to almost half an ulp."


 when they say they calculated their
 constants in arbitrary precision before rounding them for runtime use,
 which is ironically similar to what Walter suggested doing for D's CTFE
 also.
 ...


Nothing "ironic" about that. It is sometimes a good idea and I can do 
this explicitly and make sure the rounding is done correctly, just like 
they did. Also, it is a lot more flexible if I can specify the exact way 
the computation is done and the result is rounded. 80 bits might not be 
enough anyway. There is no reason for the language to apply potentially 
incorrect "hand holding" here.

Again, please understand that my point is not that lower precision is 
better. My point is that doing the same thing in every context and 
allowing the programmer to specify what happens is better.


 In this case, not increasing precision gets the more accurate result,
 but other examples could be constructed that _heavily_ favor increasing
 precision.


 Sure. In such cases, you should use higher precision. What is the
 problem? This is already supported (the compiler is not allowed to use
 lower precision than requested).


 I'm not the one with the problem, you're the one complaining.
 ...


So you see no problem with my requested semantics for the built-in 
floating point types?


 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that far being
 accurate or "reliable."



 IEEE floating point has well-defined behaviour and there is absolutely
 nothing wrong with code that delivers more accurate results just
 because it is actually aware of the actual semantics of the operations
 being carried out.


 You just made the case for Walter doing what he did. :)


No.



 May 20 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:

 On 20.05.2016 11:14, Joakim wrote:

 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 On 19.05.2016 08:04, Joakim wrote:

 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of 
 useful bits in
 half or more! It is not unusual, I have actually run into 
 those
 problems in the past, and it can break an algorithm that is 
 in Phobos
 today!


 I wouldn't call that broken.  Looking at the hex output by 
 replacing %f
 with %A in writefln, it appears the only differences in all 
 those
 results is the last byte in the significand.


 Argh...

 // ...

 void main(){
     //double[] data=[1e16,1,-9e15];
     import std.range;
     double[] data=1e16~repeat(1.0,100000000).array~(-9e15);
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
 }


 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000


 Better?

 Obviously there is more structure in the data that I invent 
 manually
 than in a real test case where it would go wrong. The 
 problems carry
 over though.


 I looked over your code a bit.  If I define sum and c as reals 
 in
 "kahanBroken" at runtime, this problem goes away.


 Yes. That's absolutely obvious, and I have noted it before, but 
 thanks. Maybe try to understand why this problem occurs in the 
 first place.


Yet you're the one arguing against increasing precision 
everywhere in CTFE.


 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point 
 to reals at
 compile-time, I don't see what you're complaining about.  Try 
 it, run
 even your original naive summation algorithm through CTFE and 
 it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...


 This example wasn't specifically about CTFE, but just imagine 
 that only part of the computation is done at CTFE, all local 
 variables are transferred to runtime and the computation is 
 completed there.


Why would I imagine that?  And this whole discussion is about 
what happens if you change the precision of all variables to real 
when doing CTFE, so what's the point of giving an example that 
isn't "specifically" about that?

And if any part of it is done at runtime using the algorithms you 
gave, which you yourself admit works fine if you use the right 
higher-precision types, you don't seem to have a point at all.


 As Don's talk pointed out,
 all floating-point calculations will see loss of precision 
 starting
 there.
 ...



 This is implicitly assuming a development model where the 
 programmer
 first writes down the computation as it would be correct in 
 the real
 number system and then naively replaces every operation by the
 rounding equivalent and hopes for the best.


 No, it is intrinsic to any floating-point calculation.
 ...


 How do you even define accuracy if you don't specify an 
 infinitely precise reference result?


There is no such thing as an infinitely precise result.  All one 
can do is compute using even higher precision and compare it to 
lower precision.


 It is a useful rule if that is what you're doing. One might 
 be doing
 something else. Consider the following paper for an example 
 where the
 last bit in the significant actually carries useful 
 information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


 Did you link to the wrong paper? ;)


 No. That paper uses multiple doubles per approximated real 
 value to implement arithmetic that is more precise than using 
 just plain doubles. If any bit in the first double is off, this 
 is no better than using a single double.


 I skimmed it and that paper
 explicitly talks about error bounds all over the place.


 It is enough to read the abstract to figure out what the 
 problem is. This demonstrates a non-contrived case where CTFE 
 using enhanced precision throughout can break your program. 
 Compute something as a double-double at compile-time, and when 
 it is transferred to runtime you lose all the nice extra 
 precision, because bits in the middle of the (conceptual) 
 mantissa are lost.


That is a very specific case where they're implementing 
higher-precision algorithms using lower-precision registers.  If 
you're going to all that trouble, you should know not to blindly 
run the same code at compile-time.


 The only mention of "the last bit" is


 This part is actually funny. Thanks for the laugh. :-)
 I was going to say that your text search was too naive, but 
 then I double-checked your claim and there are actually two 
 mentions of "the last bit", and close by to the other mention, 
 the paper says that "the first double a_0 is a double-precision 
 approximation to the number a, accurate to almost half an ulp."


Is there a point to this paragraph?


 when they say they calculated their
 constants in arbitrary precision before rounding them for 
 runtime use,
 which is ironically similar to what Walter suggested doing for 
 D's CTFE
 also.
 ...


 Nothing "ironic" about that. It is sometimes a good idea and I 
 can do this explicitly and make sure the rounding is done 
 correctly, just like they did. Also, it is a lot more flexible 
 if I can specify the exact way the computation is done and the 
 result is rounded. 80 bits might not be enough anyway. There is 
 no reason for the language to apply potentially incorrect "hand 
 holding" here.

 Again, please understand that my point is not that lower 
 precision is better. My point is that doing the same thing in 
 every context and allowing the programmer to specify what 
 happens is better.


I understand your point that sometimes the programmer wants more 
control.  But as long as the way CTFE extending precision is 
consistently done and clearly communicated, those people can 
always opt out and do it some other way.


 In this case, not increasing precision gets the more 
 accurate result,
 but other examples could be constructed that _heavily_ favor 
 increasing
 precision.


 Sure. In such cases, you should use higher precision. What is 
 the
 problem? This is already supported (the compiler is not 
 allowed to used
 lower precision than requested).


 I'm not the one with the problem, you're the one complaining.
 ...


 So you see no problem with my requested semantics for the 
 built-in floating point types?


I think it's an extreme minority use case that may not merit the 
work, though I don't know how much work it would require.  
However, I also feel that way about Walter's suggested move to 
128-bit CTFE, as dmd is x86-only anyway.



 May 20 2016



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 20.05.2016 13:32, Joakim wrote:

 On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:

 On 20.05.2016 11:14, Joakim wrote:

 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 On 19.05.2016 08:04, Joakim wrote:

 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:

 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!


 I wouldn't call that broken.  Looking at the hex output by
 replacing %f
 with %A in writefln, it appears the only differences in all those
 results is the last byte in the significand.


 Argh...

 // ...

 void main(){
     //double[] data=[1e16,1,-9e15];
     import std.range;
     double[] data=1e16~repeat(1.0,100000000).array~(-9e15);
     import std.stdio;
     writefln("%f",sum(data)); // baseline
     writefln("%f",kahan(data)); // kahan
     writefln("%f",kahanBroken(data)); // broken kahan
 }


 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000


 Better?

 Obviously there is more structure in the data that I invent manually
 than in a real test case where it would go wrong. The problems carry
 over though.


 I looked over your code a bit.  If I define sum and c as reals in
 "kahanBroken" at runtime, this problem goes away.


 Yes. That's absolutely obvious, and I have noted it before, but
 thanks. Maybe try to understand why this problem occurs in the first
 place.


 Yet you're the one arguing against increasing precision everywhere in CTFE.
 ...


High precision is usually good (use high precision, up to arbitrary 
precision or even symbolic arithmetic whenever it improves your results 
and you can afford it). *Implicitly* increasing precision during CTFE is 
bad. *Explicitly* using higher precision during CTFE than at running 
time /may or may not/ be good. In case it is good, there is often no 
reason to stop at 80 bits.


 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point to reals at
 compile-time, I don't see what you're complaining about.  Try it, run
 even your original naive summation algorithm through CTFE and it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...


 This example wasn't specifically about CTFE, but just imagine that
 only part of the computation is done at CTFE, all local variables are
 transferred to runtime and the computation is completed there.


 Why would I imagine that?


Because that's the most direct way to go from that example to one where 
implicit precision enhancement during CTFE only is bad.


 And this whole discussion is about what
 happens if you change the precision of all variables to real when doing
 CTFE, so what's the point of giving an example that isn't "specifically"
 about that?
 ...


Walter's request wasn't specifically about that, and it's the first 
thing I came up with:


 On 17.05.2016 23:07, Walter Bright wrote:

 I'd like to see an example of double rounding "completely defeating" an
 algorithm,




I don't have infinite amounts of time. The (significant) time spent 
writing up all of this competes with time spent doing things that are 
guaranteed to yield (often immediate) tangible benefits.



 And if any part of it is done at runtime using the algorithms you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,


What's "right" about them? That the compiler will not implicitly 
transform some of them to even higher precision in order to break the 
algorithm again? (I don't think that is even guaranteed.)


 you don't seem to have a point at all.
 ...


Be assured that I have a point. If you spend some time reading, or ask 
some constructive questions, I might be able to get it across to you. 
Otherwise, we might as well stop arguing.


 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting
 there.
 ...



 This is implicitly assuming a development model where the programmer
 first writes down the computation as it would be correct in the real
 number system and then naively replaces every operation by the
 rounding equivalent and hopes for the best.


 No, it is intrinsic to any floating-point calculation.
 ...


 How do you even define accuracy if you don't specify an infinitely
 precise reference result?


 There is no such thing as an infinitely precise result.  All one can do
 is compute using even higher precision and compare it to lower precision.
 ...


If I may ask, how much mathematics have you been exposed to?



 It is a useful rule if that is what you're doing. One might be doing
 something else. Consider the following paper for an example where the
 last bit in the significant actually carries useful information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf


 Did you link to the wrong paper? ;)


 No. That paper uses multiple doubles per approximated real value to
 implement arithmetic that is more precise than using just plain
 doubles. If any bit in the first double is off, this is no better than
 using a single double.


 I skimmed it and that paper
 explicitly talks about error bounds all over the place.


 It is enough to read the abstract to figure out what the problem is.
 This demonstrates a non-contrived case where CTFE using enhanced
 precision throughout can break your program. Compute something as a
 double-double at compile-time, and when it is transferred to runtime
 you lose all the nice extra precision, because bits in the middle of
 the (conceptual) mantissa are lost.


 That is a very specific case where they're implementing higher-precision
 algorithms using lower-precision registers.


If I argue in the abstract, people request examples. If I provide 
examples, people complain that they are too specific.


 If you're going to all that
 trouble, you should know not to blindly run the same code at compile-time.
 ...


The point of CTFE is to be able to run any code at compile-time that 
adheres to a well-defined set of restrictions. Not using floating point 
is not one of them.


 The only mention of "the last bit" is


 This part is actually funny. Thanks for the laugh. :-)
 I was going to say that your text search was too naive, but then I
 double-checked your claim and there are actually two mentions of "the
 last bit", and close by to the other mention, the paper says that "the
 first double a_0 is a double-precision approximation to the number a,
 accurate to almost half an ulp."


 Is there a point to this paragraph?


I don't think further explanations are required here. Maybe be more 
careful next time.


 when they say they calculated their
 constants in arbitrary precision before rounding them for runtime use,
 which is ironically similar to what Walter suggested doing for D's CTFE
 also.
 ...


 Nothing "ironic" about that. It is sometimes a good idea and I can do
 this explicitly and make sure the rounding is done correctly, just
 like they did. Also, it is a lot more flexible if I can specify the
 exact way the computation is done and the result is rounded. 80 bits
 might not be enough anyway. There is no reason for the language to
 apply potentially incorrect "hand holding" here.

 Again, please understand that my point is not that lower precision is
 better. My point is that doing the same thing in every context and
 allowing the programmer to specify what happens is better.


 I understand your point that sometimes the programmer wants more
 control.


Thanks!


 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,


It never will be clearly communicated to everyone and it will also hit 
people by accident who would have been aware of it.

What is so incredibly awesome about /implicit/ 80 bit precision as to 
justify the loss of control? If I want to use high precision for certain 
constants precomputed at compile time, I can do it just as well, 
possibly even at more than 80 bits such as to actually obtain accuracy 
up to the last bit.

Also, maybe I will need to move the computation to startup at runtime 
some time in the future because of some CTFE limitation, and then the 
additional implicit gain from 80 bit precision will be lost and cause a 
regression. The compiler just has no way to guess what precision is 
actually needed for each operation.


 those people can always opt out and do it some other way.
 ...


Sure, but now they need to _carefully_ maintain different 
implementations for CTFE and runtime, for an ugly misfeature. It's a 
silly magic trick that is not actually very useful and prone to errors.



 May 21 2016



  parent reply Joakim <dlang joakim.fea.st>  writes:


Sorry, I stopped reading this thread after my last response, as I 
felt I was wasting too much time on this discussion, so I didn't 
read your response till now.

On Saturday, 21 May 2016 at 14:38:20 UTC, Timon Gehr wrote:

 On 20.05.2016 13:32, Joakim wrote:

 Yet you're the one arguing against increasing precision 
 everywhere in CTFE.
 ...


 High precision is usually good (use high precision, up to 
 arbitrary precision or even symbolic arithmetic whenever it 
 improves your results and you can afford it). *Implicitly* 
 increasing precision during CTFE is bad. *Explicitly* using 
 higher precision during CTFE than at running time /may or may 
 not/ be good. In case it is good, there is often no reason to 
 stop at 80 bits.


It is not "implicitly increasing," Walter has said it will always 
be done for CTFE, ie it is explicit behavior for all compile-time 
calculation.  And he agrees with you about not stopping at 80 
bits, which is why he wanted to increase the precision of 
compile-time calculation even more.


 This example wasn't specifically about CTFE, but just imagine 
 that
 only part of the computation is done at CTFE, all local 
 variables are
 transferred to runtime and the computation is completed there.


 Why would I imagine that?


 Because that's the most direct way to go from that example to 
 one where implicit precision enhancement during CTFE only is 
 bad.


Obviously, but you still have not said why one would need to do 
that in some real situation, which is what I was asking for.


 And if any part of it is done at runtime using the algorithms 
 you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,


 What's "right" about them? That the compiler will not 
 implicitly transform some of them to even higher precision in 
 order to break the algorithm again? (I don't think that is even 
 guaranteed.)


What's right is that their precision is high enough to possibly 
give you the accuracy you want, and increasing their precision 
will only better that.


 you don't seem to have a point at all.
 ...


 Be assured that I have a point. If you spend some time reading, 
 or ask some constructive questions, I might be able to get it 
 across to you. Otherwise, we might as well stop arguing.


I think you don't really have a point, as your argumentation and 
examples are labored.


 No, it is intrinsic to any floating-point calculation.
 ...


 How do you even define accuracy if you don't specify an 
 infinitely
 precise reference result?


 There is no such thing as an infinitely precise result.  All 
 one can do
 is compute using even higher precision and compare it to lower 
 precision.
 ...


 If I may ask, how much mathematics have you been exposed to?


I suspect a lot more than you have.  Note that I'm talking about 
calculation and compute, which can only be done at finite 
precision.  One can manipulate symbolic math with all kinds of 
abstractions, but once you have to insert arbitrarily but 
finitely precise inputs and _compute_ outputs, you have to round 
somewhere for any non-trivial calculation.


 That is a very specific case where they're implementing 
 higher-precision
 algorithms using lower-precision registers.


 If I argue in the abstract, people request examples. If I 
 provide examples, people complain that they are too specific.


Yes, and?  The point of providing examples is to illustrate a 
general need with a specific case.  If your specific case is too 
niche, it is not a general need, ie the people you're complaining 
about can make both those statements and still make sense.


 If you're going to all that
 trouble, you should know not to blindly run the same code at 
 compile-time.
 ...


 The point of CTFE is to be able to run any code at compile-time 
 that adheres to a well-defined set of restrictions. Not using 
 floating point is not one of them.


My point is that potentially not being able to use CTFE for 
floating-point calculation that is highly specific to the 
hardware is a perfectly reasonable restriction.


 The only mention of "the last bit" is


 This part is actually funny. Thanks for the laugh. :-)
 I was going to say that your text search was too naive, but 
 then I
 double-checked your claim and there are actually two mentions 
 of "the
 last bit", and close by to the other mention, the paper says 
 that "the
 first double a_0 is a double-precision approximation to the 
 number a,
 accurate to almost half an ulp."


 Is there a point to this paragraph?


 I don't think further explanations are required here. Maybe be 
 more careful next time.


Not required because you have some unstated assumptions that we 
are supposed to read from your mind?  Specifically, you have not 
said why doing the calculation of that "double-precision 
approximation" at a higher precision and then rounding would 
necessarily throw their algorithms off.


 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,


 It never will be clearly communicated to everyone and it will 
 also hit people by accident who would have been aware of it.

 What is so incredibly awesome about /implicit/ 80 bit precision 
 as to justify the loss of control? If I want to use high 
 precision for certain constants precomputed at compile time, I 
 can do it just as well, possibly even at more than 80 bits such 
 as to actually obtain accuracy up to the last bit.


On the other hand, what is so bad about CTFE-calculated constants 
being computed at a higher precision and then rounded down?  
Almost any algorithm would benefit from that.


 Also, maybe I will need to move the computation to startup at 
 runtime some time in the future because of some CTFE 
 limitation, and then the additional implicit gain from 80 bit 
 precision will be lost and cause a regression. The compiler 
 just has no way to guess what precision is actually needed for 
 each operation.


Another scenario that I find far-fetched.


 those people can always opt out and do it some other way.
 ...


 Sure, but now they need to _carefully_ maintain different 
 implementations for CTFE and runtime, for an ugly misfeature. 
 It's a silly magic trick that is not actually very useful and 
 prone to errors.


I think the idea is to give compile-time calculations a boost in 
precision and accuracy, thus improving the constants computed at 
compile-time for almost every runtime algorithm.  There may be 
some algorithms that have problems with this, but I think Walter 
and I are saying they're so few not to worry about, ie the 
benefits greatly outweigh the costs.



 Aug 22 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 22.08.2016 20:26, Joakim wrote:

 Sorry, I stopped reading this thread after my last response, as I felt I
 was wasting too much time on this discussion, so I didn't read your
 response till now.
 ...


No problem. Would have been fine with me if it stayed that way.


 On Saturday, 21 May 2016 at 14:38:20 UTC, Timon Gehr wrote:

 On 20.05.2016 13:32, Joakim wrote:

 Yet you're the one arguing against increasing precision everywhere in
 CTFE.
 ...


 High precision is usually good (use high precision, up to arbitrary
 precision or even symbolic arithmetic whenever it improves your
 results and you can afford it). *Implicitly* increasing precision
 during CTFE is bad. *Explicitly* using higher precision during CTFE
 than at running time /may or may not/ be good. In case it is good,
 there is often no reason to stop at 80 bits.


 It is not "implicitly increasing,"


Yes it is. I don't state anywhere that I want the precision to increase. 
The default assumption is that CTFE behaves as (close to as reasonably 
possible to) runtime execution.


 Walter has said it will always be
 done for CTFE, ie it is explicit behavior for all compile-time
 calculation.


Well, you can challenge the definition of words I am using if you want, 
but what's the point?


  And he agrees with you about not stopping at 80 bits,
 which is why he wanted to increase the precision of compile-time
 calculation even more.
 ...


I'd rather not think of someone reaching that conclusion as agreeing 
with me.


 This example wasn't specifically about CTFE, but just imagine that
 only part of the computation is done at CTFE, all local variables are
 transferred to runtime and the computation is completed there.


 Why would I imagine that?


 Because that's the most direct way to go from that example to one
 where implicit precision enhancement during CTFE only is bad.


 Obviously, but you still have not said why one would need to do that in
 some real situation, which is what I was asking for.
 ...


It seems you think your use cases are real, but mine are not, so there 
is no way to give you a "real" example. I can just hope that Murphy's 
law strikes and you eventually run into the problems yourself.



 And if any part of it is done at runtime using the algorithms you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,


 What's "right" about them? That the compiler will not implicitly
 transform some of them to even higher precision in order to break the
 algorithm again? (I don't think that is even guaranteed.)


 What's right is that their precision is high enough to possibly give you
 the accuracy you want, and increasing their precision will only better
 that.
 ...


I have explained why this is not true. (There is another explanation 
further below.)



 No, it is intrinsic to any floating-point calculation.
 ...


 How do you even define accuracy if you don't specify an infinitely
 precise reference result?


 There is no such thing as an infinitely precise result.  All one can do
 is compute using even higher precision and compare it to lower
 precision.
 ...


 If I may ask, how much mathematics have you been exposed to?


 I suspect a lot more than you have.


I would not expect anyone familiar with the real number system to make a 
remark like "there is no such thing as an infinitely precise result".


 Note that I'm talking about
 calculation and compute, which can only be done at finite precision.


I wasn't, and it was my post pointing out the implicit assumption that 
floating point algorithms are thought of as operating on real numbers 
that started this subthread, if you remember. Then you said that my 
point was untrue without any argumentation, and I asked a very specific 
question in order to figure out how you reached your conclusion. Then 
you wrote a comment that didn't address my question at all and was 
obviously untrue from where I stood. Therefore I suspected that we might 
be using incompatible terminology, hence I asked how familiar you are 
with mathematical language, which you didn't answer either.


 One can manipulate symbolic math with all kinds of abstractions, but
 once you have to insert arbitrarily but finitely precise inputs and
 _compute_ outputs, you have to round somewhere for any non-trivial
 calculation.
 ...


You don't need to insert any concrete values to make relevant 
definitions and draw conclusions. My question was how you define 
accuracy, because this is crucial for understanding and/or refuting your 
point. It's a reasonable question that you ought to be able to answer if 
you use the term in an argument repeatedly.


 That is a very specific case where they're implementing higher-precision
 algorithms using lower-precision registers.


 If I argue in the abstract, people request examples. If I provide
 examples, people complain that they are too specific.


 Yes, and?  The point of providing examples is to illustrate a general
 need with a specific case.  If your specific case is too niche, it is
 not a general need, ie the people you're complaining about can make both
 those statements and still make sense.
 ...


I think the problem is that they don't see the general need from the 
example.


 If you're going to all that
 trouble, you should know not to blindly run the same code at
 compile-time.
 ...


 The point of CTFE is to be able to run any code at compile-time that
 adheres to a well-defined set of restrictions. Not using floating
 point is not one of them.


 My point is that potentially not being able to use CTFE for
 floating-point calculation that is highly specific to the hardware is a
 perfectly reasonable restriction.
 ...


I meant that the restriction is not enforced by the language definition. 
I.e. it is not a compile-time error to compute with built-in floating 
point types in CTFE.

Anyway, it is unfortunate but true that performance requirements might 
make it necessary to allow the results to be slightly hardware-specific; 
I agree that some compromises might be necessary. Arbitrarily using 
higher precision even in cases where the target hardware actually 
supports all features of IEEE floats and doubles does not seem like a 
good compromise though, it's completely unforced.


 The only mention of "the last bit" is


 This part is actually funny. Thanks for the laugh. :-)
 I was going to say that your text search was too naive, but then I
 double-checked your claim and there are actually two mentions of "the
 last bit", and close by to the other mention, the paper says that "the
 first double a_0 is a double-precision approximation to the number a,
 accurate to almost half an ulp."


 Is there a point to this paragraph?


 I don't think further explanations are required here. Maybe be more
 careful next time.


 Not required because you have some unstated assumptions that we are
 supposed to read from your mind?


Because anyone with a suitable pdf reader can verify that "the last bit" 
is mentioned twice inside that pdf document, and that the mention that 
you didn't see supports my point.


 Specifically, you have not said why
 doing the calculation of that "double-precision approximation" at a
 higher precision and then rounding would necessarily throw their
 algorithms off.
 ...


I did somewhere in this thread. (Using ASCII-art graphics even.)

Basically, the number is represented using two doubles with a 
non-overlapping mantissa. I'll try to explain using a decimal 
floating-point type to maybe illustrate it better. E.g. assume that the 
higher-precision type has a 4-digit mantissa, and the lower-precision 
type has a 3-digit mantissa:

The computation could have resulted in the double-double (1234e2, 56.78) 
representing the number 123456.78 (which is the exact sum of the two 
components.)

If we now round both components to lower precision independently, we are 
left with (123e3, 56.7) which represents the number 123056.7, which has 
only 3 accurate mantissa digits.

If OTOH, we had used the lower-precision type from the start, we would 
get a more accurate result, such as (123e3, 457) representing the number 
123457.

This might be slightly counter-intuitive, but it is not that uncommon 
for floating point-specific algorithms to actually rely on floating 
point specifics.

Here, the issue is that the compiler has no way to know the correct way 
to transform the set of higher-precision floating point numbers to a 
corresponding set of lower-precision floating point numbers; it does not 
know how the values are actually interpreted by the program.


 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,


 It never will be clearly communicated to everyone and it will also hit
 people by accident who would have been aware of it.

 What is so incredibly awesome about /implicit/ 80 bit precision as to
 justify the loss of control? If I want to use high precision for
 certain constants precomputed at compile time, I can do it just as
 well, possibly even at more than 80 bits such as to actually obtain
 accuracy up to the last bit.


 On the other hand, what is so bad about CTFE-calculated constants being
 computed at a higher precision and then rounded down?  Almost any
 algorithm would benefit from that.
 ...


Some will subtly break, for some it won't matter and the others will 
work for reasons mostly hidden to the programmer and might therefore 
break later. Sometimes the programmer is aware of the funny language 
semantics and exploiting it cleverly, using 'float' during CTFE 
deliberately for performing 80-bit computations, confusing readers about 
the actual precision being used.



 Also, maybe I will need to move the computation to startup at runtime
 some time in the future because of some CTFE limitation, and then the
 additional implicit gain from 80 bit precision will be lost and cause
 a regression. The compiler just has no way to guess what precision is
 actually needed for each operation.


 Another scenario that I find far-fetched.
 ...


Well, it's not. (The CTFE limitation could be e.g. performance.)

Basically, anytime that a programmer has wrong assumptions about why 
their code works correctly, this is slightly dangerous. It's not a good 
thing if the compiler tries to outsmart the programmer, because the 
compiler is not (supposed to be) smarter than the programmer.


 those people can always opt out and do it some other way.
 ...


 Sure, but now they need to _carefully_ maintain different
 implementations for CTFE and runtime, for an ugly misfeature. It's a
 silly magic trick that is not actually very useful and prone to errors.


 I think the idea is to give compile-time calculations a boost in
 precision and accuracy, thus improving the constants computed at
 compile-time for almost every runtime algorithm.  There may be some
 algorithms that have problems with this, but I think Walter and I are
 saying they're so few not to worry about, ie the benefits greatly
 outweigh the costs.


There are no benefits, because I can just explicitly compute at the 
precision I need myself, and I would prefer others to do the same, such 
that I have some clues about their reasoning when reading their code. 
Give me what I ask for. If you think I asked for the wrong thing, give 
me that wrong thing. If it is truly the wrong thing, I will see it and 
fix it.

If you still disagree, that's fine, just don't claim that I don't have a 
point, thanks.



 Aug 22 2016



prev sibling  parent reply Johan Engelen <j j.nl>  writes:


On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 
 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000

 Better?


Ignore if you think it's not relevant.
I am a bit lost in this thread. I thought this was about floating 
point CTFE, but I think what you are seeing here (in the 2nd 
case) is the DMD optimizer allowing reassociation of fp 
expressions? LDC does not allow that (per default) and produces 
the 1000000100000000.000000 result also with optimization on.

-Johan



 May 20 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 20.05.2016 14:34, Johan Engelen wrote:

 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:

 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000

 Better?


 Ignore if you think it's not relevant.
 I am a bit lost in this thread. I thought this was about floating point
 CTFE,


Yes, this is how it started out. :)
At some point, I made the comment that the compiler should never be 
allowed to use a higher precision than the one specified, which 
broadened the scope of the discussion somewhat. But the kinds of 
problems caused by using higher precision than requested can be similar 
for CTFE and runtime.


 but I think what you are seeing here (in the 2nd case) is the DMD
 optimizer allowing reassociation of fp expressions?


I don't believe it does (Walter knows /very/ well that a+(b+c) is not 
the same as (a+b)+c), but I have not looked at the assembly output. The 
result becomes wrong when 80 bit precision is used behind the scenes for 
some (but not all) of the computation, so this could be the explanation.


 LDC does not allow
 that (per default) and produces the 1000000100000000.000000 result also
 with optimization on.

 -Johan


Thanks!



 May 21 2016



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2016 10:10 AM, Timon Gehr wrote:

 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;


           double y = roundToDouble(x - c);

         double t=sum+y;


           double t = roundToDouble(sum + y);

         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }


Those are the only two points in the program that depend on rounding. If you're 
implementing Kahan, you'd know that, so there shouldn't be anything difficult 
about it.

There's no reason to make the rest of the program suffer inaccuracies.



 May 19 2016



 next sibling parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Thursday, 19 May 2016 at 07:09:30 UTC, Walter Bright wrote:

 On 5/18/2016 10:10 AM, Timon Gehr wrote:

 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;


           double y = roundToDouble(x - c);

         double t=sum+y;


           double t = roundToDouble(sum + y);

         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }


 Those are the only two points in the program that depend on 
 rounding. If you're implementing Kahan, you'd know that, so 
 there shouldn't be anything difficult about it.

 There's no reason to make the rest of the program suffer 
 inaccuracies.


People are trying to get across that, if they wanted to maximize 
accuracy, they would request the most precise type explicitly. D 
has 'real' for that. This thread has shown unequivocally that the 
semantics you are insisting on is bound to cause endless 
confusion and complaints.



 May 19 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2016 12:49 AM, Max Samukha wrote:

 People are trying to get across that, if they wanted to maximize accuracy, they
 would request the most precise type explicitly. D has 'real' for that. This
 thread has shown unequivocally that the semantics you are insisting on is bound
 to cause endless confusion and complaints.


C++ programs do the same thing, and have for decades, and such behavior is 
allowed by the C++ Standard.

People have been confused by and complained about floating point behavior at 
least since I started programming 40 years ago.

I have not invented anything new here.



 May 19 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 20.05.2016 08:25, Walter Bright wrote:

 On 5/19/2016 12:49 AM, Max Samukha wrote:

 People are trying to get across that, if they wanted to maximize
 accuracy, they
 would request the most precise type explicitly. D has 'real' for that.
 This
 thread has shown unequivocally that the semantics you are insisting on
 is bound
 to cause endless confusion and complaints.


 C++ programs do the same thing, and have for decades, and such behavior
 is allowed by the C++ Standard.
 ...


And reported as a gcc bug regularly, and there seems to be a flag to 
turn x87-induced strangeness off completely. Also, the C99 standard 
rules out compiler-specific results. (And allows the program to query 
whether it is being compiled with the set of flags it requires: 
https://en.wikipedia.org/wiki/C99#IEEE.C2.A0754_floating_point_support .)


 People have been confused by and complained about floating point
 behavior at least since I started programming 40 years ago.
 ...


That's mostly an incidental problem. (Sharing language syntax with 
non-rounding operations, needing to map to mis-designed hardware, etc.)


 I have not invented anything new here.


You may not have invented anything new, but the old invention is broken 
and in the (slow and painful) process of getting fixed. There is no 
reason to build new things that use the same principle, as there is 
already a better way available.



 May 21 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 19.05.2016 09:09, Walter Bright wrote:

 On 5/18/2016 10:10 AM, Timon Gehr wrote:

 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;


          double y = roundToDouble(x - c);


Those two lines producing different results is unexpected, because you 
are explicitly saying that y is a double, and the first line also does 
implicit rounding (probably -- on all compilers and targets that will be 
relevant in the near future -- to double). It's obviously bad language 
design on multiple levels.


         double t=sum+y;


          double t = roundToDouble(sum + y);

         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }


 Those are the only two points in the program that depend on rounding.


If you say so. I would like to see an example that demonstrates that the 
first roundToDouble is required.

Also, in case that the compiler chooses to internally use higher 
precision for all local variables in that program, the 'roundToDouble's 
you have inserted will reduce precision in comparison to the case where 
they are not inserted, reducing magical precision enhancement that would 
otherwise happen. (I.e., assuming that some of the ideas are valid and 
it is in fact desirable to have variable precision enhancement depending 
on target, if I find a precision bug I can fix by adding roundToDouble, 
this introduces a potential regression on other targets. And as none of 
it was guaranteed to work in the first place, the resulting mess is all 
my fault.)


 If you're implementing Kahan,


And if you are not? (I find the standard assumption that counterexamples 
to wrong statements are one-off special cases tiresome. This is not 
usually the case, even if you cannot construct other examples right away.)


 you'd know that,


I would, but what often happens in practice is that people don't. For 
example because they wouldn't expect roundToDouble to be required 
anywhere in code that only uses doubles. They are right.


 so there shouldn't be anything difficult about it.
 ...


https://issues.dlang.org/show_bug.cgi?id=13474

Creating useful programs in D shouldn't require knowing (or thinking 
about) an excess amount of D-specific implicit arcane details of the 
language semantics. Being easy to pick up and use is a major selling 
point for D.

I'm not convinced there is nothing difficult about it. It's certainly a 
generator of incidental complexity. Also, error prone is not the same as 
difficult. Almost everybody makes some mistakes occasionally.


 There's no reason to make the rest of the program suffer inaccuracies.


Indeed there isn't. I'm not suggesting to allow using a precision lower 
than the one specified.

If you are talking about programs that "benefit" from automatic 
extension of precision: do you think their authors are aware that e.g. 
their 32 bit release builds with gdc/ldc are producing wrong results?



 May 19 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2016 1:26 PM, Timon Gehr wrote:

 Those two lines producing different results is unexpected, because you are
 explicitly saying that y is a double, and the first line also does implicit
 rounding (probably -- on all compilers and targets that will be relevant in the
 near future -- to double).
 [...]
 It's obviously bad language design on multiple levels.


Floating point semantics simply are not that simple, on any compiler, language 
or hardware. I recommend reading what the C/C++ Standards say about it, and
look 
at the plethora of compiler switches for VC++/g++/clang++.

The people who design these things are not fools, and there are good reasons
for 
the way things are.

Sorry, I don't agree.


 If you say so. I would like to see an example that demonstrates that the first
 roundToDouble is required.


That's beside the point. If there are spots in the program that require 
rounding, what is wrong with having to specify it? Why compromise speed & 
accuracy in the rest of the code that does not require it? (And yes, rounding
in 
certain spots can and does compromise both speed and accuracy.)



 And if you are not? (I find the standard assumption that counterexamples to
 wrong statements are one-off special cases tiresome.


You might, but I've implemented a number of FP algorithms, and none of them 
required rounding to a lower precision. I've also seen many that failed due to 
premature rounding.



 Creating useful programs in D shouldn't require knowing (or thinking about) an
 excess amount of D-specific implicit arcane details of the language semantics.
 Being easy to pick up and use is a major selling point for D.


As pointed out innumerable times here, the same issues are in C and C++ (with 
even more variability!). When designing an algorithm that you know requires 
rounding in a certain place, you are already significantly beyond implementing
a 
naive algorithm, and it would be time to start paying attention to how FP 
actually works.

I'm curious if you know of any language that meets your requirements. (Java 1.0 
did, but Sun was forced to abandon that.)



 May 19 2016



 next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 20.05.2016 08:12, Walter Bright wrote:

 On 5/19/2016 1:26 PM, Timon Gehr wrote:

 Those two lines producing different results is unexpected, because you
 are
 explicitly saying that y is a double, and the first line also does
 implicit
 rounding (probably -- on all compilers and targets that will be
 relevant in the
 near future -- to double).


  > [...]

 It's obviously bad language design on multiple levels.


 Floating point semantics simply are not that simple,


This is about how the language defines floating point semantics. They 
can be that simple if the specification says so.


 on any compiler,


gdc/ldc seem to get it right. David, Iain, is this the case?


 language


Anybody can invent a language and IEEE 754 semantics are possible to 
support. Recent hardware does it.


 or hardware. I recommend reading what the C/C++ Standards say
 about it, and look at the plethora of compiler switches for
 VC++/g++/clang++.
 ...


I don't really need to look at examples of how to do it wrong, but I 
will read whatever relevant link you can give me carefully. In D you 
don't need compiler switches to change the semantics of your code. We 
have more or less sane conditional compilation.


 The people who design these things are not fools,


I don't know those people, but I would like to know their thoughts about 
the current discussion.


 and there are good reasons for the way things are.
 ...


It is not unusual that there is some good reason for a design mistake.


 Sorry, I don't agree.


 If you say so. I would like to see an example that demonstrates that
 the first
 roundToDouble is required.


 That's beside the point.


Absolutely not.


 If there are spots in the program that require
 rounding, what is wrong with having to specify it?


I explained what is wrong with it.

What is wrong with having to specify the precision that is /required for 
your code to run correctly/?


 Why compromise speed & accuracy in the rest of the code


You don't compromise speed or accuracy if the rest of the code just 
correctly specifies the precision it should be run at.


 that does not require it? (And yes,
 rounding in certain spots can and does compromise both speed and accuracy.)
 ...


Because it is the only sane thing to do, as explained from many 
different angles in this thread. If it does compromise accuracy, that 
means the code is bad (it relies on precision enhancements that are not 
guaranteed to happen, hence it is hardly correct).


 And if you are not? (I find the standard assumption that
 counterexamples to
 wrong statements are one-off special cases tiresome.


 You might, but I've implemented a number of FP algorithms, and none of
 them required rounding to a lower precision.


double x=..., z=...;
double y=roundToDouble(x+z);

Lower than what precision? It is not a 'lower' precision, it is the 
precision that was specified.


 I've also seen many that
 failed due to premature rounding.
 ...


Yes. Your results might be garbage more often if you run at too low 
precision. I and everyone else are very aware of this. I am not arguing 
against any of this. There seems to be a total communication failure here.


 Creating useful programs in D shouldn't require knowing (or thinking
 about) an
 excess amount of D-specific implicit arcane details of the language
 semantics.
 Being easy to pick up and use is a major selling point for D.


 As pointed out innumerable times here, the same issues are in C and C++
 (with even more variability!).


As also pointed out here, it does not matter. I'm *not* trying to make 
the point that C and C++ do it better or anything like that. (But well, 
in practice, the actual commonly used C and C++ implementations probably 
do get it right by default.)



 When designing an algorithm that you know
 requires rounding in a certain place, you are already significantly
 beyond implementing a naive algorithm, and it would be time to start
 paying attention to how FP actually works.

 I'm curious if you know of any language that meets your requirements.
 (Java 1.0 did, but Sun was forced to abandon that.)


x86_64 assembly language.



 May 20 2016



  parent  HorseWrangler <horse wrangler.org>  writes:


On Friday, 20 May 2016 at 11:22:48 UTC, Timon Gehr wrote:

 On 20.05.2016 08:12, Walter Bright wrote:

 I'm curious if you know of any language that meets your 
 requirements.
 (Java 1.0 did, but Sun was forced to abandon that.)


 x86_64 assembly language.


Similar discussion for Rust: 
https://internals.rust-lang.org/t/pre-rfc-dealing-with-broken-floating-point/2673/1



 May 20 2016



prev sibling next sibling parent reply Tobias =?UTF-8?B?TcO8bGxlcg==?= <troplin bluewin.ch>  writes:


On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:

 On 5/19/2016 1:26 PM, Timon Gehr wrote:

 Those two lines producing different results is unexpected, 
 because you are
 explicitly saying that y is a double, and the first line also 
 does implicit
 rounding (probably -- on all compilers and targets that will 
 be relevant in the
 near future -- to double).
 [...]
 It's obviously bad language design on multiple levels.


 Floating point semantics simply are not that simple, on any 
 compiler, language or hardware. I recommend reading what the 
 C/C++ Standards say about it, and look at the plethora of 
 compiler switches for VC++/g++/clang++.

 The people who design these things are not fools, and there are 
 good reasons for the way things are.

 Sorry, I don't agree.


Let me cite Prof. John L Gustafson from his Book "The End of 
Error" (it's worth reading):
---
The lesson taught by all these attempts at clever hidden 
scratchpad work is that computer users want *consistent* answers, 
and want to be *permitted the trade-of between speed and 
accuracy* depending on the situation. There is a subtle arrogance 
in any computer system that takes the position "You asked for a 
quick, cheap, approximate calculation, but I know what is good 
for you and will instead do a better job even if it takes more 
time and storage and energy and gives you a different answer. 
You're welcome."
---



 May 20 2016



  parent reply Tobias M <troplin bluewin.ch>  writes:


On Friday, 20 May 2016 at 12:32:40 UTC, Tobias Müller wrote:

 Let me cite Prof. John L Gustafson


Not "Prof." but "Dr.", sorry about that. Still an authority, 
though.



 May 20 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


If we're going to use the fallacy of appeal to authority, may I present Kahan 
who concurrently designed the IEEE 754 spec and the x87.



 May 20 2016



 next sibling parent reply Tobias =?UTF-8?B?TcO8bGxlcg==?= <troplin bluewin.ch>  writes:


On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority, may I 
 present Kahan who concurrently designed the IEEE 754 spec and 
 the x87.


Actually cited this *because* of you mentioning Kahan several 
times. And because you said "The people who design these things 
are not fools, and there are good reasons for the way things are."

Obviously, the designer of x87 and IEEE 754 (which both 
explicitly allow such "magic" behind the back of the programmer) 
thinks this is a good thing.

But after reading the book, I tend to agree with Gustafson. In 
the book this is elaborated more in detail, I just cited the 
conclusion.



 May 21 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/21/2016 2:26 AM, Tobias Müller wrote:

 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority, may I present Kahan
 who concurrently designed the IEEE 754 spec and the x87.


 Actually cited this *because* of you mentioning Kahan several times. And
because
 you said "The people who design these things are not fools, and there are good
 reasons for the way things are."


I meant two things by this:

1. Do the homework before disagreeing with someone who literally wrote the book 
and designed the hardware for it.

2. Complaining that the x87 is not IEEE compliant, when the guy that designed 
the x87 wrote the spec at the same time, suggests a misunderstanding the spec. 
I.e. again, gotta do the homework first.

Dismissing several decades of FP designs, and every programming language, as 
being "obviously wrong" and "insane" is an extraordinary claim, and so requires 
extraordinary evidence.

After all, what would your first thought be when a sophomore physics student 
tells you that Feynman got it all wrong? It's good to revisit existing dogma
now 
and then, and challenge the underlying assumptions of it, but again, you gotta 
understand the existing dogma all the way down first.

If you don't, you're very likely to miss something fundamental and produce a 
design that is less usable.



 May 21 2016



 next sibling parent reply Tobias M <troplin bluewin.ch>  writes:


On Saturday, 21 May 2016 at 17:58:49 UTC, Walter Bright wrote:

 On 5/21/2016 2:26 AM, Tobias Müller wrote:

 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority, may 
 I present Kahan
 who concurrently designed the IEEE 754 spec and the x87.


 Actually cited this *because* of you mentioning Kahan several 
 times. And because
 you said "The people who design these things are not fools, 
 and there are good
 reasons for the way things are."


 I meant two things by this:

 1. Do the homework before disagreeing with someone who 
 literally wrote the book and designed the hardware for it.

 2. Complaining that the x87 is not IEEE compliant, when the guy 
 that designed the x87 wrote the spec at the same time, suggests 
 a misunderstanding the spec. I.e. again, gotta do the homework 
 first.


Sorry but this is a misrepresentation. I never claimed that the 
x87 doesn't conform to the IEEE standard. That's completely 
missing the point. Again.


 Dismissing several decades of FP designs, and every programming 
 language, as being "obviously wrong" and "insane" is an 
 extraordinary claim, and so requires extraordinary evidence.

 After all, what would your first thought be when a sophomore 
 physics student tells you that Feynman got it all wrong? It's 
 good to revisit existing dogma now and then, and challenge the 
 underlying assumptions of it, but again, you gotta understand 
 the existing dogma all the way down first.

 If you don't, you're very likely to miss something fundamental 
 and produce a design that is less usable.


The point is, that is IS possible to provide fairly reasonable 
and consistent semantics within the existing standards (C, C++, 
IEEE, ...). They provide a certain degree of freedom to 
accomodate for different hardware, but this doesn't mean that 
software should use this freedom to do arbitrary things.

Regarding the decades of FP design, the initial edition of K&R C 
contained the following clause:
"Notice that all floats in an expression are converted to double; 
all floating point arithmethic in C is done in double precision".
That passus was removed quite quickly because users complained 
about it.



 May 21 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/21/2016 11:36 AM, Tobias M wrote:

 Sorry but this is a misrepresentation. I never claimed that the x87 doesn't
 conform to the IEEE standard.


My point was directed to more than just you. Sorry I didn't make that clear.



 The point is, that is IS possible to provide fairly reasonable and consistent
 semantics within the existing standards (C, C++, IEEE, ...).


That implies what I propose, which is what many C/C++ compilers do, is 
unreasonable, inconsistent, not Standard compliant, and not IEEE. I.e. that the 
x87 is not conformant :-)

Read the documentation on the FP switches for VC++, g++, clang, etc. You'll see 
there are tradeoffs. There is no "obvious, sane" way to do it.

There just isn't.



 They provide a
 certain degree of freedom to accomodate for different hardware, but this
doesn't
 mean that software should use this freedom to do arbitrary things.


Nobody is suggesting doing arbitrary things, but to write portable fp, take
into 
account what the Standard says rather than what your version of the compiler 
does with various default and semi-documented switches.



 Regarding the decades of FP design, the initial edition of K&R C contained the
 following clause:
 "Notice that all floats in an expression are converted to double; all floating
 point arithmethic in C is done in double precision".
 That passus was removed quite quickly because users complained about it.


It was changed to allow floats to be computed as floats, not require it. And
the 
reason at the time, as I recall, was to get faster floating point ops, not 
because anyone desired reduced precision.



 May 21 2016



  parent  Tobias M <troplin bluewin.ch>  writes:


On Saturday, 21 May 2016 at 21:56:02 UTC, Walter Bright wrote:

 On 5/21/2016 11:36 AM, Tobias M wrote:

 Sorry but this is a misrepresentation. I never claimed that 
 the x87 doesn't
 conform to the IEEE standard.


 My point was directed to more than just you. Sorry I didn't 
 make that clear.



 The point is, that is IS possible to provide fairly reasonable 
 and consistent
 semantics within the existing standards (C, C++, IEEE, ...).


 That implies what I propose, which is what many C/C++ compilers 
 do, is unreasonable, inconsistent, not Standard compliant, and 
 not IEEE. I.e. that the x87 is not conformant :-)


I'm trying to understand what you want to say here, but I just 
don't get it. Can you maybe formulate it differently?


 Read the documentation on the FP switches for VC++, g++, clang, 
 etc. You'll see there are tradeoffs. There is no "obvious, 
 sane" way to do it.

 There just isn't.


As I see it, the only real trade off is speed/optimization vs 
correctness.


 They provide a
 certain degree of freedom to accomodate for different 
 hardware, but this doesn't
 mean that software should use this freedom to do arbitrary 
 things.


 Nobody is suggesting doing arbitrary things, but to write 
 portable fp, take into account what the Standard says rather 
 than what your version of the compiler does with various 
 default and semi-documented switches.


https://gcc.gnu.org/wiki/FloatingPointMath

Seems relatively straight forward to me and well documented to 
me...
Dangerous optimizations like reordering expressions are all 
opt-in.

Sure it's probably not 100% consistent across 
implementations/platforms, but it's also *that* bad. And it's 
certainly not an excuse to make it even worse.

And yes, I think that in such an underspecified domain like FP, 
you cannot just rely on the standard but have to take the 
individual implementations into account.
Again, this is not ideal, but let's not make it even worse.


 Regarding the decades of FP design, the initial edition of K&R 
 C contained the
 following clause:
 "Notice that all floats in an expression are converted to 
 double; all floating
 point arithmethic in C is done in double precision".
 That passus was removed quite quickly because users complained 
 about it.


 It was changed to allow floats to be computed as floats, not 
 require it. And the reason at the time, as I recall, was to get 
 faster floating point ops, not because anyone desired reduced 
 precision.


I don't think that anyone has argued that lower precision is 
better. But a compiler should just do what it is told, not trying 
to be too clever.



 May 21 2016



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 21.05.2016 19:58, Walter Bright wrote:

 On 5/21/2016 2:26 AM, Tobias Müller wrote:

 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority, may I
 present Kahan
 who concurrently designed the IEEE 754 spec and the x87.


 Actually cited this *because* of you mentioning Kahan several times.
 And because
 you said "The people who design these things are not fools, and there
 are good
 reasons for the way things are."


 I meant two things by this:

 1. Do the homework before disagreeing with someone who literally wrote
 the book and designed the hardware for it.
 ...


Sure.


 2. Complaining that the x87 is not IEEE compliant, when the guy that
 designed the x87 wrote the spec at the same time, suggests a
 misunderstanding the spec.


Who claimed that the x87 is not IEEE compliant? Anyway, this is easy to 
resolve. IEEE 754-2008 requires FMA. x87 has no FMA.


Also, in practice, it is used to produce non-compliant results, as it 
has a default mode of being used that gives you results that differ from 
the specified results for single and double precision sources and 
destinations.

 From IEEE 754-2008:

"1.2 Purpose
1.2.0
This standard provides a method for computation with floating-point 
numbers that will yield the same result whether the processing is done 
in hardware, software, or a combination of the two."

I.e. the only stated purpose of IEEE 754 is actually reproducibility.



"shall indicates mandatory requirements strictly to be followed in order 
to conform to the standard and from which no deviation is permitted 
(“shall” means “is required to”"

"3.1.2 Conformance
3.1.2 .0
A  conforming  implementation  of  any  supported  format  shall 
provide means to initialize that format and shall provide conversions 
between that format and all other supported formats.

A conforming implementation of a supported arithmetic format shall 
provide all the operations of this standard defined in Clause 5, for 
that format.

A conforming implementation of a supported interchange format shall 
provide means to read and write that
format using a specific encoding defined in this clause, for that format.

A programming environment conforms to this standard, in a particular 
radix, by implementing one or more of the basic formats of that radix as 
both a supported arithmetic format and a supported interchange format."


For the original IEEE 754-1985, the x87 seems to support all of those 
clauses for float, double and extended (if you use it in the right way, 
which is inefficient, as you need to spill the result to memory after 
each operation, and it is not the default way), but it also supports 
further operations that fulfill similar functions in a non-conforming 
manner, and compiler implementations use it that way.

Another way to read it is that the x86 conforms by supporting float and 
double as interchange formats and extended as arithmetic format.

What is the correct way to interpret the x87 as conforming?



 I.e. again, gotta do the homework first.

 Dismissing several decades of FP designs, and every programming
 language, as being "obviously wrong" and "insane"


No need to unduly generalize the scope of my claims.


 is an extraordinary claim,


Not really. There don't seem to be so many popular systems in computing 
that are not broken in one way or another. (And even when they are, they 
are a lot more useful than nothing. I'd rather have the x87 than no 
hardware floating point support at all.)


 and so requires extraordinary evidence.
 ...


There are several decades of experience with the x87 to draw from. SSE 
does not suffer from those issues anymore. This is because flaws in the 
design were identified and fixed. Why is this evidence not extraordinary 
enough?


 After all, what would your first thought be when a sophomore physics
 student tells you that Feynman got it all wrong?


It's an oddly unspecific statement (what about his work is wrong?) and 
it's a completely different case. Feynman getting everything wrong is 
not a popular sentiment in the Physics community AFAICT.


 It's good to revisit
 existing dogma now and then, and challenge the underlying assumptions of
 it, but again, you gotta understand the existing dogma all the way down
 first.
 ...


What parts of "existing dogma" justify all the problems that x87 causes?
Also, if e.g.implicit 80 bit precision for CTFE floats truly is mandated 
by "floating point dogma" then "floating point dogma" clashes with 
"language design dogma".


 If you don't, you're very likely to miss something fundamental and
 produce a design that is less usable.


I haven't invented anything new either. SSE already fixes the issues.

(But yes, it is probably hard to support systems that only have the x87 
in a sane way. Tradeoffs might be necessary. Deliberately copying 
mistakes that x87 made to other contexts is not the right course of 
action though.)



 May 21 2016



prev sibling next sibling parent  Tobias M <troplin bluewin.ch>  writes:


On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority, may I 
 present Kahan who concurrently designed the IEEE 754 spec and 
 the x87.


Since I'm just in the mood of appealing to authorities, let me 
cite Wikipedia:

The argument from authority (Latin: argumentum ad verecundiam) 
also appeal to authority, is a common argument form which can be 
fallacious, *such as when an authority is cited on a topic 
outside their area of expertise, when the authority cited is not 
a true expert or if the cited authority is stating a contentious 
or controversial position.*

(Emphasis is mine)



 May 21 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 21.05.2016 00:22, Walter Bright wrote:

 On 5/20/2016 5:36 AM, Tobias M wrote:

 Still an authority, though.


 If we're going to use the fallacy of appeal to authority,


Authorities are not always wrong, the fallacy is to argue that they are 
right *because* they are authorities. However, in this case, I don't 
think you have provided a quote from Kahan that is applicable to the 
case we have here. The Gustafson quote is formulated generally enough to 
apply to the current case directly.


 may I present
 Kahan who concurrently designed the IEEE 754 spec and the x87.


The x87 is by far not the slam-dunk design you seem to make it out to 
be. (And you don't need to take my word for it. Otherwise, why would 
hardware designers prefer to phase it out in favour of systems that 
don't do the implicit extra scratch space thing?)

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

Bug report for this problem in gcc. The discussion raises an issue I 
wasn't aware of so far (or have forgotten about): Apparently the x87 
does not even round to double correctly, it only rounds the mantissa but 
does not deal with out-of-range-exponents correctly. What a mess.


http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html

x87 experience report.


http://www.vedetta.com/x87-fpu-php-bug-causes-infinite-loop-affected-websites-vulnerable-to-dos-via-php-get-function-2.2250738585072011e-308

DOS attack possible due to usage of x87 instructions.


http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it

Many complaints about x87, and some concerns about 80 bit floating point 
support in the future.


http://www.vinc17.org/research/extended.en.html

Good summary of some x87 problems. (Slightly broken English though.)


http://www.realworldtech.com/physx87/

x87 instructions used to make the GPU look faster compared to the CPU. 
(Not /directly/ a x87 design problem.)


http://www.cims.nyu.edu/~dbindel/class/cs279/87stack.pdf
http://www.cims.nyu.edu/~dbindel/class/cs279/stack87.pdf

Kahan criticizing the design of the register stack on the x87.


I haven't found any statement from Kahan on gcc bug 323, but this would 
be really interesting, if anyone can find anything.

I think there is ample evidence that x87 is ultimately a design failure, 
even though the intentions were good and based on real expertise.



 May 21 2016



  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 21.05.2016 15:45, Timon Gehr wrote:

 On 21.05.2016 00:22, Walter Bright wrote:

 ...
 may I present
 Kahan who concurrently designed the IEEE 754 spec and the x87.


 The x87 is by far not the slam-dunk design you seem to make it out to
 be. ...


https://hal.archives-ouvertes.fr/hal-00128124v5/document
Check out section 5 for some convincing examples showing why the x87 is 
horrible.



 May 21 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/21/2016 10:03 AM, Timon Gehr wrote:

 Check out section 5 for some convincing examples showing why the x87 is
horrible.


The polio vaccine winds up giving a handful of people polio, too.

It's good to list traps for the unwary in FP usage. It's disingenuous to list 
only problems with one design and pretend there are no traps in another design.



 May 21 2016



 next sibling parent  Fool <fool dlang.org>  writes:


Reasons have been alleged. What's your final decision?



 May 21 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 21.05.2016 20:14, Walter Bright wrote:

 On 5/21/2016 10:03 AM, Timon Gehr wrote:

 Check out section 5 for some convincing examples showing why the x87
 is horrible.


 The polio vaccine winds up giving a handful of people polio, too.
 ...


People don't get vaccinated without consent.


 It's good to list traps for the unwary in FP usage. It's disingenuous to
 list only problems with one design and pretend there are no traps in
 another design.


Some designs are much better than others.



 May 21 2016



  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Saturday, 21 May 2016 at 22:05:31 UTC, Timon Gehr wrote:

 On 21.05.2016 20:14, Walter Bright wrote:

 It's good to list traps for the unwary in FP usage. It's 
 disingenuous to
 list only problems with one design and pretend there are no 
 traps in
 another design.


 Some designs are much better than others.


Indeed. There are actually _only_ problems with D's take on 
floating point. It even prevents implementing higher precision 
double-double and quad-double math libraries by error-correction 
techniques where you get 106 bit/212 bit mantissas:

C++ w/2 x 64 bits adder and conservative settings
--> GOOD ACCURACY/ ~106 significant bits:

#include <iostream>
int main()
{
const double a = 1.23456;
const double b = 1.3e-18;
double hi = a+b;
double lo = -((a - ((a+b) - ((a+b) - a))) - (b + ((a+b) - a)));
std::cout << hi << std::endl; // 1.23456
std::cout << lo << std::endl; // 1.3e-18 SUCCESS!
std::cout << (hi-a) <<std::endl; // 0
}



D w/2 x 64/80 bits adder
--> BAD ACCURACY

import std.stdio;
void main()
{
const double a = 1.23456;
const double b = 1.3e-18;
double hi = a+b;
double lo = -((a - ((a+b) - ((a+b) - a))) - (b + ((a+b) - a)));
writeln(hi);  // 1.23456
writeln(lo); // 2.60104e-18 FAILURE!
writeln(hi-a); // 0
}


Add to this that compiler-backed emulation of 128 bit floats are 
twice as fast as 80-bit floats in hardware... that's how 
incredibly slow it 80 bit floats are on modern hardware.

I don't even understand why this is a topic as there is not one 
single rational to keep it the way it is. Not one.



 Jun 06 2016



prev sibling  parent  Max Samukha <maxsamukha gmail.com>  writes:


On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:


 If you say so. I would like to see an example that 
 demonstrates that the first
 roundToDouble is required.


 That's beside the point. If there are spots in the program that 
 require rounding, what is wrong with having to specify it?


Because the programmer has already specified it with a type that 
requires rounding!


 Why compromise speed & accuracy in the rest of the code that 
 does not require it? (And yes, rounding in certain spots can 
 and does compromise both speed and accuracy.)


Accuracy and speed won't be compromised if the programmer chooses 
an appropriate type - 'real' for the highest precision supported 
by the implementation.

BTW, you need to remove 'hardware' from the definition of 'real' 
in the spec.



 May 20 2016



prev sibling  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 18.05.2016 19:10, Timon Gehr wrote:

 implementation-defined behaviour


Maybe that wasn't the right term (it's worse than that; I think the 
documentation of the implementation is not even required to tell you 
precisely what it does).



 May 19 2016



prev sibling  parent  Fool <fool dlang.org>  writes:


On Tuesday, 17 May 2016 at 21:07:21 UTC, Walter Bright wrote:

 [...] why an unusual case of producing a slightly worse answer 
 trumps the usual case of producing better answers.


'Sometimes worse' is not 'better', but that's not the point, here.

Even if you managed to consistently produce not less accurate 
results: if the results computed at compile time differ from 
those computed at run time you cannot safely substitute run time 
calls with compile time calls without altering the result. Even 
slightest differences can lead to completely different outcomes.

For example, if we are interested in the number of real roots of 
a quadratic polynomial we compute its discriminant D. There are 
no solutions if D is negative, there is one solution if D is zero 
and there are two solutions if D is positive. In a floating point 
environment the crucial test is often implemented with a certain 
tolerance and good implementations compute the discriminant with 
increased precision. In any case, the critical point is that we 
have to make a decision based on a single computed value. A 
difference of one unit in the last place can change the result.

Note that this in not about correctness of the result but about 
reproducibility and substitutability across run time and compile 
time boundaries.

Please make a decision, document it, and move on.

Fool



 May 21 2016



prev sibling  parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:


 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended 
 precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not 
 reasonable
 that CTFE has higher precision applied than the same code at 
 runtime.
 CTFE must give the exact same result as runtime execution of 
 the function.


You are not even guaranteed to get the same result on two 
different x86 implementations. AMD64:

"The processor produces a floating-point result defined by the 
IEEE standard to be infinitely precise.
This result may not be representable exactly in the destination 
format, because only a subset of the
continuum of real numbers finds exact representation in any 
particular floating-point format."



 May 15 2016



 next sibling parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 15:49, Max Samukha via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:


 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.


 You are not even guaranteed to get the same result on two different x86
 implementations. AMD64:

 "The processor produces a floating-point result defined by the IEEE standard
 to be infinitely precise.
 This result may not be representable exactly in the destination format,
 because only a subset of the
 continuum of real numbers finds exact representation in any particular
 floating-point format."


I think that is to be interpreted that the result will be the closest
representation possible (rounding up), and that's not fuzzy.
I've never heard of an IEEE float unit that gives different results
than another one...?



 May 15 2016



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 16.05.2016 07:49, Max Samukha wrote:

 On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:


 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the
 function.


 You are not even guaranteed to get the same result on two different x86
 implementations.


Without reading the x86 specification, I think it is safe to claim that 
you actually are guaranteed to get the same result.


 AMD64:

 "The processor produces a floating-point result defined by the IEEE
 standard to be infinitely precise.
 This result may not be representable exactly in the destination format,
 because only a subset of the
 continuum of real numbers finds exact representation in any particular
 floating-point format."


This just says that results of computations will need to be rounded to 
fit into constant-size storage.



 May 16 2016



  parent reply Max Samukha <maxsamukha gmail.com>  writes:


On Monday, 16 May 2016 at 19:01:19 UTC, Timon Gehr wrote:


 You are not even guaranteed to get the same result on two 
 different x86
 implementations.


 Without reading the x86 specification, I think it is safe to 
 claim that you actually are guaranteed to get the same result.


Correct. Sorry for the noise.


 AMD64:

 "The processor produces a floating-point result defined by the 
 IEEE
 standard to be infinitely precise.
 This result may not be representable exactly in the 
 destination format,
 because only a subset of the
 continuum of real numbers finds exact representation in any 
 particular
 floating-point format."


 This just says that results of computations will need to be 
 rounded to fit into constant-size storage.


Right.



 May 17 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 17.05.2016 20:09, Max Samukha wrote:

 On Monday, 16 May 2016 at 19:01:19 UTC, Timon Gehr wrote:


 You are not even guaranteed to get the same result on two different x86
 implementations.


 Without reading the x86 specification, I think it is safe to claim
 that you actually are guaranteed to get the same result.


 Correct. Sorry for the noise.


Interestingly, I was actually wrong:
https://hal.archives-ouvertes.fr/hal-00128124v5/document

I think that document is worth a read in any case.

Page 18: "Note, also, that different processors within the same 
architecture can implement the same transcendental functions with 
different accuracies. We already noted the difference between the AMD-K5 
and the K6 and following processors with respect to angular reduction. 
Intel also notes that the algorithms’ precision was improved between the 
80387 / i486DX processors and the Pentium processors. [Int, 1997,§7.5.10]"

The language should ideally not leak such differences to the user.



 May 21 2016



prev sibling  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 06:02, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 I'm not interested in C/C++, I gave some anecdotes where it's gone
 wrong for me too, but regardless; generally, they do match, and I
 can't think of a single modern example where that's not true. If you
 *select* fast-math, then you may generate code that doesn't match, but
 that's a deliberate selection.

 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types) out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.


It isn't all bad.  Most the time you'll never notice. :-)

I can't think of a case of the top of my head where too much precision
caused a surprise.  It's always when there is too little:

https://issues.dlang.org/show_bug.cgi?id=16026

And I think the it's about that time of the year when I remind people
of gcc bug 323, and this lovely blog post.

http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html



 May 15 2016



 next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Monday, 16 May 2016 at 06:48:19 UTC, Iain Buclaw wrote:

 I can't think of a case of the top of my head where too much 
 precision caused a surprise.  It's always when there is too 
 little:


Wrong. You obviously don't do much system level programming using 
floats.



 May 16 2016



  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 09:06, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 16 May 2016 at 06:48:19 UTC, Iain Buclaw wrote:

 I can't think of a case of the top of my head where too much precision
 caused a surprise.  It's always when there is too little:



 Wrong. You obviously don't do much system level programming using floats.


Feel free to give me any bug report or example.  None that you've done
so far in this thread relate to CTFE.



 May 16 2016



  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Monday, 16 May 2016 at 07:16:41 UTC, Iain Buclaw wrote:

 Feel free to give me any bug report or example.  None that 
 you've done so far in this thread relate to CTFE.


Of course it is the same issue. Constant folding is 
"compile-time-function-evaluation", e.g. an expression is a 
function. Compiler internals does not change that.



 May 16 2016



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 11:48 PM, Iain Buclaw via Digitalmars-d wrote:

 I can't think of a case of the top of my head where too much precision
 caused a surprise.  It's always when there is too little:

 https://issues.dlang.org/show_bug.cgi?id=16026

 And I think the it's about that time of the year when I remind people
 of gcc bug 323, and this lovely blog post.

 http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html


Manu, read those!

More:

https://randomascii.wordpress.com/2012/03/21/intermediate-floating-point-precision/

Ironically, this Microsoft article argues for greater precision for
intermediate 
calculations, although Microsoft ditched 80 bits:

https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx



 May 16 2016



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 3:31 AM, Walter Bright wrote:

 Ironically, this Microsoft article argues for greater precision for
 intermediate calculations, although Microsoft ditched 80 bits:

 https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx


Do you have an explanation on why Microsoft ditched 80-bit floats? -- Andrei



 May 16 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 3:30 AM, Andrei Alexandrescu wrote:

 On 5/16/16 3:31 AM, Walter Bright wrote:

 Ironically, this Microsoft article argues for greater precision for
 intermediate calculations, although Microsoft ditched 80 bits:

 https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx


 Do you have an explanation on why Microsoft ditched 80-bit floats? -- Andrei


At the time they were porting NT to some other architecture which didn't have
80 
bits. I suspect they dumped it in order to enhance code compatibility between 
the x86 compiler and the other compiler. I think Microsoft tools had an
internal 
mandate to make the differences between the machines as small as possible.

The lack of 80 bit SIMD support likely gave it a good shove off the curb as 
well. Even dmd no longer generates x87 code for float/double for 64 bit
targets, 
meaning no more 80 bit temporaries.

It's kinda sad, really. The x87 was a triumph of engineering when it came out. 
The comments that "64 bits is all anyone will ever need" are not made by people 
who do numerical work, where one constantly battles catastrophic loss of
precision.

I've often wondered how NASA calculates trajectories out to Jupiter, because it 
has to be done iteratively, and that means cumulative loss of precision. I
wrote 
some orbit software for fun in college, and catastrophic loss of precision
would 
make the orbits go completely bonkers after just one orbit. I invented several 
schemes to fix it, but nothing worked.

When I was doing numerical analysis for my you-know-what job, I had to invert 
matrices all the time. I only knew the math algorithm, and that would produce 
nonsense on anything larger than 14*14 or so using doubles. Better algorithms 
exist that compensate for the errors, but this was pre-internet and I didn't 
know where to get such a solution.

Those two experiences shaped my attitudes about the value of precision, as well 
as the TTL boards I designed where the design had to still work if faster parts 
were swapped in.



 May 16 2016



prev sibling next sibling parent  Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 13 May 2016 at 07:12, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/12/2016 4:32 PM, Marco Leise wrote:

 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results



 I've actually been thinking of writing a 128 bit float emulator, and then
 using that in the compiler internals to do all FP computation with.


 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.


I have some bad news for you about CTFE then. This already happens in
DMD even though float is not emulated.  :-o



 May 13 2016



prev sibling  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 14 May 2016 at 00:00, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 13 May 2016 at 07:12, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/12/2016 4:32 PM, Marco Leise wrote:

 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results



 I've actually been thinking of writing a 128 bit float emulator, and then
 using that in the compiler internals to do all FP computation with.


 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.


 I have some bad news for you about CTFE then. This already happens in
 DMD even though float is not emulated.  :-o


O_O

Are you saying 'float' in CTFE is not 'float'? I protest this about as
strongly as I can muster...



 May 15 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:

 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...


I imagine you'll be devastated when you discover that the C++ Standard does not 
require 32 bit floats to have 32 bit precision either, and never did.

:-)



 May 15 2016



 next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:

 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...



 I imagine you'll be devastated when you discover that the C++ Standard does
 not require 32 bit floats to have 32 bit precision either, and never did.

 :-)


I've never read the C++ standard, but I have more experience with a
wide range of real-world compilers than most, and it is rarely very
violated. The times it is, we've known about it, and it has made us
all very, very angry.



 May 15 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:

 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.


It has on every C/C++ compiler for x86 machines that used the x87, which was 
true until SIMD, and is still true for x86 CPUs that don't target SIMD.


 The times it is, we've known about it, and it has made us
 all very, very angry.


The C/C++ standard is written that way for a reason.

I'd like to hear what terrible problem is caused by having more accurate values.



 May 15 2016



  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 14:31, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:

 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.



 It has on every C/C++ compiler for x86 machines that used the x87, which was
 true until SIMD, and is still true for x86 CPUs that don't target SIMD.


It has what? Reinterpreted your constant-folding to execute in 80bits
internally for years? Again, if that's true, I expect that's only true
in the context that the compiler also takes care to maintain the IEEE
conformant bit pattern, or at very least, it works because the
opportunity for FP constant folding in C++ is almost non-existent
compared to CTFE, such that it's never resulted in a problem case in
my experience.
In D, we will (do) use CTFE for table generation all the time (this
has never been done in C++). If those tables were generated with
entirely different precision than the runtime functions, that's just
begging for trouble.


 The times it is, we've known about it, and it has made us
 all very, very angry.



 The C/C++ standard is written that way for a reason.

 I'd like to hear what terrible problem is caused by having more accurate
 values.


In the majority of my anecdotes, if they don't match, there are
cracks/seams in the world. That is a show-stopping bug. We have had
many late nights, even product launch delays due to by these problems.
They have been a nightmare to solve in the past.
Obviously the solution in this case is a relatively simple
work-around; don't use CTFE (ie, lean on the LLVM runtime codegen
instead to do the right thing with the float precision), but that's a
tragic solution to a problem that should never happen in the first
place.



 May 15 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 10:24 PM, Manu via Digitalmars-d wrote:

 On 16 May 2016 at 14:31, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:

 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.



 It has on every C/C++ compiler for x86 machines that used the x87, which was
 true until SIMD, and is still true for x86 CPUs that don't target SIMD.


 It has what? Reinterpreted your constant-folding to execute in 80bits
 internally for years? Again, if that's true, I expect that's only true
 in the context that the compiler also takes care to maintain the IEEE
 conformant bit pattern, or at very least, it works because the
 opportunity for FP constant folding in C++ is almost non-existent
 compared to CTFE, such that it's never resulted in a problem case in
 my experience.


Check out my other message to you quoting the VC++ manual. It does constant 
folding at higher precision. So does gcc:

http://stackoverflow.com/questions/7295861/enabling-strict-floating-point-mode-in-gcc



 In D, we will (do) use CTFE for table generation all the time (this
 has never been done in C++). If those tables were generated with
 entirely different precision than the runtime functions, that's just
 begging for trouble.


You can generate the tables at runtime at program startup by using a 'static 
this()' constructor.



 In the majority of my anecdotes, if they don't match, there are
 cracks/seams in the world. That is a show-stopping bug. We have had
 many late nights, even product launch delays due to by these problems.
 They have been a nightmare to solve in the past.
 Obviously the solution in this case is a relatively simple
 work-around; don't use CTFE (ie, lean on the LLVM runtime codegen
 instead to do the right thing with the float precision), but that's a
 tragic solution to a problem that should never happen in the first
 place.


You're always going to have tragic problems if you expect FP to behave like 
conventional mathematics. Just look at all the switches VC++ has that influence 
FP behavior. Do you really understand all that?

Tell me you understand all the gcc floating point switches?

https://gcc.gnu.org/wiki/FloatingPointMath

How about g++ getting different results based on optimization switches?

http://stackoverflow.com/questions/7517588/different-floating-point-result-with-optimization-enabled-compiler-bug

-------

Or, you could design the code so that there aren't cracks because it is more 
tolerant of slight differences, i.e. look for so many mantissa bits matching 
instead of all of them matching. This can be done by subtracting the operands 
and looking at the magnitude of the difference.

The fact that you have had "many late nights" and were not using D means that 
the compiler switches you were using were not adequate or were not doing what 
you thought they were doing.



 May 15 2016



prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:

 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:

 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...



 I imagine you'll be devastated when you discover that the C++ Standard does
 not require 32 bit floats to have 32 bit precision either, and never did.

 :-)


 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated. The times it is, we've known about it, and it has made us
 all very, very angry.


Holy shit, it's just occurred to me that 'real' is only 64bits on arm
(and every non-x86 platform)...
That means a compiler running on an arm host will produce a different
binary than a compiler running on an x86 host!! O_O



 May 15 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 9:06 PM, Manu via Digitalmars-d wrote:

 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...


There are some that support 128 bits.


 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O


That's probably why VC++ dropped 80 bit long double support entirely.

Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU, we're 
going to give you only 64 bits."

It's also why I'd like to build a 128 soft fp emulator in dmd for all compile 
time float operations.



 May 15 2016



 next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 14:37, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/15/2016 9:06 PM, Manu via Digitalmars-d wrote:

 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...



 There are some that support 128 bits.


 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O



 That's probably why VC++ dropped 80 bit long double support entirely.

 Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU, we're
 going to give you only 64 bits."


No, you'll give me 80bit _when I type "real"_. Otherwise, if I type
'double', you'll give me that. I don't understand how that can be
controversial.
I know you love the x87, but I'm pretty sure you're among a small
minority. Personally, I don't want a single line of code that goes
anywhere near the x87 to be emit in any binary I produce. It's a
deprecated old crappy piece of hardware, and transfers between x87 and
sse regs are slow.


 It's also why I'd like to build a 128 soft fp emulator in dmd for all
 compile time float operations.


And I did also realise the reason for your desire to implement 128bit
soft-float the same moment I realised this. The situation that
different DMD builds operate at different precisions internally (based
on the host arch) is a whole new level of astonishment.
I support this soft-float, but please, build a soft-float for all
precisions, and use them everywhere that a hardware float for _the
specified precision_ is not available ;)



 May 15 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/15/2016 10:37 PM, Manu via Digitalmars-d wrote:

 No, you'll give me 80bit _when I type "real"_. Otherwise, if I type
 'double', you'll give me that. I don't understand how that can be
 controversial.


Because, as I explained, that results in a 2x or more speed degradation (along 
with the loss of accuracy).



 I know you love the x87, but I'm pretty sure you're among a small
 minority. Personally, I don't want a single line of code that goes
 anywhere near the x87 to be emit in any binary I produce. It's a
 deprecated old crappy piece of hardware, and transfers between x87 and
 sse regs are slow.


I used to do numerics work professionally. Most of the troubles I had were 
catastrophic loss of precision. Accumulated roundoff errors when doing
numerical 
integration or matrix inversion are major problems. 80 bits helps dramatically 
with that.



 It's also why I'd like to build a 128 soft fp emulator in dmd for all
 compile time float operations.


 And I did also realise the reason for your desire to implement 128bit
 soft-float the same moment I realised this. The situation that
 different DMD builds operate at different precisions internally (based
 on the host arch) is a whole new level of astonishment.


Then you're probably also astonished at the links I posted to you on how g++
and 
VC++ behave with FP.



 May 15 2016



 next sibling parent  Mike James <foo bar.com>  writes:


On Monday, 16 May 2016 at 06:46:59 UTC, Walter Bright wrote:

 On 5/15/2016 10:37 PM, Manu via Digitalmars-d wrote:

 [...]


 Because, as I explained, that results in a 2x or more speed 
 degradation (along with the loss of accuracy).



 [...]


 I used to do numerics work professionally. Most of the troubles 
 I had were catastrophic loss of precision. Accumulated roundoff 
 errors when doing numerical integration or matrix inversion are 
 major problems. 80 bits helps dramatically with that.


A classic example... :-)


http://www.stsci.edu/~lbradley/seminar/butterfly.html



 May 16 2016



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 2:46 AM, Walter Bright wrote:

 I used to do numerics work professionally. Most of the troubles I had
 were catastrophic loss of precision. Accumulated roundoff errors when
 doing numerical integration or matrix inversion are major problems. 80
 bits helps dramatically with that.


Aren't good algorithms helping dramatically with that?

Also, do you have a theory that reconciles your assessment of the 
importance of 80-bit math with the fact that the computing world is 
moving away from it? 
http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it


Andrei



 May 16 2016



 next sibling parent reply deadalnix <deadalnix gmail.com>  writes:


On Monday, 16 May 2016 at 10:29:02 UTC, Andrei Alexandrescu wrote:

 On 5/16/16 2:46 AM, Walter Bright wrote:

 I used to do numerics work professionally. Most of the 
 troubles I had
 were catastrophic loss of precision. Accumulated roundoff 
 errors when
 doing numerical integration or matrix inversion are major 
 problems. 80
 bits helps dramatically with that.


 Aren't good algorithms helping dramatically with that?

 Also, do you have a theory that reconciles your assessment of 
 the importance of 80-bit math with the fact that the computing 
 world is moving away from it? 
 http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it


 Andrei


Regardless of the compiler actually doing it or not, the argument 
that extra precision is a problem is self defeating. I don't 
think argument for speed have been raised so far.



 May 16 2016



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 7:33 AM, deadalnix wrote:

 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.


I agree that this whole "we need less precision" argument would be 
difficult to accept.


 I don't think argument
 for speed have been raised so far.


This may be the best angle in this discussion. For all I can tell 80 bit 
is slow as molasses and on the road to getting slower. Isn't that enough 
of an argument to move away from it?


Andrei



 May 16 2016



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 4:35 AM, Andrei Alexandrescu wrote:

 This may be the best angle in this discussion. For all I can tell 80 bit is
slow
 as molasses and on the road to getting slower. Isn't that enough of an argument
 to move away from it?


We are talking CTFE here, not runtime.



 May 16 2016



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 8:19 AM, Walter Bright wrote:

 On 5/16/2016 4:35 AM, Andrei Alexandrescu wrote:

 This may be the best angle in this discussion. For all I can tell 80
 bit is slow
 as molasses and on the road to getting slower. Isn't that enough of an
 argument
 to move away from it?


 We are talking CTFE here, not runtime.


I have big plans with floating-point CTFE and all are elastic: the 
faster CTFE FP is, the more and better things we can do. Things that 
other languages can't dream to do, like interpolation tables for 
transcendental functions. So a slowdown of FP CTFE would be essentially 
a strategic loss. -- Andrei



 May 16 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 6:15 AM, Andrei Alexandrescu wrote:

 On 5/16/16 8:19 AM, Walter Bright wrote:

 We are talking CTFE here, not runtime.


 I have big plans with floating-point CTFE and all are elastic: the faster CTFE
 FP is, the more and better things we can do. Things that other languages can't
 dream to do, like interpolation tables for transcendental functions. So a
 slowdown of FP CTFE would be essentially a strategic loss. -- Andrei


Based on my experience with soft float on DOS, and the fact that CPUs are what, 
1000 times faster today, I have a hard time thinking of a case where that would 
be a big problem.

I can't see someone running a meteorological prediction using CTFE :-)



 May 16 2016



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 05/16/2016 01:16 PM, Walter Bright wrote:

 On 5/16/2016 6:15 AM, Andrei Alexandrescu wrote:

 On 5/16/16 8:19 AM, Walter Bright wrote:

 We are talking CTFE here, not runtime.


 I have big plans with floating-point CTFE and all are elastic: the
 faster CTFE
 FP is, the more and better things we can do. Things that other
 languages can't
 dream to do, like interpolation tables for transcendental functions. So a
 slowdown of FP CTFE would be essentially a strategic loss. -- Andrei


 Based on my experience with soft float on DOS


I seriously think all experience accumulated last century needs to be 
thoroughly scrutinized. The world of software has changed and is 
changing fast enough to make all tricks older than a a decade virtually 
useless. Although core structures and algorithms stay the same, even 
some fundamentals are changing - e.g. chasing pointers is no longer 
faster than seemingly slow operations on implicit data structures on 
arrays, and so on and so forth.

Yes, there was a time when one setvbuf() call would make I/O one order 
of magnitude faster. Those days are long gone, and the only value of 
that knowledge industry is tepid anecdotes.

Fast compile-time FP is good, and the faster it is, the better it is. 
It's a huge differentiating factor for us. A little marginal precision 
is not. Please don't spend time on making compile-time FP slower.


Thanks,

Andrei



 May 16 2016



prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 21:35, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/16/16 7:33 AM, deadalnix wrote:

 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.



 I agree that this whole "we need less precision" argument would be difficult
 to accept.


 I don't think argument
 for speed have been raised so far.


 This may be the best angle in this discussion. For all I can tell 80 bit is
 slow as molasses and on the road to getting slower. Isn't that enough of an
 argument to move away from it?


It really has though!
At Remedy, the first thing we noticed when compiling code with DMD is
that our float code produced x87 ops, and with the x64 calling
convention (floats in SSE regs), this means pushing every function
argument to the stack, loading them into x87 regs, doing the work
(slowly), pushing them back, and popping them into a return reg. The
codegen is insanely long and inefficient.
The only reason it wasn't a critical blocker was because of the
relative small level of use of D, and that it wasn't in any
time-critical path.
If Ethan and Remedy want to expand their use of D, the compiler CAN
NOT emit x87 code. It's just a matter of time before a loop is in a
hot path.



 May 16 2016



 next sibling parent  Ethan Watson <gooberman gmail.com>  writes:


On Tuesday, 17 May 2016 at 02:00:24 UTC, Manu wrote:

 If Ethan and Remedy want to expand their use of D, the compiler 
 CAN
 NOT emit x87 code. It's just a matter of time before a loop is 
 in a
 hot path.


I really need to see what the codegen for the latest DMD looks 
like, I have no idea what the current state is. But this isn't 
just true for Remedy, it's true for any engine programmer in 
another company thinking of using D.

In context of this entire discussion though, a compiler switch to 
give the codegen I want is my preference. I have no desire to 
dictate how other people should use D/DMD, I just want the option 
to use it the way I need to.



 May 17 2016



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 7:00 PM, Manu via Digitalmars-d wrote:

 If Ethan and Remedy want to expand their use of D, the compiler CAN
 NOT emit x87 code. It's just a matter of time before a loop is in a
 hot path.


dmd no longer emits x87 code for float/double on 64 bit models, and hasn't for 
years.



 May 17 2016



prev sibling  parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 17 May 2016 at 12:00, Manu <turkeyman gmail.com> wrote:

 On 16 May 2016 at 21:35, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/16/16 7:33 AM, deadalnix wrote:

 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.



 I agree that this whole "we need less precision" argument would be difficult
 to accept.


 I don't think argument
 for speed have been raised so far.


 This may be the best angle in this discussion. For all I can tell 80 bit is
 slow as molasses and on the road to getting slower. Isn't that enough of an
 argument to move away from it?


 It really has though!
 At Remedy, the first thing we noticed when compiling code with DMD is
 that our float code produced x87 ops, and with the x64 calling
 convention (floats in SSE regs), this means pushing every function
 argument to the stack, loading them into x87 regs, doing the work
 (slowly), pushing them back, and popping them into a return reg. The
 codegen is insanely long and inefficient.
 The only reason it wasn't a critical blocker was because of the
 relative small level of use of D, and that it wasn't in any
 time-critical path.
 If Ethan and Remedy want to expand their use of D, the compiler CAN
 NOT emit x87 code. It's just a matter of time before a loop is in a
 hot path.


That said, I recall a conversation where Walter said this has been
addressed? I haven't worked on this in some time now, so I haven't
tested where it's at. Just saying that complaints about x87
performance have definitely been made.



 May 16 2016



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 3:29 AM, Andrei Alexandrescu wrote:

 Aren't good algorithms helping dramatically with that?


Yup. But they are not textbook math algorithms, tend to be complex and strange, 
and are not very discoverable by regular programmers (I tried and failed).

Extended precision is a simple, straightforward fix to precision problems in 
straightforward code.



 May 16 2016



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 12:37 AM, Walter Bright wrote:

 Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU,
 we're going to give you only 64 bits."


I'm not sure about this. My understanding is that all SSE has hardware 
for 32 and 64 bit floats, and the the 80-bit hardware is pretty much 
cut-and-pasted from the x87 days without anyone really looking in 
improving it. And that's been the case for more than a decade. Is that 
correct?

I'm looking for example at 
http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see that 
on all Intel and compatible hardware, the speed of 80-bit floating point 
operations ranges between much slower and disastrously slower.

I think it's time to revisit our attitudes to floating point, which was 
formed last century in the heydays of x87. My perception is the world 
has moved to SSE and 32- and 64-bit float; the "real" type is a 
distraction for D; the whole let's do things in 128-bit during 
compilation is a time waster; and many of the original things we want to 
do with floating point are different without a distinction, and a 
further waste of our resources.


Andrei



 May 16 2016



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:

 I'm not sure about this. My understanding is that all SSE has hardware for 32
 and 64 bit floats, and the the 80-bit hardware is pretty much cut-and-pasted
 from the x87 days without anyone really looking in improving it. And that's
been
 the case for more than a decade. Is that correct?


I believe so.


 I'm looking for example at
 http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see that on all
 Intel and compatible hardware, the speed of 80-bit floating point operations
 ranges between much slower and disastrously slower.


It's not a totally fair comparison. A matrix inversion algorithm that 
compensates for cumulative precision loss involves executing a lot more FP 
instructions (don't know the ratio).



 I think it's time to revisit our attitudes to floating point, which was formed
 last century in the heydays of x87. My perception is the world has moved to SSE
 and 32- and 64-bit float; the "real" type is a distraction for D; the whole
 let's do things in 128-bit during compilation is a time waster; and many of the
 original things we want to do with floating point are different without a
 distinction, and a further waste of our resources.


Some counter points:

1. Go uses 256 bit soft float for constant folding.

2. Speed is hardly the only criterion. Quickly getting the wrong answer (and
not 
just a few bits off, but total loss of precision) is of no value.

3. Supporting 80 bit reals does not take away from the speed of floats/doubles 
at runtime.

4. Removing 80 bit reals will consume resources (adapting the test suite, 
rewriting the math library, ...).

5. Other languages not supporting it means D has a capability they don't have. 
My experience with selling products is that if you have an exclusive feature 
that a particular customer needs, it's a slam dunk sale.

6. My other experience with feature sets is if you drop things that make your 
product different, and concentrate on matching feature checklists with Major 
Brand X, customers go with Major Brand X.

7. 80 bit reals are there and they work. The support is mature, and is rarely 
worked on, i.e. it does not consume resources.

8. Removing it would break an unknown amount of code, and there's no reasonable 
workaround for those that rely on it.



 May 16 2016



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/16/16 8:37 AM, Walter Bright wrote:

 On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:

 I'm looking for example at
 http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see
 that on all
 Intel and compatible hardware, the speed of 80-bit floating point
 operations
 ranges between much slower and disastrously slower.


 It's not a totally fair comparison. A matrix inversion algorithm that
 compensates for cumulative precision loss involves executing a lot more
 FP instructions (don't know the ratio).


It is rare to need to actually compute the inverse of a matrix. Most of 
the time it's of interest to solve a linear equation of the form Ax = b, 
for which a variety of good methods exist that don't entail computing 
the actual inverse.

I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a 
lot of other evidence. This is what happened with input vs. forward C++ 
iterators as the main motivator for a variety of concepts designs.


 Some counter points:


Glad to see these!


 1. Go uses 256 bit soft float for constant folding.


Go can afford it because it does no interesting things during 
compilation. We can't.


 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no value.


Of course. But it turns out the precision argument loses to the speed 
argument.

A. It's been many many years and very few if any people commend D for 
its superior approach to FP precision.

B. In contrast, a bunch of folks complain about anything slow, be it 
during compilation or at runtime.

Good algorithms lead to good precision, not 16 additional bits. 
Precision is overrated. Speed isn't.


 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.


Fast compile-time floats are of strategic importance to us. Give me fast 
FP during compilation, I'll make it go slow (whilst put to do amazing work).


 4. Removing 80 bit reals will consume resources (adapting the test
 suite, rewriting the math library, ...).


I won't argue with that! Just let's focus on the right things: good fast 
streamlined computing using the appropriate hardware.


 5. Other languages not supporting it means D has a capability they don't
 have. My experience with selling products is that if you have an
 exclusive feature that a particular customer needs, it's a slam dunk sale.


Again: I'm not seeing people coming out of the woodwork to praise D's 
precision. What they would indeed enjoy is amazing FP use during 
compilation, and that can be done only if CTFE FP is __FAST__. That 
_is_, indeed the capability others are missing!


 6. My other experience with feature sets is if you drop things that make
 your product different, and concentrate on matching feature checklists
 with Major Brand X, customers go with Major Brand X.


This is true in principle but too vague to be relevant. Again, what 
evidence do you have that D's additional precision is revered? I see 
none, over like a decade.


 7. 80 bit reals are there and they work. The support is mature, and is
 rarely worked on, i.e. it does not consume resources.


Yeah, I just don't want it used in any new code. Please. It's using lead 
for building boats.


 8. Removing it would break an unknown amount of code, and there's no
 reasonable workaround for those that rely on it.


Let's do what everybody did to x87: continue supporting, but slowly and 
surely drive it to obsolescence.

That's the right way.


Andrei



 May 16 2016



 next sibling parent  jmh530 <john.michael.hall gmail.com>  writes:


On Monday, 16 May 2016 at 14:32:55 UTC, Andrei Alexandrescu wrote:

 Let's do what everybody did to x87: continue supporting, but 
 slowly and surely drive it to obsolescence.

 That's the right way.


This is a long thread that has covered a lot of ground. There has 
been a lot of argument, but few concrete take-a-ways (this was 
one, why I'm quoting it).

As far as I see it, the main topics have been:

1) Change in precision during comparisons (the original)
2) Change in precision to real during intermediate floating point 
calculations
3) Floating point precision in CTFE


a solution. I have not seen any concrete recommendations.

Personally, I am not a fan of implicit conversions. It's one more 
set of rules I need to remember. I would prefer a compile-time 
error with a recommendation to explicitly cast to whatever is the 
best precision. It may even make sense for the error to also 
recommend approxEqual.


emphasized the importance of precision, while others emphasized 
determinism and speed.

As someone who sometimes does matrix inversions and gets 
eigenvalues, I'm sympathetic to Walter's points about precision. 
For some applications, additional precision is essential. 
However, others clearly want the option to disable this behavior.

Walter stressed Java's difficulty in resolving the issue. What 
about their solution? They have strictfp, which allows exactly 
the behavior people like Manu seem to want. In D, you could make 
it an attribute. The benefit of adding the new annotation is that 
no code would get broken. You could write

 strictfp:

at the top of a file instead of a compiler flag. You could also 
add a  defaultfp (or !strictfp or  strictfp(false)) as its 
complement.

I'm not sure whether it should be applied recursively. Perhaps? A 
recursive approach can be difficult. However, a non-recursive 
approach would be limited in some ways. You'd have to do some 
work on std.math to get it working.


topic seems to be the least fully formed. Improved CTFE seems 
universally regarded as a positive.



 May 16 2016



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:

 It is rare to need to actually compute the inverse of a matrix. Most of the
time
 it's of interest to solve a linear equation of the form Ax = b, for which a
 variety of good methods exist that don't entail computing the actual inverse.


I was solving n equations with n unknowns.


 I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a lot of
 other evidence. This is what happened with input vs. forward C++ iterators as
 the main motivator for a variety of concepts designs.


What I did was implement the algorithm out of my calculus textbook. Sure, it's
a 
naive algorithm - but it is highly unlikely that untrained FP programmers know 
intuitively how to deal with precision loss. I bring up our very own Phobos sum 
algorithm, which was re-implemented later with the Kahan method to reduce 
precision loss.



 1. Go uses 256 bit soft float for constant folding.


 Go can afford it because it does no interesting things during compilation. We
 can't.


The we can't is conjecture at the moment.



 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no value.


 Of course. But it turns out the precision argument loses to the speed argument.

 A. It's been many many years and very few if any people commend D for its
 superior approach to FP precision.

 B. In contrast, a bunch of folks complain about anything slow, be it during
 compilation or at runtime.


D's support for reals does not negatively impact the speed of float or double 
computations.



 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.


 Fast compile-time floats are of strategic importance to us. Give me fast FP
 during compilation, I'll make it go slow (whilst put to do amazing work).


I still have a hard time seeing what you plan to do at compile time that would 
involve tens of millions of FP calculations.



 6. My other experience with feature sets is if you drop things that make
 your product different, and concentrate on matching feature checklists
 with Major Brand X, customers go with Major Brand X.


 This is true in principle but too vague to be relevant. Again, what evidence do
 you have that D's additional precision is revered? I see none, over like a
decade.


Fortran supports Quad (128 bit floats) as standard.

https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format



 May 16 2016



 next sibling parent  jmh530 <john.michael.hall gmail.com>  writes:


On Monday, 16 May 2016 at 18:57:24 UTC, Walter Bright wrote:

 On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:

 It is rare to need to actually compute the inverse of a 
 matrix. Most of the time
 it's of interest to solve a linear equation of the form Ax = 
 b, for which a
 variety of good methods exist that don't entail computing the 
 actual inverse.


 I was solving n equations with n unknowns.


LU decomposition is the more common approach.

atlab has a backslash operator to solve systems of equations with 
LU decomposition:

http://www.mathworks.com/help/matlab/ref/inv.html#bu6sfy8-1



 May 16 2016



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 05/16/2016 02:57 PM, Walter Bright wrote:

 On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:

 It is rare to need to actually compute the inverse of a matrix. Most
 of the time
 it's of interest to solve a linear equation of the form Ax = b, for
 which a
 variety of good methods exist that don't entail computing the actual
 inverse.


 I was solving n equations with n unknowns.


That's the worst way to go about it. I've seen students fail exams over 
it. Solving a linear equations sytems by computing the inverse is 
inferior to just about any other method. See e.g. 
http://www.mathworks.com/help/matlab/ref/inv.html

"It is seldom necessary to form the explicit inverse of a matrix. A 
frequent misuse of inv arises when solving the system of linear 
equations Ax = b. One way to solve the equation is with x = inv(A)*b. A 
better way, from the standpoint of both execution time and numerical 
accuracy, is to use the matrix backslash operator x = A\b. This produces 
the solution using Gaussian elimination, without explicitly forming the 
inverse. See mldivide for further information."

You have long been advocating that the onus is on the engineer to 
exercise good understanding of what's going on when using 
domain-specific code such as UTF, linear algebra, etc. So if you 
exercised it now, you need to discount this argument.


 I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a
 lot of
 other evidence. This is what happened with input vs. forward C++
 iterators as
 the main motivator for a variety of concepts designs.


 What I did was implement the algorithm out of my calculus textbook.
 Sure, it's a naive algorithm - but it is highly unlikely that untrained
 FP programmers know intuitively how to deal with precision loss.


As someone else said: a few bits of extra precision ain't gonna help 
them. I thought that argument was closed.


 I bring
 up our very own Phobos sum algorithm, which was re-implemented later
 with the Kahan method to reduce precision loss.


Kahan is clear, ingenous, and understandable and a great part of the 
stdlib. I don't see what the point is here. Naive approaches aren't 
going to take anyone far, regardless of precision.


 1. Go uses 256 bit soft float for constant folding.


 Go can afford it because it does no interesting things during
 compilation. We
 can't.


 The we can't is conjecture at the moment.


We can't and we shouldn't invest time in investigating whether we can. 
It's a waste even if the project succeeded 100% and exceeded anyone's 
expectations.


 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no
 value.


 Of course. But it turns out the precision argument loses to the speed
 argument.

 A. It's been many many years and very few if any people commend D for its
 superior approach to FP precision.

 B. In contrast, a bunch of folks complain about anything slow, be it
 during
 compilation or at runtime.


 D's support for reals does not negatively impact the speed of float or
 double computations.


Then let's not do more of it.


 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.


 Fast compile-time floats are of strategic importance to us. Give me
 fast FP
 during compilation, I'll make it go slow (whilst put to do amazing work).


 I still have a hard time seeing what you plan to do at compile time that
 would involve tens of millions of FP calculations.


Give those to me and you'll be surprised.


Andrei



 May 16 2016



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 16.05.2016 22:10, Andrei Alexandrescu wrote:

 I bring
 up our very own Phobos sum algorithm, which was re-implemented later
 with the Kahan method to reduce precision loss.


 Kahan is clear, ingenous, and understandable and a great part of the
 stdlib. I don't see what the point is here. Naive approaches aren't
 going to take anyone far, regardless of precision.


Kahan summation is not guaranteed to work in D.



 May 17 2016



prev sibling  parent reply Ethan Watson <gooberman gmail.com>  writes:


On Monday, 16 May 2016 at 14:32:55 UTC, Andrei Alexandrescu wrote:

 It is rare to need to actually compute the inverse of a matrix.


Unless you're doing game/graphics work ;-) 4x3 or 4x4 matrices 
are commonly used to represent transforms in 3D space in every 3D 
polygon-based rendering pipeline I know of. It's even a 
requirement for fixed-function OpenGL 1.x.

Video games - also known around here as "The Exception To The 
Rule".

(Side note: My own preference is to represent transforms as a 
quaternion and vector. Inverting such a transform is a simple 
matter of negating a few components. Generating a matrix from 
such a transform for rendering purposes is trivial compared to 
matrix inversion.)



 May 17 2016



  parent  jmh530 <john.michael.hall gmail.com>  writes:


On Tuesday, 17 May 2016 at 07:47:58 UTC, Ethan Watson wrote:

 Unless you're doing game/graphics work ;-) 4x3 or 4x4 matrices 
 are commonly used to represent transforms in 3D space in every 
 3D polygon-based rendering pipeline I know of. It's even a 
 requirement for fixed-function OpenGL 1.x.

 Video games - also known around here as "The Exception To The 
 Rule".

 (Side note: My own preference is to represent transforms as a 
 quaternion and vector. Inverting such a transform is a simple 
 matter of negating a few components. Generating a matrix from 
 such a transform for rendering purposes is trivial compared to 
 matrix inversion.)


I don't know much about computer graphics, but if you're solving 
equations, then you can use the techniques mentioned above.

Nevertheless, I'm not really sure what would be the fastest 
approach to inverting small matrices. I would definitely try the 
LU or Cholesky approaches. It might be that for a small matrix a 
Gaussian reduction approach would be fast. There are some 
analytic tricks you could use if you have some information about 
them, like when you can represent them as blocks. If some of the 
blocks are zero or identity matrices, then it simplifies the 
calculations too.



 May 17 2016



prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com>  writes:


On Mon, May 16, 2016 at 05:37:58AM -0700, Walter Bright via Digitalmars-d wrote:

 On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:


[...]

I think it's time to revisit our attitudes to floating point, which
was formed last century in the heydays of x87. My perception is the
world has moved to SSE and 32- and 64-bit float; the "real" type is a
distraction for D; the whole let's do things in 128-bit during
compilation is a time waster; and many of the original things we want
to do with floating point are different without a distinction, and a
further waste of our resources.


 
 Some counter points:
 
 1. Go uses 256 bit soft float for constant folding.
 
 2. Speed is hardly the only criterion. Quickly getting the wrong
 answer (and not just a few bits off, but total loss of precision) is
 of no value.


An algorithm that uses 80-bit but isn't written properly to account for
FP total precision loss will also produce wrong results, just like an
algorithm that uses 64-bit and isn't written to account for FP total
precision loss.


[...]

 5. Other languages not supporting it means D has a capability they
 don't have. My experience with selling products is that if you have an
 exclusive feature that a particular customer needs, it's a slam dunk
 sale.


It's also the case that maintaining a product with feature X that nobody
uses is an unnecessary drain on developmental resources.



 6. My other experience with feature sets is if you drop things that
 make your product different, and concentrate on matching feature
 checklists with Major Brand X, customers go with Major Brand X.


[...]

If I were presented with product A having an archaic feature X that
works slowly and that I don't even need in the first place, vs. product
B having exactly the feature set I need without the baggage of archaic
feature X, I'd go with product B.


T

-- 
Real Programmers use "cat > a.out".



 May 16 2016



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 7:37 AM, H. S. Teoh via Digitalmars-d wrote:

 An algorithm that uses 80-bit but isn't written properly to account for
 FP total precision loss will also produce wrong results, just like an
 algorithm that uses 64-bit and isn't written to account for FP total
 precision loss.


That is correct. But in most routine cases, it's enough more precision that 
going to more heroic algorithm changes become unnecessary.



 If I were presented with product A having an archaic feature X that
 works slowly and that I don't even need in the first place, vs. product
 B having exactly the feature set I need without the baggage of archaic
 feature X, I'd go with product B.


I have business experience with what people actually choose, and it's often not 
what they say they would choose. Sorry :-)

Social proof (i.e. what your colleagues use) is a very, very important factor. 
But having a needed feature not available in the more popular product trumps 
social proof. It doesn't need to appeal to everyone, but it can be the wedge 
that drives a product into the mainstream.

The needs of numerical analysts have often been neglected by the C/C++ 
community. The community has been very slow to recognize IEEE, by about a 
decade. It wasn't until fairly recently that C/C++ compilers even handled NaN 
properly. It's no surprise that FORTRAN reigned as the choice of numerical
analysts.

(On the other hand, the needs of game developers have received strong support.)

The dismissal of concerns about precision as "archaic" is something I have seen 
for a long time.

As my calculator anecdote illustrates, even engineers do not recognize loss of 
precision when it happens. I'm not talking about a few bits in last place, I'm 
talking about not recognizing total loss of precision.

I sometimes wonder how many utter garbage FP results are being generated and 
treated as correct answers by researchers who confuse textbook math with FP 
math. The only field I can think of where a sprinkling of garbage results don't 
matter as long as it is fast is game programming.

Even if you select doubles for speed, it's nice to be able to temporarily swap 
in reals as a check to see if the similar results are computed. If not, you 
surely have a loss of precision problem.



 May 16 2016



  parent reply Guillaume Piolat <first.last gmail.com>  writes:


On Monday, 16 May 2016 at 17:50:22 UTC, Walter Bright wrote:

 The needs of numerical analysts have often been neglected by 
 the C/C++ community. The community has been very slow to 
 recognize IEEE, by about a decade. It wasn't until fairly 
 recently that C/C++ compilers even handled NaN properly. It's 
 no surprise that FORTRAN reigned as the choice of numerical 
 analysts.

 (On the other hand, the needs of game developers have received 
 strong support.)


I have regression scripts that check audio computations against 
the previous baseline. The goal is to ensure the results are 
consistent between compilers and bitness, and to check for the 
validity of sound-altering optimization (usually within 80 dB 
peak difference is OK). And of course not to utterly break things 
during optimization work.

I've been harmed once by the x87 promotion of a float to real, 
which made an algorithm sound better on 32-bit than 64-bit. On 
64-bit DMD used SSE which avoided the internal precision boost. 
If I had tested only in 32-bit I could have inferred float 
precision was enough.

This is one time I would have preferred if D would never promoted 
float to double or to real. I understand that the x87 code would 
go much slower if restricted precision was ensured. So how about 
using SSE like in 64-bit code?

Free bits of precision may lead to think things are better than 
they are.



 May 16 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 1:43 PM, Guillaume Piolat wrote:

 So how about using SSE like in 64-bit code?


It does for 32 bits on OSX, because SSE is guaranteed to be available on Macs. 
But this is not true in general, and so it does not for other x86 systems.



 May 17 2016



prev sibling next sibling parent reply Daniel Murphy <yebbliesnospam gmail.com>  writes:


On 16/05/2016 10:37 PM, Walter Bright wrote:

 Some counter points:

 1. Go uses 256 bit soft float for constant folding.


Then we should use 257 bit soft float!



 May 16 2016



 next sibling parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 16.05.2016 17:11, Daniel Murphy wrote:

 On 16/05/2016 10:37 PM, Walter Bright wrote:

 Some counter points:

 1. Go uses 256 bit soft float for constant folding.


 Then we should use 257 bit soft float!


I don't see how you reach that conclusion, as 258 bit soft float is 
clearly superior.



 May 16 2016



  parent  Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 18:31, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 16.05.2016 17:11, Daniel Murphy wrote:

 On 16/05/2016 10:37 PM, Walter Bright wrote:

 Some counter points:

 1. Go uses 256 bit soft float for constant folding.


 Then we should use 257 bit soft float!


 I don't see how you reach that conclusion, as 258 bit soft float is clearly
 superior.


You should be more pragmatic, 224 bits is all you need. :-)



 May 16 2016



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 8:11 AM, Daniel Murphy wrote:

 On 16/05/2016 10:37 PM, Walter Bright wrote:

 1. Go uses 256 bit soft float for constant folding.


 Then we should use 257 bit soft float!



I like the cut of your jib!



 May 16 2016



prev sibling  parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Monday, 16 May 2016 at 15:11:21 UTC, Daniel Murphy wrote:

 On 16/05/2016 10:37 PM, Walter Bright wrote:

 Some counter points:

 1. Go uses 256 bit soft float for constant folding.


 Then we should use 257 bit soft float!


Go uses at least 288 bits!!!!! (256 bits mantissa and 32 bits 
exponent).

But Go has a completely different take on constant expressions, 
and much better than the C family that D belongs to (but still 
far from ideal). From https://golang.org/ref/spec:

«
Numeric constants represent exact values of arbitrary precision 
and do not overflow. Consequently, there are no constants 
denoting the IEEE-754 negative zero, infinity, and not-a-number 
values.

Constants may be typed or untyped. Literal constants, true, 
false, iota, and certain constant expressions containing only 
untyped constant operands are untyped.

A constant may be given a type explicitly by a constant 
declaration or conversion, or implicitly when used in a variable 
declaration or an assignment or as an operand in an expression. 
It is an error if the constant value cannot be represented as a 
value of the respective type. For instance, 3.0 can be given any 
integer or any floating-point type, while 2147483648.0 (equal to 
1<<31) can be given the types float32, float64, or uint32 but not 
int32 or string.

An untyped constant has a default type which is the type to which 
the constant is implicitly converted in contexts where a typed 
value is required, for instance, in a short variable declaration 
such as i := 0 where there is no explicit type. The default type 
of an untyped constant is bool, rune, int, float64, complex128 or 
string respectively, depending on whether it is a boolean, rune, 
integer, floating-point, complex, or string constant.

Implementation restriction: Although numeric constants have 
arbitrary precision in the language, a compiler may implement 
them using an internal representation with limited precision. 
That said, every implementation must:

Represent integer constants with at least 256 bits.
Represent floating-point constants, including the parts of a 
complex constant, with a mantissa of at least 256 bits and a 
signed exponent of at least 32 bits.
Give an error if unable to represent an integer constant 
precisely.
Give an error if unable to represent a floating-point or complex 
constant due to overflow.
Round to the nearest representable constant if unable to 
represent a floating-point or complex constant due to limits on 
precision.
These requirements apply both to literal constants and to the 
result of evaluating constant expressions.
»

See also https://golang.org/ref/spec#Constant_expressions



 May 17 2016



  parent reply Matthias Bentrup <matthias.bentrup googlemail.com>  writes:


If you try to make compile-time FP math behave exactly like 
run-time FP math, you'd not only have to use the same precision 
in the compiler, but also the same rounding mode, denormal 
handling etc., which can be changed at run time 
(http://dlang.org/phobos/core_stdc_fenv.html), so the exact same 
piece of code can yield different results based on the run-time 
FP-context.



 May 17 2016



  parent  Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?=  writes:


On Tuesday, 17 May 2016 at 14:05:54 UTC, Matthias Bentrup wrote:

 If you try to make compile-time FP math behave exactly like 
 run-time FP math, you'd not only have to use the same precision 
 in the compiler, but also the same rounding mode, denormal 
 handling etc., which can be changed at run time 
 (http://dlang.org/phobos/core_stdc_fenv.html), so the exact 
 same piece of code can yield different results based on the 
 run-time FP-context.


Yes, heading rounding mode is an absolute requirement. If the 
system don't heed the rounding mode then it will change the 
semantics and break the program when computing upper vs lower 
bounds.

I've never liked the stateful approach to rounding. If I was to 
design a language I probably would make it part of the type or 
operator or something along those lines.



 May 17 2016



prev sibling next sibling parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/16/2016 5:37 AM, Walter Bright wrote:

 [...]


9. Both clang and gcc offer 80 bit long doubles. It's Microsoft VC++ that is
out 
of step. Not having an 80 bit type in D means diminished interoperability with 
very popular C and C++ compilers.



 May 16 2016



prev sibling  parent reply Wyatt <wyatt.epp gmail.com>  writes:


On Monday, 16 May 2016 at 12:37:58 UTC, Walter Bright wrote:

 7. 80 bit reals are there and they work. The support is mature, 
 and is rarely worked on, i.e. it does not consume resources.


This may not be true for too much longer-- both Intel and AMD are 
slowly phasing the x86 FPU out.  I think Intel already announced 
a server chip that omits it entirely, though I can't find the 
corroborating link.

-Wyatt



 May 17 2016



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/17/2016 7:08 AM, Wyatt wrote:

 On Monday, 16 May 2016 at 12:37:58 UTC, Walter Bright wrote:

 7. 80 bit reals are there and they work. The support is mature, and is rarely
 worked on, i.e. it does not consume resources.


 This may not be true for too much longer-- both Intel and AMD are slowly
phasing
 the x86 FPU out.  I think Intel already announced a server chip that omits it
 entirely, though I can't find the corroborating link.



Consider that clang and g++ support 80 bit long doubles. D is hardly unique. 
It's VC++ that does not.



 May 17 2016



prev sibling  parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 20:27, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 5/16/16 12:37 AM, Walter Bright wrote:

 Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU,
 we're going to give you only 64 bits."



 I'm not sure about this. My understanding is that all SSE has hardware for
 32 and 64 bit floats, and the the 80-bit hardware is pretty much
 cut-and-pasted from the x87 days without anyone really looking in improving
 it. And that's been the case for more than a decade. Is that correct?


It's not only correct, but it's also deprecated, and may be removed
from silicone (ie, emulated/microcoded, is it's not already...) at
some unknown point in the future.



 May 16 2016



prev sibling next sibling parent  Max Samukha <maxsamukha gmail.com>  writes:


On Monday, 16 May 2016 at 10:25:33 UTC, Andrei Alexandrescu wrote:

 On 5/16/16 12:37 AM, Walter Bright wrote:

 Me, I think of that as "Who cares that you paid $$$ for an 80 
 bit CPU,
 we're going to give you only 64 bits."


 I'm not sure about this. My understanding is that all SSE has 
 hardware for 32 and 64 bit floats, and the the 80-bit hardware 
 is pretty much cut-and-pasted from the x87 days without anyone 
 really looking in improving it. And that's been the case for 
 more than a decade. Is that correct?

 I'm looking for example at 
 http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and 
 see that on all Intel and compatible hardware, the speed of 
 80-bit floating point operations ranges between much slower and 
 disastrously slower.

 I think it's time to revisit our attitudes to floating point, 
 which was formed last century in the heydays of x87. My 
 perception is the world has moved to SSE and 32- and 64-bit 
 float; the "real" type is a distraction for D; the whole let's 
 do things in 128-bit during compilation is a time waster; and 
 many of the original things we want to do with floating point 
 are different without a distinction, and a further waste of our 
 resources.

 It is a bit ironic that we worry about autodecoding (I'll 
 destroy that later) whilst a landslide loss of speed and 
 predictability in floating point math doesn't raise an eyebrow.


 Andrei


The AMD64 programmer's manual discourages the use of x87:

"For media and scientific programs that demand floating-point 
operations, it is often easier and more
powerful to use SSE instructions. Such programs perform better 
than x87 floating-point programs,
because the YMM/XMM register file is flat rather than 
stack-oriented, there are twice as many
registers (in 64-bit mode), and SSE instructions can operate on 
four or eight times the number of
floating-point operands as can x87 instructions. This ability to 
operate in parallel on multiple pairs of
floating-point elements often makes it possible to remove local 
temporary variables that would
otherwise be needed in x87 floating-point code."



 May 16 2016



prev sibling  parent  Ethan Watson <gooberman gmail.com>  writes:


On Monday, 16 May 2016 at 10:25:33 UTC, Andrei Alexandrescu wrote:

 I'm not sure about this. My understanding is that all SSE has 
 hardware for 32 and 64 bit floats, and the the 80-bit hardware 
 is pretty much cut-and-pasted from the x87 days without anyone 
 really looking in improving it. And that's been the case for 
 more than a decade. Is that correct?


Pretty much. On the OS side, Windows has officially deprecated 
x87 for the 64-bit version in desktop mode, and it's flat out 
forbidden in kernel mode. All development focus from Intel has 
been on improving the SSE/AVX instruction set and pipeline.

And on a gamedev side, we generally go for fast over precise. Or, 
more to the point, an acceptable loss in precision. The C++ 
codegen spits out SSE/AVX code by default in our builds, and I 
hand optimise with appropriate intrinsics certain functions that 
get inlined. SIMD is even more an appropriate point to bring up 
here - gaming is trending towards more parallel operations, 
operating on a single float at a time is not the correct way to 
get the best performance out of your system.

This is one of those things where I can see the point for the D 
compiler to do things its own way - but only when it expects to 
operate in a pure D environment. We have heavy interop between 
C++ and D. If simple functions can give different results at 
compile time without a way for me to configure the compiler on 
both sides, what actual benefits does that give me?



 May 16 2016



prev sibling next sibling parent  Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 06:06, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:

 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:

 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...



 I imagine you'll be devastated when you discover that the C++ Standard does
 not require 32 bit floats to have 32 bit precision either, and never did.

 :-)


 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated. The times it is, we've known about it, and it has made us
 all very, very angry.


 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...
 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O


Which is why gcc/g++ (ergo gdc) uses floating point emulation.
Getting consistent results at compile time regardless of whatever
host/target/cross configuration trumps doing it natively.



 May 15 2016



prev sibling  parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 16 May 2016 at 16:09, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 16 May 2016 at 06:06, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:

 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:

 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...



 I imagine you'll be devastated when you discover that the C++ Standard does
 not require 32 bit floats to have 32 bit precision either, and never did.

 :-)


 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated. The times it is, we've known about it, and it has made us
 all very, very angry.


 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...
 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O


 Which is why gcc/g++ (ergo gdc) uses floating point emulation.
 Getting consistent results at compile time regardless of whatever
 host/target/cross configuration trumps doing it natively.


Certainly. I understand this, and desire it in the frontend too.



 May 16 2016



prev sibling  parent  Marco Leise <Marco.Leise gmx.de>  writes:


Content-Disposition: inline

I wondered how the precision of typical SSE vs FPU code would
compare. SSE uses floats and doubles as the main mean to
control precision, while the FPU is configured via x87 control
word.

The attached program calculates 5^-441 iteratively using these
methods, then converts to double and prints the mantissa:

  Compiled with Digital Mars D in 64-bit
  float SSE : 00000000000000000000000000000000000000000000000000000
  float x87 : 00100000101010100111001110000000000000000000000000000
  double SSE: 00100000101010100111001101111011011110011011110010001
  double x87: 00100000101010100111001101111011011110011011110010000
  real x87  : 00100000101010100111001101111011011110011011110001111

Take-aways:
- SSE results generally differ from x87 results at the same
  precision
- x87 produces accurate single precision results, whereas
  the error with SSE is fatal (and that's not a flush-to-zero;
  the greatest power that still produces 1 bit of mantissa
  is 5^-64(!))

-- 
Marco



 May 16 2016