www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CustomFloat

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I found myself in need for smaller and specialized floating point types, 
where I configure the exact configuration (sign, mantissa, exponent, 
bias). See http://en.wikipedia.org/wiki/Half_precision for a half 
precision number slated for inclusion in IEEE 754r.

Would it be interesting to add a CustomFloat template to phobos? I'm 
thinking along the lines of:

template CustomFloat!(bool sign, uint mantissa,
         uint exponent, uint bias)
{
     ...
}

So half-precision numbers are:

alias CustomFloat!(true, 5, 10, 15) HalfFloat;

There are quite a few details to kink out but this is definitely doable. 
Numbers like 24-bit floating point and even 8-bit floating point would 
be easy to support too. For now CustomFloat would be intended 
exclusively as a compact storage mechanism; only conversion to the 
standard floating points would be implemented. Later, maybe we can get 
to implement some operations natively at least on machines that support 
them in hardware. I wanted to gauge interest in the topic.


Andrei
Oct 16 2008
next sibling parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Fri, Oct 17, 2008 at 1:30 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 I found myself in need for smaller and specialized floating point types,
 where I configure the exact configuration (sign, mantissa, exponent, bias).
 See http://en.wikipedia.org/wiki/Half_precision for a half precision number
 slated for inclusion in IEEE 754r.

 Would it be interesting to add a CustomFloat template to phobos? I'm
 thinking along the lines of:

 template CustomFloat!(bool sign, uint mantissa,
        uint exponent, uint bias)
 {
    ...
 }

 So half-precision numbers are:

 alias CustomFloat!(true, 5, 10, 15) HalfFloat;

 There are quite a few details to kink out but this is definitely doable.
 Numbers like 24-bit floating point and even 8-bit floating point would be
 easy to support too. For now CustomFloat would be intended exclusively as a
 compact storage mechanism; only conversion to the standard floating points
 would be implemented. Later, maybe we can get to implement some operations
 natively at least on machines that support them in hardware. I wanted to
 gauge interest in the topic.



In theory it would be great to have this in Phobos.  I say in theory
because I haven't actually been in need of such a thing recently, but
it is definitely handy to have if you're working with high dynamic
range images (like from openEXR), or doing other GPU-related things.
Is that the kind of thing you need them for too?  I've heard of using
anything beyond 16-bit and 24-bit floats in the GPU/HDR world, though.
 A full generic solution is probably overkill for that.

--bb
Oct 16 2008
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:

 On Fri, Oct 17, 2008 at 1:30 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 I found myself in need for smaller and specialized floating point types,
 where I configure the exact configuration (sign, mantissa, exponent, bias).
 See http://en.wikipedia.org/wiki/Half_precision for a half precision number
 slated for inclusion in IEEE 754r.

 Would it be interesting to add a CustomFloat template to phobos? I'm
 thinking along the lines of:

 template CustomFloat!(bool sign, uint mantissa,
        uint exponent, uint bias)
 {
    ...
 }

 So half-precision numbers are:

 alias CustomFloat!(true, 5, 10, 15) HalfFloat;

 There are quite a few details to kink out but this is definitely doable.
 Numbers like 24-bit floating point and even 8-bit floating point would be
 easy to support too. For now CustomFloat would be intended exclusively as a
 compact storage mechanism; only conversion to the standard floating points
 would be implemented. Later, maybe we can get to implement some operations
 natively at least on machines that support them in hardware. I wanted to
 gauge interest in the topic.


 
 
 In theory it would be great to have this in Phobos.  I say in theory
 because I haven't actually been in need of such a thing recently, but
 it is definitely handy to have if you're working with high dynamic
 range images (like from openEXR), or doing other GPU-related things.
 Is that the kind of thing you need them for too?  I've heard of using
 anything beyond 16-bit and 24-bit floats in the GPU/HDR world, though.
  A full generic solution is probably overkill for that.


Yah, graphics is a big target for such types (and a big pusher for 
standardizing them). I need such numbers for different purposes, i.e. 
storing large arrays of probability distribution. In that case the range 
is [0, 1] and I need to cram those numbers in as little space as 
possible. I could definitely use a customized floating point layout.

Note that a fully generic implementation is not overkill; if there are 
two layouts to support, you might as well support any.


Andrei
Oct 16 2008
next sibling parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Fri, Oct 17, 2008 at 1:56 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 Bill Baxter wrote:

 On Fri, Oct 17, 2008 at 1:30 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 I found myself in need for smaller and specialized floating point types,
 where I configure the exact configuration (sign, mantissa, exponent,
 bias).
 See http://en.wikipedia.org/wiki/Half_precision for a half precision
 number
 slated for inclusion in IEEE 754r.

 Would it be interesting to add a CustomFloat template to phobos? I'm
 thinking along the lines of:

 template CustomFloat!(bool sign, uint mantissa,
       uint exponent, uint bias)
 {
   ...
 }

 So half-precision numbers are:

 alias CustomFloat!(true, 5, 10, 15) HalfFloat;

 There are quite a few details to kink out but this is definitely doable.
 Numbers like 24-bit floating point and even 8-bit floating point would be
 easy to support too. For now CustomFloat would be intended exclusively as
 a
 compact storage mechanism; only conversion to the standard floating
 points
 would be implemented. Later, maybe we can get to implement some
 operations
 natively at least on machines that support them in hardware. I wanted to
 gauge interest in the topic.



 In theory it would be great to have this in Phobos.  I say in theory
 because I haven't actually been in need of such a thing recently, but
 it is definitely handy to have if you're working with high dynamic
 range images (like from openEXR), or doing other GPU-related things.
 Is that the kind of thing you need them for too?  I've heard of using
 anything beyond 16-bit and 24-bit floats in the GPU/HDR world, though.
  A full generic solution is probably overkill for that.


 Yah, graphics is a big target for such types (and a big pusher for
 standardizing them). I need such numbers for different purposes, i.e.
 storing large arrays of probability distribution. In that case the range is
 [0, 1] and I need to cram those numbers in as little space as possible. I
 could definitely use a customized floating point layout.

 Note that a fully generic implementation is not overkill; if there are two
 layouts to support, you might as well support any.


Yeh, mostly thinking about ASM tweaks to make them fast.

That and handling of denormal numbers.  Not sure how regular denormals
are across different precisions of floating point numbers.   Been a
while since I looked at how denormals are defined, though.  Maybe it's
easy.

But writing optimized ASM for math ops is probably not as easy to do
in a generic way.  Don's probably going to prove me wrong though.  :-)

--bb
Oct 16 2008
parent reply Don <nospam nospam.com.au> writes:
Bill Baxter wrote:

 On Fri, Oct 17, 2008 at 1:56 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 Bill Baxter wrote:

 On Fri, Oct 17, 2008 at 1:30 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 I found myself in need for smaller and specialized floating point types,
 where I configure the exact configuration (sign, mantissa, exponent,
 bias).
 See http://en.wikipedia.org/wiki/Half_precision for a half precision
 number
 slated for inclusion in IEEE 754r.








754r became official two months ago. It's now IEEE 754-2008, and it 
includes 16 bit floats.


 Would it be interesting to add a CustomFloat template to phobos? I'm
 thinking along the lines of:

 template CustomFloat!(bool sign, uint mantissa,
       uint exponent, uint bias)
 {
   ...
 }

 So half-precision numbers are:

 alias CustomFloat!(true, 5, 10, 15) HalfFloat;

 There are quite a few details to kink out but this is definitely doable.
 Numbers like 24-bit floating point and even 8-bit floating point would be
 easy to support too. For now CustomFloat would be intended exclusively as
 a
 compact storage mechanism; only conversion to the standard floating
 points
 would be implemented. Later, maybe we can get to implement some
 operations
 natively at least on machines that support them in hardware. I wanted to
 gauge interest in the topic.


 In theory it would be great to have this in Phobos.  I say in theory
 because I haven't actually been in need of such a thing recently, but
 it is definitely handy to have if you're working with high dynamic
 range images (like from openEXR), or doing other GPU-related things.
 Is that the kind of thing you need them for too?  I've heard of using
 anything beyond 16-bit and 24-bit floats in the GPU/HDR world, though.
  A full generic solution is probably overkill for that.


 Yah, graphics is a big target for such types (and a big pusher for
 standardizing them). I need such numbers for different purposes, i.e.
 storing large arrays of probability distribution. In that case the range is
 [0, 1] and I need to cram those numbers in as little space as possible. I
 could definitely use a customized floating point layout.




Wouldn't it be better to use fixed point for that application? Or are 
the numbers distributed uniformly in IEEE space (ie, as many between 
0.0025 and 0.005, as between 0.5 and 0.1)?


 Note that a fully generic implementation is not overkill; if there are two
 layouts to support, you might as well support any.


 
 Yeh, mostly thinking about ASM tweaks to make them fast.
 
 That and handling of denormal numbers.  Not sure how regular denormals
 are across different precisions of floating point numbers.   Been a
 while since I looked at how denormals are defined, though.  Maybe it's
 easy.


Yeah. If it's in the denormal range you can't treat the mantissa and 
exponent independently. NaNs are a pain, too.
Actually I think most GPUs don't support denormals, infinity, or NaN.


 
 But writing optimized ASM for math ops is probably not as easy to do
 in a generic way.  Don's probably going to prove me wrong though.  :-)


It's very hairy if you're going to support all the rounding modes.

On processors with SSE2, you can do a load/shift/or sequence to shuffle 
the bits of a halffloat[4] into a float[4], so the performance shouldn't 
be too terrible for array operations, for example. Need to check for 
denormals, though, so it gets ugly.
The amount to shift and the bitmasks should be the only things which 
change. Perfect application for mixin asm!

There's no SSE shift instructions, so it would be very slow on earlier 
x86 machines.
Oct 17 2008
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:

 In theory it would be great to have this in Phobos.  I say in theory
 because I haven't actually been in need of such a thing recently, but
 it is definitely handy to have if you're working with high dynamic
 range images (like from openEXR), or doing other GPU-related things.
 Is that the kind of thing you need them for too?  I've heard of using
 anything beyond 16-bit and 24-bit floats in the GPU/HDR world, though.
  A full generic solution is probably overkill for that.


 Yah, graphics is a big target for such types (and a big pusher for
 standardizing them). I need such numbers for different purposes, i.e.
 storing large arrays of probability distribution. In that case the 
 range is
 [0, 1] and I need to cram those numbers in as little space as 
 possible. I
 could definitely use a customized floating point layout.




 
 Wouldn't it be better to use fixed point for that application? Or are 
 the numbers distributed uniformly in IEEE space (ie, as many between 
 0.0025 and 0.005, as between 0.5 and 0.1)?


The latter. Relative precision has to be about constant within the whole 
range. In fact I could store logprobs in fixed-point format, but that 
makes addition slow.


Andrei
Oct 17 2008
prev sibling parent Benji Smith <dlanguage benjismith.net> writes:
Andrei Alexandrescu wrote:

 Yah, graphics is a big target for such types (and a big pusher for 
 standardizing them). I need such numbers for different purposes, i.e. 
 storing large arrays of probability distribution. In that case the range 
 is [0, 1] and I need to cram those numbers in as little space as 
 possible. I could definitely use a customized floating point layout.


I'll be damned. That was my exact same use case!

I didn't implement a standard half-precision float, though. Knowing that 
I'd never have negative values, my type didn't have a sign bit at all. I 
think I had a 11-bit mantissa and a 5-bit exponent... or something like 
that.

--benji
Oct 17 2008
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Fri, 17 Oct 2008 00:30:33 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Would it be interesting to add a CustomFloat template to phobos?


Yes, I do work on the GPU a fair amount and this would be useful either as  
a stand alone file or as part of phobos.
Oct 17 2008
prev sibling next sibling parent reply Paul D. Anderson <paul.d.removethis.anderson comcast.andthis.net> writes:
Andrei Alexandrescu Wrote:


 I found myself in need for smaller and specialized floating point types, 
 where I configure the exact configuration (sign, mantissa, exponent, 
 bias). See http://en.wikipedia.org/wiki/Half_precision for a half 
 precision number slated for inclusion in IEEE 754r.
 
 Would it be interesting to add a CustomFloat template to phobos? I'm 
 thinking along the lines of:
 
 template CustomFloat!(bool sign, uint mantissa,
          uint exponent, uint bias)
 {
      ...
 }
 
 So half-precision numbers are:
 
 alias CustomFloat!(true, 5, 10, 15) HalfFloat;
 
 There are quite a few details to kink out but this is definitely doable. 
 Numbers like 24-bit floating point and even 8-bit floating point would 
 be easy to support too. For now CustomFloat would be intended 
 exclusively as a compact storage mechanism; only conversion to the 
 standard floating points would be implemented. Later, maybe we can get 
 to implement some operations natively at least on machines that support 
 them in hardware. I wanted to gauge interest in the topic.
 
 
 Andrei


I've done some work on a set of decimal floating point numbers consistent with
the new 754r decimal formats approved in June. Some of that work would be
applicable here, I think.

It would probably make sense to add the decimal formats at the same time. Let
me know if I can help.

Paul
Oct 17 2008
next sibling parent Paul D. Anderson <paul.d.removethis.anderson comcast.andthis.net> writes:
Paul D. Anderson Wrote:


 Andrei Alexandrescu Wrote:
 

 I found myself in need for smaller and specialized floating point types, 
 where I configure the exact configuration (sign, mantissa, exponent, 
 bias). See http://en.wikipedia.org/wiki/Half_precision for a half 
 precision number slated for inclusion in IEEE 754r.
 
 Would it be interesting to add a CustomFloat template to phobos? I'm 
 thinking along the lines of:
 
 template CustomFloat!(bool sign, uint mantissa,
          uint exponent, uint bias)
 {
      ...
 }
 
 So half-precision numbers are:
 
 alias CustomFloat!(true, 5, 10, 15) HalfFloat;
 
 There are quite a few details to kink out but this is definitely doable. 
 Numbers like 24-bit floating point and even 8-bit floating point would 
 be easy to support too. For now CustomFloat would be intended 
 exclusively as a compact storage mechanism; only conversion to the 
 standard floating points would be implemented. Later, maybe we can get 
 to implement some operations natively at least on machines that support 
 them in hardware. I wanted to gauge interest in the topic.
 
 
 Andrei


 
 I've done some work on a set of decimal floating point numbers consistent with
the new 754r decimal formats approved in June. Some of that work would be
applicable here, I think.
 
 It would probably make sense to add the decimal formats at the same time. Let
me know if I can help.
 
 Paul
 


Here's a link to info regarding the decimal formats:
http://speleotrove.com/decimal/
Oct 17 2008
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Paul D. Anderson wrote:

 Andrei Alexandrescu Wrote:
 

 I found myself in need for smaller and specialized floating point
 types, where I configure the exact configuration (sign, mantissa,
 exponent, bias). See http://en.wikipedia.org/wiki/Half_precision
 for a half precision number slated for inclusion in IEEE 754r.
 
 Would it be interesting to add a CustomFloat template to phobos?
 I'm thinking along the lines of:
 
 template CustomFloat!(bool sign, uint mantissa, uint exponent, uint
 bias) { ... }
 
 So half-precision numbers are:
 
 alias CustomFloat!(true, 5, 10, 15) HalfFloat;
 
 There are quite a few details to kink out but this is definitely
 doable. Numbers like 24-bit floating point and even 8-bit floating
 point would be easy to support too. For now CustomFloat would be
 intended exclusively as a compact storage mechanism; only
 conversion to the standard floating points would be implemented.
 Later, maybe we can get to implement some operations natively at
 least on machines that support them in hardware. I wanted to gauge
 interest in the topic.
 
 
 Andrei


 
 I've done some work on a set of decimal floating point numbers
 consistent with the new 754r decimal formats approved in June. Some
 of that work would be applicable here, I think.
 
 It would probably make sense to add the decimal formats at the same
 time. Let me know if I can help.


Sounds interesting, and thanks for the offer. I'd be glad to add decimal 
types too, in case there's enough interest. I think monetary 
calculations can be helped, and what don't we do nowadays to help people 
involved in monetary calculations? :o)

Andrei
Oct 17 2008
prev sibling next sibling parent Benji Smith <dlanguage benjismith.net> writes:
Andrei Alexandrescu wrote:

 I found myself in need for smaller and specialized floating point types, 
 where I configure the exact configuration (sign, mantissa, exponent, 
 bias). See http://en.wikipedia.org/wiki/Half_precision for a half 
 precision number slated for inclusion in IEEE 754r.


I would definitely find that useful. Not so much for runtime calculation 
(though I wouldn't object to it) but for serialization.

A few years ago, I implement a half-precision float for Java because I 
needed to send a bazillion measurements across a slow connection. The 
nature of the data was such that I didn't mind losing precision, as long 
as I could send twice as many values.

Having that functionality in D, without having to write it myself, would 
be very handy.

--benji
Oct 17 2008
prev sibling parent Jay Norwood <jayn io.com> writes:
This guy has done some arbitrary precision floating point
classes ... maybe something useful there.  It's all template code,
but other than that, it's ok.

http://www.hvks.com/Numerical/arbitrary_precision.htm
Oct 23 2008