www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is there a good lib out there to handle large integer of know size ?

reply deadalnix <deadalnix gmail.com> writes:
In my case, I'm interested in 160 and 256 bits integers. Using 
BigInt seems wasteful as I know the integer size I want. Do we 
have something like this around ?
Jan 06 2016
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 09:25:45PM +0000, deadalnix via Digitalmars-d wrote:
 In my case, I'm interested in 160 and 256 bits integers. Using BigInt
 seems wasteful as I know the integer size I want. Do we have something
 like this around ?
If not, it may be something nice to add to Phobos. Integer width can be passed as a compile-time parameter, making it completely generic. T -- Right now I'm having amnesia and deja vu at the same time. I think I've forgotten this before.
Jan 06 2016
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
 In my case, I'm interested in 160 and 256 bits integers. Using 
 BigInt seems wasteful as I know the integer size I want. Do we 
 have something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
Jan 06 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 6 January 2016 at 21:57:13 UTC, Guillaume Piolat 
wrote:
 On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
 In my case, I'm interested in 160 and 256 bits integers. Using 
 BigInt seems wasteful as I know the integer size I want. Do we 
 have something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
That is awesome ! Won't do the 160 bits case, but I can deal with that. Did you consider this as a worthy addition to phobos ? I think it is.
Jan 06 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/6/16 5:18 PM, deadalnix wrote:
 On Wednesday, 6 January 2016 at 21:57:13 UTC, Guillaume Piolat wrote:
 On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
 In my case, I'm interested in 160 and 256 bits integers. Using BigInt
 seems wasteful as I know the integer size I want. Do we have
 something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
That is awesome ! Won't do the 160 bits case, but I can deal with that. Did you consider this as a worthy addition to phobos ? I think it is.
Yes, we need to add that to phobos. Who could be the champion? -- Andrei
Jan 06 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 05:23:12PM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:
 On 1/6/16 5:18 PM, deadalnix wrote:
On Wednesday, 6 January 2016 at 21:57:13 UTC, Guillaume Piolat wrote:
On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
In my case, I'm interested in 160 and 256 bits integers. Using
BigInt seems wasteful as I know the integer size I want. Do we have
something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
That is awesome ! Won't do the 160 bits case, but I can deal with that. Did you consider this as a worthy addition to phobos ? I think it is.
Yes, we need to add that to phobos. Who could be the champion? --
Here are a few things that should probably be addressed before merging to Phobos: - Better error message when user attempts to instantiate a wide int with non-power-of-2 bits. (Currently spews a whole bunch of internal compile errors.) - toString() needs to: 1) Be the non-allocating overload: void toString(scope void delegate(const(char)[]) dg) {...} 2) Output in decimal by default instead of hex. - Need a nice way of initializing wide ints from literals. - [optional] Division performance can probably be improved. T -- I don't trust computers, I've spent too long programming to think that they can get anything right. -- James Miller
Jan 06 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/06/2016 05:47 PM, H. S. Teoh via Digitalmars-d wrote:
 On Wed, Jan 06, 2016 at 05:23:12PM -0500, Andrei Alexandrescu via
Digitalmars-d wrote:
 On 1/6/16 5:18 PM, deadalnix wrote:
 On Wednesday, 6 January 2016 at 21:57:13 UTC, Guillaume Piolat wrote:
 On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
 In my case, I'm interested in 160 and 256 bits integers. Using
 BigInt seems wasteful as I know the integer size I want. Do we have
 something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
That is awesome ! Won't do the 160 bits case, but I can deal with that. Did you consider this as a worthy addition to phobos ? I think it is.
Yes, we need to add that to phobos. Who could be the champion? --
Here are a few things that should probably be addressed before merging to Phobos: - Better error message when user attempts to instantiate a wide int with non-power-of-2 bits. (Currently spews a whole bunch of internal compile errors.) - toString() needs to: 1) Be the non-allocating overload: void toString(scope void delegate(const(char)[]) dg) {...} 2) Output in decimal by default instead of hex. - Need a nice way of initializing wide ints from literals. - [optional] Division performance can probably be improved.
This is a good list, and not difficult to implement. Thanks! -- Andrei
Jan 06 2016
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 02:47:24PM -0800, H. S. Teoh via Digitalmars-d wrote:
[...]
 - Better error message when user attempts to instantiate a wide int with
   non-power-of-2 bits. (Currently spews a whole bunch of internal
   compile errors.)
[...] https://github.com/d-gamedev-team/gfm/pull/143 T -- Ph.D. = Permanent head Damage
Jan 08 2016
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 02:47:24PM -0800, H. S. Teoh via Digitalmars-d wrote:
[...]
 - Need a nice way of initializing wide ints from literals.
[...] https://github.com/d-gamedev-team/gfm/pull/144 T -- Try to keep an open mind, but not so open your brain falls out. -- theboz
Jan 08 2016
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 02:47:24PM -0800, H. S. Teoh via Digitalmars-d wrote:
[...]
 - toString() needs to:
   1) Be the non-allocating overload:
 
   	void toString(scope void delegate(const(char)[]) dg) {...}
 
   2) Output in decimal by default instead of hex.
[...] https://github.com/d-gamedev-team/gfm/pull/145 T -- Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian W. Kernighan
Jan 08 2016
parent reply rumbu <rumbu rumbu.ro> writes:
Here is what I've done as a side project for a decimal data type:

https://github.com/rumbu13/fixed/blob/master/src/fixedtest/fixed.d

- signed and unsigned fixed width types;
- all current operators supported (even signed shift or power);
- optimized division and multiplication;
- formatting with specifiers;
- conversion (currently 16 and 10-based numbers)
- casting;
- endianess aware;
- minimal dependendencies; (std.traits, std.format - needed for 
FormatSpec).

Working on:
- floating point conversion optimization;
- random base conversion;
- rangify current strings conversions;
- some bugs for the signed data type;
Jan 09 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/9/16 6:31 AM, rumbu wrote:
 Here is what I've done as a side project for a decimal data type:

 https://github.com/rumbu13/fixed/blob/master/src/fixedtest/fixed.d
Who can champion ONE fixed large integer library for Phobos? -- Andrei
Jan 09 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 9 January 2016 at 20:28:26 UTC, Andrei Alexandrescu 
wrote:
 Who can champion ONE fixed large integer library for Phobos? -- 
 Andrei
Got a possible one. My implementation is heavy on assembly language to take advantage of x86 features (like the carry flag, and giving you 64 & 128bit inputs/results). Still building it, but it's promising, mostly got cleanup on it to do and more unittests. Honestly the hardest part is writing the division code, which I think I just got done.
Jun 10 2017
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/10/17 3:28 PM, Era Scarecrow wrote:
 On Saturday, 9 January 2016 at 20:28:26 UTC, Andrei Alexandrescu wrote:
 Who can champion ONE fixed large integer library for Phobos? -- Andrei
Got a possible one. My implementation is heavy on assembly language to take advantage of x86 features (like the carry flag, and giving you 64 & 128bit inputs/results). Still building it, but it's promising, mostly got cleanup on it to do and more unittests. Honestly the hardest part is writing the division code, which I think I just got done.
That's cool as long as the assembler is guarded by version(X_86) and has a portable alternative. Where's the code? -- Andrei
Jun 10 2017
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 10 June 2017 at 19:40:47 UTC, Andrei Alexandrescu 
wrote:
 On 6/10/17 3:28 PM, Era Scarecrow wrote:
   Got a possible one. My implementation is heavy on assembly 
 language to take advantage of x86 features
That's cool as long as the assembler is guarded by version(X_86) and has a portable alternative. Where's the code? -- Andrei
My computer, and no portable alternative yet. I've got a ways before that's an option, probably end up doing 32bit math with ulongs to make it work reliably and portably. Worse it will probably be 10-50x slower.
Jun 10 2017
parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 10 June 2017 at 20:19:22 UTC, Era Scarecrow wrote:
 On Saturday, 10 June 2017 at 19:40:47 UTC, Andrei Alexandrescu 
 wrote:
 On 6/10/17 3:28 PM, Era Scarecrow wrote:
   Got a possible one. My implementation is heavy on assembly 
 language to take advantage of x86 features
That's cool as long as the assembler is guarded by version(X_86) and has a portable alternative. Where's the code? -- Andrei
My computer, and no portable alternative yet. I've got a ways before that's an option, probably end up doing 32bit math with ulongs to make it work reliably and portably. Worse it will probably be 10-50x slower.
I ended up doing my own. There are just no way to do it well without cent/ucent . Weka is running into the same problem for error correction.
Jun 11 2017
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 11 June 2017 at 08:52:37 UTC, deadalnix wrote:
 I ended up doing my own. There are just no way to do it well 
 without cent/ucent . Weka is running into the same problem for 
 error correction.
And what timing, I just finished getting it working with assembly and compatible versions. Seems 5x slower rather than the dreaded 50, but that's from testing factorial up to 100 (200ms vs 1,200ms). Anyways, here's how it is currently, most features are avaliable, need more tests and find occurrences where it doesn't work. https://github.com/rtcvb32/Side-Projects/blob/master/scaledint.d One note, to get the x86 assembly versions, use -version=Intel, I have it disabled so I can concentrate on the compatible version and switch between them on my machine. -debug includes a unittest that runs a factorial test
Jun 11 2017
next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 11 June 2017 at 18:01:41 UTC, Era Scarecrow wrote:
  Anyways, here's how it is currently, most features are 
 avaliable, need more tests and find occurrences where it 
 doesn't work.
I should probably mention that's only for UCent, Cent isn't fully tested or implemented yet.
Jun 11 2017
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 11 June 2017 at 18:01:41 UTC, Era Scarecrow wrote:
  And what timing, I just finished getting it working with 
 assembly and compatible versions. Seems 5x slower rather than 
 the dreaded 50, but that's from testing factorial up to 100 
 (200ms vs 1,200ms).
Doing a 10240bit int I ran though 1,000 factorials in 6 seconds (this includes printing the values out). I got to about 400 in 16 minutes using the generic code. So like the wideint implementation I saw a while ago, larger types aren't recommended, but the speed penalty you can live with to use cent/ucent. Also got Cent working (mostly). In a few more days I should probably have this done. If there's somewhere else i should post my work regarding this or updates, do let me know.
Jun 11 2017
next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 11 June 2017 at 21:52:40 UTC, Era Scarecrow wrote:
  Also got Cent working (mostly).
It's complete, more or less now. I just a matter of filling in any missing methods that I haven't programmed for, and probably adding filler/forwarding functions from ScaledInt to UScaledInt. Although I'm quite happy to change (ScaledInt) if someone has a better idea name idea. I'll be in the D IRC chat for the next week or so, so if anyone wants to talk to me directly... When it's good enough I'll submit my code to the std.experimental branch. Also I need to compile/test the asm, I don't have it set up here (can compile, can't run).
Jun 11 2017
prev sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 11 June 2017 at 21:52:40 UTC, Era Scarecrow wrote:
  Doing a 10240bit int I ran though 1,000 factorials in 6 
 seconds (this includes printing the values out). I got to about 
 400 in 16 minutes using the generic code.
Got the compatible/generic code to run far faster, about 7 times slower vs the assembly language version! Probably the last of those optimizations I'll be making, don't think i can squeeze too much more out of it.
Jun 11 2017
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 June 2017 at 18:01:41 UTC, Era Scarecrow wrote:
 On Sunday, 11 June 2017 at 08:52:37 UTC, deadalnix wrote:
 I ended up doing my own. There are just no way to do it well 
 without cent/ucent . Weka is running into the same problem for 
 error correction.
And what timing, I just finished getting it working with assembly and compatible versions. Seems 5x slower rather than the dreaded 50, but that's from testing factorial up to 100 (200ms vs 1,200ms). Anyways, here's how it is currently, most features are avaliable, need more tests and find occurrences where it doesn't work. https://github.com/rtcvb32/Side-Projects/blob/master/scaledint.d One note, to get the x86 assembly versions, use -version=Intel, I have it disabled so I can concentrate on the compatible version and switch between them on my machine. -debug includes a unittest that runs a factorial test
You misunderstood. We need cent/ucent supported by the compiler to get to larger integral types efficiently. There are no ways around it. There are a ton of operations such as the X86 MUL which are able to produce a large multiplication into 2 registers. There are no way to leverage theses without compiler provided cent/ucent.
Jun 12 2017
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 12 June 2017 at 13:41:36 UTC, deadalnix wrote:
 You misunderstood. We need cent/ucent supported by the compiler 
 to get to larger integral types efficiently. There are no ways 
 around it. There are a ton of operations such as the X86 MUL 
 which are able to produce a large multiplication into 2 
 registers. There are no way to leverage these without compiler 
 provided cent/ucent.
Agreed. But until larger types are natively avaliable (either by simulation built into the compiler or by hardware registers) you gotta work with what you got.
Jun 12 2017
parent deadalnix <deadalnix gmail.com> writes:
On Monday, 12 June 2017 at 15:41:03 UTC, Era Scarecrow wrote:
 On Monday, 12 June 2017 at 13:41:36 UTC, deadalnix wrote:
 You misunderstood. We need cent/ucent supported by the 
 compiler to get to larger integral types efficiently. There 
 are no ways around it. There are a ton of operations such as 
 the X86 MUL which are able to produce a large multiplication 
 into 2 registers. There are no way to leverage these without 
 compiler provided cent/ucent.
Agreed. But until larger types are natively avaliable (either by simulation built into the compiler or by hardware registers) you gotta work with what you got.
I work with SDC on that one. That's the only reasonable path forward.
Jun 12 2017
prev sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 10 June 2017 at 19:40:47 UTC, Andrei Alexandrescu 
wrote:
 That's cool as long as the assembler is guarded by 
 version(X_86) and has a portable alternative. Where's the code?
Should i do a pull request and push it under std.experimental? Also if it's going to get renamed to a better struct name, gotta decide that. I think ScaledInt is a bad name to describe what it really is. The code's about 50k... I could instead push for an alternative version if you'd like, rebuild from the skeleton of this code to put the assembly versions of add/sub/mul/div to handle any size, and the template would only generate code for how big the struct is and passing buffers around... might be cleaner that way. Would also be easier for ctfe to handle calling for statically known values at compile time...
Jun 12 2017
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 10 June 2017 at 19:28:30 UTC, Era Scarecrow wrote:
 On Saturday, 9 January 2016 at 20:28:26 UTC, Andrei 
 Alexandrescu wrote:
 Who can champion ONE fixed large integer library for Phobos? 
 -- Andrei
Got a possible one. My implementation is heavy on assembly language to take advantage of x86 features (like the carry flag, and giving you 64 & 128bit inputs/results). Still building it, but it's promising, mostly got cleanup on it to do and more unittests.
You should be able to use the overflow intrinsics to similar effect
Jun 10 2017
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 6 January 2016 at 22:18:48 UTC, deadalnix wrote:
 On Wednesday, 6 January 2016 at 21:57:13 UTC, Guillaume Piolat 
 wrote:
 On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
 In my case, I'm interested in 160 and 256 bits integers. 
 Using BigInt seems wasteful as I know the integer size I 
 want. Do we have something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
That is awesome ! Won't do the 160 bits case, but I can deal with that. Did you consider this as a worthy addition to phobos ? I think it is.
Well I've never ever used them after writing. Perhaps should be in their own dub package. Need wideint literals at a minimum!
Jan 06 2016
prev sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 6 January 2016 at 22:18:48 UTC, deadalnix wrote:
 Won't do the 160 bits case
Now that I think of it, wideint could be used as a base for fixed point and 160 bit integers, much like unsigned BigInt is implemented with signed BigInt (or perhaps it's the other way around). I've tried with https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/fixedpoint.d but it's limited.
Jan 06 2016
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 06, 2016 at 09:57:13PM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 On Wednesday, 6 January 2016 at 21:25:45 UTC, deadalnix wrote:
In my case, I'm interested in 160 and 256 bits integers. Using BigInt
seems wasteful as I know the integer size I want. Do we have
something like this around ?
https://github.com/d-gamedev-team/gfm/blob/master/math/gfm/math/wideint.d public domain. Slow division. Only power-of-2 bits supported.
Looks nice. With a bit of work, it could be a potential Phobos addition IMO. T -- Perhaps the most widespread illusion is that if we were in power we would behave very differently from those who now hold it---when, in truth, in order to get power we would have to become very much like them. -- Unknown
Jan 06 2016