• TJB (10/10) Sep 19 2012 D Helpers,
• Justin Whear (6/19) Sep 19 2012 Take a look at std.zip and std.zlib. Personally, I use LZO which is

"TJB" <broughtj gmail.com> writes:

D Helpers,

I have a ton of huge financial data to process and analyze 
statistically.  I am looking for the best way to store/retrieve 
the data for processing. I am wondering how to write the data to 
a compressed binary file and then later read from that same file 
for processing.

I can read/write the binary data just fine, but how to do it with 
compression?

I appreciate your help and suggestions.

TJB

Justin Whear <justin economicmodeling.com> writes:

On Wed, 19 Sep 2012 19:48:49 +0200, TJB wrote:

 D Helpers,
 
 I have a ton of huge financial data to process and analyze
 statistically.  I am looking for the best way to store/retrieve the data
 for processing. I am wondering how to write the data to a compressed
 binary file and then later read from that same file for processing.
 
 I can read/write the binary data just fine, but how to do it with
 compression?
 
 I appreciate your help and suggestions.
 
 TJB

Take a look at std.zip and std.zlib.  Personally, I use LZO which is 
incredibly fast and pretty easy to wrap (it's a C lib).  Using LZO takes 
a little more work (you need split the data into chunks, manually store 
compressed/uncompressed lengths, etc.), so the easy route is to use 
std.zlib.