www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - buffered output to files

reply Gerald Jansen <gjansen ownmail.net> writes:
In this great article [1] there is a brief section on buffered 
output to files. Also in this thread [2] I was advised to use 
explicitly buffered output for maximum performance. This left me 
perplexed: surely any high-level routines already use buffered 
IO, no?

[1] 
http://nomad.so/2015/09/working-with-files-in-the-d-programming-language/
[2] 
http://forum.dlang.org/post/ydgmzhlspvvvrbeemrqf forum.dlang.org

In order to get a better understanding of the issue, I decided to 
do a bit of simple testing with kind with the kind of file 
writing I have to do very often: millions of shortish formatted 
data records. My simple test program is here:
http://dpaste.dzfl.pl/329023e651c4.

Time to write 10 million formatted records (average of 10 runs) 
using format ("%d %d %0.8f", i, i*13, i/1e7) with different 
methods:
        GDC    DMD
f0:  12.0s  15.2s  std.stdio.writef
f1:  10.0s  13.6s  1Mb buffer std.array.appender; 
std.stdio.rawWrite
f2:  10.3s  13.6s  300Mb buffer std.outbuffer; std.file.write
f3:   4.9s   5.0s  std.stdc.fprintf
f4:   4.8s   4.8s  30b buffer std.stdc.sprintf (no file output)

* gdc (Debian 5.2.1-17) 5.2.1 20150911 :: -O3 -frelease 
-march=native
* dmd DMD64 D Compiler v2.068.1 :: -O -release -inline

Some observations:
* the two double-buffered output approaches (f1 and f2) are 
roughly the same
* the performance gain from the extra buffering was not huge 
(about 20%)
* good old fprintf seems to be my best bet :-(

These results may be very specific to my conditions, but I throw 
them out there just in case others may be interested.
Oct 03 2015
parent Gerald Jansen <gjansen ownmail.net> writes:
On Saturday, 3 October 2015 at 22:21:08 UTC, Gerald Jansen wrote:

 My simple test program is here: 
 http://dpaste.dzfl.pl/329023e651c4.


An alternative link to the program (that doesn't try to run it)
http://codepad.org/FbHJJqYM.
Oct 04 2015