digitalmars.D.learn - Buffered output
- =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= (22/22) Oct 24 2005 I need to create a very simple program module that prints some runtime
- Kris (17/38) Oct 24 2005 You may wish to try Mango.io (Stdout)? The IO model is designed to take...
- Sean Kelly (12/21) Oct 24 2005 ..
- =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= (4/13) Oct 25 2005 writef is very useful and flexible, but also pretty slow. The
- Ben Hinkle (7/28) Oct 24 2005 printf and writef are already buffered - I believe stdout is line-buffer...
- =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= (9/15) Oct 25 2005 Yes, you're right. writef & printf are already buffered. One can see
I need to create a very simple program module that prints some runtime
generated strings to the standard output. Ok, that's not a problem. The
real problem is that I need to output these as fast as possible. The
output consists of big string blocks (>100.000 chars) that mostly
contain only two different characters.
I've ported the program from C. The original program doesn't use any
sort of buffering, just some printf:s like for(i=1;i<n;i++) if
(foobar()) printf("*.-");
Here are some simple benchmarks using the GNU time utility:
dmd test.c -o test -O3 -s
time ./test_c_printf > /dev/null
real 0m0.117s
dmd test.d -O -release -inline -version=printf
time ./test > /dev/null
real 0m0.140s
dmd test.d -O -release -inline -version=writef
time ./test > /dev/null
real 0m0.225s
How come writef() is so much slower than printf()?
Is there a fast way to do buffered output. I tried to use dynamic &
static arrays and printf the full arrays, but that wasn't actually any
faster.
Oct 24 2005
"Kris" <fu bar.com>
You may wish to try Mango.io (Stdout)? The IO model is designed to take
advantage of buffering, and I once ran into a problem with the Win32 console
where it barfed on more than 32K at one time :-)
To illustrate, there's a specific subclass to handle that condition at the
end of this module:
http://svn.dsource.org/projects/mango/trunk/mango/io/DeviceConduit.d
Mango.io has formatting options similar to printf(), the full gamut of
discrete IO methods, and a whole lot more besides. The mango.io equivalent
to writef() should be faster in that it emits arrays of utf8, rather than
one dchar at a time; and Stdout employs buffering by default. It's worth
taking a look here http://mango.dsource.org/ and here
http://www.dsource.org/forums/viewtopic.php?t=148
Those who prefer OOP may well prefer Mango.io over the phobos IO package(s)
~ they're designed for different audiences.
- Kris
"Jari-Matti Mäkelä" <jmjmak invalid_utu.fi> wrote in message
news:djjlb6$1l22$1 digitaldaemon.com...
I need to create a very simple program module that prints some runtime
generated strings to the standard output. Ok, that's not a problem. The
real problem is that I need to output these as fast as possible. The output
consists of big string blocks (>100.000 chars) that mostly contain only two
different characters.
I've ported the program from C. The original program doesn't use any sort
of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar())
printf("*.-");
Here are some simple benchmarks using the GNU time utility:
dmd test.c -o test -O3 -s
time ./test_c_printf > /dev/null
real 0m0.117s
dmd test.d -O -release -inline -version=printf
time ./test > /dev/null
real 0m0.140s
dmd test.d -O -release -inline -version=writef
time ./test > /dev/null
real 0m0.225s
How come writef() is so much slower than printf()?
Is there a fast way to do buffered output. I tried to use dynamic & static
arrays and printf the full arrays, but that wasn't actually any faster.
Oct 24 2005
In article <djjlb6$1l22$1 digitaldaemon.com>, =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= says...I need to create a very simple program module that prints some runtime generated strings to the standard output. Ok, that's not a problem. The real problem is that I need to output these as fast as possible. The output consists of big string blocks (>100.000 chars) that mostly contain only two different characters...How come writef() is so much slower than printf()?writef does some fancy things that printf doesn't, though I had expected performance to be comparable to printf for ASCII character strings. The implementation has changed since I last looked at it, but you might want to look at the source for writef in std.format.d (doFormat) to see if you can find any hangups.Is there a fast way to do buffered output. I tried to use dynamic & static arrays and printf the full arrays, but that wasn't actually any faster.If you just want to print ASCII strings as fast as possible, printf and writef are probably not ideal. You may be better off with puts or fwrite, and possibly even messing with the buffer settings using setbuf/setvbuf. Sean
Oct 24 2005
Sean Kelly wrote:writef is very useful and flexible, but also pretty slow. The std.format.d is too complex for me to optimize. I need to take a second look at it sometime, but for now another approach may be more useful.How come writef() is so much slower than printf()?writef does some fancy things that printf doesn't, though I had expected performance to be comparable to printf for ASCII character strings. The implementation has changed since I last looked at it, but you might want to look at the source for writef in std.format.d (doFormat) to see if you can find any hangups.
Oct 25 2005
printf and writef are already buffered - I believe stdout is line-buffered.
Posting the actual code might help people help you. I also recommend using
the -profile compiler option and/or std.perf. If you like you can also wrap
a std.stream.Stream with a std.stream.BufferedStream but I suspect that you
can do what you want with regular C buffering. See for example setvbuf.
"Jari-Matti Mäkelä" <jmjmak invalid_utu.fi> wrote in message
news:djjlb6$1l22$1 digitaldaemon.com...
I need to create a very simple program module that prints some runtime
generated strings to the standard output. Ok, that's not a problem. The
real problem is that I need to output these as fast as possible. The output
consists of big string blocks (>100.000 chars) that mostly contain only two
different characters.
I've ported the program from C. The original program doesn't use any sort
of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar())
printf("*.-");
Here are some simple benchmarks using the GNU time utility:
dmd test.c -o test -O3 -s
time ./test_c_printf > /dev/null
real 0m0.117s
dmd test.d -O -release -inline -version=printf
time ./test > /dev/null
real 0m0.140s
dmd test.d -O -release -inline -version=writef
time ./test > /dev/null
real 0m0.225s
How come writef() is so much slower than printf()?
Is there a fast way to do buffered output. I tried to use dynamic & static
arrays and printf the full arrays, but that wasn't actually any faster.
Oct 24 2005
Ben Hinkle wrote:printf and writef are already buffered - I believe stdout is line-buffered. Posting the actual code might help people help you. I also recommend using the -profile compiler option and/or std.perf. If you like you can also wrap a std.stream.Stream with a std.stream.BufferedStream but I suspect that you can do what you want with regular C buffering. See for example setvbuf.Yes, you're right. writef & printf are already buffered. One can see this with multithreaded CLI applications. Unfortunately the original code isn't mine. I need to ask for permission if I want to post some code here. Ok, I'll give a shot at trying the std.stream-functions. Thanks. The strangest thing here is that according to the time program the c-version does much greater amount of work in kernel mode. How is it possible at all that the D program seems to do the same I/O in user mode?
Oct 25 2005