www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Tango I/O bug?

reply Jason House <jason.james.house gmail.com> writes:
My friend using a mac has had severe issues getting my project (recently ported
to Tango) to run on his machine.

Looking at debug output and doing some tests, it appears that Stdout is not
flushing.  

Typical offending code:
auto outStream = new Print!(char)(new Layout!(char),Stdout);
outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes


Sometimes I get a non-response too, but restarting the program usually fixes
it.  It appears that on the mac this is not the case.  Any ideas what could be
wrong?  I do not know which version of tango my friend is using.  I think I'm
using Tango 0.99 rc1 with dmd 1.020.
Sep 17 2007
parent reply Sean Kelly <sean f4.ca> writes:
Jason House wrote:

 My friend using a mac has had severe issues getting my project (recently
ported to Tango) to run on his machine.
 
 Looking at debug output and doing some tests, it appears that Stdout is not
flushing.  
 
 Typical offending code:
 auto outStream = new Print!(char)(new Layout!(char),Stdout);


Is this really the line used?  The second parameter is supposed to be an 
OutputStream, and Stdout is not.


 outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes


Seems okay, though it's difficult to say from just that line.  Is output 
redirected?  Are objects being re-used?  Is .close() perhaps not being 
called if output is to a file?


Sean
Sep 17 2007
parent reply Jason House <jason.james.house gmail.com> writes:
Sean Kelly Wrote:


 Jason House wrote:

 Typical offending code:
 auto outStream = new Print!(char)(new Layout!(char),Stdout);


 
 Is this really the line used?  The second parameter is supposed to be an 
 OutputStream, and Stdout is not.



No, I did a quick hack to make it short.  The code was more generic with the
last input parameter being passed in from elsewhere.  I took a quick swag at it
without verifying with other code.  The real code compiles ;)

Here's the key lines from the class:

Print!(char) outStream;
this(InputStream _inStream=Cin.stream, OutputStream _outStream=Cout.stream,
double _pollPeriod=0.1){
  outStream = new Print!(char)(new Layout!(char),_outStream);
  ...



 outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes


 
 Seems okay, though it's difficult to say from just that line.  Is output 
 redirected?  Are objects being re-used?  Is .close() perhaps not being 
 called if output is to a file?



Output is going to Cout.stream and is never closed.  The code relies on
formatln to flush the output.  It does real time communication in plain text
with a 3rd party controlling application.  This line really was from the
program.  In this particular example, valueToReturn is of type char[] and is
literally "HouseBot"
Sep 17 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Jason House wrote:

 Sean Kelly Wrote:
 

 Jason House wrote:

 outStream.formatln("= {}\n", valueToReturn); // adds extra \n and flushes


 Seems okay, though it's difficult to say from just that line.  Is output 
 redirected?  Are objects being re-used?  Is .close() perhaps not being 
 called if output is to a file?


 
 
 Output is going to Cout.stream and is never closed.  The code relies on
formatln to flush the output.  It does real time communication in plain text
with a 3rd party controlling application.  This line really was from the
program.  In this particular example, valueToReturn is of type char[] and is
literally "HouseBot"


This part is likely your problem: "It does real time communication in 
plain text with a 3rd party controlling application"
That probably means the output is redirected and not sent to an actual 
console, right?
The default behavior for Stdout is to auto-flush only if the output is 
sent to an actual console (i.e. isn't redirected).

To make sure flushing occurs, execute "Stdout.flush = true;" somewhere 
at the start of your program (main(), a static this(), or just right 
before the first output).
Sep 17 2007
next sibling parent reply kris <foo bar.com> writes:
Frits van Bommel wrote:

 Jason House wrote:

 Sean Kelly Wrote:


 Jason House wrote:

 outStream.formatln("= {}\n", valueToReturn); // adds extra \n and 
 flushes


 Seems okay, though it's difficult to say from just that line.  Is 
 output redirected?  Are objects being re-used?  Is .close() perhaps 
 not being called if output is to a file?



 Output is going to Cout.stream and is never closed.  The code relies 
 on formatln to flush the output.  It does real time communication in 
 plain text with a 3rd party controlling application.  This line really 
 was from the program.  In this particular example, valueToReturn is of 
 type char[] and is literally "HouseBot"


 
 This part is likely your problem: "It does real time communication in 
 plain text with a 3rd party controlling application"
 That probably means the output is redirected and not sent to an actual 
 console, right?
 The default behavior for Stdout is to auto-flush only if the output is 
 sent to an actual console (i.e. isn't redirected).
 
 To make sure flushing occurs, execute "Stdout.flush = true;" somewhere 
 at the start of your program (main(), a static this(), or just right 
 before the first output).



Yeah, Stdout and Cout inhibit *automatic* flush when the console is 
redirected. If you think about it, the whole automatic flush thing is a 
'shortcut' so that output to the real console doesn't need an explicit 
call to flush() each time. It's a royal PITA to support that :)

Anyway, the most effective route here is simply to invoke flush() 
explicitly, whenever you actually need to. For example:
Sep 17 2007
parent reply Jason House <jason.james.house gmail.com> writes:
kris wrote:

 Anyway, the most effective route here is simply to invoke flush() 
 explicitly, whenever you actually need to. For example:
 




I thought I checked to verify that formatln calls newline which calls 
flush.  I therefore concluded that an explicit flush call was useless. 
My friend had the issue, so I couldn't really test.  When I hear more 
about testing fixes, I'll post back on the list.
Sep 17 2007
parent kris <foo bar.com> writes:
Jason House wrote:

 
 kris wrote:

 Anyway, the most effective route here is simply to invoke flush() 
 explicitly, whenever you actually need to. For example:




 
 
 I thought I checked to verify that formatln calls newline which calls 
 flush.  I therefore concluded that an explicit flush call was useless. 
 My friend had the issue, so I couldn't really test.  When I hear more 
 about testing fixes, I'll post back on the list.



Well, newline *does* call flush, but only if the console has not been 
redirected. This was not the case in some earlier Tango releases, but 
the change was made to accommodate requests for faster throughput (flush 
is very expensive on some platforms). In short, we may have caused your 
app to break. Please accept my apologies, and I hope it didn't cause too 
much trouble for you
Sep 17 2007
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Sadly, this simple fix isn't the solution.  Here's what I got back (via
e-mail)...


 This part is likely your problem: "It does real time communication in
 plain text with a 3rd party controlling application"
 That probably means the output is redirected and not sent to an actual
 console, right?
 The default behavior for Stdout is to auto-flush only if the output is
 sent to an actual console (i.e. isn't redirected).


Well, it is sent to an actual console. I mean I just start housebot
by hand
and interact with it.


 To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
 at the start of your program (main(), a static this(), or just right
 before the first output).


I tried that. I added the import for tango.io.Stdout to housebot.d
and added
"Stdout.flush = true;" at the beginning of main(), just after the
variable
declarations. But nothing changed.
Sep 18 2007
parent reply Jason House <jason.james.house gmail.com> writes:
Given that enabling a flush doesn't solve the problem, what is the path forward
for figuring out how to fix this?  It works on all other systems tested with
dmd, but doesn't work on mac (with gdc).  (I've had big problems getting gdc
and tango to play nicely with each other in a cygwin environment, but that's
probably another thread)

Jason House Wrote:


 Sadly, this simple fix isn't the solution.  Here's what I got back (via
e-mail)...
 

 This part is likely your problem: "It does real time communication in
 plain text with a 3rd party controlling application"
 That probably means the output is redirected and not sent to an actual
 console, right?
 The default behavior for Stdout is to auto-flush only if the output is
 sent to an actual console (i.e. isn't redirected).


 
 Well, it is sent to an actual console. I mean I just start housebot
 by hand
 and interact with it.
 

 To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
 at the start of your program (main(), a static this(), or just right
 before the first output).


 
 I tried that. I added the import for tango.io.Stdout to housebot.d
 and added
 "Stdout.flush = true;" at the beginning of main(), just after the
 variable
 declarations. But nothing changed.
Sep 18 2007
parent reply Sean Kelly <sean f4.ca> writes:
For what it's worth, flush() merely writes all data buffered by Tango to 
the underlying device.  On non-Win32 platforms, there is no actual 
kernel call performed to force an immediate flush of the data to 
wherever.  Is it possible that OSX does buffering within the kernel and 
the data is lingering in there?  And if so, do you know of a routine 
that could be called to perform a hard flush?  Typically, the commit() 
method is used for this purpose, but it's empty on Posix systems right now.

Jason House wrote:

 Given that enabling a flush doesn't solve the problem, what is the path
forward for figuring out how to fix this?  It works on all other systems tested
with dmd, but doesn't work on mac (with gdc).  (I've had big problems getting
gdc and tango to play nicely with each other in a cygwin environment, but
that's probably another thread)
 
 Jason House Wrote:
 

 Sadly, this simple fix isn't the solution.  Here's what I got back (via
e-mail)...


 This part is likely your problem: "It does real time communication in
 plain text with a 3rd party controlling application"
 That probably means the output is redirected and not sent to an actual
 console, right?
 The default behavior for Stdout is to auto-flush only if the output is
 sent to an actual console (i.e. isn't redirected).


 Well, it is sent to an actual console. I mean I just start housebot
 by hand
 and interact with it.


 To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
 at the start of your program (main(), a static this(), or just right
 before the first output).


 I tried that. I added the import for tango.io.Stdout to housebot.d
 and added
 "Stdout.flush = true;" at the beginning of main(), just after the
 variable
 declarations. But nothing changed.
Sep 18 2007
parent reply Jason House <jason.james.house gmail.com> writes:
You already know more than I do... even after a while trying to google 
on the topic.  What would be the right forum to even ask questions like 
that?

Sean Kelly wrote:

 For what it's worth, flush() merely writes all data buffered by Tango to 
 the underlying device.  On non-Win32 platforms, there is no actual 
 kernel call performed to force an immediate flush of the data to 
 wherever.  Is it possible that OSX does buffering within the kernel and 
 the data is lingering in there?  And if so, do you know of a routine 
 that could be called to perform a hard flush?  Typically, the commit() 
 method is used for this purpose, but it's empty on Posix systems right now.
 
 Jason House wrote:

 Given that enabling a flush doesn't solve the problem, what is the 
 path forward for figuring out how to fix this?  It works on all other 
 systems tested with dmd, but doesn't work on mac (with gdc).  (I've 
 had big problems getting gdc and tango to play nicely with each other 
 in a cygwin environment, but that's probably another thread)

 Jason House Wrote:


 Sadly, this simple fix isn't the solution.  Here's what I got back 
 (via e-mail)...


 This part is likely your problem: "It does real time communication in
 plain text with a 3rd party controlling application"
 That probably means the output is redirected and not sent to an actual
 console, right?
 The default behavior for Stdout is to auto-flush only if the output is
 sent to an actual console (i.e. isn't redirected).


 Well, it is sent to an actual console. I mean I just start housebot
 by hand
 and interact with it.


 To make sure flushing occurs, execute "Stdout.flush = true;" somewhere
 at the start of your program (main(), a static this(), or just right
 before the first output).


 I tried that. I added the import for tango.io.Stdout to housebot.d
 and added
 "Stdout.flush = true;" at the beginning of main(), just after the
 variable
 declarations. But nothing changed.
Sep 18 2007
parent reply kris <foo bar.com> writes:
Jason,

This is trivial to test, if you'd be prepared to do that? switch to the 
tango examples dir, and then the console dir (examples/console/)

Compile hello.d

execute it

What happens?
Sep 18 2007
parent reply Urban Hafner <urban bettong.net> writes:
kris wrote:

 Jason,


I guess it's easier if I start answering instead of having Jason relay 
messages.


 This is trivial to test, if you'd be prepared to do that? switch to the 
 tango examples dir, and then the console dir (examples/console/)
 
 Compile hello.d


That didn't quite work for me, with all the include and library files, 
so I instead did a "dsss build" in the examples directory.


 execute it
 
 What happens?


u koala:~/Programmieren/D/tango/example/console$ ./hello
hello, sweetheart ☺
u koala:~/Programmieren/D/tango/example/console$

Is that what should happen?


Urban
Sep 19 2007
next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Urban Hafner wrote:

 kris wrote:

 Jason,


 
 I guess it's easier if I start answering instead of having Jason relay 
 messages.
 

 This is trivial to test, if you'd be prepared to do that? switch to 
 the tango examples dir, and then the console dir (examples/console/)

 Compile hello.d


 
 That didn't quite work for me, with all the include and library files, 
 so I instead did a "dsss build" in the examples directory.
 

 execute it

 What happens?


 
 u koala:~/Programmieren/D/tango/example/console$ ./hello
 hello, sweetheart ☺
 u koala:~/Programmieren/D/tango/example/console$
 
 Is that what should happen?
 
 
 Urban


If I understand things correctly, Urban does see program output once the 
program exits...  He just doesn't see any while it's running.  When I 
look at Hello.d, it looks like it prints and then exits... potentially 
hiding this particular problem.
Sep 19 2007
parent kris <foo bar.com> writes:
Jason House wrote:


 
 If I understand things correctly, Urban does see program output once the 
 program exits...  He just doesn't see any while it's running.  When I 
 look at Hello.d, it looks like it prints and then exits... potentially 
 hiding this particular problem.



Well, the code does exactly the same thing when running or exiting. All 
the happens at exit is a flush() call on Stdout, just in case the user 
hadn't done one already (courtesy of Frits).

This, Any flush() should operate just fine. If the example had done 
nothing, it would have been more suspicious
Sep 19 2007
prev sibling parent kris <foo bar.com> writes:
Urban Hafner wrote:

 kris wrote:

 Jason,


 
 I guess it's easier if I start answering instead of having Jason relay 
 messages.
 

 This is trivial to test, if you'd be prepared to do that? switch to 
 the tango examples dir, and then the console dir (examples/console/)

 Compile hello.d


 
 That didn't quite work for me, with all the include and library files, 
 so I instead did a "dsss build" in the examples directory.
 

 execute it

 What happens?


 
 u koala:~/Programmieren/D/tango/example/console$ ./hello
 hello, sweetheart ☺
 u koala:~/Programmieren/D/tango/example/console$
 
 Is that what should happen?
 
 
 Urban



Yes, that's exactly what should happen :)
Sep 19 2007