www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - piping processes

reply "NMS" <nathanmswan gmail.com> writes:
Is there a cross-platform way to create a new process and get its i/o 
streams? Like java.lang.Process?
Nov 30 2011
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 30 Nov 2011 23:35:58 -0000, NMS <nathanmswan gmail.com> wrote:


 Is there a cross-platform way to create a new process and get its i/o  
 streams? Like java.lang.Process?


No.  It different on windows and unix platforms, tho most/many of the unix  
platforms are similar.

std.process is rather limited in it's current incarnation but I think  
Steve is working on a bit update..

I have misplaced the pipestream and processstream code I wrote ages ago  
which did what you want, otherwise I'd just send it to you.  So, best I  
can do is describe the process.

On windows you call Win32 functions, requiring you link with the  
appropriate windows libs - to find out which ones search for the function  
names I am about to describe in MSDN online.

To start a child process with input handles..
  - You call CreatePipe for each input (stdin, stdout, stderr) this gives  
you 2 handles for each pipe, one handle is for the child process, the  
other for the parent (think of the cans on string you used to use as a kid)
  - Pro tip; you now want to call DuplicateHandle on the parent handle for  
each pipe to remove 'inheritance' - this stops the child process  
inheriting the handle (which would result in 2 open handles to the pipe,  
and closure of the parent handle would not be detected by the child).   
After duplication you close the original parent side inheritable handles.
  - Call CreateProcess to start the process.  In flags specify  
STARTF_USESTDHANDLES and assign the child end of each of the pipes to each  
input.
  - Close the child pipe handles (in the parent) that were passed to  
CreateProcess.

To read from the child
  - Call PeekNamedPipe or ReadFile on the parent end of the pipe that was  
passed to the child as it's stdout.

To write to the child
  - Call WriteFile on the parent end of the pipe that was passed to the  
child as it's stdin.

To detect the child termination
  - CreateProcess returns a thread handle and a process handle, you can  
wait on either with WaitForSingleObject (I typically use the process  
handle and close the thread handle immediately after starting the child).


I haven't done this sort of thing on unix for ages and I have no good code  
to hand so someone else will have to help you out with this..

R


-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
We talked about that too on IRC - the conclusion was - we should have 
std.pipe module with (at least) AnonymousPipe, and NamedPipe classes.
Dec 01 2011
parent "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 01 Dec 2011 11:28:50 -0000, Dejan Lekic <dejan.lekic gmail.com>  
wrote:


 We talked about that too on IRC - the conclusion was - we should have
 std.pipe module with (at least) AnonymousPipe, and NamedPipe classes.


Definitely.  I will have a hunt for my misplaced code but it had some sort  
of pipe class which was used by the subprocess class or similar.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Dec 2011 06:10:00 -0500, Regan Heath <regan netmail.co.nz>  
wrote:


 On Wed, 30 Nov 2011 23:35:58 -0000, NMS <nathanmswan gmail.com> wrote:


 Is there a cross-platform way to create a new process and get its i/o  
 streams? Like java.lang.Process?


 No.  It different on windows and unix platforms, tho most/many of the  
 unix platforms are similar.

 std.process is rather limited in it's current incarnation but I think  
 Steve is working on a bit update..


It's ready for review, but I'm waiting on myself to get a pull request in  
for dmc...

Going to kick myself to do it now.


   - Pro tip; you now want to call DuplicateHandle on the parent handle  
 for each pipe to remove 'inheritance' - this stops the child process  
 inheriting the handle (which would result in 2 open handles to the pipe,  
 and closure of the parent handle would not be detected by the child).   
 After duplication you close the original parent side inheritable handles.


There's also SetHandleInformation (which I forgot to do in my  
implementation, thanks for bringing this up)

http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935(v=vs.85).aspx

-Steve
Dec 01 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Dec 2011 08:24:25 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Thu, 01 Dec 2011 06:10:00 -0500, Regan Heath <regan netmail.co.nz>  
 wrote:

 std.process is rather limited in it's current incarnation but I think  
 Steve is working on a bit update..


 It's ready for review, but I'm waiting on myself to get a pull request  
 in for dmc...

 Going to kick myself to do it now.


Just done, we'll see if Walter merges anytime soon.

-Steve
Dec 01 2011
prev sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
NMS wrote:


 Is there a cross-platform way to create a new process and get its i/o
 streams? Like java.lang.Process?


I doubt. However, you can use platform-specific popen() (POSIX) and _popen() 
(Windows) for that. I recently talked about this on IRC. At the moment 
Phobos uses system() , redirects the output of a command to a temp file, and 
reads the file (that is how the shell() is implemented on Windows). My 
question was - why not using _popen() ???
Dec 01 2011
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I can't find my thread in the archives yet, but I made a similar
inquiry a few days ago.

You can see a few examples for Windows here:
https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandle
https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandleThreaded
Dec 01 2011
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Thu, 01 Dec 2011 11:29:52 -0000, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 I can't find my thread in the archives yet, but I made a similar
 inquiry a few days ago.

 You can see a few examples for Windows here:
 https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandle
 https://github.com/AndrejMitrovic/DWinProgramming/tree/master/Samples/Extra/RedirectChildHandleThreaded


Oddly I haven't used the SetHandleInformation function for this.. I wonder  
if it arrived /after/ I learnt all this :p

I know these are basic examples, but I think in RedirectChildHandle.d,  
CreateChildProcess, you should (ideally) be closing childStdoutWrite and  
childStdinRead /after/ CreateProcess.  If you don't you get 2 copies of  
them, one in the child and one in the parent.  If the child closes it's  
copy the parent will not notice the pipe close (when reading from the  
other end of the pipe - for example).  If you close it, it will mean that  
you will actually drop out of a blocking read in the parent immediately  
(when the child terminates), rather than timing out and then discovering  
the child has terminated (using the thread or process handle).

Regan

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Dec 01 2011
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/1/11, Regan Heath <regan netmail.co.nz> wrote:

 I know these are basic examples, but I think in RedirectChildHandle.d,
 CreateChildProcess, you should (ideally) be closing childStdoutWrite and
 childStdinRead /after/ CreateProcess.  If you don't you get 2 copies of
 them, one in the child and one in the parent.  If the child closes it's
 copy the parent will not notice the pipe close (when reading from the
 other end of the pipe - for example).  If you close it, it will mean that
 you will actually drop out of a blocking read in the parent immediately
 (when the child terminates), rather than timing out and then discovering
 the child has terminated (using the thread or process handle).


That example comes straight from MSDN's documentation, although I
might have made some modifications. Thanks for the info though, I'll
see about fixing it.
Dec 01 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
 On 12/1/11, Regan Heath <regan netmail.co.nz> wrote:

 you should (ideally) be closing childStdoutWrite and
 childStdinRead /after/ CreateProcess.  If you don't you get 2 copies of
 them, one in the child and one in the parent.




Ok so I should move CloseHandle(childStdoutWrite) immediately after a
successful call to CreateProcess? And add CloseHandle for
childStdinRead. I've noticed now childStdinRead was never even closed.
Dec 01 2011