• Cameron Reid (11/11) Nov 23 2015 I'm rather new to D, so apologies if this is a silly question:
• Quentin Ladeveze (5/16) Nov 23 2015 Take a look at message passing concurrency
• Adam D. Ruppe (51/53) Nov 23 2015 Yes, though the D stdlib doesn't help a whole lot, unless you
• Jesse Phillips (7/18) Nov 24 2015 pipeProcess will give you the input/output pair (and believe the

Cameron Reid <csreid.cr gmail.com> writes:

I'm rather new to D, so apologies if this is a silly question:

I'd like to be able to fork a number of instances of a process, 
write to their stdins and read from their stdouts in parallel. 
That is, I want to write some data to the stdin of all these 
processes and collect lines written to their stdouts as soon as 
they're emitted by the process.

I tried for quite a while last night using permutations of 
pipeProcess and i/o redirection but nothing did exactly what I 
was looking for.

Is such a thing possible? If so, where might I go to educate 
myself?

Quentin Ladeveze <ladeveze.quentin openmailbox.org> writes:

On Monday, 23 November 2015 at 20:02:16 UTC, Cameron Reid wrote:
 I'm rather new to D, so apologies if this is a silly question:

 I'd like to be able to fork a number of instances of a process, 
 write to their stdins and read from their stdouts in parallel. 
 That is, I want to write some data to the stdin of all these 
 processes and collect lines written to their stdouts as soon as 
 they're emitted by the process.

 I tried for quite a while last night using permutations of 
 pipeProcess and i/o redirection but nothing did exactly what I 
 was looking for.

 Is such a thing possible? If so, where might I go to educate 
 myself?

Take a look at message passing concurrency 
(http://ddili.org/ders/d.en/concurrency.html). You can make a 
program that automatically send what its stdin as a message to 
the owner thread and that can receive messages to print to stdout

Adam D. Ruppe <destructionator gmail.com> writes:

On Monday, 23 November 2015 at 20:02:16 UTC, Cameron Reid wrote:
 Is such a thing possible? If so, where might I go to educate 
 myself?

Yes, though the D stdlib doesn't help a whole lot, unless you 
want to use threads and that's blargh, I hate using threads and 
recommend you avoid them when you can. If you do want to try them 
you can just spawn a new thread for each child then block until 
it talks to you.

But without threads, well, Phobos helps a little, but you'll want 
to get platform-specific to go the rest of the way (and at that 
point, you might as well just use the OS functions all the way 
down but whatever).

Creating the process is simple enough: you can DIY with 
pipe/fork/exec/dup2 (on Posix), CreatePipe/CreateProcess (on 
Windows), or go ahead and let Phobos help you with pipeProcess.

Whatever you're more comfortable with at that point. Now the 
program is running and you have a handle to the pipes to talk to 
it.

Next, you want to be alerted when one of them is ready. This is 
where Phobos won't help much (though a lib like vibe.d might, I 
don't know though) and you want to use an operating system 
function.

On Posix, one of the select/poll family of functions will do what 
you want. If you have just a handful of processes, select is easy 
enough to use and will do the job. On Windows, you'll want 
WaitForMultipleObjects which works similarly (or you could use 
overlapped I/O on the pipe which is actually pretty elegant but 
pretty different to use too).

These functions take an array of file numbers/file handles and 
tells you when one of them is ready to read. If you pass all your 
read pipes from the children to them, you'll be alerted when any 
of them writes back to you. You will handle them serially when a 
message comes in (so you prolly want to handle it quickly if you 
can), but all the children run in parallel.

I find this generally easier to code than the threaded solution 
and it is also generally more efficient while keeping a lot of 
the same benefits.



Anyway, if you used Phobos' pipeProcess to create the pipes, you 
get the platform-specific handles through these functions: 

and File.windowsHandle for Windows (use `version(Windows) { win 
version } else version(Posix) { posix version } else static 
assert(0, "unsupported OS");`).


Once you have the handles, you call the OS functions just like 
you would in C. If you need an example, I can write one up, but I 
suggest just looking up a C example and remembering:

import core.sys.posix.sys.select; // same as 
#include<sys/select.h> in a C example

import core.sys.windows.windows; // has WaitForMultipleObjects 
from C in there


will get you started. So import them, add the files to your list, 
then call the function so the OS waits and tells you when any of 
them are ready, then loop over it and handle the input again.

Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:

On Monday, 23 November 2015 at 20:02:16 UTC, Cameron Reid wrote:
 I'm rather new to D, so apologies if this is a silly question:

 I'd like to be able to fork a number of instances of a process, 
 write to their stdins and read from their stdouts in parallel. 
 That is, I want to write some data to the stdin of all these 
 processes and collect lines written to their stdouts as soon as 
 they're emitted by the process.

 I tried for quite a while last night using permutations of 
 pipeProcess and i/o redirection but nothing did exactly what I 
 was looking for.

 Is such a thing possible? If so, where might I go to educate 
 myself?

pipeProcess will give you the input/output pair (and believe the 
output is accessible at the available time the program writes a 
line). To publish the same input to multiple pipes will be 
something have to create along with a receiver of multiple 
outputs.

That is to say, you can do it, but managing read/write is on you.