• Anthony (24/28) Jan 28 2021 ```
• =?UTF-8?Q?Ali_=c3=87ehreli?= (44/47) Jan 28 2021 That would work if `cat` received the *contents* of the files (and with
• Anthony (41/93) Jan 28 2021 Thanks Ali.
• =?UTF-8?Q?Ali_=c3=87ehreli?= (8/20) Jan 28 2021 Cool but there should be one improvement because I don't think end() is
• Anthony (10/34) Jan 28 2021 I take it you're referring to missing scope guards like in your

Anthony <anthoq88 gmail.com> writes:

This post https://dlang.org/library/std/process/pipe.html 
mentions:

Pipes can, for example, be used for interprocess communication 
by spawning a new process and passing one end of the pipe to the 
child, while the parent uses the other end. (See also 
pipeProcess and pipeShell for an easier way of doing this.)

```
auto p = pipe();
auto outFile = File("D downloads.txt", "w");
auto cpid = spawnProcess(["curl", 
"http://dlang.org/download.html"],
                          stdin, p.writeEnd);
scope(exit) wait(cpid);
auto gpid = spawnProcess(["grep", "-o", `http://\S*\.zip`],
                          p.readEnd, outFile);
scope(exit) wait(gpid);
```

How am I able to compose the pipes from pipeProcess if I can't 
pass in an existing pipe into the function. Eg.

```
auto p = pipeProcess("ls");
auto q = pipeProcess("cat", stdin = p.stdout); //it would be good 
to do this or something like it
```

Do I need to manually extract the output from pipes.stdin.readln 
and place it in pipes.stdout.writeln?
Or is there an easier way to do this?

Thanks

=?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:

On 1/28/21 2:16 AM, Anthony wrote:

 auto p = pipeProcess("ls");
 auto q = pipeProcess("cat", stdin = p.stdout); //it would be good to do

That would work if `cat` received the *contents* of the files (and with 
a "-" command line switch). Since `ls` produces file names, you would 
have to make the complete `cat` command line from `ls`'s output.

 Do I need to manually extract the output from pipes.stdin.readln

Seems to be so for the `ls | cat` case. But the following `find | grep` 
example shows how two ends of pipes can be connected:

import std.stdio;
import std.process;
import std.range;

// BONUS: Enable one of the following lines to enjoy an issue.
// version = bonus_bug;
// version = bonus_bug_but_this_works;

void main() {
   // Writes to 'a':
   auto a = pipe();
   auto lsPid = spawnProcess([ "find", "."], stdin, a.writeEnd);
   scope (exit) wait(lsPid);

   // Reads from 'a', writes to 'b':
   auto b = pipe();
   auto catPid = spawnProcess([ "grep", "-e", `\.d$` ], a.readEnd, 
b.writeEnd);
   scope (exit) wait(catPid);

   version (bonus_bug) {
     // Fails with the following error.
     //
     // "/usr/include/dmd/phobos/std/typecons.d(6540): Error:
     // `"Attempted to access an uninitialized payload."`"
     writefln!"Some of the D source files under the current 
directory:\n%-(  %s\n%)"(
       b.readEnd.byLine);

   } else version (bonus_bug_but_this_works) {
     // Note .take at the end:
     writefln!"Some of the D source files under the current 
directory:\n%-(  %s\n%)"(
       b.readEnd.byLine.take(1000));

   } else {
     // The results are read from 'b':
     writeln(b.readEnd.byLine);
   }
}

I've discovered a strange issue, which can be observed by uncommenting 
the 'version = bonus_bug;' line above. But comment that one back in and 
uncomment the next line, now it works. (?)

Ali

Anthony <anthoq88 gmail.com> writes:

On Thursday, 28 January 2021 at 17:18:46 UTC, Ali Çehreli wrote:
 On 1/28/21 2:16 AM, Anthony wrote:

 auto p = pipeProcess("ls");
 auto q = pipeProcess("cat", stdin = p.stdout); //it would be

 good to do

 That would work if `cat` received the *contents* of the files 
 (and with a "-" command line switch). Since `ls` produces file 
 names, you would have to make the complete `cat` command line 
 from `ls`'s output.

 Do I need to manually extract the output from

 pipes.stdin.readln

 Seems to be so for the `ls | cat` case. But the following `find 
 | grep` example shows how two ends of pipes can be connected:

 import std.stdio;
 import std.process;
 import std.range;

 // BONUS: Enable one of the following lines to enjoy an issue.
 // version = bonus_bug;
 // version = bonus_bug_but_this_works;

 void main() {
   // Writes to 'a':
   auto a = pipe();
   auto lsPid = spawnProcess([ "find", "."], stdin, a.writeEnd);
   scope (exit) wait(lsPid);

   // Reads from 'a', writes to 'b':
   auto b = pipe();
   auto catPid = spawnProcess([ "grep", "-e", `\.d$` ], 
 a.readEnd, b.writeEnd);
   scope (exit) wait(catPid);

   version (bonus_bug) {
     // Fails with the following error.
     //
     // "/usr/include/dmd/phobos/std/typecons.d(6540): Error:
     // `"Attempted to access an uninitialized payload."`"
     writefln!"Some of the D source files under the current 
 directory:\n%-(  %s\n%)"(
       b.readEnd.byLine);

   } else version (bonus_bug_but_this_works) {
     // Note .take at the end:
     writefln!"Some of the D source files under the current 
 directory:\n%-(  %s\n%)"(
       b.readEnd.byLine.take(1000));

   } else {
     // The results are read from 'b':
     writeln(b.readEnd.byLine);
   }
 }

 I've discovered a strange issue, which can be observed by 
 uncommenting the 'version = bonus_bug;' line above. But comment 
 that one back in and uncomment the next line, now it works. (?)

 Ali


Thanks Ali.
I was messing around and below seems to work well enough for me.

```

struct AccumulatorPipe {
     Pid[] pids;

     File stdin;
     File stdout;
}

AccumulatorPipe run(string cmd) {
     AccumulatorPipe acc;

     auto p = P.pipeShell(cmd, P.Redirect.stdout);

     acc.pids ~= p.pid;
     acc.stdout = p.stdout;

     return acc;
}

AccumulatorPipe pipe(AccumulatorPipe acc0, string cmd) {
     AccumulatorPipe acc;
     Pipe p = P.pipe();

     acc.stdin = p.writeEnd;
     acc.stdout = p.readEnd;

     auto pid = P.spawnShell(cmd, acc0.stdout, acc.stdin);
     acc.pids = acc0.pids ~ pid;

     return acc;
}

void end(AccumulatorPipe acc) {
     auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);

     foreach (pid; pids) {
         P.wait(pid);
     }
}
```


So now I can do something like:
```
run("find source -name '*.d'")
         .pipe("entr ./make.d tests")
         .end(),
```


 That would work if `cat` received the *contents* of the files 
 (and with a "-" command line switch)

I was actually trying to use cat to just spit out the filenames 
of the directory as a test.
But I see what you mean.

=?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:

On 1/28/21 3:45 PM, Anthony wrote:

 void end(AccumulatorPipe acc) {
      auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);

      foreach (pid; pids) {
          P.wait(pid);
      }
 }
 ```


 So now I can do something like:
 ```
 run("find source -name '*.d'")
          .pipe("entr ./make.d tests")
          .end(),

Cool but there should be one improvement because I don't think end() is 
guaranteed to be executed in that code, which may leave zombie processes 
around. From 'man waitpid':

   "A child that terminates, but has not been waited for becomes a "zombie".

Which is relayed to std.process documentation as "to avoid child 
processes becoming "zombies"".

Ali

Anthony <anthoq88 gmail.com> writes:

On Friday, 29 January 2021 at 03:49:38 UTC, Ali Çehreli wrote:
 On 1/28/21 3:45 PM, Anthony wrote:

 void end(AccumulatorPipe acc) {
      auto pids = acc.pids ~ P.spawnShell("cat", acc.stdout);

      foreach (pid; pids) {
          P.wait(pid);
      }
 }
 ```


 So now I can do something like:
 ```
 run("find source -name '*.d'")
          .pipe("entr ./make.d tests")
          .end(),

 Cool but there should be one improvement because I don't think 
 end() is guaranteed to be executed in that code, which may 
 leave zombie processes around. From 'man waitpid':

   "A child that terminates, but has not been waited for becomes 
 a "zombie".

 Which is relayed to std.process documentation as "to avoid 
 child processes becoming "zombies"".

 Ali

I take it you're referring to missing scope guards like in your 
code
`scope (exit) wait(lsPid);`

Yeah, that is a tricky one. I can't think of a way to have a nice 
interface that also closes the pids on exit since scope guards 
are handled on function exit in this case.

Perhaps thats just the nature of the problem though.

I'll take a look at what scriptlike does 
https://github.com/Abscissa/scriptlike#script-style-shell-commands