www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to break from parallel foreach?

reply Andrey <saasecondbox yandex.ru> writes:
Hello,
How to break from parallel foreach?
More general question - how to control such loop?
Feb 26 2019
next sibling parent Alex <sascha.orlov gmail.com> writes:
On Tuesday, 26 February 2019 at 19:58:24 UTC, Andrey wrote:

 Hello,
 How to break from parallel foreach?
 More general question - how to control such loop?


Which expectation do you have on such a break?
Feb 26 2019
prev sibling next sibling parent reply Jordan Wilson <wilsonjord gmail.com> writes:
On Tuesday, 26 February 2019 at 19:58:24 UTC, Andrey wrote:

 Hello,
 How to break from parallel foreach?
 More general question - how to control such loop?


A basic way would be to use a flag:

shared stopWork=false;
foreach (wordBag; wordBags.parallel) {
     if (!stopWork) {
         // do work
         if (wordBag.canFind("myword")) stopWork=true
     }
}

I'd say it's probably not an elegant solution to "break" from 
parallel forloops, but that's up to you to decide.

Jordan
Feb 26 2019
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 02/26/2019 01:36 PM, Jordan Wilson wrote:

 On Tuesday, 26 February 2019 at 19:58:24 UTC, Andrey wrote:

 Hello,
 How to break from parallel foreach?
 More general question - how to control such loop?


 
 A basic way would be to use a flag:
 
 shared stopWork=false;
 foreach (wordBag; wordBags.parallel) {
      if (!stopWork) {
          // do work
          if (wordBag.canFind("myword")) stopWork=true
      }
 }
 
 I'd say it's probably not an elegant solution to "break" from parallel 
 forloops, but that's up to you to decide.
 
 Jordan
 
 


I assume stopWork needs to be checked periodically, which requires an 
inner loop. So, I wrote the following. break breaks from the inner loop, 
which ends the outer loop and that's what makes sense in this case.


import std.stdio;
import std.random;
import std.parallelism;
import std.datetime;
import std.range;
import core.thread;
import std.conv;

enum workerCount = 10;
enum workTime = 5.seconds;

bool someEndCondition(int oneInN) {
   return uniform(0, oneInN) == 0;
}

void main() {
   int[][workerCount] arrays;

   foreach (i; workerCount.iota.parallel) {
     enum sleepTime = 100.msecs;
     enum loopCount = workTime / sleepTime;

     foreach (j; loopCount.iota) {
       if (someEndCondition(loopCount / 2)) {
         break;
       }
       arrays[i] ~= j.to!int;
       Thread.sleep(sleepTime);
     }
     writefln!"Worker %s finished"(i);
   }

   foreach (i; workerCount.iota) {
     writefln!"Worker %s:\n%(%s %)"(i, arrays[i]);
   }
}

Ali
Feb 26 2019
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-02-26 at 21:36 +0000, Jordan Wilson via Digitalmars-d-learn
wrote:

 On Tuesday, 26 February 2019 at 19:58:24 UTC, Andrey wrote:

 Hello,
 How to break from parallel foreach?
 More general question - how to control such loop?


=20
 A basic way would be to use a flag:
=20
 shared stopWork=3Dfalse;
 foreach (wordBag; wordBags.parallel) {
      if (!stopWork) {
          // do work
          if (wordBag.canFind("myword")) stopWork=3Dtrue
      }
 }
=20
 I'd say it's probably not an elegant solution to "break" from=20
 parallel forloops, but that's up to you to decide.
=20
 Jordan
=20


Isn't this really an example of exactly how not to use parallel for loops? =
The
existence of the global data (with respect to the loop body) is surely
attempting to impose (and almost certainly failing) a sequential solution o=
n
what is a parallel computation.

Given the scatter/gather semantics of parallel for loops, each loop body wi=
ll
evaluate stopWork and act accordingly, i.e. they will all evaluate and the
gather (and possible reduction) happen.

Certainly, because of finite resources, there is an element of sequential
behaviour in a parallel for, but this is implementation detail. Conceptuall=
y,
all loop bodies of a parallel for are executed at the same time.=20

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Feb 26 2019
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-02-26 at 19:58 +0000, Andrey via Digitalmars-d-learn wrote:

 Hello,
 How to break from parallel foreach?
 More general question - how to control such loop?


It is not clear to me that the concept of "breaking" from a parallel loop
makes sense.

For a sequential loop each loop body is executed one after the other, and s=
o
the concept of "break" makes some sort of sense as an early exit.

For a parallel loop, each loop body is (at least conceptually) executed at =
the
same time, there is no sequential activity, thus no relevance of "break". A
parallel for is really a scatter of all the loop bodies which then execute,
and then there is a gather, and maybe even a reduction. The notion of "brea=
k"
seems to have no meaning in such a computation.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Feb 26 2019