• Oliver Dathe (34/34) Jul 16 2008 Hello,
• Saaa (5/5) Jul 18 2008 Sorry, it goes just a bit over my head..

Oliver Dathe <o.dathe gmx.de> writes:

Hello,

I just tryed to get a foreach parallelized in a neat way. This is an 
early preliminary implementation.

Here is how it can be used:

   foreach (i, item, ...; parallel(iterableobject)) {
     ...
   }

The code can be found in:
https://ftp.tu-ilmenau.de/index.php?id=03f3fba06a22658a2cca1793830d18a0&login=f4cfe3f230bc868842ad012d6e4d5e47

And here is how it works:
parallel(iterableobject) generates a thin wrapper around the object. It 
inspects typeof(iterableobject).opApply() regarding its parametertuple 
and generates an opApply with the same signature at compile time. This 
wrapping opApply calls the original opApply with an inner function 
fakedbody(), that has the same signature as the passed delegate. It 
packs the arguments, it is called with, in a CallBox for later 
execution, puts this on a thread that calls the original foreachbody 
with the given boxed arguments. The purpose of this arguments boxing 
with CallBox is that the stackframe/arguments may go out of scope or 
change context before an actuall call with the now boxed parameters may 
happend.

There is plenty of room for further discussion:

(1) Only the very first opApply of the iterable object can be wrapped. 
Is there a way to inspect the others, if there are more?

(2) Is there a more neat way to implement CallBox? Currently it parses 
ParameterTupleOf!(WrappedType.opApply).stringof and generates the 
wrapping put() and call() methods via mixin. Is there another way to 
check for each of the parameter if it is callbyreference or not.

(3) There are numerous ways to do the thread synchronization / job 
distribution. The goal should be to minimize the overhead.

(4) In a parallel foreach break should not be supported - any other 
opinions?

(5) Are there any other attempts around?

-Oliver

"Saaa" <empty needmail.com> writes:

Sorry, it goes just a bit over my head..
But I'd love to see a simple to use parallel foreach.

Maybe add a little example to show off the speed gains :)

Dsource scrapple might be a good place for this.
Although I'd wish more scrapple modules had examples.