www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Are spawn'ed threads waited automatically?

reply =?ISO-8859-1?Q?Ali_=C7ehreli?= <acehreli yahoo.com> writes:
First, the answer may be as simple as "use core.thread.thread_joinAll". 
Is that the proper way of waiting for all threads?

Second, my question may not be a valid example as starting a thread 
without communicating with it may be under the umbrella of 
parallelization. Maybe in concurrency, threads communicate with each 
other so that the following situation should not occur in practice.

Third is my question: :) If I spawn a single thread in main, the single 
thread seems to run to completion.

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
     foreach (i; 0 .. 5) {
         Thread.sleep(dur!"msecs"(500));
         writeln(i, " foo");
     }
}

void main()
{
     spawn(&foo);
     writeln("main done");
}

I get all of foo's output after "main done":

main done
0 foo
1 foo
2 foo
3 foo
4 foo

If I introduce an intermediate thread that spawns the foo thread, now 
foo sometimes terminates early:

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
     foreach (i; 0 .. 5) {
         Thread.sleep(dur!"msecs"(500));
         writeln(i, " foo");
     }
}

void intermediate()
{
     spawn(&foo);
     writeln("intermediate done");
}

void main()
{
     spawn(&intermediate);
     writeln("main done");
}

The output is inconsistent. Sometimes there is nothing from foo:

main done
intermediate done

Sometimes it runs fully:

main done
intermediate done
0 foo
1 foo
2 foo
3 foo
4 foo

Is the inconsistency a bug or a natural consequence of something? :) 
(Perhaps even the first example that seems to run correctly just has a 
higher probability of showing this behavior.)

I am aware of thread_joinAll(). Is that the recommended way of waiting 
for all threads?

Thank you,
Ali
Jun 06 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> wrote:


 First, the answer may be as simple as "use core.thread.thread_joinAll".  
 Is that the proper way of waiting for all threads?


main (the C main, not D main) does this already:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512

But note that "daemonized" threads will not be included:

http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon

-Steve
Jun 06 2011
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:

 On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> wrote:


 First, the answer may be as simple as "use
 core.thread.thread_joinAll". Is that the proper way of waiting for all
 threads?


 main (the C main, not D main) does this already:

 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512


 But note that "daemonized" threads will not be included:

 http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon

 -Steve


Thank you but it doesn't explain the inconsistent behavior. It seems 
like thread_joinAll() has a different idea at different times. Now I 
also print the result of isDaemon():

import std.stdio;
import std.concurrency;
import core.thread;

void tell_daemon_state(string name)
{
     writeln(name, " isDaemon: ", Thread.getThis.isDaemon);
}

void foo()
{
     tell_daemon_state("foo");

     foreach (i; 0 .. 5) {
         Thread.sleep(dur!"msecs"(500));
         writeln(i, " foo");
     }
}

void intermediate()
{
     tell_daemon_state("intermediate");

     spawn(&foo);
     writeln("intermediate done");
}

void main()
{
     tell_daemon_state("main");

     spawn(&intermediate);
     writeln("main done");
}

I see that only the main thread is a daemon:

main isDaemon: true
main done
intermediate isDaemon: false
intermediate done
foo isDaemon: false
0 foo
1 foo
2 foo
3 foo
4 foo

That makes sense.

There is a race condition: Just because I added the printing of the 
isDaemon state, now foo() runs to completion seemingly everytime I start 
the program. When I remove the printing AND run the program under 
'time', I get inconsistent behavior.

The following are two consecutive runs:

$ time ./deneme
main done
intermediate done
0 foo                    <--- foo()'s output is present
1 foo
2 foo
3 foo
4 foo

real	0m2.504s
user	0m0.000s
sys	0m0.000s

$ time ./deneme
main done
intermediate done        <--- foo()'s output is missing

real	0m0.003s
user	0m0.000s
sys	0m0.000s

As if thread_joinAll() misses the fact that there is still the 
non-daemon foo() thread.

Note that it's not failing to flush stdout either. The program runs 
shorter in the case where foo()'s output is missing.

Thank you,
Ali
Jun 06 2011
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-06-06 14:37, Ali Çehreli wrote:

 On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:

 On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> 




wrote:

 First, the answer may be as simple as "use
 core.thread.thread_joinAll". Is that the proper way of waiting for all
 threads?


 
 main (the C main, not D main) does this already:
 
 https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dma
 in2.d#L512
 
 
 But note that "daemonized" threads will not be included:
 
 http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon
 
 -Steve


 
 Thank you but it doesn't explain the inconsistent behavior. It seems
 like thread_joinAll() has a different idea at different times. Now I
 also print the result of isDaemon():
 
 import std.stdio;
 import std.concurrency;
 import core.thread;
 
 void tell_daemon_state(string name)
 {
 writeln(name, " isDaemon: ", Thread.getThis.isDaemon);
 }
 
 void foo()
 {
 tell_daemon_state("foo");
 
 foreach (i; 0 .. 5) {
 Thread.sleep(dur!"msecs"(500));
 writeln(i, " foo");
 }
 }
 
 void intermediate()
 {
 tell_daemon_state("intermediate");
 
 spawn(&foo);
 writeln("intermediate done");
 }
 
 void main()
 {
 tell_daemon_state("main");
 
 spawn(&intermediate);
 writeln("main done");
 }
 
 I see that only the main thread is a daemon:
 
 main isDaemon: true
 main done
 intermediate isDaemon: false
 intermediate done
 foo isDaemon: false
 0 foo
 1 foo
 2 foo
 3 foo
 4 foo
 
 That makes sense.
 
 There is a race condition: Just because I added the printing of the
 isDaemon state, now foo() runs to completion seemingly everytime I start
 the program. When I remove the printing AND run the program under
 'time', I get inconsistent behavior.
 
 The following are two consecutive runs:
 
 $ time ./deneme
 main done
 intermediate done
 0 foo <--- foo()'s output is present
 1 foo
 2 foo
 3 foo
 4 foo
 
 real 0m2.504s
 user 0m0.000s
 sys 0m0.000s
 
 $ time ./deneme
 main done
 intermediate done <--- foo()'s output is missing
 
 real 0m0.003s
 user 0m0.000s
 sys 0m0.000s
 
 As if thread_joinAll() misses the fact that there is still the
 non-daemon foo() thread.
 
 Note that it's not failing to flush stdout either. The program runs
 shorter in the case where foo()'s output is missing.


Unless the code has changed (and Sean was working on it a couple of months 
back, so I'm not sure what the current state is), on Linux, none of the 
spawned threads ever get joined, and they're all joinable - which causes 
problems. As I understand it, they should all be detached (as in the pthread 
concept of detached, not detached from the GC like core.Thread talks about) 
rather than joinable.

At this point, I don't trust spawn at all (on Linux at least). I've had too 
many problems with it. But I don't know what the current state is, because 
Sean was at least working on improving the situation. It's possible that the 
joinable issues and whatnot were worked out, but I don't know and kind of 
doubt it.

Regardless, spawned threads aren't intended to be joined by you. They should 
run until they're done doing whatever they're doing and then exit. And the 
program should wait for them all to exit, even if main finishes. If that's not 
happening, then there are bugs that need to be fixed. You shouldn't ever have 
to worry about joining spawned threads.

- Jonathan M Davis
Jun 06 2011
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/06/2011 03:52 PM, Jonathan M Davis wrote:


 At this point, I don't trust spawn at all (on Linux at least). I've 


had too

 many problems with it.


Thank you.

I've spawned threads from within threads and now received 
ThreadException and segmentation faults as well:

   http://d.puremagic.com/issues/show_bug.cgi?id=6116

Ali
Jun 06 2011