www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threads not independent...

reply Larry Cowan <Larry_member pathlink.com> writes:
| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[i].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can thnk of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?
Apr 04 2005
next sibling parent Larry Cowan <Larry_member pathlink.com> writes:
Corrected error in code given, bad typing...

In article <d2sdsg$9ca$1 digitaldaemon.com>, Larry Cowan says...

| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[j].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can think of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?
Apr 04 2005
prev sibling next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:

 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?


In the above all the unnamed delegates share the same stack. Containing  
the same 'i' variable.

The dodgy bit, from where I'm sitting is that 'i' only exists for the  
duration of the for loop, so by the time you 'start' the threads... I must  
be missing something here? Perhaps the stack is not 'free'd (?) unless no  
more 'refences'(?) to it exist?


 Do I
 have to make one for them on the heap to actually get independent  
 activity,
 then have them pick up an index from a locked counter?


I would write a class i.e.

class Foo {
   int index;
   this(int i) { index = i; }
   .. run() ..
}

then

|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(new Foo(i).run);

(not sure if the above is legal, but you get the idea?)

This should create a delegate that's object pointer points to a Foo class  
which has it's own 'index' integer, not shared between all the threads.

You could instead derive a clas from thread eg.

class Foo : Thread {
   int index;
   this(int i, ?delegate?) { super(?delegate?); index = i; }
   ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?)  
type method?
}

"?delegate?" above as I cannot remember the syntax off hand.

Regan
Apr 04 2005
next sibling parent reply Larry Cowan <Larry_member pathlink.com> writes:
See below.

In article <opsoqa0osi23k2f5 nrage.netwin.co.nz>, Regan Heath says...

On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:

 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?


In the above all the unnamed delegates share the same stack. Containing  
the same 'i' variable.

The dodgy bit, from where I'm sitting is that 'i' only exists for the  
duration of the for loop, so by the time you 'start' the threads... I must  
be missing something here? Perhaps the stack is not 'free'd (?) unless no  
more 'refences'(?) to it exist?


Its not "free"ed, but it is reused, maybe j then uses the same place - the
effect would be the same, but I can test this.  Anyway, I know what is
happening re the main() sction of the stack, but I was asking for an idea
about how best to staticize the index for each thread, and I will try your
"new"ing it idea, but the ref is still in main()'s stack area and may have
the same problem.


 Do I
 have to make one for them on the heap to actually get independent  
 activity,
 then have them pick up an index from a locked counter?




Also I wanted assurance that when the threads actually
start they have independent stacks without my help - not just as if they
were called by main() and got the next stack frame - each getting the same
one.


I would write a class i.e.

class Foo {
   int index;
   this(int i) { index = i; }
   .. run() ..
}

then

|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(new Foo(i).run);

(not sure if the above is legal, but you get the idea?)

This should create a delegate that's object pointer points to a Foo class  
which has it's own 'index' integer, not shared between all the threads.

You could instead derive a clas from thread eg.

class Foo : Thread {
   int index;
   this(int i, ?delegate?) { super(?delegate?); index = i; }
   ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?)  
type method?
}

"?delegate?" above as I cannot remember the syntax off hand.

Regan


I'll report back after some more experimenting...

-Larry
Apr 04 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 4 Apr 2005 22:42:21 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:

 In article <opsoqa0osi23k2f5 nrage.netwin.co.nz>, Regan Heath says...

 On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan
 <Larry_member pathlink.com> wrote:

 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but  
 they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?


 In the above all the unnamed delegates share the same stack. Containing
 the same 'i' variable.

 The dodgy bit, from where I'm sitting is that 'i' only exists for the
 duration of the for loop, so by the time you 'start' the threads... I  
 must
 be missing something here? Perhaps the stack is not 'free'd (?) unless  
 no
 more 'refences'(?) to it exist?


 Its not "free"ed, but it is reused, maybe j then uses the same place -  
 the
 effect would be the same, but I can test this.


Good point. This seems likely to me.


 Anyway, I know what is
 happening re the main() sction of the stack, but I was asking for an idea
 about how best to staticize the index for each thread, and I will try  
 your
 "new"ing it idea, but the ref is still in main()'s stack area and may  
 have
 the same problem.


No, my ideas do not use main()'s stack area. Each class is allocated on  
the heap, and the reference to the class is not stored except in the  
delegate passed to the threads constructor. (or in t[] if you extend  
Thread with your own class - which still means it is not stored in mains  
stack area - only the t[] reference is).

Either way the stack frame of the thread code is not main() it is the  
stack frame of the function in the class.


 Do I
 have to make one for them on the heap to actually get independent
 activity,
 then have them pick up an index from a locked counter?




 Also I wanted assurance that when the threads actually
 start they have independent stacks without my help - not just as if they
 were called by main() and got the next stack frame - each getting the  
 same
 one.


IIRC creating an unnamed delegate assigns the current stack frame to the  
delegates object pointer.
So, creating an unnamed delegate in "main()" assigns mains stack frame to  
the delegate.
A class method delegate has it's object pointer assigned to the class  
instance.
The stack frame for a class delegate is the stack frame created when the  
method is called.


 I would write a class i.e.

 class Foo {
   int index;
   this(int i) { index = i; }
   .. run() ..
 }

 then

 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(new Foo(i).run);

 (not sure if the above is legal, but you get the idea?)

 This should create a delegate that's object pointer points to a Foo  
 class
 which has it's own 'index' integer, not shared between all the threads.

 You could instead derive a clas from thread eg.

 class Foo : Thread {
   int index;
   this(int i, ?delegate?) { super(?delegate?); index = i; }
   ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?)
 type method?
 }

 "?delegate?" above as I cannot remember the syntax off hand.

 Regan


 I'll report back after some more experimenting...


I am interested to know what you find. I have done very little  
experimentation with threads myself, so my advice may not be entirely  
acurate. IIRC Russ Lewis is the person I see most often posting about  
delegates and stack frames, his advice you can trust (I believe).

Regan
Apr 04 2005
prev sibling parent reply Larry Cowan <Larry_member pathlink.com> writes:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;




+| class Index { int n; this(int n) { this.n = n; } }

 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
+|    for (int i=0; i < 5 ;i++) {




+|        Index ix = new Index(i);

 |        t[i] = new Thread(
 |            delegate int() {
+|                printf("  thread %d\n",ix.n);
 |                allDone.countDown();
 |                return 0;
+|            });




+|    int test = 99;

 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[j].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

+++++++ Now prints:

 |Starting threads.
 |  thread 4
 |  thread 4
 |  thread 4
 |  thread 4
 |  thread 4
 |All done.




So they all ref the same (last one) new'ed Index.  The ref apparently is still
sitting on the stack in main()'s frame, even though "test" should have stomped
on it as I understand D's interspersed allocation method - apparently the
internal blocks get their own frame above the main() one, and they still are
there, but there is space reserved in main ()'s for the "test" allocation.


 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?




New try with results below:

|// dmd test -L locks.lib
|module test;
|import std.thread;
|import std.stdio;
|import std.c.time;
|import locks.countdown;
|
||class Indexed {
|
|    private int n;
|    private Thread t;
|    private CountDownLatch* cdlp;
|
|    this ( int n, CountDownLatch* cdlp )
|    {
|        this.n = n;
|        this.cdlp = cdlp;
|        this.t = new Thread(delegate int() {
|                                printf("  thread %d\n",n);
|                                cdlp.countDown();  /* (Access Violation */
|                                return 0;
|                                           }
|                           );
|    }
|    void run () { t.start(); }
|}
|
|void main ( char[][] args )
|{
|    CountDownLatch allDone = new CountDownLatch(5);
|    Indexed[5] ixt;
|
|    for (int i=0; i < 5 ;i++)
|        ixt[i] = new Indexed(i,&allDone);
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        ixt[j].run();
|
|//    allDone.wait(); --> (never ends)

|    printf("All done.\n");
|}

New output is:

|Starting threads.
|  thread 7
|Error: Access Violation
|  thread 7
|Error: Access Violation
|  thread 7
|Error: Access Violation
|  thread 7
|Error: Access Violation
|  thread 7
|Error: Access Violation
|All done.

This is just weird - I'm abandoning this path.  See below:


From Ben Hinkle:



Glad to see someone using the Locks lib. When I was testing things out I ran 
into the same conundrum you did. A simple way (eg not very generic but it 
gets the job done) of solving the problem is to loop over t looking for the 
thread in question:

t[i] = new Thread(
 delegate int() {
  int n;
  for(n=0;n<5;n++) {
    if (Thread.getThis() is t[n]) break;
  }
  printf("  thread %d\n",n);
  allDone.countDown();
  return 0;
});


I am trying to pass a work index into the thread, and this will probably solve
that part.  It works for me as you coded above.

Thanks for all the mintl stuff, especially the concurrent and locks.  I don't
understand the trouble I had with CountDownLatch - I haven't investigated it
yet, but it does work in your method above.

I still have some problems in my multithreaded multiple ping, but I think I'm
getting independent thread activity now.

-Larry
Apr 04 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Try this:

import std.c.windows.windows;
import std.thread;
import std.stdio;

class RThread : Thread
{
	int index;
	
	this(int i)
	{
		super(&run);
		index = i;
	}

	int run()
	{
		writefln("thread %d",index);
		//allDone.countDown();
		return 0;
	}
}

int main(char[][] args)
{
	//CountDownLatch allDone = new CountDownLatch(5);
	RThread[5] threads;
	
	for (int i = 0; i < threads.length; i++)
		threads[i] = new RThread(i);
	
	foreach(RThread t; threads)
		t.start();
	
	//allDone.wait();
	Sleep(10000);
	printf("All done.\n");
	return 0;
}

Note: I dont have the concurrent stuff so I have included it but commented  
it out.

Regan
Apr 04 2005
prev sibling next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Alternate solution using seperate object, not derived from Thread:

import std.c.windows.windows;
import std.thread;
import std.stdio;

class Other
{
	int index;
	
	this(int i)
	{
		index = i;
	}
	
	int run()
	{
		writefln("thread %d",index);
		//allDone.countDown();
		return 0;
	}
}

int main(char[][] args)
{
	//CountDownLatch allDone = new CountDownLatch(5);
	Thread[5] threads;
	Other o;
	
	for (int i = 0; i < threads.length; i++) {
		o = new Other(i);
		threads[i] = new Thread(&o.run);
	}
	
	foreach(Thread t; threads)
		t.start();
	
	//allDone.wait();
	Sleep(10000);
	printf("All done.\n");
	return 0;
}

Regan
Apr 04 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Ooops. One important correction is that the 'allDone' variable actually  
needs to be global, or otherwise accessable from the RThread or Other  
classes. eg.

import ..etc..

//global variable/singleton
CountDownLatch allDone;

//module static ctor, executed when program starts
static this()
{
	allDone = new CountDownLatch(5);
}

class RThread : Thread
{
   ..etc..
}

class Other
{
   ..etc..
}

int main(char[][] args)
{
   ..etc..
}

Regan
Apr 04 2005
prev sibling next sibling parent Larry Cowan <Larry_member pathlink.com> writes:
Corrected typing error in code...

In article <d2sdsg$9ca$1 digitaldaemon.com>, Larry Cowan says...

| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[j].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can thnk of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?
Apr 04 2005
prev sibling next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
Glad to see someone using the Locks lib. When I was testing things out I ran 
into the same conundrum you did. A simple way (eg not very generic but it 
gets the job done) of solving the problem is to loop over t looking for the 
thread in question:


 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });


t[i] = new Thread(
 delegate int() {
  int n;
  for(n=0;n<5;n++) {
    if (Thread.getThis() is t[n]) break;
  }
  printf("  thread %d\n",n);
  allDone.countDown();
  return 0;
});
Apr 04 2005
prev sibling next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
I should add that if you are on Linux and you start seeing seg-v's on 
program exit that is because D doesn't wait for all the threads to finish 
before running gc_term and exiting. So basically you need be very careful 
before you return from your main() to make sure the threads are completely 
done and - not just by waiting for a latch, either, since the threads you 
create still need to execute for a bit after they decrement the latch count. 
Mike Swieton and I had some posts ages ago about proposals to change how D 
exits w.r.t. waiting for child threads but nothing changed. Just thought I'd 
mention it because it can be very confusing trying to debug that behavior. 
See for example 
http://www.digitalmars.com/d/archives/digitalmars/D/bugs/156.html

"Larry Cowan" <Larry_member pathlink.com> wrote in message 
news:d2sdsg$9ca$1 digitaldaemon.com...

| module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}
Apr 04 2005
prev sibling parent "Carlos Santander B." <csantander619 gmail.com> writes:
Larry Cowan wrote:

 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}
 
 prints:
 
 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.
 
 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?  Do I
 have to make one for them on the heap to actually get independent activity,
 then have them pick up an index from a locked counter?
 
 
 
 
 


The problem is, you're asking the threads to print the value of "i", but 
by the time they've started, "i" doesn't have the value it used to have, 
so that's why they're all printing 5.

Extending Thread (as Regan said) is the way I've done things like this.

-- 
Carlos Santander Bernal

JP2, you'll always live in our minds
Apr 04 2005