www.digitalmars.com         C & C++   DMDScript  

D - GC & multiple thread causes program hang

reply dickl <dickl_member pathlink.com> writes:
When an application has multiple threads and if the garbage collection gets
called, the GC pauses all threads. The problem is if the GC was called from
inside a thread that thread is suspended, hanging the application.
If the thread has a long life and is repeatedly allocating/deallocating memory,
GC get called automatically.

--------------------

import thread;
import gc;
import string;

int main(){

Thread foo=new Thread(&testThread,null);
Thread foo1=new Thread(&testThread1,null);
foo.start();
foo1.start();
Sleep(10000);
return 1;
}

int testThread(void *p)
{
while(true)
{
char [] Str;
for(int i=0;i<20;i++)
{
Str="Loop1:"~string.toString(i);
printf("%s\n",cast(char*)Str);
}
fullCollect();
}
return 1;
}

int testThread1(void *p)
{
while(true)
{
char [] Str;
for(int i=0;i<20;i++)
{
Str="Loop2:"~string.toString(i);
printf("%s\n",cast(char*)Str);
}
genCollect();
}
return 1;
}
Sep 23 2003
parent reply jhenzie mac.com writes:
That is very interesting my friend.

It seems that the first call to the garbage collectors causes the deadlock.  The
interesting thing is 
that the Main thread is not deadlocked, amend your example and add the following
after 
Sleep(10000);

Thread.resumeAll()

The application exits as you would expect which tends to point to another
problem. that  of 
threads that are sleeping being ignored as part of a Thread.suspendAll()

I am new to D so it may take me some time to figure this out but you are not
alone.

Walter if you are reading, is there any reason why sychronization semantics are
not present in 
Thread.suspendAll.  I fairly sure you are handling the locking lower down but it
would appear 
logica;l to implement the lock on the Thread class.  But again I'm new here so
feel free to disabuse 
of that notion. 

More soon.

Justin





In article <bkpuc1$me5$1 digitaldaemon.com>, dickl says...

When an application has multiple threads and if the garbage collection gets
called, the GC pauses all threads. The problem is if the GC was called from
inside a thread that thread is suspended, hanging the application.
If the thread has a long life and is repeatedly allocating/deallocating memory,
GC get called automatically.

--------------------

import thread;
import gc;
import string;

int main(){

Thread foo=new Thread(&testThread,null);
Thread foo1=new Thread(&testThread1,null);
foo.start();
foo1.start();
Sleep(10000);
return 1;
}

int testThread(void *p)
{
while(true)
{
char [] Str;
for(int i=0;i<20;i++)
{
Str="Loop1:"~string.toString(i);
printf("%s\n",cast(char*)Str);
}
fullCollect();
}
return 1;
}

int testThread1(void *p)
{
while(true)
{
char [] Str;
for(int i=0;i<20;i++)
{
Str="Loop2:"~string.toString(i);
printf("%s\n",cast(char*)Str);
}
genCollect();
}
return 1;
}
Sep 23 2003
parent reply "Walter" <walter digitalmars.com> writes:
<jhenzie mac.com> wrote in message news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to another
 problem. that  of
 threads that are sleeping being ignored as part of a Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you are


not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower down


but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 23 2003
parent reply jhenzie mac.com writes:
Walter

Let me just say that I am new to D but I can not tell you how much I appeciate
your efforts.  It has 
given me a sense of excitement that I have not felt since Objective-C.
[Objective-c retain] by the 
way.

In any case my concerns with the Thread class are as follows.

1.  It seems to me that resumeAll and pauseAll ought to be considered atomic
operations.  In my 
opinion, a call to pauseAll should leave the calling thread as the only running
thread in the 
process and it should be serial. That is calls to pauseAll should be serialized
such that if thread1 
calls pauseAll before thread2, thread1 will be the controlling thread when all
is said and done.  A 
resumeAll will then allow thread2 to take control.  

To my mind this type of call should be extremely rare and only used by people
with GURU Status, 
after all suspending other threads ignores the possibility that they hold a lock
on a specific 
resource that the calling thread may soon need.  This is the reason suspend and
resume have been 
deprecated in Java.

NB:  I realize this may be a somewhat naive view and am willing to be educated.
<smile/>

The most basic approach would be (using resumeAll)

static void resumeAll()
{
if (nthreads > 1)
{
Thread tthis = getThis();

synchronized (threadlock) {
for (int i = 0; i < allThreadsDim; i++)
{   Thread t;

t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.resume();
}
}
}
}

of course this would require a recursive lock on Thread::resume Thread::pause.

2.  Given the sychronized(object) stnrax ir would seem that every object can act
as mutex in D. If 
this is the case perhaps the addition of wait, notify and notifyall, a la java,
on object would allow 
for a more flexible and complete approach to multithreading.  Of course this
would require the 
Thread::wait method to be renamed Thread::join <smile/>.  No flames please.  

I will consider it further and hopefully give as much feedback as I can and is
wanted.

I will keep looking for the cause of the deadlock and let you know what I find.

What are people using as a D Debugger, is there such a beast?

Justin



In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to another
 problem. that  of
 threads that are sleeping being ignored as part of a Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you are


not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower down


but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 23 2003
parent reply "Walter" <walter digitalmars.com> writes:
I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 23 2003
next sibling parent reply jhenzie mac.com writes:
with pleasure.

n article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
parent jhenzie mac.com writes:
Has anyone run the test code on Linux?

Ok  I have added synchronization to pause(All), resume(All) so we have atomicity
here.  I think this is a good thing but it is not the problem.

In fact the problem is not even in the Garbage collection code. no sir.

pausing and resuming threads in fact works like a champ, so much so that I think
it might be worth removing the synchronization or at least offering an
alternative strategy.  But that's for the future.

The problem is that calling SuspendThread on allThreads[0], the primary thread,
halts the application in its tracks.  The call to SuspendThread never returns.
Removing all synchronization results in the same, the SuspendThread method must
have its own syunchronization semantics whereas the ones I have implemented are
more relevant to expected behavior.

Any win32 Gurus who want to interject, please do, it was myunderstanding that
the primary thread of an application could be suspended, although one always
runs the risk of suspending the thread in a critical section, say memory
allocation, bringing the app to its knees.

I'll keep looking but any ideas would be welcome, I'll try linux tonight.

Justin 


Walter is there anything significant about the 

In article <bks4on$mq2$1 digitaldaemon.com>, jhenzie mac.com says...

with pleasure.

n article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
prev sibling parent reply dickl <dickl_member pathlink.com> writes:
I've added synchronized() to Thread.pauseAll. I have also found that you can not
pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the children.

A better solution would be to put checks in the gc routines so that any memory
alloc/dealloc waits for collectfull() to finish. There is no reason to pause
threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{   
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
next sibling parent reply jhenzie mac.com writes:
I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that you can not
pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the children.

A better solution would be to put checks in the gc routines so that any memory
alloc/dealloc waits for collectfull() to finish. There is no reason to pause
threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{   
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
parent reply dickl <dickl_member pathlink.com> writes:
I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from doing stuff
while a child thread is re-arranging memory. Its probably an very uncommon for
this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that you can not
pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the children.

A better solution would be to put checks in the gc routines so that any memory
alloc/dealloc waits for collectfull() to finish. There is no reason to pause
threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{   
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
parent reply jhenzie mac.com writes:
I have written a small C test app on XP and suspending the primary thread has no
effect on the process, execution continues. this makes me think that the primary
is in a critical, and common section when the pause is executed.

My synchronization implementation in thread.d is entirely independent of the gc
synchronization, thread uses threadLock, gc uses gclock, ir seem that the gc,
nenory allocator and thread suspension/resume might need to share a global
lock(s)

I'll let you know how I get on with Linux tonight.

Justin

In article <bkt5ue$25tp$1 digitaldaemon.com>, dickl says...

I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from doing stuff
while a child thread is re-arranging memory. Its probably an very uncommon for
this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that you can not
pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the children.

A better solution would be to put checks in the gc routines so that any memory
alloc/dealloc waits for collectfull() to finish. There is no reason to pause
threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{   
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 24 2003
next sibling parent jhenzie mac.com writes:
Ok so Linux hangs as well but that should not be that much of a suprise since a
process is active in unix and so suspending it means that the process will not
be allocated an cpu time until woken up so it can't schedule threads, of course
this depends on the threading model.  More annoyingly SIGINT is blocked so you
can't ctrl-c tha application, must change that.

I am going to look at the windows code again today because the only reason a
thread would cease to run if its parent thread were paused woul dbe if it were a
fibre.

Let me know if you make any progress.


Justin 






In article <bkt5ue$25tp$1 digitaldaemon.com>, dickl says...

I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from doing stuff
while a child thread is re-arranging memory. Its probably an very uncommon for
this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that you can not
pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the children.

A better solution would be to put checks in the gc routines so that any memory
alloc/dealloc waits for collectfull() to finish. There is no reason to pause
threads which are not alloc/dealloc-ing memory. I don't understand the gc enough
yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{   
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since Objective-C.
 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be considered


atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the only


running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling thread when


all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used by


people

 with GURU Status,
 after all suspending other threads ignores the possibility that they hold


a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to be


educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every object


can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall, a la


java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of course


this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I can


and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know what I


find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and add the


following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but you






are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking lower






down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again I'm new


here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 25 2003
prev sibling parent reply "dickl" <dick221z yahoo.com> writes:
I've looked a the GC code in more detail. It appears anything to do anything
with dynamic memory,  has a call to synchronized() in it. So, it theory, any
thread making a call will wait for any GC in progress. So, I don't see a
need for the calls to thread.pauseAll and thread.resumeAll.

I commented them out in gcx.fullCollect. So far I have not seen any
problems. I've
run my app for a for a few hours with no problems. I also then added another
thread which
calls fullCollect every 1 Sec and have my worker thread calling  fullCollect
every loop (about  every 50mS). Didn't see a problem there either.

For all the tests above, I put that for-loop in pauseAll back the way it
was. I'm not using thread.pauseAll in my app. More checking on the Thread
code is needed to see if pauseAll works ok when used by itself (outside of
the GC code).

The gc code is somewhat disruptive to my worker thread since it is doing
real time signal processing, so I am looking at removing all dynamic memory
stuff from it.


<jhenzie mac.com> wrote in message news:bkt7p3$28e7$1 digitaldaemon.com...

 I have written a small C test app on XP and suspending the primary thread


has no

 effect on the process, execution continues. this makes me think that the


primary

 is in a critical, and common section when the pause is executed.

 My synchronization implementation in thread.d is entirely independent of


the gc

 synchronization, thread uses threadLock, gc uses gclock, ir seem that the


gc,

 nenory allocator and thread suspension/resume might need to share a global
 lock(s)

 I'll let you know how I get on with Linux tonight.

 Justin

 In article <bkt5ue$25tp$1 digitaldaemon.com>, dickl says...

I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from doing




stuff

while a child thread is re-arranging memory. Its probably an very




uncommon for

this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that








you can not

pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the








children.

A better solution would be to put checks in the gc routines so that any








memory

alloc/dealloc waits for collectfull() to finish. There is no reason to








pause

threads which are not alloc/dealloc-ing memory. I don't understand the








gc enough

yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message










news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since












Objective-C.

 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be












considered

atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the












only

running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling












thread when

all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used












by

people

 with GURU Status,
 after all suspending other threads ignores the possibility that they












hold

a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to












be

educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every












object

can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall,












a la

java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of












course

this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I












can

and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know












what I

find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and
















add the

following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but
















you

are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking
















lower

down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again
















I'm new

here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 25 2003
parent reply jhenzie mac.com writes:
Totally agree the synchonized semantics are unnecessary in Thread.

The problem is clearly to do with suspending the primary thread.  On unix one
might expect the app to hang given that what D uses as allThreads[0] is actually
the process, but this does not explain the windows hang since windows processes
are not active.

It is very strange.  

Justin


In article <bkvl68$1s3f$1 digitaldaemon.com>, dickl says...

I've looked a the GC code in more detail. It appears anything to do anything
with dynamic memory,  has a call to synchronized() in it. So, it theory, any
thread making a call will wait for any GC in progress. So, I don't see a
need for the calls to thread.pauseAll and thread.resumeAll.

I commented them out in gcx.fullCollect. So far I have not seen any
problems. I've
run my app for a for a few hours with no problems. I also then added another
thread which
calls fullCollect every 1 Sec and have my worker thread calling  fullCollect
every loop (about  every 50mS). Didn't see a problem there either.

For all the tests above, I put that for-loop in pauseAll back the way it
was. I'm not using thread.pauseAll in my app. More checking on the Thread
code is needed to see if pauseAll works ok when used by itself (outside of
the GC code).

The gc code is somewhat disruptive to my worker thread since it is doing
real time signal processing, so I am looking at removing all dynamic memory
stuff from it.


<jhenzie mac.com> wrote in message news:bkt7p3$28e7$1 digitaldaemon.com...

 I have written a small C test app on XP and suspending the primary thread


has no

 effect on the process, execution continues. this makes me think that the


primary

 is in a critical, and common section when the pause is executed.

 My synchronization implementation in thread.d is entirely independent of


the gc

 synchronization, thread uses threadLock, gc uses gclock, ir seem that the


gc,

 nenory allocator and thread suspension/resume might need to share a global
 lock(s)

 I'll let you know how I get on with Linux tonight.

 Justin

 In article <bkt5ue$25tp$1 digitaldaemon.com>, dickl says...

I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from doing




stuff

while a child thread is re-arranging memory. Its probably an very




uncommon for

this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that








you can not

pause allThreads[0] or the app will freeze. I believe this is because
allThreads[0] is the main thread and pausing it will pause all the








children.

A better solution would be to put checks in the gc routines so that any








memory

alloc/dealloc waits for collectfull() to finish. There is no reason to








pause

threads which are not alloc/dealloc-ing memory. I don't understand the








gc enough

yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message










news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how much I


appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since












Objective-C.

 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be












considered

atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as the












only

running

 thread in the
 process and it should be serial. That is calls to pauseAll should be


serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling












thread when

all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only used












by

people

 with GURU Status,
 after all suspending other threads ignores the possibility that they












hold

a lock

 on a specific
 resource that the calling thread may soon need.  This is the reason


suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing to












be

educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every












object

can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and notifyall,












a la

java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of












course

this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No flames


please.

 I will consider it further and hopefully give as much feedback as I












can

and is

 wanted.

 I will keep looking for the cause of the deadlock and let you know












what I

find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes the


deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and
















add the

following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point to






another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out but
















you

are

not

 alone.

 Walter if you are reading, is there any reason why sychronization


semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking
















lower

down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again
















I'm new

here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 25 2003
parent reply "dickl" <dick221z yahoo.com> writes:
The Thread class looks like it needs some work. There isn't a way to kill a
thread and remove if from the list of threads, if the thread completes, you
can't re-start it later on.

Walter, whats the best way to address these issues with the Thread class ?

1) suspending allthread[0] suspends the application in pauseAll()
2) doing a delete on a thread causes problems (no destructor to remove the
thread from the list)
3) letting a thread complete and then restarting it causes problems.(no way
to mark a thread as terminated and allow it  to restarted)
4) the call to pauseAll() and resumeAll() does not seem to be necessary in
gcx.fullcollect()


<jhenzie mac.com> wrote in message news:bkvq9i$2325$1 digitaldaemon.com...

 Totally agree the synchonized semantics are unnecessary in Thread.

 The problem is clearly to do with suspending the primary thread.  On unix


one

 might expect the app to hang given that what D uses as allThreads[0] is


actually

 the process, but this does not explain the windows hang since windows


processes

 are not active.

 It is very strange.

 Justin


 In article <bkvl68$1s3f$1 digitaldaemon.com>, dickl says...

I've looked a the GC code in more detail. It appears anything to do




anything

with dynamic memory,  has a call to synchronized() in it. So, it theory,




any

thread making a call will wait for any GC in progress. So, I don't see a
need for the calls to thread.pauseAll and thread.resumeAll.

I commented them out in gcx.fullCollect. So far I have not seen any
problems. I've
run my app for a for a few hours with no problems. I also then added




another

thread which
calls fullCollect every 1 Sec and have my worker thread calling




fullCollect

every loop (about  every 50mS). Didn't see a problem there either.

For all the tests above, I put that for-loop in pauseAll back the way it
was. I'm not using thread.pauseAll in my app. More checking on the Thread
code is needed to see if pauseAll works ok when used by itself (outside




of

the GC code).

The gc code is somewhat disruptive to my worker thread since it is doing
real time signal processing, so I am looking at removing all dynamic




memory

stuff from it.


<jhenzie mac.com> wrote in message




news:bkt7p3$28e7$1 digitaldaemon.com...

 I have written a small C test app on XP and suspending the primary






thread

has no

 effect on the process, execution continues. this makes me think that






the

primary

 is in a critical, and common section when the pause is executed.

 My synchronization implementation in thread.d is entirely independent






of

the gc

 synchronization, thread uses threadLock, gc uses gclock, ir seem that






the

gc,

 nenory allocator and thread suspension/resume might need to share a






global

 lock(s)

 I'll let you know how I get on with Linux tonight.

 Justin

 In article <bkt5ue$25tp$1 digitaldaemon.com>, dickl says...

I'm running XP.

I don't think the synchronized() call is needed in resumeAll..
The synchronized call in pauseAll should keep the main thread from








doing

stuff

while a child thread is re-arranging memory. Its probably an very




uncommon for

this to happen.

I plan on taking a more serious look at the gc tomorrow..
--------------
In article <bkt1ne$1vvi$1 digitaldaemon.com>, jhenzie mac.com says...

I have found the same results, see previous.

Are you running on Win or Lin?




In article <bkt0sq$1umd$1 digitaldaemon.com>, dickl says...

I've added synchronized() to Thread.pauseAll. I have also found that








you can not

pause allThreads[0] or the app will freeze. I believe this is












because

allThreads[0] is the main thread and pausing it will pause all the








children.

A better solution would be to put checks in the gc routines so that












any

memory

alloc/dealloc waits for collectfull() to finish. There is no reason












to

pause

threads which are not alloc/dealloc-ing memory. I don't understand












the

gc enough

yet to suggest a change.



static void pauseAll()
{
if (nthreads > 1)
{
synchronized (threadLock)
{
Thread tthis = getThis();
for (int i = 1; i < allThreadsDim; i++)
{
Thread t;
t = allThreads[i];
if (t && t !== tthis && t.state == TS.RUNNING)
t.pause();
}
}
}
}
=============================
In article <bkrhoh$2vhg$1 digitaldaemon.com>, Walter says...

I think you're on the right track here. If you want to make the
modifications to thread.d, that'd be a great contribution to D!

<jhenzie mac.com> wrote in message










news:bkrbr7$2mhf$1 digitaldaemon.com...

 Walter

 Let me just say that I am new to D but I can not tell you how
















much I

appeciate

 your efforts.  It has
 given me a sense of excitement that I have not felt since












Objective-C.

 [Objective-c retain] by the
 way.

 In any case my concerns with the Thread class are as follows.

 1.  It seems to me that resumeAll and pauseAll ought to be












considered

atomic

 operations.  In my
 opinion, a call to pauseAll should leave the calling thread as
















the

only

running

 thread in the
 process and it should be serial. That is calls to pauseAll should
















be

serialized

 such that if thread1
 calls pauseAll before thread2, thread1 will be the controlling












thread when

all

 is said and done.  A
 resumeAll will then allow thread2 to take control.

 To my mind this type of call should be extremely rare and only
















used

by

people

 with GURU Status,
 after all suspending other threads ignores the possibility that
















they

hold

a lock

 on a specific
 resource that the calling thread may soon need.  This is the
















reason

suspend and

 resume have been
 deprecated in Java.

 NB:  I realize this may be a somewhat naive view and am willing
















to

be

educated.

 <smile/>

 The most basic approach would be (using resumeAll)

 static void resumeAll()
 {
 if (nthreads > 1)
 {
 Thread tthis = getThis();

 synchronized (threadlock) {
 for (int i = 0; i < allThreadsDim; i++)
 {   Thread t;

 t = allThreads[i];
 if (t && t !== tthis && t.state == TS.RUNNING)
 t.resume();
 }
 }
 }
 }

 of course this would require a recursive lock on Thread::resume


Thread::pause.

 2.  Given the sychronized(object) stnrax ir would seem that every












object

can act

 as mutex in D. If
 this is the case perhaps the addition of wait, notify and
















notifyall,

a la

java,

 on object would allow
 for a more flexible and complete approach to multithreading.  Of












course

this

 would require the
 Thread::wait method to be renamed Thread::join <smile/>.  No
















flames

please.

 I will consider it further and hopefully give as much feedback as
















I

can

and is

 wanted.

 I will keep looking for the cause of the deadlock and let you
















know

what I

find.

 What are people using as a D Debugger, is there such a beast?

 Justin



 In article <bkr2gh$29sc$1 digitaldaemon.com>, Walter says...

<jhenzie mac.com> wrote in message




news:bkqu8s$24dq$1 digitaldaemon.com...

 It seems that the first call to the garbage collectors causes




















the

deadlock.  The

 interesting thing is
 that the Main thread is not deadlocked, amend your example and
















add the

following

 after
 Sleep(10000);

 Thread.resumeAll()

 The application exits as you would expect which tends to point




















to

another

 problem. that  of
 threads that are sleeping being ignored as part of a






Thread.suspendAll()

 I am new to D so it may take me some time to figure this out




















but

you

are

not

 alone.

 Walter if you are reading, is there any reason why




















sychronization

semantics are

 not present in
 Thread.suspendAll.  I fairly sure you are handling the locking
















lower

down

but it

 would appear
 logica;l to implement the lock on the Thread class.  But again
















I'm new

here so

 feel free to disabuse
 of that notion.

 More soon.

 Justin


Can you suggest any fixes to thread.d?
Sep 26 2003
next sibling parent jhenzie mac.com writes:
At least on windows.  I was so excited by the find that I have not refactored to
make it clean but here is the problem.

On windows GetCurrentThread returns -2  This is a pseudo handle that always
represents the current thread so which ever thread is calling SuspendThread with

allThreads[0].hdl is effectively pausing itself.

The following additions will resolve the problem, walter is there an official
submission form?

***windows.d***

extern (Windows)
{
export BOOL DuplicateHandle (HANDLE sourceProcess, HANDLE sourceThread,
HANDLE targetProcessHandle, HANDLE *targetHandle, DWORD access, 
BOOL inheritHandle, DWORD options);
}

***thread.d***

static this()
{
threadLock = new Object();

Thread t = new Thread();

t.state = TS.RUNNING;
t.id = GetCurrentThreadId();

thread_hdl currentThread = GetCurrentThread();

HANDLE process = (HANDLE)-1;        

DWORD access = (DWORD)0x00000002;

DuplicateHandle(process, currentThread, process, 
&(t.hdl), (DWORD)0, TRUE, access);

t.stackBottom = os_query_stackBottom();
synchronized (threadLock)
{
assert(!allThreads[0]);
allThreads[0] = t;
allThreadsDim = 1;
t.idx = 0;
}
}
Sep 26 2003
prev sibling parent jhenzie mac.com writes:
At least on windows.  I was so excited by the find that I have not refactored to
make it clean but here is the problem.

On windows GetCurrentThread returns -2  This is a pseudo handle that always
represents the current thread so which ever thread is calling SuspendThread with

allThreads[0].hdl is effectively pausing itself.

The following additions will resolve the problem, walter is there an official
submission form?

***windows.d***

extern (Windows)
{
export BOOL DuplicateHandle (HANDLE sourceProcess, HANDLE sourceThread,
HANDLE targetProcessHandle, HANDLE *targetHandle, DWORD access, 
BOOL inheritHandle, DWORD options);
}

***thread.d***

static this()
{
threadLock = new Object();

Thread t = new Thread();

t.state = TS.RUNNING;
t.id = GetCurrentThreadId();

thread_hdl currentThread = GetCurrentThread();

HANDLE process = (HANDLE)-1;        

DWORD access = (DWORD)0x00000002;

DuplicateHandle(process, currentThread, process, 
&(t.hdl), (DWORD)0, TRUE, access);

t.stackBottom = os_query_stackBottom();
synchronized (threadLock)
{
assert(!allThreads[0]);
allThreads[0] = t;
allThreadsDim = 1;
t.idx = 0;
}
}
Sep 26 2003
prev sibling parent "Walter" <walter digitalmars.com> writes:
"dickl" <dickl_member pathlink.com> wrote in message
news:bkt0sq$1umd$1 digitaldaemon.com...

 There is no reason to pause
 threads which are not alloc/dealloc-ing memory.


Yes, there is, because they may hold (on their stacks) a reference to
allocated memory.
Oct 01 2003