www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - spawn bug?

reply Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
The attached code usually hangs (running on OSX, dmd 2.051).

It uses spawn to create a new thread.  All the thread does is create a large
array of doubles. In one case it uses the Array!double from std.container.  In
the other case it uses the built in dynamic double[].  It declares the array,
sets the length, then sends a message to main saying it is finished and exits.

The Array!double case always runs fine.  The built-in dynamic array case
usually hangs, though not always in the same place.

And the array has to be sufficiently big (in my case 200000 or bigger I think)
or the problem doesn't occur.

I've no idea why.  It took me a *long* time to find this since it manifested
as very mysterious hanging in a more complex piece of threaded code.  Now I've
switched to using Array!double in my code and it all works fine.


Adam

(An example run)

Spawned foo 1
Spawned foo 2
Spawned foo 3
Spawned foo 4
Finished foo 1
Finished foo 2
Finished foo 3
Finished foo 4
Spawned bar 1
Spawned bar 2
Spawned bar 3
(stays here without finishing)

begin 644 spawn.d
M:6UP;W)T('-T9"YC;VYC=7)R96YC>3L*:6UP;W)T('-T9"YS=&1I;SL*:6UP
M;W)T('-T9"YC;VYT86EN97(["FEM<&]R="!C;W)E+G1H<F5A9#L*"G9O:60 
M9F]O*'5I;G0 :RP =6EN="!L+%1I9"!M5&ED*2`*>PH ($%R<F%Y(61O=6)L
M92!A<G(["B` 87)R+FQE;F=T:"AL*3L*("!S96YD*&U4:60L:RD["GT*" IV
M;VED(&)A<BAU:6YT(&LL('5I;G0 ;"Q4:60 ;51I9"D "GL*("!D;W5B;&5;
M72!A<G(["B` 87)R+FQE;F=T:"`](&P["B` <V5N9"AM5&ED+&LI.PI]" H*
M=F]I9"!M86EN*"D "GL*("!I;6UU=&%B;&4 =6EN="!M=6QT:7!L:65R(#T 


M96%D<RLQ*2D >PH ("` 875T;R!D=6UM>5]4:60 /2!S<&%W;B F9F]O+&LL
M:RIM=6QT:7!L:65R+&UA:6Y4:60I.PH ("` =W)I=&5F;&XH(E-P87=N960 
M9F]O("5S(BQK*3L*("!]" H (&9O<F5A8V H:SL ,"XN;E1H<F5A9',I"B` 
M("!R96-E:79E*"AU:6YT(&HI('L =W)I=&5F;&XH(D9I;FES:&5D(&9O;R`E
M<R(L:BD[('TI.PH*("!F;W)E86-H("AK.R`Q+BXH;E1H<F5A9',K,2DI('L*
M("` (&%U=&\ 9'5M;7E?5&ED(#T <W!A=VXH)F)A<BQK+&LJ;75L=&EP;&EE
M<BQM86EN5&ED*3L*("` ('=R:71E9FQN*")3<&%W;F5D(&)A<B`E<R(L:RD[
M"B` ?0H (&9O<F5A8V H:SL ,"XN;E1H<F5A9',I"B` ("!R96-E:79E*"AU
M:6YT(&HI('L =W)I=&5F;&XH(D9I;FES:&5D(&)A<B`E<R(L:BD[('TI.PH 
&(`H (`I]
`
end
Jan 23 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 23 January 2011 20:34:16 Adam Conner-Sax wrote:

 The attached code usually hangs (running on OSX, dmd 2.051).
 
 It uses spawn to create a new thread.  All the thread does is create a
 large array of doubles. In one case it uses the Array!double from
 std.container.  In the other case it uses the built in dynamic double[]. 
 It declares the array, sets the length, then sends a message to main
 saying it is finished and exits.
 
 The Array!double case always runs fine.  The built-in dynamic array case
 usually hangs, though not always in the same place.
 
 And the array has to be sufficiently big (in my case 200000 or bigger I
 think) or the problem doesn't occur.
 
 I've no idea why.  It took me a *long* time to find this since it
 manifested as very mysterious hanging in a more complex piece of threaded
 code.  Now I've switched to using Array!double in my code and it all works
 fine.
 
 
 Adam
 
 (An example run)
 
 Spawned foo 1
 Spawned foo 2
 Spawned foo 3
 Spawned foo 4
 Finished foo 1
 Finished foo 2
 Finished foo 3
 Finished foo 4
 Spawned bar 1
 Spawned bar 2
 Spawned bar 3
 (stays here without finishing)
 
 begin 644 spawn.d
 M:6UP;W)T('-T9"YC;VYC=7)R96YC>3L*:6UP;W)T('-T9"YS=&1I;SL*:6UP
 M;W)T('-T9"YC;VYT86EN97(["FEM<&]R="!C;W)E+G1H<F5A9#L*"G9O:60 
 M9F]O*'5I;G0 :RP =6EN="!L+%1I9"!M5&ED*2`*>PH ($%R<F%Y(61O=6)L
 M92!A<G(["B` 87)R+FQE;F=T:"AL*3L*("!S96YD*&U4:60L:RD["GT*" IV
 M;VED(&)A<BAU:6YT(&LL('5I;G0 ;"Q4:60 ;51I9"D "GL*("!D;W5B;&5;
 M72!A<G(["B` 87)R+FQE;F=T:"`](&P["B` <V5N9"AM5&ED+&LI.PI]" H*
 M=F]I9"!M86EN*"D "GL*("!I;6UU=&%B;&4 =6EN="!M=6QT:7!L:65R(#T 


 M96%D<RLQ*2D >PH ("` 875T;R!D=6UM>5]4:60 /2!S<&%W;B F9F]O+&LL
 M:RIM=6QT:7!L:65R+&UA:6Y4:60I.PH ("` =W)I=&5F;&XH(E-P87=N960 
 M9F]O("5S(BQK*3L*("!]" H (&9O<F5A8V H:SL ,"XN;E1H<F5A9',I"B` 
 M("!R96-E:79E*"AU:6YT(&HI('L =W)I=&5F;&XH(D9I;FES:&5D(&9O;R`E
 M<R(L:BD[('TI.PH*("!F;W)E86-H("AK.R`Q+BXH;E1H<F5A9',K,2DI('L*
 M("` (&%U=&\ 9'5M;7E?5&ED(#T <W!A=VXH)F)A<BQK+&LJ;75L=&EP;&EE
 M<BQM86EN5&ED*3L*("` ('=R:71E9FQN*")3<&%W;F5D(&)A<B`E<R(L:RD[
 M"B` ?0H (&9O<F5A8V H:SL ,"XN;E1H<F5A9',I"B` ("!R96-E:79E*"AU
 M:6YT(&HI('L =W)I=&5F;&XH(D9I;FES:&5D(&)A<B`E<R(L:BD[('TI.PH 
 &(`H (`I]
 `
 end


Please don't post to the bug list. Ask about bugs on D.Learn or D, or just 
report them in the tracker. This list is only meant for receiving messages from 
the bug tracker.

There are several spawn bugs: 
http://d.puremagic.com/issues/buglist.cgi?quicksearch=spawn

This one seems a likely culprit: 
http://d.puremagic.com/issues/show_bug.cgi?id=4307

- Jonathan M Davis
Jan 23 2011
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
Adam Conner-Sax Wrote:


 The attached code usually hangs (running on OSX, dmd 2.051).
 
 It uses spawn to create a new thread.  All the thread does is create a large
 array of doubles. In one case it uses the Array!double from std.container.  In
 the other case it uses the built in dynamic double[].  It declares the array,
 sets the length, then sends a message to main saying it is finished and exits.
 
 The Array!double case always runs fine.  The built-in dynamic array case
 usually hangs, though not always in the same place.
 
 And the array has to be sufficiently big (in my case 200000 or bigger I think)
 or the problem doesn't occur.
 
 I've no idea why.  It took me a *long* time to find this since it manifested
 as very mysterious hanging in a more complex piece of threaded code.  Now I've
 switched to using Array!double in my code and it all works fine.


This one is weird, and doesn't appear related to 4307.  One of the threads
(thread A) is in a GC collection and blocked trying to acquire the mutex
protecting the global thread list within thread_resumeAll.  Another thread
(thread B) is also blocked trying to acquire this mutex for other reasons.  My
best guess is that pthread_mutex in OSX is trying to give ownership of the lock
to thread B, and since thread B is suspended it effectively blocks thread A
from acquiring it to resume execution after the GC cycle.

Please submit a new issue for this.  I suspect this is OSX-specific, since OSX
doesn't use signaling to coordinate stop-the-world collections (signaling
*should* prevent a thread from being suspended while waiting for a mutex, while
the suspend mechanism in OSX doesn't seem to).
Jan 25 2011
parent reply Sean Kelly <sean invisibleduck.org> writes:
Sean Kelly Wrote:


 
 This one is weird, and doesn't appear related to 4307.  One of the threads
(thread A) is in a GC collection and blocked trying to acquire the mutex
protecting the global thread list within thread_resumeAll.  Another thread
(thread B) is also blocked trying to acquire this mutex for other reasons.  My
best guess is that pthread_mutex in OSX is trying to give ownership of the lock
to thread B, and since thread B is suspended it effectively blocks thread A
from acquiring it to resume execution after the GC cycle.


After some testing, it looks like I was right.  I have a fix for this, but it's
far from ideal (though the diff is small): require everything but
thread_resumeAll to acquire two locks in sequence, while thread_resumeAll only
acquires the second.  I'll try to come up with something better.
Jan 25 2011
next sibling parent Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
I tried to get a bugzilla account but it never emailed me the verification. 
I'll
try again and then try to figure out how to post this there.

Thanks for looking into it.

Adam
Jan 25 2011
prev sibling parent Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
Okay, figured out puremagic a little.

http://d.puremagic.com/issues/show_bug.cgi?id=5488
Jan 25 2011