www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threading bugs

reply Tristam MacDonald <swiftcoder gmail.com> writes:
So, I am sure that I must be missing something simple, that is causing my
multi-threading code in D fail miserably. I have cut it down to the smallest
code sample that exhibits the problem, and I would be grateful if someone could
have a look and let me know why...

import std.thread;
import std.stdio;

class Main
{
	this() {
		writefln("starting");

		Thread worker = new Thread(&workerMain);
		worker.start();
	}
	~this() {
		writefln("ending");
	}
	
	int workerMain()
	{
// -- disabled in case of deadlock in std.format, but never reached anyway
//		writefln("In Thread");
		return 0;
	}
	
	Thread worker;
}

int main()
{
	Main m = new Main();
		
	return 0;
}
Jun 26 2007
next sibling parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
And score 10 for forgetting to explain the problem :)

Anyway, this code sample should print "starting", star a thread that exits
immediately, and finally print "ending" as the destructor is called at program
termination.
Instead, it prints "starting" and then does nothing - or more precisely hogs
the entirety of only one of my 2 cores - and never exits.

BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
pthread specific problem perhaps?

Tristam MacDonald Wrote:


 So, I am sure that I must be missing something simple, that is causing my
multi-threading code in D fail miserably. I have cut it down to the smallest
code sample that exhibits the problem, and I would be grateful if someone could
have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 	this() {
 		writefln("starting");
 
 		Thread worker = new Thread(&workerMain);
 		worker.start();
 	}
 	~this() {
 		writefln("ending");
 	}
 	
 	int workerMain()
 	{
 // -- disabled in case of deadlock in std.format, but never reached anyway
 //		writefln("In Thread");
 		return 0;
 	}
 	
 	Thread worker;
 }
 
 int main()
 {
 	Main m = new Main();
 		
 	return 0;
 }
Jun 26 2007
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:


 And score 10 for forgetting to explain the problem :)
 
 Anyway, this code sample should print "starting", star a thread that exits
 immediately, and finally print "ending" as the destructor is called at
 program termination. Instead, it prints "starting" and then does nothing -
 or more precisely hogs the entirety of only one of my 2 cores - and never
 exits.
 
 BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
 pthread specific problem perhaps?
 
 Tristam MacDonald Wrote:
 

 So, I am sure that I must be missing something simple, that is causing my
 multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be grateful
 if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //           writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 return 0;
 }
 
 




If you change the main method to this the thread will be run.

int main()
{
        Main m = new Main();

        foreach(t;Thread.getAll())
                if(!t.isSelf())
                        t.wait();

        return 0;
}

The current behavior seams to be that all threads exits when main returns
(or when std.c.stdlib.exit is run). The constructor is not run because the
class is not deleted by you or the garbage collector, there was talk about
this behavior some time ago (maybe last summer) and this is by design,
search the newsgroup or changelog if you want to follow the discussion.
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).

I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?

The point I don't understand, is why is this only the case when I am using
threads? And I think the thread implementation may be a little buggy here
anyway, why on earth would the assert statement below cause a 'Bus Error'?

class Main
{
	this() {
		writefln("starting");

		Thread worker = new Thread(&workerMain);
		worker.start();
	}
	~this() {
		writefln("ending");
	}
	
	int workerMain()
	{
		writefln("In Thread");
		return 0;
	}
	
	Thread worker;
}

int main()
{
	Main m = new Main();

	assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
		
	return 0;
}

Johan Granberg Wrote:


 Tristam MacDonald wrote:
 

 And score 10 for forgetting to explain the problem :)
 
 Anyway, this code sample should print "starting", star a thread that exits
 immediately, and finally print "ending" as the destructor is called at
 program termination. Instead, it prints "starting" and then does nothing -
 or more precisely hogs the entirety of only one of my 2 cores - and never
 exits.
 
 BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
 pthread specific problem perhaps?
 
 Tristam MacDonald Wrote:
 

 So, I am sure that I must be missing something simple, that is causing my
 multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be grateful
 if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //           writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 return 0;
 }
 
 




 
 If you change the main method to this the thread will be run.
 
 int main()
 {
         Main m = new Main();
 
         foreach(t;Thread.getAll())
                 if(!t.isSelf())
                         t.wait();
 
         return 0;
 }
 
 The current behavior seams to be that all threads exits when main returns
 (or when std.c.stdlib.exit is run). The constructor is not run because the
 class is not deleted by you or the garbage collector, there was talk about
 this behavior some time ago (maybe last summer) and this is by design,
 search the newsgroup or changelog if you want to follow the discussion.
Jun 26 2007
next sibling parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:


 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the object
 had a non trivial destructor (dispose of shared memory, flush an iostream,
 etc.)?
 
 The point I don't understand, is why is this only the case when I am using
 threads? And I think the thread implementation may be a little buggy here
 anyway, why on earth would the assert statement below cause a 'Bus Error'?


Because you are redeclaring worker in the Main constructor, change this line
 Thread worker = new Thread(&workerMain);
to this
 worker = new Thread(&workerMain);

and it will probably work.


 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
 
 return 0;
 }
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Oh cripes, right, with that fixed your wait solution works fine, but I still
can't find why destructors are not  called at termination with threads?

Johan Granberg Wrote:


 Tristam MacDonald wrote:
 

 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the object
 had a non trivial destructor (dispose of shared memory, flush an iostream,
 etc.)?
 
 The point I don't understand, is why is this only the case when I am using
 threads? And I think the thread implementation may be a little buggy here
 anyway, why on earth would the assert statement below cause a 'Bus Error'?


 
 Because you are redeclaring worker in the Main constructor, change this line
  Thread worker = new Thread(&workerMain);
 to this
  worker = new Thread(&workerMain);
 
 and it will probably work.
 

 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
 
 return 0;
 }
Jun 26 2007
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:


 Oh cripes, right, with that fixed your wait solution works fine, but I
 still can't find why destructors are not  called at termination with
 threads?


An old newsgroup post by Walter
http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=11455
i don't like this behavior myself but that's it.


 Johan Granberg Wrote:
 

 Tristam MacDonald wrote:
 

 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the
 object had a non trivial destructor (dispose of shared memory, flush an
 iostream, etc.)?
 
 The point I don't understand, is why is this only the case when I am
 using threads? And I think the thread implementation may be a little
 buggy here anyway, why on earth would the assert statement below cause
 a 'Bus Error'?


 
 Because you are redeclaring worker in the Main constructor, change this
 line
  Thread worker = new Thread(&workerMain);
 to this
  worker = new Thread(&workerMain);
 
 and it will probably work.
 

 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not
 done!");
 
 return 0;
 }
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Would explicitly running a full collect cycle at the end of main (and wrapping
logic in an inner function work around this? More likely some would still be
left though.

Johan Granberg Wrote:


 Tristam MacDonald wrote:
 

 Oh cripes, right, with that fixed your wait solution works fine, but I
 still can't find why destructors are not  called at termination with
 threads?


 
 An old newsgroup post by Walter
 http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=11455
 i don't like this behavior myself but that's it.
 

 Johan Granberg Wrote:
 

 Tristam MacDonald wrote:
 

 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the
 object had a non trivial destructor (dispose of shared memory, flush an
 iostream, etc.)?
 
 The point I don't understand, is why is this only the case when I am
 using threads? And I think the thread implementation may be a little
 buggy here anyway, why on earth would the assert statement below cause
 a 'Bus Error'?


 
 Because you are redeclaring worker in the Main constructor, change this
 line
  Thread worker = new Thread(&workerMain);
 to this
  worker = new Thread(&workerMain);
 
 and it will probably work.
 

 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not
 done!");
 
 return 0;
 }
Jun 26 2007
next sibling parent Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:


 Would explicitly running a full collect cycle at the end of main (and
 wrapping logic in an inner function work around this? More likely some
 would still be left though.


I think it would work if you ensure that no valid references is left in
scope when you does so. (set all globals and the local references in main
to null)
Jun 26 2007
prev sibling parent Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:

 Would explicitly running a full collect cycle at the end of main (and wrapping
logic in an inner function work around this? More likely some would still be
left though.


You can call fullCollectNoStack() to collect everything whether there is 
a valid reference to it or not.


Sean
Jun 26 2007
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:

 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?


Running the dtors of all objects on exit is problematic.  Should they 
run before or after the module dtors?  What if they are run after the 
module dtors but the object in question relied on the module's dtor not 
yet having been run?  In Tango, uncollected objects not specifically 
cleaned up in a module dtor are not guaranteed to be collected for this 
reason.  An alternative would be to run a collection after main() exits 
as a part of the cleanup process.  This would get your Main object 
below, but it would slow the shutdown process for the sake of collecting 
only a very few objects, and I'm not sure it's worthwhile to do so.


 The point I don't understand, is why is this only the case when I am using
threads? And I think the thread implementation may be a little buggy here
anyway, why on earth would the assert statement below cause a 'Bus Error'?


I have no idea.  On Tango, your program outputs:

     starting
     In Thread

Here's the converted code:


     import tango.core.Thread;
     import tango.stdc.stdio;

     class Main
     {
     	this() {
     		printf("starting\n");

     		Thread worker = new Thread(&workerMain);
     		worker.start();
     	}
     	~this() {
     		printf("ending\n");
     	}

     	void workerMain()
     	{
     		printf("In Thread\n");
     	}

     	Thread worker;
     }

     int main()
     {
     	Main m = new Main();

     	return 0;
     }

Sean
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
I guess coming from a C++ background (i.e. no GC), I am having trouble with the
whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.

To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.

AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 

Sean Kelly Wrote:

 Tristam MacDonald wrote:

 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?


 
 Running the dtors of all objects on exit is problematic.  Should they 
 run before or after the module dtors?  What if they are run after the 
 module dtors but the object in question relied on the module's dtor not 
 yet having been run?  In Tango, uncollected objects not specifically 
 cleaned up in a module dtor are not guaranteed to be collected for this 
 reason.  An alternative would be to run a collection after main() exits 
 as a part of the cleanup process.  This would get your Main object 
 below, but it would slow the shutdown process for the sake of collecting 
 only a very few objects, and I'm not sure it's worthwhile to do so.
Jun 27 2007
next sibling parent Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:

 I guess coming from a C++ background (i.e. no GC), I am having trouble with
the whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.


They can do both, but if it's important that those resources are 
explicitly released then you should take steps to manage the object's 
lifetime.  I could still be convinced that all lingering objects should 
be collected on shutdown, but I haven't come up with an approach I'm 
entirely happy with (the most obvious being to simply collect everything 
and if an exception is thrown then give up, but continue the rest of the 
shutdown process).


 To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.


Yup.  It doesn't help that there is no way to implement smart pointers 
in D, given the lack of copy semantics in structs.


 AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 


FWIW, Tango has a means of hooking the collection routine, which may be 
used as a means of detecting resource 'leaks'.


Sean
Jun 27 2007
prev sibling parent Myron Alexander <someone somewhere.com> writes:
Tristam MacDonald wrote:

 I guess coming from a C++ background (i.e. no GC), I am having trouble with
the whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.
 
 To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.
 
 AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 
 
 Sean Kelly Wrote:

 Tristam MacDonald wrote:

 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).

 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?


 Running the dtors of all objects on exit is problematic.  Should they 
 run before or after the module dtors?  What if they are run after the 
 module dtors but the object in question relied on the module's dtor not 
 yet having been run?  In Tango, uncollected objects not specifically 
 cleaned up in a module dtor are not guaranteed to be collected for this 
 reason.  An alternative would be to run a collection after main() exits 
 as a part of the cleanup process.  This would get your Main object 
 below, but it would slow the shutdown process for the sake of collecting 
 only a very few objects, and I'm not sure it's worthwhile to do so.




I'm not quite sure how D is different from C++ in this instance.

C++ on heap:
main... {
   SomeObject so = new SomeObject ();
   ...
}

End of main, program exits and the destructor of SomeObject is not 
executed. You have to 'delete so;'.

C++ on stack:
main... {
   SomeObject so();
   ...
}

End of main, stack unwound, so::~SomeObject() is executed. If 'so' were 
in a function, and you wanted to return 'so', then you need a copy 
constructor; 'so' is destructed at end of function.

D on heap:
main... {
   SomeObject so = new SomeObject ();
   ...
}

D on stack:
main... {
   scope SomeObject so = new SomeObject ();
   ...
}

I have not tried it yet, but you can write a class copy constructor for 
D and, afaik,  constructors are coming to structs.

I have no idea how to solve the raised gc issues.


Regards,

Myron.
Jun 29 2007
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Tristam,


 So, I am sure that I must be missing something simple, that is causing
 my multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be
 grateful if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 class Main
 {
 this() {
 writefln("starting");
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //		writefln("In Thread");
 return 0;
 }
 Thread worker;
 }
 int main()
 {
 Main m = new Main();
 return 0;
 }


If I had to guess, I'd say that the program is exiting before the thread 
starts. But then I'm also guessing what the problem is.
Jun 26 2007
parent BCS <ao pathlink.com> writes:
Reply to Benjamin,


  But then I'm also guessing what the problem is.
 



scrath that
Jun 26 2007