www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A thread without GC

reply "John Colvin" <john.loughran.colvin gmail.com> writes:
I would post this in d.learn, but I suspect there isn't an easy 
answer so it would be good to have some serious discussion here.

Is there any way to create a thread that is totally free from the 
garbage collector?

I.e.
Nothing in that thread will ever be scanned by the GC.
Therefore, when the GC "stops the world", that thread can just 
keep on going.

Obviously one could create a separate process, but it would be 
nice to have it encapsulated within a single process for optimum 
speed of communication and not having to mess around with pipes.

P.S. Yes I realise how careful one would have to be when using 
this.
Jul 08 2013
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
If you make a thread using the operating system functions 
directly, D would never know (I'm pretty sure) and thus it won't 
be on the gc nor the list the gc uses to pause all threads.
Jul 08 2013
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 July 2013 at 14:11:15 UTC, Adam D. Ruppe wrote:

 If you make a thread using the operating system functions 
 directly, D would never know (I'm pretty sure) and thus it 
 won't be on the gc nor the list the gc uses to pause all 
 threads.


Huh. Well that was easy. Presumably that means I have to resort 
to doing all my communication with OS-level functions (e.g. 
pthread_****).

I wonder if one could somehow register a pre-existing thread with 
std.concurrency, being careful not to introduce any reference 
that lets in the garbage collector.
Jul 08 2013
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 8 July 2013 at 14:53:25 UTC, John Colvin wrote:

 I wonder if one could somehow register a pre-existing thread 
 with std.concurrency, being careful not to introduce any 
 reference that lets in the garbage collector.


There is a thread_attachThis() in core.thread that "Registers the 
calling thread for use with the D Runtime. " and returns a Thread 
object.

However, I think that registers it with the GC too so it defeats 
the purpose of skipping Thread in the first place.

Looking at the source of std.concurrency, it looks like it just 
has a thread local mailbox for each one, that is created when you 
call thisTid() the first time.

idk if that's actually all that is done, or if TLS needs any 
special work (I've never actually used pthreads directly), but 
maybe you could try calling that thisTid() function in the thread 
you make yourself and then send it a message and see what happens.
Jul 08 2013
parent Sean Kelly <sean invisibleduck.org> writes:
On Jul 8, 2013, at 8:08 AM, Adam D. Ruppe <destructionator gmail.com> =
wrote:


 On Monday, 8 July 2013 at 14:53:25 UTC, John Colvin wrote:

 I wonder if one could somehow register a pre-existing thread with =




std.concurrency, being careful not to introduce any reference that lets =
in the garbage collector.

=20
 There is a thread_attachThis() in core.thread that "Registers the =


calling thread for use with the D Runtime. " and returns a Thread =
object.

=20
 However, I think that registers it with the GC too so it defeats the =


purpose of skipping Thread in the first place.

Right.  The point of that routine is to make it so an external thread =
can safely reference garbage collected memory.



 Looking at the source of std.concurrency, it looks like it just has a =


thread local mailbox for each one, that is created when you call =
thisTid() the first time.

=20
 idk if that's actually all that is done, or if TLS needs any special =


work (I've never actually used pthreads directly), but maybe you could =
try calling that thisTid() function in the thread you make yourself and =
then send it a message and see what happens.

You'd really need to call thread_attachThis() first, since =
std.concurrency uses garbage collected memory for message passing.=
Jul 09 2013
prev sibling next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Monday, 8 July 2013 at 14:04:41 UTC, John Colvin wrote:

 I would post this in d.learn, but I suspect there isn't an easy 
 answer so it would be good to have some serious discussion here.

 Is there any way to create a thread that is totally free from 
 the garbage collector?

 I.e.
 Nothing in that thread will ever be scanned by the GC.
 Therefore, when the GC "stops the world", that thread can just 
 keep on going.

 Obviously one could create a separate process, but it would be 
 nice to have it encapsulated within a single process for 
 optimum speed of communication and not having to mess around 
 with pipes.

 P.S. Yes I realise how careful one would have to be when using 
 this.



We really need to implement a better GC. That's the very hard but 
better solution.
Jul 08 2013
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 July 2013 at 14:11:16 UTC, w0rp wrote:

 On Monday, 8 July 2013 at 14:04:41 UTC, John Colvin wrote:

 I would post this in d.learn, but I suspect there isn't an 
 easy answer so it would be good to have some serious 
 discussion here.

 Is there any way to create a thread that is totally free from 
 the garbage collector?

 I.e.
 Nothing in that thread will ever be scanned by the GC.
 Therefore, when the GC "stops the world", that thread can just 
 keep on going.

 Obviously one could create a separate process, but it would be 
 nice to have it encapsulated within a single process for 
 optimum speed of communication and not having to mess around 
 with pipes.

 P.S. Yes I realise how careful one would have to be when using 
 this.



 We really need to implement a better GC. That's the very hard 
 but better solution.


We need to have a <b>choice</b> of GCs. There is no single design 
that is right for every use case. Sometimes (admittedly rather 
rarely) conservative stop-the-world is actually a good thing!
Jul 08 2013
parent "w0rp" <devw0rp gmail.com> writes:
On Monday, 8 July 2013 at 14:16:46 UTC, John Colvin wrote:

 On Monday, 8 July 2013 at 14:11:16 UTC, w0rp wrote:

 We really need to implement a better GC. That's the very hard 
 but better solution.


 We need to have a <b>choice</b> of GCs. There is no single 
 design that is right for every use case. Sometimes (admittedly 
 rather rarely) conservative stop-the-world is actually a good 
 thing!


Yes, this is true. Java offers a few options for tweaking the GC, 
so we could do the same.
Jul 08 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 8 July 2013 at 14:04:41 UTC, John Colvin wrote:

 Is there any way to create a thread that is totally free from 
 the garbage collector?


Sure, just few pull request here, some pull requests there... ;)
Jul 08 2013
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jul 8, 2013, at 7:04 AM, John Colvin <john.loughran.colvin gmail.com> =
wrote:


 I would post this in d.learn, but I suspect there isn't an easy answer =


so it would be good to have some serious discussion here.

=20
 Is there any way to create a thread that is totally free from the =


garbage collector?

=20
 I.e.
 Nothing in that thread will ever be scanned by the GC.
 Therefore, when the GC "stops the world", that thread can just keep on =


going.

=20
 Obviously one could create a separate process, but it would be nice to =


have it encapsulated within a single process for optimum speed of =
communication and not having to mess around with pipes.

=20
 P.S. Yes I realise how careful one would have to be when using this.



There's a pull request for this that's been discussed on the Druntime =
list and while I'm willing to accept it so long as sufficient safeguards =
are in place, I still really don't understand the need for this feature. =
 Once you eliminate the GC support code there really isn't much left in =
core.thread.  What advantage is there of being able to create Thread =
objects that aren't managed by the GC (which I'll call juggernaut =
threads)?=
Jul 09 2013