www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Has anyone ever tested CDGC (Concurrent D GC)?

reply Trass3r <un known.com> writes:
I have just come to know that there is an experimental branch in druntime  
with a concurrent GC:  
https://github.com/D-Programming-Language/druntime/tree/CDGC
I really wonder why this wasn't announced here so people actually try it  
out, find bugs etc.

Has anyone tried it yet?
Jul 17 2011
parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jul 17, 2011, at 4:34 PM, Trass3r wrote:


 I have just come to know that there is an experimental branch in =


druntime with a concurrent GC: =
https://github.com/D-Programming-Language/druntime/tree/CDGC

 I really wonder why this wasn't announced here so people actually try =


it out, find bugs etc.

=20
 Has anyone tried it yet?


I made sure it was functional back when I added the branch.  The runtime =
code is horribly out of date though.  What needs to happen is for the =
branch to be re-created from main.  I also need to see if Leandro has =
updated his CDGC since the branch was created.
Jul 19 2011
parent reply Trass3r <un known.com> writes:
 I made sure it was functional back when I added the branch.  The runtime  
 code is horribly out of date though.  What needs to happen is for the  
 branch to be re-created from main.  I also need to see if Leandro has  
 updated his CDGC since the branch was created.


What we really need is to put it into the master branch in its own  
directory and add some makefile magic so you can easily switch GCs when  
compiling druntime (just like Tango does)
instead of having it rot in some branch that nobody even knows about.

I'm already working on this and managed to compile CDGC with latest  
druntime.
Now it needs extensive testing and bug fixing.
Jul 19 2011
parent reply "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Tue, 19 Jul 2011 21:53:01 +0900, Trass3r <un known.com> wrote:


 I made sure it was functional back when I added the branch.  The  
 runtime code is horribly out of date though.  What needs to happen is  
 for the branch to be re-created from main.  I also need to see if  
 Leandro has updated his CDGC since the branch was created.


 What we really need is to put it into the master branch in its own  
 directory and add some makefile magic so you can easily switch GCs when  
 compiling druntime (just like Tango does)
 instead of having it rot in some branch that nobody even knows about.

 I'm already working on this and managed to compile CDGC with latest  
 druntime.
 Now it needs extensive testing and bug fixing.


I am interested in Concurrent GC.

But, I have a question about CDGC.
AFAIK, other language runtimes use thread for concurrent processing.
Why use fork()? What is the merit of fork() approach?


Masahiro
Jul 19 2011
parent reply Trass3r <un known.com> writes:
 I am interested in Concurrent GC.

 But, I have a question about CDGC.
 AFAIK, other language runtimes use thread for concurrent processing.
 Why use fork()? What is the merit of fork() approach?


I don't know.
On Windows you can't use fork anyway and we have to figure out an  
alternative way.

Maybe he explains it in his thesis, but it's only available in Spanish:
http://www.llucax.com.ar/proj/dgc/index.html
Jul 19 2011
parent reply "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Tue, 19 Jul 2011 22:57:47 +0900, Trass3r <un known.com> wrote:


 I am interested in Concurrent GC.

 But, I have a question about CDGC.
 AFAIK, other language runtimes use thread for concurrent processing.
 Why use fork()? What is the merit of fork() approach?


 I don't know.
 On Windows you can't use fork anyway and we have to figure out an  
 alternative way.

 Maybe he explains it in his thesis, but it's only available in Spanish:
 http://www.llucax.com.ar/proj/dgc/index.html


Thanks for the link!
But, I didn't read Spanish...
Jul 19 2011
parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 19 Jul 2011 10:19:09 -0400, Masahiro Nakagawa <repeatedly gmail.com>
wrote:


 On Tue, 19 Jul 2011 22:57:47 +0900, Trass3r <un known.com> wrote:


 I am interested in Concurrent GC.

 But, I have a question about CDGC.
 AFAIK, other language runtimes use thread for concurrent processing.
 Why use fork()? What is the merit of fork() approach?


 I don't know.
 On Windows you can't use fork anyway and we have to figure out an
 alternative way.

 Maybe he explains it in his thesis, but it's only available in Spanish:
 http://www.llucax.com.ar/proj/dgc/index.html


 Thanks for the link!
 But, I didn't read Spanish...


He explains a lot of it in his blog. CDGC is an example of a snapshot GC. In
short, when a collection is triggered, a snapshot of the program's entire
memory subsystem is created. The GC can then trace using this snapshot without
having to worry about program writes, etc, messing it up. When finished, it
passes back to the main program what objects are garbage and the snapshot is
discarded. Now, a userland implementation of this is horribly inefficient, so
CDGC uses fork to leverage the OS's ability to do copy-on-write at the page
level. I believe CDGC also uses a memory mapped file for the GC's meta info, to
avoid the update message passing overhead.

While Windows does support allocating memory in a COW manner, there's no way to
do add this dynamically to an existing page. Windows does support page-level
write tracking (since 2K SP3), which is very useful for incremental GC. i.e.
the GC can use OS to only trace through modified pages.

There are two other alternative, modern GCs that I know of which fit system
programming languages like D. One used a kernel patch to trap hardware writes
efficiently, allowing one to bolt a traditional concurrent GC onto a system's
language. Which, while cool, isn't practical until OS APIs support it out of
the box. The other is thread-local GCs, which according to Apple, have roughly
equivalent performance to existing concurrent GCs. Given shared and immutable,
thread-local GC's make a lot of sense for D and can be combined with other
concurrent options should they be/become available.
Jul 19 2011
parent reply "Masahiro Nakagawa" <repeatedly gmail.com> writes:
On Wed, 20 Jul 2011 00:25:58 +0900, Robert Jacques <sandford jhu.edu>  
wrote:


 On Tue, 19 Jul 2011 10:19:09 -0400, Masahiro Nakagawa  
 <repeatedly gmail.com> wrote:


 On Tue, 19 Jul 2011 22:57:47 +0900, Trass3r <un known.com> wrote:


 I am interested in Concurrent GC.

 But, I have a question about CDGC.
 AFAIK, other language runtimes use thread for concurrent processing.
 Why use fork()? What is the merit of fork() approach?


 I don't know.
 On Windows you can't use fork anyway and we have to figure out an
 alternative way.

 Maybe he explains it in his thesis, but it's only available in Spanish:
 http://www.llucax.com.ar/proj/dgc/index.html


 Thanks for the link!
 But, I didn't read Spanish...


 snapshot GC


I have heard this by "A Real-Time Garbage Collection for Java using  
Snapshot Algorithm".
I understand a basic mechanism(above article uses a thread instead of  
fork()).
Hmm... I want to see a comparison of fork() and thread.


 There are two other alternative, modern GCs that I know of which fit  
 system programming languages like D. One used a kernel patch to trap  
 hardware writes efficiently, allowing one to bolt a traditional  
 concurrent GC onto a system's language. Which, while cool, isn't  
 practical until OS APIs support it out of the box. The other is  
 thread-local GCs, which according to Apple, have roughly equivalent  
 performance to existing concurrent GCs. Given shared and immutable,  
 thread-local GC's make a lot of sense for D and can be combined with  
 other concurrent options should they be/become available.


I didn't know the former. Thanks for the information.

Latter approach is similar to Erlang.
Erlang provides GC per-process and I like this approach.
Is Thread-local GC in D realistic?
D allows global mutable state...
Jul 19 2011
next sibling parent Trass3r <un known.com> writes:
 I have heard this by "A Real-Time Garbage Collection for Java using  
 Snapshot Algorithm".
 I understand a basic mechanism(above article uses a thread instead of  
 fork()).
 Hmm... I want to see a comparison of fork() and thread.


Well someone has to implement it ;)
Jul 19 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 19 Jul 2011 13:28:04 -0400, Masahiro Nakagawa <repeatedly gmail.com>
wrote:

 On Wed, 20 Jul 2011 00:25:58 +0900, Robert Jacques <sandford jhu.edu>
 wrote:


 On Tue, 19 Jul 2011 10:19:09 -0400, Masahiro Nakagawa
 <repeatedly gmail.com> wrote:


 On Tue, 19 Jul 2011 22:57:47 +0900, Trass3r <un known.com> wrote:


 I am interested in Concurrent GC.

 But, I have a question about CDGC.
 AFAIK, other language runtimes use thread for concurrent processing.
 Why use fork()? What is the merit of fork() approach?


 I don't know.
 On Windows you can't use fork anyway and we have to figure out an
 alternative way.

 Maybe he explains it in his thesis, but it's only available in Spanish:
 http://www.llucax.com.ar/proj/dgc/index.html


 Thanks for the link!
 But, I didn't read Spanish...


 snapshot GC


 I have heard this by "A Real-Time Garbage Collection for Java using
 Snapshot Algorithm".
 I understand a basic mechanism(above article uses a thread instead of
 fork()).
 Hmm... I want to see a comparison of fork() and thread.


 There are two other alternative, modern GCs that I know of which fit
 system programming languages like D. One used a kernel patch to trap
 hardware writes efficiently, allowing one to bolt a traditional
 concurrent GC onto a system's language. Which, while cool, isn't
 practical until OS APIs support it out of the box. The other is
 thread-local GCs, which according to Apple, have roughly equivalent
 performance to existing concurrent GCs. Given shared and immutable,
 thread-local GC's make a lot of sense for D and can be combined with
 other concurrent options should they be/become available.


 I didn't know the former. Thanks for the information.

 Latter approach is similar to Erlang.
 Erlang provides GC per-process and I like this approach.
 Is Thread-local GC in D realistic?
 D allows global mutable state...


Well, given that D has separate types for immutable and shared data,
implementing a thread-local GC in D is a while lot easier and safer than other
languages like Objective-C and Java, etc for which people have already written
thread-local GCs. But, unlike Erlang, D would also have to have a global shared
GC in addition to the thread local GCs, in order to handle shared/immutable
objects. Though how often the shared GC would actually run would be dependent
on the form of concurrency you're using.
Jul 19 2011