www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - [GSoC] Precise GC

reply Jeremy DeHaan <dehaan.jeremiah gmail.com> writes:
Hi everyone,

I know I'm super late to the party for this, and sorry for that. 
While my work on the precise GC didn't go as planned, it is 
closer than it was to be getting merged.

My open PR for the actual inclusion of the precise GC is here:
https://github.com/dlang/druntime/pull/1603

Even though GSoC is over and it isn't quite ready to be merged, I 
will still continue working on it in the hopes it'll get in. I 
have about a month until school starts up again, and I'm going to 
try to get it merged before that point.

Through the work I did and the research of a couple of GC topics, 
I discovered that I really enjoyed working on the garbage 
collector and I plan on continuing that. I was recently accepted 
to the University of Washington's computer science program and I 
am currently working on getting some independent study classes 
set up for future GC work. Once I have some better programming 
chops I'm hoping to help take D's GC to new heights.

Thanks to Martin Nowak and Adam Wilson for being my mentors and 
for keeping me going, to Rainer Schuetze for doing a significant 
portion of code review, and to the D community for giving me the 
opportunity to do this work.

     Jeremy
Sep 01 2016
next sibling parent Dsby <dushibaiyu yahoo.com> writes:
On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 [...]


wait for merge
Sep 01 2016
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 02/09/2016 3:25 PM, Jeremy DeHaan wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for that. While
 my work on the precise GC didn't go as planned, it is closer than it was
 to be getting merged.

 My open PR for the actual inclusion of the precise GC is here:
 https://github.com/dlang/druntime/pull/1603

 Even though GSoC is over and it isn't quite ready to be merged, I will
 still continue working on it in the hopes it'll get in. I have about a
 month until school starts up again, and I'm going to try to get it
 merged before that point.

 Through the work I did and the research of a couple of GC topics, I
 discovered that I really enjoyed working on the garbage collector and I
 plan on continuing that. I was recently accepted to the University of
 Washington's computer science program and I am currently working on
 getting some independent study classes set up for future GC work. Once I
 have some better programming chops I'm hoping to help take D's GC to new
 heights.

 Thanks to Martin Nowak and Adam Wilson for being my mentors and for
 keeping me going, to Rainer Schuetze for doing a significant portion of
 code review, and to the D community for giving me the opportunity to do
 this work.

     Jeremy


Awesome, please do keep going with it! It would be great over the next 
few years to get some research papers out of this.
Sep 01 2016
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 My open PR for the actual inclusion of the precise GC is here:
 https://github.com/dlang/druntime/pull/1603


So what's its current state, how is it different from the version 
Rainer had years ago? Is stack scan precise? Are closures scanned 
precisely? What about unions? We'd be happy to see some overview 
of this version, what it does and how it succeeds (or fails).
Sep 01 2016
parent reply Jeremy DeHaan <dehaan.jeremiah gmail.com> writes:
On Friday, 2 September 2016 at 05:19:57 UTC, thedeemon wrote:

 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 My open PR for the actual inclusion of the precise GC is here:
 https://github.com/dlang/druntime/pull/1603


 So what's its current state, how is it different from the 
 version Rainer had years ago? Is stack scan precise? Are 
 closures scanned precisely? What about unions?


The version in the pull request is not different from the one 
Rainer presented at all. It's actually the same one, just updated 
with what has been changed in druntime recently and my attempts 
at making the marking phase faster. It isn't any faster, but I 
didn't realize this due to benchmarking differences on AMD and 
Intel platforms. I should correct this in the next couple of days 
and performance can be assessed again.

Scanning the stack would require some support from the compiler. 
Precisely scanning unions is tricky since they could mix pointer 
and non pointer types. I'm not sure about closures. If I've done 
anything with closures I'm not aware of it. I have yet to 
actually work with the compiler, but in the future I will have to 
I suppose.


 We'd be happy to see some overview of this version, what it 
 does and how it succeeds (or fails).


It is essentially in a similar state that it was when Rainer 
presented it at DConf2013. I spent a lot of time doing research 
on techniques to improve its performance and I did start working 
on a sort of proof of concept for some of these ideas, but 
implementing it in a cohesive way would have required a lot more 
than was possible in the GSoC timeframe and it just didn't 
happen. It's pretty late for me here, but I will definitely 
explain more about some of these things later. It's essentially 
stuff that I do on my own time.
Sep 02 2016
parent reply Rory McGuire via Digitalmars-d-announce writes:
On Fri, Sep 2, 2016 at 9:46 AM, Jeremy DeHaan via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:


 [snip]. Precisely scanning unions is tricky since they could mix pointer
 and non pointer types. [snip]


Can we rather just make a special tagged union that is scanned... rather
avoid trying to make something that should be minimal more and more
complex? The compiler already treats some parts of phobos as "special" as
far as I know.
Sep 02 2016
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 2 September 2016 at 08:14:33 UTC, Rory McGuire wrote:

 Can we rather just make a special tagged union that is 
 scanned... rather avoid trying to make something that should be 
 minimal more and more complex? The compiler already treats some 
 parts of phobos as "special" as far as I know.


I had asked about unions in  safe code in other threads, but I 
don't recall anyone responding.

 safe functions cannot access unions with pointers or references 
overlapping with other types. In effect, there is already a 
special tagged union that can be scanned. I would think that the 
garbage collector already identifies these types of unions and 
doesn't just treat every union with the same conservative 
approach, but I have really no idea how it works and don't recall 
hearing back about my questions.

Anyway, with  safe unions, my thinking is that it would mean that 
the garbage collector can be made precise in  safe code in a way 
that it can't in  system code (assuming unions with pointers 
aren't snuck in through  trusted).
Sep 02 2016
parent reply thedeemon <dlang thedeemon.com> writes:
On Friday, 2 September 2016 at 14:55:26 UTC, jmh530 wrote:


 Anyway, with  safe unions, my thinking is that it would mean 
 that the garbage collector can be made precise in  safe code in 
 a way that it can't in  system code (assuming unions with 
 pointers aren't snuck in through  trusted).


GC (and runtime in general) has no idea what code is safe and 
what code is system. GC works with data at run-time. All 
 safe-related stuff is about code (not data!) and happens at 
compile-time. They are completely orthogonal and independent, as 
I understand.
Sep 03 2016
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 3 September 2016 at 12:22:25 UTC, thedeemon wrote:

 GC (and runtime in general) has no idea what code is safe and 
 what code is system. GC works with data at run-time. All 
  safe-related stuff is about code (not data!) and happens at 
 compile-time. They are completely orthogonal and independent, 
 as I understand.


I don't see why you wouldn't be able to use compile-time 
information like __traits with the runtime.

In my head, I imagine that at compile-time you can figure out 
which unions are in  safe functions, add a UDA to each (so you're 
marking data, not code), and then read that information at 
run-time (like with __traits).
Sep 06 2016
parent thedeemon <dlang thedeemon.com> writes:
On Tuesday, 6 September 2016 at 14:56:15 UTC, jmh530 wrote:


 GC (and runtime in general) has no idea what code is safe and 
 what code is system. GC works with data at run-time. All 
  safe-related stuff is about code (not data!) and happens at 
 compile-time. They are completely orthogonal and independent, 
 as I understand.


 I don't see why you wouldn't be able to use compile-time 
 information like __traits with the runtime.

 In my head, I imagine that at compile-time you can figure out 
 which unions are in  safe functions, add a UDA to each (so 
 you're marking data, not code), and then read that information 
 at run-time (like with __traits).


UDAs are also purely compile-time, they don't exist as run-time 
data.
Sep 07 2016
prev sibling parent David Nadlinger <code klickverbot.at> writes:
On Saturday, 3 September 2016 at 12:22:25 UTC, thedeemon wrote:

 On Friday, 2 September 2016 at 14:55:26 UTC, jmh530 wrote:

 Anyway, with  safe unions, my thinking is that it would mean 
 that the garbage collector can be made precise in  safe code 
 in a way that it can't in  system code (assuming unions with 
 pointers aren't snuck in through  trusted).


 GC (and runtime in general) has no idea what code is safe and 
 what code is system. GC works with data at run-time. All 
  safe-related stuff is about code (not data!) and happens at 
 compile-time. They are completely orthogonal and independent, 
 as I understand.


This is correct, but when designing a GC – in particular, a 
precise GC –, having the compiler emit additional helpful 
metadata to binaries is always an option worth considering.

  — David
Sep 08 2016
prev sibling next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 09/01/2016 08:25 PM, Jeremy DeHaan wrote:


 I will still continue working on it in the hopes it'll get in.


Great news! :)


 I
 discovered that I really enjoyed working on the garbage collector


If that's not motivation enough...


 I was recently accepted to the University of Washington's
 computer science program


... now we have your address. We will hunt you if you don't complete the 
GC work. :p

Ali
Sep 01 2016
parent Jeremy DeHaan <dehaan.jeremiah gmail.com> writes:
On Friday, 2 September 2016 at 06:54:57 UTC, Ali Çehreli wrote:

 On 09/01/2016 08:25 PM, Jeremy DeHaan wrote:


 I will still continue working on it in the hopes it'll get in.


 Great news! :)


 I
 discovered that I really enjoyed working on the garbage


 collector

 If that's not motivation enough...


 I was recently accepted to the University of Washington's
 computer science program


 ... now we have your address. We will hunt you if you don't 
 complete the GC work. :p

 Ali


There's more than one UW campus! Take that!
Sep 02 2016
prev sibling next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:

 Through the work I did and the research of a couple of GC 
 topics, I discovered that I really enjoyed working on the 
 garbage collector and I plan on continuing that. I was recently 
 accepted to the University of Washington's computer science 
 program and I am currently working on getting some independent 
 study classes set up for future GC work. Once I have some 
 better programming chops I'm hoping to help take D's GC to new 
 heights.


Hey thanks a lot for working on this!
Sep 02 2016
prev sibling next sibling parent reply Dsby <dushibaiyu yahoo.com> writes:
On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 [...]


In Mac 32 bit. the test is not pass.
Sep 06 2016
parent Jeremy DeHaan <dehaan.jeremiah gmail.com> writes:
On Wednesday, 7 September 2016 at 02:15:30 UTC, Dsby wrote:

 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 [...]


 In Mac 32 bit. the test is not pass.


I didn't have a mac to test this on at the time. I currently have 
some things ordered so that I can see what is actually going on 
and fix this.
Sep 08 2016
prev sibling parent reply FrankLike <1150015857 qq.com> writes:
On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 [...]


On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan wrote:
Hi,how about the precise GC, now?
Oct 13 2016
parent reply Dsby <dushibaiyu yahoo.com> writes:
On Friday, 14 October 2016 at 03:26:31 UTC, FrankLike wrote:

 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, it 
 is closer than it was to be getting merged.

 [...]


 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:
 Hi,how about the precise GC, now?


I want to known too.
Oct 16 2016
parent reply Jeremy DeHaan <dehaan.jeremiah gmail.com> writes:
On Monday, 17 October 2016 at 02:59:15 UTC, Dsby wrote:

 On Friday, 14 October 2016 at 03:26:31 UTC, FrankLike wrote:

 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, 
 it is closer than it was to be getting merged.

 [...]


 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:
 Hi,how about the precise GC, now?


 I want to known too.


I was asked the same question on github, but I'll answer it here 
too with a couple more details.

I've done a little work, but not enough to getit finished. I 
started school a couple of weeks ago and I'm still trying to get 
my schedule figured out, but I'm hoping to start putting some 
regular effort into it starting next week.
Oct 22 2016
parent qznc <qznc web.de> writes:
On Sunday, 23 October 2016 at 05:34:08 UTC, Jeremy DeHaan wrote:

 On Monday, 17 October 2016 at 02:59:15 UTC, Dsby wrote:

 On Friday, 14 October 2016 at 03:26:31 UTC, FrankLike wrote:

 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:

 Hi everyone,

 I know I'm super late to the party for this, and sorry for 
 that. While my work on the precise GC didn't go as planned, 
 it is closer than it was to be getting merged.

 [...]


 On Friday, 2 September 2016 at 03:25:33 UTC, Jeremy DeHaan 
 wrote:
 Hi,how about the precise GC, now?


 I want to known too.


 I was asked the same question on github, but I'll answer it 
 here too with a couple more details.

 I've done a little work, but not enough to getit finished. I 
 started school a couple of weeks ago and I'm still trying to 
 get my schedule figured out, but I'm hoping to start putting 
 some regular effort into it starting next week.


Did anything happen?
Dec 20 2016