www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - More recent work on GC

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On my reading list:

http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf

http://users.cecs.anu.edu.au/~steveb/downloads/pdf/immix-pldi-2008.pdf 
(this has been mentioned before)


Andrei
Jan 13 2015
next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 14 January 2015 at 06:15:09 UTC, Andrei 
Alexandrescu wrote:

 On my reading list:

 http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf

 http://users.cecs.anu.edu.au/~steveb/downloads/pdf/immix-pldi-2008.pdf 
 (this has been mentioned before)


 Andrei


Thanks for the heads up.
Jan 14 2015
prev sibling next sibling parent reply "Dominikus Dittes Scherkl" writes:
On Wednesday, 14 January 2015 at 06:15:09 UTC, Andrei 
Alexandrescu wrote:

 On my reading list:

 http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf


Interesting. non-exact GC but with about the same performance.
The bad part for D would be, that this GC collects periodically, 
so even if you don't use GC'ed memory it will be invoked :-/
On the other hand it stops the world more often, but only for 
much shorter time - at least in interactive programms a better 
choice I think.
Jan 14 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 14 January 2015 at 16:09:06 UTC, Dominikus Dittes 
Scherkl wrote:

 On Wednesday, 14 January 2015 at 06:15:09 UTC, Andrei 
 Alexandrescu wrote:

 On my reading list:

 http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf


 Interesting. non-exact GC but with about the same performance.
 The bad part for D would be, that this GC collects 
 periodically, so even if you don't use GC'ed memory it will be 
 invoked :-/


You could still GC.disable(); /* realtime work here */ 
GC.enable(); of course
Jan 14 2015
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 14, 2015 at 05:37:41PM +0000, John Colvin via Digitalmars-d wrote:

 On Wednesday, 14 January 2015 at 16:09:06 UTC, Dominikus Dittes Scherkl
 wrote:

On Wednesday, 14 January 2015 at 06:15:09 UTC, Andrei Alexandrescu wrote:

On my reading list:

http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf






Fascinating! This paper seems to be describing a *partially*
compacting/copying GC.  That's a very clever way of circumventing the
constraints imposed by a conservative GC, which D can't around with due
to interfacing with C.  IOW, if a D program just so happens to have no
unambiguous references, then the GC will effectively become a fully
compacting/copying GC. If there are some ambiguous references, then it
"gracefully degrades" into a non-compacting/copying conservative GC.
Looking forward to see what will come of this!



Interesting. non-exact GC but with about the same performance.  The
bad part for D would be, that this GC collects periodically, so even
if you don't use GC'ed memory it will be invoked :-/


 
 You could still GC.disable(); /* realtime work here */ GC.enable(); of
 course


Recently in one of my projects I found that I can gain a huge
performance improvement just by calling GC.disable() at the beginning of
the program and never calling GC.enable() again, but instead manually
calling GC.collect() at strategic points in the code. Obviously, YMMV,
but I managed to get a 40% performance improvement, which is pretty big
for such a relatively simple change.


T

-- 
What do you call optometrist jokes? Vitreous humor.
Jan 14 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 14 January 2015 at 18:01:22 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Recently in one of my projects I found that I can gain a huge
 performance improvement just by calling GC.disable() at the 
 beginning of
 the program and never calling GC.enable() again, but instead 
 manually
 calling GC.collect() at strategic points in the code. 
 Obviously, YMMV,
 but I managed to get a 40% performance improvement, which is 
 pretty big
 for such a relatively simple change.


Interesting that you need to disable to get the effect. That mean 
our heuristic for the GC collection to kick in sucks quite badly.
Jan 14 2015
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jan 14, 2015 at 08:07:37PM +0000, deadalnix via Digitalmars-d wrote:

 On Wednesday, 14 January 2015 at 18:01:22 UTC, H. S. Teoh via Digitalmars-d
 wrote:

Recently in one of my projects I found that I can gain a huge
performance improvement just by calling GC.disable() at the beginning
of the program and never calling GC.enable() again, but instead
manually calling GC.collect() at strategic points in the code.
Obviously, YMMV, but I managed to get a 40% performance improvement,
which is pretty big for such a relatively simple change.


 
 Interesting that you need to disable to get the effect. That mean our
 heuristic for the GC collection to kick in sucks quite badly.


Well, I'm not sure what the real cause is, but what happened was that I
was working on optimizing performance, and gprof indicated that a lot of
time was spent in the GC collection cycle. That led me to a lot of
needless GC allocations that, after I eliminated them, netted me a huge
performance boost. However, I noticed that there was still a lot of time
spent in the GC collection cycle -- less than before, but still a big
chunk of my running times. So as an experiment I decided to turn off the
GC completely to see what happens -- found that running times improved
by 40-50%, which is pretty huge!

Of course, that also meant I was leaking memory and the program was
soaking up too much RAM, so the second thought I had was to still run
the GC collection cycles, but at a much reduced frequency. This is
specific to my program's memory usage patterns (an ever-increasing
amount of allocations that remain live until the end of the program,
plus a comparatively much smaller number of temporary allocations that
need to get cleaned up every now and then to keep total memory use under
control); I'm not sure how generally applicable it is. In my particular
case, one of the major factors in poor GC performance was the increasing
bulk of allocations that are known to remain live until the end of the
program, that the GC must scan every collection cycle because it doesn't
know that most of them are going to remain live for a long time.
Consequently, collection cycles become slower and slower as the program
progresses, with most of the work being unnecessary since the growing
bulk of allocations aren't going away anytime soon.

This problem would be instantly solved by a generational GC, since after
a few cycles most of the bulk of the long-lived allocations will get
pushed to the oldest generations and the young collection cycles won't
be bogged down scanning them unnecessarily.

I'm not holding my breath for D to get a generational GC, though. :-P

Alternatively, since I already know exactly which allocations are going
to persist until the end, I could just use malloc instead. However this
is a bit annoying to implement since these allocations are coming from a
(very large) AA that I'm adding stuff to (nothing is ever removed). But
since I'm already working on replacing this AA with something else with
better cache-friendliness (and also disk-cacheability to transcend
current memory limitations), there's no point trying to improve AA
performance at this time.


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are
gone to milk the bull. -- Sam. Johnson
Jan 15 2015
prev sibling parent "safety0ff" <safety0ff.dev gmail.com> writes:
On Wednesday, 14 January 2015 at 06:15:09 UTC, Andrei 
Alexandrescu wrote:

 On my reading list:

 http://research.microsoft.com/pubs/230708/conservative-gc-oopsla-2014.pdf

 http://users.cecs.anu.edu.au/~steveb/downloads/pdf/immix-pldi-2008.pdf 
 (this has been mentioned before)


 Andrei


These are probably worth re-mentioning since the 2014 paper 
builds upon them:

http://research.microsoft.com/pubs/202163/rcix-oopsla-2013.pdf

http://users.cecs.anu.edu.au/~steveb/downloads/pdf/rc-ismm-2012.pdf

They've been mentioned here before.
Jan 15 2015