www.digitalmars.com         C & C++   DMDScript  

D - Improving garbage collection

reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
I wonder if it would be possible to get more information from the 
garbage collector in some sort of debug mode.  Outputs of where things 
were allocated, where circular references were found, etc.  This would 
be useful in situations where the garbage collector can only be run 
occasionally or not at all, and for the optimization of memory usage.  I 
don't really have the clearest idea of how it would work, but I think 
that the python GC has something similar, so that might be a good place 
to start looking.  I can see utility in things like being able to 
inspect the stuff being kept track of, being able to get various 
statistics about collection, and getting a list of who is holding on to 
which object, etc.   None of this sounds too insanely complicated, so if 
no one else has the time to work on something like this (or more useful 
experience with it, since I'm a newbie with GC), I'll take a crack at 
adding it to the current GC and submitting a patch.

OK, I just looked at the code for the GC (was going on the documentation 
  when writing the above) and there is a gc.GetStats() function.  The 
other suggestions still stand though.  Anyone else think that this is a 
good idea?

Evan
Oct 23 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:ap62it$1qkq$1 digitaldaemon.com...

 OK, I just looked at the code for the GC (was going on the documentation
   when writing the above) and there is a gc.GetStats() function.  The
 other suggestions still stand though.  Anyone else think that this is a
 good idea?


The gc does have some statistics gathering functionality, and I've used that
for performance tuning of it.
Oct 23 2002
parent reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
Walter wrote:

 "Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
 news:ap62it$1qkq$1 digitaldaemon.com...
 

OK, I just looked at the code for the GC (was going on the documentation
  when writing the above) and there is a gc.GetStats() function.  The
other suggestions still stand though.  Anyone else think that this is a
good idea?


 
 
 The gc does have some statistics gathering functionality, and I've used that
 for performance tuning of it.


I realize that.  When I said other suggestions, i meant things like 
dumping all of the garbage to a struct or printing it out, and being 
able to print out an object's referrers.  Not terribly useful day to 
day, I think, but nice when it comes to memory optimization.  I'll try 
to think up an example where this would be concretely useful, but slow 
systems without much memory and tough time constraints (rt imbedded 
systems and game consoles) will likely need something like this.

Evan
Oct 24 2002
parent reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
The game programmers that I know personally or through the internet are 
almost pathologically averse to using garbage collection.  There are 
some good reasons for this, but they're less important nowaday.  Still, 
opinions are slow to change.   There's only one company out there that I 
know of that uses a language other than C or C++ (occasionally with some 
python mixed in).  They use lisp, and since I worked with one of their 
founders for a while and have seen a lot of their commentary on working 
with lisp and garbage collection, I'd like to comment on what makes 
people not choose garbage collected languages for their games(Intertia 
is obvious, so I'll ignore it here).

Since memory is tight, most game developers want to have as close a 
level of control over the memory that they're using as possible.  They 
don't want something lurking around, waiting for the most inappropriate 
moment to choose to take a nondeterministic amount of time cleaning up. 
  Sure, you can disable garbage collection for a while, but eventually, 
unless you're very, very careful, that's going to bite you in the ass, 
and you're sunk there too.  Memory constraints, too, make people think 
badly of garbage collectors.  If you're worried about getting everything 
to fit, a lot of people think that giving away the control of your 
allocation and deallocation to a collector it tantamount to suicide, 
regardless of the fact that a well tuned and optimized GC'd program 
might be just as fast and more memory efficient than one written in hand 
allocated C.  The biggest barriers to entry then, are myths and 
misconceptions, but ones that have been around for ages and ages, which 
aren't going to go away anytime soon.  The biggest of them is that if 
you're going to do something that's big and fast, you really have to do 
it in C or C++ or some other manually allocated, low-level lanugage.

I think that in order to make people coming from non-collected languages 
less leery of using them, we should make the garbage collecter as open 
and helpful for after the fact optimization as possible.  Since the best 
  way to cut down on garbage collection times is to not generate any 
garbage, the GC should help the programmer to learn what s/he's doing to 
generate the garbage, and to replace those sections in the code with 
things that create less or no garbage to collect, making the program run 
in a more deterministic amount of space and time.  Statistics are a 
goods start, but being able to more intimately inspect the items in the 
garbage would be really nice for some programmers.  Garbage collection 
makes programming easier, no doubt, but it inserts some other variables 
into the overall equasion, and I think that it would be unwise to ignore 
them, and say, people will figure it out.  I don't think that many 
active C/C++ programmers have a good enough idea of what garbage 
collection is and does and how to just figure it out for a few 
statistics.  I think that having, initially and openly, a framework that 
would make it easy to write a garbage inspector that could be run 
occasionally throughout the debug mode of a program would be a really 
helpful thing, and would help people just coming into the language say, 
"oh, stupid me, I have a foo object that's hangnig around too long and 
preventing the collection of this half-meg of memory declared at line 15 
  of file monkey.d!" instead of saying, "stupid garbage collection, 
we're never going to get this to run quickly enough and in a small 
enough amout of space.  Forget D for this then.".  IDE support for these 
features would also be something almost without precedent.  I'm not 
carping, or trying to push this for my own good.  Unless I start my own 
studio (hah!) or go to work for someone who is likewise open-minded and 
interested in working in a truly modern lanugage, I'm not going to be 
writing D professionally for quite some time. I'm simply trying to put 
this forward as an idea that might help acceptance of the language and 
speed its growth.  Let me know if you think differently.

Evan
Oct 25 2002
next sibling parent reply "anderson" <anderson firestar.com.au> writes:
I think it's a good idea,  if you can extend the functionally that is
already there for garbage collection satistics.  Parhaps it could even
output some sort of graph for various things. Of course the graph component
wouldn't be simply limited to the GC statics.  Parhaps there could be a
statisics reporting module, that includes GUI stuff like graphs or something
simular to win98 defrag layout (for program memory).  So debugging widgets
could be centerlized and use a common mechinism.

Of cause that's probably going a bit overboard.  Any improvement you can
come up with would be good.

Anyway just a thaught anyway.

"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:apbe6u$2ohe$1 digitaldaemon.com...

 The game programmers that I know personally or through the internet are
 almost pathologically averse to using garbage collection.  There are
 some good reasons for this, but they're less important nowaday.  Still,
 opinions are slow to change.   There's only one company out there that I
 know of that uses a language other than C or C++ (occasionally with some
 python mixed in).  They use lisp, and since I worked with one of their
 founders for a while and have seen a lot of their commentary on working
 with lisp and garbage collection, I'd like to comment on what makes
 people not choose garbage collected languages for their games(Intertia
 is obvious, so I'll ignore it here).

 Since memory is tight, most game developers want to have as close a
 level of control over the memory that they're using as possible.  They
 don't want something lurking around, waiting for the most inappropriate
 moment to choose to take a nondeterministic amount of time cleaning up.
   Sure, you can disable garbage collection for a while, but eventually,
 unless you're very, very careful, that's going to bite you in the ass,
 and you're sunk there too.  Memory constraints, too, make people think
 badly of garbage collectors.  If you're worried about getting everything
 to fit, a lot of people think that giving away the control of your
 allocation and deallocation to a collector it tantamount to suicide,
 regardless of the fact that a well tuned and optimized GC'd program
 might be just as fast and more memory efficient than one written in hand
 allocated C.  The biggest barriers to entry then, are myths and
 misconceptions, but ones that have been around for ages and ages, which
 aren't going to go away anytime soon.  The biggest of them is that if
 you're going to do something that's big and fast, you really have to do
 it in C or C++ or some other manually allocated, low-level lanugage.

 I think that in order to make people coming from non-collected languages
 less leery of using them, we should make the garbage collecter as open
 and helpful for after the fact optimization as possible.  Since the best
   way to cut down on garbage collection times is to not generate any
 garbage, the GC should help the programmer to learn what s/he's doing to
 generate the garbage, and to replace those sections in the code with
 things that create less or no garbage to collect, making the program run
 in a more deterministic amount of space and time.  Statistics are a
 goods start, but being able to more intimately inspect the items in the
 garbage would be really nice for some programmers.  Garbage collection
 makes programming easier, no doubt, but it inserts some other variables
 into the overall equasion, and I think that it would be unwise to ignore
 them, and say, people will figure it out.  I don't think that many
 active C/C++ programmers have a good enough idea of what garbage
 collection is and does and how to just figure it out for a few
 statistics.  I think that having, initially and openly, a framework that
 would make it easy to write a garbage inspector that could be run
 occasionally throughout the debug mode of a program would be a really
 helpful thing, and would help people just coming into the language say,
 "oh, stupid me, I have a foo object that's hangnig around too long and
 preventing the collection of this half-meg of memory declared at line 15
   of file monkey.d!" instead of saying, "stupid garbage collection,
 we're never going to get this to run quickly enough and in a small
 enough amout of space.  Forget D for this then.".  IDE support for these
 features would also be something almost without precedent.  I'm not
 carping, or trying to push this for my own good.  Unless I start my own
 studio (hah!) or go to work for someone who is likewise open-minded and
 interested in working in a truly modern lanugage, I'm not going to be
 writing D professionally for quite some time. I'm simply trying to put
 this forward as an idea that might help acceptance of the language and
 speed its growth.  Let me know if you think differently.

 Evan
Oct 25 2002
parent Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
anderson wrote:

 I think it's a good idea,  if you can extend the functionally that is
 already there for garbage collection satistics.  Parhaps it could even
 output some sort of graph for various things. Of course the graph component
 wouldn't be simply limited to the GC statics.  Parhaps there could be a
 statisics reporting module, that includes GUI stuff like graphs or something
 simular to win98 defrag layout (for program memory).  So debugging widgets
 could be centerlized and use a common mechinism.


While I think that some sort of GUI inspector for inclusion in a 
debugging suite would be nice, I don't think that it belongs in the core 
of the language.  Not to mention that we don't seem to have a IDE aimed 
at D atm.  I think that it would make a lot more sense just to have some 
  data structure that describes (but doesn't allow manipulation of) the 
internal structure of the GC owned space an objects.  That, combined 
with a function that gives the referrers of any object described by the 
structure above, would provide a good enough base framework for 90% of 
the inspection that people would need to do, at least in terms of 
building on top of it.  Since the lib is open source anyway, someone who 
needed something on top of that 90% could add it to the library 
themselves without distorting the behavior of the GC for the rest of us.

Evan


 Of cause that's probably going a bit overboard.  Any improvement you can
 come up with would be good.
 
 Anyway just a thaught anyway.
Oct 29 2002
prev sibling next sibling parent reply "chris jones" <flak clara.co.uk> writes:
Did you know that the Unreal engine uses Garbage Colection? Has done since
Unreal Tournemant, so for about 4 years i think.

chris



"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:apbe6u$2ohe$1 digitaldaemon.com...

 The game programmers that I know personally or through the internet are
 almost pathologically averse to using garbage collection.  There are
 some good reasons for this, but they're less important nowaday.  Still,
 opinions are slow to change.   There's only one company out there that I
 know of that uses a language other than C or C++ (occasionally with some
 python mixed in).  They use lisp, and since I worked with one of their
 founders for a while and have seen a lot of their commentary on working
 with lisp and garbage collection, I'd like to comment on what makes
 people not choose garbage collected languages for their games(Intertia
 is obvious, so I'll ignore it here).

 Since memory is tight, most game developers want to have as close a
 level of control over the memory that they're using as possible.  They
 don't want something lurking around, waiting for the most inappropriate
 moment to choose to take a nondeterministic amount of time cleaning up.
   Sure, you can disable garbage collection for a while, but eventually,
 unless you're very, very careful, that's going to bite you in the ass,
 and you're sunk there too.  Memory constraints, too, make people think
 badly of garbage collectors.  If you're worried about getting everything
 to fit, a lot of people think that giving away the control of your
 allocation and deallocation to a collector it tantamount to suicide,
 regardless of the fact that a well tuned and optimized GC'd program
 might be just as fast and more memory efficient than one written in hand
 allocated C.  The biggest barriers to entry then, are myths and
 misconceptions, but ones that have been around for ages and ages, which
 aren't going to go away anytime soon.  The biggest of them is that if
 you're going to do something that's big and fast, you really have to do
 it in C or C++ or some other manually allocated, low-level lanugage.

 I think that in order to make people coming from non-collected languages
 less leery of using them, we should make the garbage collecter as open
 and helpful for after the fact optimization as possible.  Since the best
   way to cut down on garbage collection times is to not generate any
 garbage, the GC should help the programmer to learn what s/he's doing to
 generate the garbage, and to replace those sections in the code with
 things that create less or no garbage to collect, making the program run
 in a more deterministic amount of space and time.  Statistics are a
 goods start, but being able to more intimately inspect the items in the
 garbage would be really nice for some programmers.  Garbage collection
 makes programming easier, no doubt, but it inserts some other variables
 into the overall equasion, and I think that it would be unwise to ignore
 them, and say, people will figure it out.  I don't think that many
 active C/C++ programmers have a good enough idea of what garbage
 collection is and does and how to just figure it out for a few
 statistics.  I think that having, initially and openly, a framework that
 would make it easy to write a garbage inspector that could be run
 occasionally throughout the debug mode of a program would be a really
 helpful thing, and would help people just coming into the language say,
 "oh, stupid me, I have a foo object that's hangnig around too long and
 preventing the collection of this half-meg of memory declared at line 15
   of file monkey.d!" instead of saying, "stupid garbage collection,
 we're never going to get this to run quickly enough and in a small
 enough amout of space.  Forget D for this then.".  IDE support for these
 features would also be something almost without precedent.  I'm not
 carping, or trying to push this for my own good.  Unless I start my own
 studio (hah!) or go to work for someone who is likewise open-minded and
 interested in working in a truly modern lanugage, I'm not going to be
 writing D professionally for quite some time. I'm simply trying to put
 this forward as an idea that might help acceptance of the language and
 speed its growth.  Let me know if you think differently.

 Evan
Oct 25 2002
parent reply "Sean L. Palmer" <seanpalmer directvinternet.com> writes:
It (UT2K3) only collects between levels.  I had a problem once during a
5-hour CTF match where the game started bogging down bigtime because of
this.

Sean

"chris jones" <flak clara.co.uk> wrote in message
news:apds5h$2mbd$1 digitaldaemon.com...

 Did you know that the Unreal engine uses Garbage Colection? Has done since
 Unreal Tournemant, so for about 4 years i think.

 chris
Oct 26 2002
parent reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
Sean L. Palmer wrote:

 It (UT2K3) only collects between levels.  I had a problem once during a
 5-hour CTF match where the game started bogging down bigtime because of
 this.
 
 Sean
 
 "chris jones" <flak clara.co.uk> wrote in message
 news:apds5h$2mbd$1 digitaldaemon.com...
 

Did you know that the Unreal engine uses Garbage Colection? Has done since
Unreal Tournemant, so for about 4 years i think.

chris




That UT has garbage collection is all well and good, but a few games do 
not a trend make.  Sean's point that it only collects between levels is 
indicative of the fact that it isn't trusted, and I think that it 
supports my opinion that a well integrated and easy to inspect GC system 
would be a boon to the lanugage.

Evan
Oct 29 2002
next sibling parent reply "chris jones" <flak clara.co.uk> writes:
"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:aplsm1$2glk$1 digitaldaemon.com...

 Sean L. Palmer wrote:

 It (UT2K3) only collects between levels.  I had a problem once during a
 5-hour CTF match where the game started bogging down bigtime because of
 this.

 Sean

 "chris jones" <flak clara.co.uk> wrote in message
 news:apds5h$2mbd$1 digitaldaemon.com...


Did you know that the Unreal engine uses Garbage Colection? Has done






since

Unreal Tournemant, so for about 4 years i think.

chris




 That UT has garbage collection is all well and good, but a few games do
 not a trend make.


I was only saying GC can work well for games.


 Sean's point that it only collects between levels is
 indicative of the fact that it isn't trusted,


I dont agree, it colects between levels because it is the best way to do it.
Why collect as you go if you dont need to? Between levels is the perfect
time to collect and then *only* do it in game if it runs low on resources.


and I think that it
 supports my opinion that a well integrated and easy to inspect GC system
 would be a boon to the lanugage.


I agree with you. But if you are developing a large game engine you would
probably write your own resource management code anyway wouldnt you? A GC
writen specificly for games would be better that a general GC would it not?

chris
Oct 29 2002
parent Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
chris jones wrote:

 "Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
 news:aplsm1$2glk$1 digitaldaemon.com...
 

Sean L. Palmer wrote:


It (UT2K3) only collects between levels.  I had a problem once during a
5-hour CTF match where the game started bogging down bigtime because of
this.

Sean

"chris jones" <flak clara.co.uk> wrote in message
news:apds5h$2mbd$1 digitaldaemon.com...



Did you know that the Unreal engine uses Garbage Colection? Has done






 since
 

Unreal Tournemant, so for about 4 years i think.

chris




That UT has garbage collection is all well and good, but a few games do
not a trend make.


 
 
 I was only saying GC can work well for games.


Of course it can, that's what I want to say.  I'm just saying that it 
isn't popular nor well trusted.


Sean's point that it only collects between levels is
indicative of the fact that it isn't trusted,


 
 I dont agree, it colects between levels because it is the best way to do it.
 Why collect as you go if you dont need to? Between levels is the perfect
 time to collect and then *only* do it in game if it runs low on resources.


what if you don't want levels?  Where do you do the collection if you're 
writing a seamless game like dungeon siege?


and I think that it
supports my opinion that a well integrated and easy to inspect GC system
would be a boon to the lanugage.


 
 I agree with you. But if you are developing a large game engine you would
 probably write your own resource management code anyway wouldnt you? A GC
 writen specificly for games would be better that a general GC would it not?


I think that having to write a large amount of resource management code 
is a sign that something is broken in the environment in which you're 
working.   I feel like automatic mem management can and should work 
better.  It is pretty mechanical, so we should be able to automate most 
of it away, I feel.  Maybe we're not at that stage yet, but I feel that 
many different approaches should be tried.


Evan
Oct 30 2002
prev sibling parent reply "Sean L. Palmer" <seanpalmer directvinternet.com> writes:
People can always turn the GC off.  But I don't think the D languages gives
us enough power to replace it with anything else.

My first change would be to add in per-object refcounting.  Say what you
want about cycles, I still have far less problems with refcounted objects
than I'd have with manually managed objects, which is the only alternative
if you turn off GC.  I don't think we have language hooks for calling
alternate versions of new and delete.

I still don't trust this spooky voodoo GC stuff.  It's not predictable
enough.  I suppose the language could be the best choice for managing memory
but what if it doesn't do a good job, what then?  Switch back to C++?  An
alternative would be nice.  GC doesn't give you enough guarantees about
object lifetimes and doesn't help with other types of resources than memory.
It is only a partial solution;  if done well, perhaps only marginally better
than refcounting.

Admittedly a good compiler could in theory generate really concise GC code
by only scanning exactly where it knows the objects might be (i.e. GC keeps
track of types of live objects and uses info about the types to scan only
places where the data might legally be).  And that might be the most
efficient way possible, who knows?  This topic is still up for debate.
Someone will invent a better kind of resource management soon I bet.

You know the kind of language research that's going on right now.  There are
bright minds working on this and one of them is sure to hit pay dirt at some
point.

Refcounting actually works in real production object systems like COM.  I
can't remember the last time I leaked a texture or something in D3D.  And we
even have to manually call Release(), or write a RAII wrapper and try to
force the team to use it.  If it's implicit in the language it'd work
better.  Tacking it on later isn't a great option (though with D's new
generics we could probably manage it).

One thing someone will want to do in D when replacing its memory manager is
get the size of the maximum free block the system can possibly allocate.
You allocate all that and then write your own memory manager.  If you can
plug that into new and delete then great.  Otherwise we need to be able to
call constructors and destructors manually.  I suppose it's a lot of work,
but there's a reason this stuff is overloadable in C++.

Sean

"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:aplsm1$2glk$1 digitaldaemon.com...

 Sean L. Palmer wrote:

 It (UT2K3) only collects between levels.  I had a problem once during a
 5-hour CTF match where the game started bogging down bigtime because of
 this.

 Sean

 "chris jones" <flak clara.co.uk> wrote in message
 news:apds5h$2mbd$1 digitaldaemon.com...


Did you know that the Unreal engine uses Garbage Colection? Has done






since

Unreal Tournemant, so for about 4 years i think.

chris




 That UT has garbage collection is all well and good, but a few games do
 not a trend make.  Sean's point that it only collects between levels is
 indicative of the fact that it isn't trusted, and I think that it
 supports my opinion that a well integrated and easy to inspect GC system
 would be a boon to the lanugage.

 Evan
Oct 29 2002
parent reply Mac Reiter <Mac_member pathlink.com> writes:
In article <apo1k4$kkq$1 digitaldaemon.com>, Sean L. Palmer says...

People can always turn the GC off.  But I don't think the D languages gives
us enough power to replace it with anything else.

My first change would be to add in per-object refcounting.  Say what you
want about cycles, I still have far less problems with refcounted objects
than I'd have with manually managed objects, which is the only alternative
if you turn off GC.  I don't think we have language hooks for calling
alternate versions of new and delete.

I still don't trust this spooky voodoo GC stuff.  It's not predictable
enough.  I suppose the language could be the best choice for managing memory
but what if it doesn't do a good job, what then?  Switch back to C++?  An
alternative would be nice.  GC doesn't give you enough guarantees about
object lifetimes and doesn't help with other types of resources than memory.
It is only a partial solution;  if done well, perhaps only marginally better
than refcounting.


First, let me start by saying that I also would like to see reference counting
as an option, using a "this class is reference counted" keyword.  I would not
want reference counting used for all objects, because the runtime cost is too
high.  Reference counting systems incur a cost per dereference, since the
reference counting has to occur at the pointer.  And the refcounting cost tends
to be in cache misses and other memory coherency issues, so you can't just look
at the code and say "but it only took two extra cycles to do these
instructions".

I picked up a book on garbage collection theory.  It is somewhat dated, but
still useful.  Garbage collection is not "perhaps only marginally better than
refcounting" if your metric is overall system processing speed.  Unfortunately,
we hit the old batch/multitasking issue again.  GC provides higher throughput
over the long haul -- even over extremely long haul so that you are forcing some
garbage collection cycles in there.  (Over the short haul, GC can end up costing
nothing, because if you never run out of memory, you may never need to collect,
so the cost is 0 cycles.  Hence, you need a long haul study.)  Unfortunately, GC
tends to be "spiky" in CPU usage.  Incremental, generational scavenging GC's,
which are the most common today, are much less spiky than the older "wait til
we're full and gather it all up" systems.  But they are still spiky.
Refcounting, at least in theory, has much lower latency and a more even
distribution of cost.  The total cost is noticably higher (10% or more) than GC,
but it is spread around so that you don't have moments when your system appears
to lock up for a second or so.  That means that if your goal is, say, numeric
computation, you want GC -- absolute maximum efficiency, even if we periodically
stop for a few seconds.  If your goal is a semi-realtime, smooth experience,
like games or multimedia editing, you are more interested in a more evenly
amortized cost, even if the total cost is higher.  Purchasing a house outright
is certainly cheaper, as long as you can afford the $150,000 lump sum.  If you
can't afford it, you may have to pay $250,000 overall, but you do it over a long
time and it doesn't hurt as much.

Because the "right" system depends on the goal, I would really like to see both
options available, so that the developer can choose the appropriate solution for
his or her application.  The developer could even mix and match -- refcounting
for high volume but short-lived objects, and GC for everything else, or some
other mix.

And just as a final comment in defense of GC -- you *cannot* leak memory in a
properly designed GC system.  If it is not reachable, it *will* be collected
(eventually).  It is very easy to leak memory in a refcounted system through
circular references.  If you have to manually Release(), it is even easier to
lose things by forgetting a Release, at which point it will never get fixed
unless a GC comes along and notices it.  Presumably, language support for
refcounting would automate the Release so that it could not be forgotten.

And if you have the foo object that is hanging around too long and holding an
entire tree of objects in memory (as mentioned in an earlier post), neither GC
nor refcounting can help you -- so the memory manager information interface
would also be a good idea.

Mac
Oct 30 2002
parent Mac Reiter <Mac_member pathlink.com> writes:
In article <app6mq$1k1t$1 digitaldaemon.com>, Mac Reiter says...

First, let me start by saying that I also would like to see reference counting
as an option, using a "this class is reference counted" keyword.  I would not
want reference counting used for all objects, because the runtime cost is too


OK, before I get smacked around again for being loose with my terms : when I say
"I would not want reference counting used for all objects" I mean "for all
objects, or for all references, or for all classes".  I don't really care, for
purposes of this level of discussion.  I think reference counting would be a
good option, but it must be optional.  Exactly where it would be specified would
be a whole other discussion...

Mac
Oct 30 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
Your ideas and reasoning are right on target. (I, too, used to think badly
of gc, until I implemented a gc for Java.)

"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:apbe6u$2ohe$1 digitaldaemon.com...

 The game programmers that I know personally or through the internet are
 almost pathologically averse to using garbage collection.  There are
 some good reasons for this, but they're less important nowaday.  Still,
 opinions are slow to change.   There's only one company out there that I
 know of that uses a language other than C or C++ (occasionally with some
 python mixed in).  They use lisp, and since I worked with one of their
 founders for a while and have seen a lot of their commentary on working
 with lisp and garbage collection, I'd like to comment on what makes
 people not choose garbage collected languages for their games(Intertia
 is obvious, so I'll ignore it here).

 Since memory is tight, most game developers want to have as close a
 level of control over the memory that they're using as possible.  They
 don't want something lurking around, waiting for the most inappropriate
 moment to choose to take a nondeterministic amount of time cleaning up.
   Sure, you can disable garbage collection for a while, but eventually,
 unless you're very, very careful, that's going to bite you in the ass,
 and you're sunk there too.  Memory constraints, too, make people think
 badly of garbage collectors.  If you're worried about getting everything
 to fit, a lot of people think that giving away the control of your
 allocation and deallocation to a collector it tantamount to suicide,
 regardless of the fact that a well tuned and optimized GC'd program
 might be just as fast and more memory efficient than one written in hand
 allocated C.  The biggest barriers to entry then, are myths and
 misconceptions, but ones that have been around for ages and ages, which
 aren't going to go away anytime soon.  The biggest of them is that if
 you're going to do something that's big and fast, you really have to do
 it in C or C++ or some other manually allocated, low-level lanugage.

 I think that in order to make people coming from non-collected languages
 less leery of using them, we should make the garbage collecter as open
 and helpful for after the fact optimization as possible.  Since the best
   way to cut down on garbage collection times is to not generate any
 garbage, the GC should help the programmer to learn what s/he's doing to
 generate the garbage, and to replace those sections in the code with
 things that create less or no garbage to collect, making the program run
 in a more deterministic amount of space and time.  Statistics are a
 goods start, but being able to more intimately inspect the items in the
 garbage would be really nice for some programmers.  Garbage collection
 makes programming easier, no doubt, but it inserts some other variables
 into the overall equasion, and I think that it would be unwise to ignore
 them, and say, people will figure it out.  I don't think that many
 active C/C++ programmers have a good enough idea of what garbage
 collection is and does and how to just figure it out for a few
 statistics.  I think that having, initially and openly, a framework that
 would make it easy to write a garbage inspector that could be run
 occasionally throughout the debug mode of a program would be a really
 helpful thing, and would help people just coming into the language say,
 "oh, stupid me, I have a foo object that's hangnig around too long and
 preventing the collection of this half-meg of memory declared at line 15
   of file monkey.d!" instead of saying, "stupid garbage collection,
 we're never going to get this to run quickly enough and in a small
 enough amout of space.  Forget D for this then.".  IDE support for these
 features would also be something almost without precedent.  I'm not
 carping, or trying to push this for my own good.  Unless I start my own
 studio (hah!) or go to work for someone who is likewise open-minded and
 interested in working in a truly modern lanugage, I'm not going to be
 writing D professionally for quite some time. I'm simply trying to put
 this forward as an idea that might help acceptance of the language and
 speed its growth.  Let me know if you think differently.

 Evan
Oct 31 2002
next sibling parent reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
Walter wrote:

 Your ideas and reasoning are right on target. (I, too, used to think badly
 of gc, until I implemented a gc for Java.)



Been reading up a lot on GC lately, and this paper 
(http://cs.anu.edu.au/~Steve.Blackburn/pubs/papers/beltway-pldi-2002.pdf)
looks interesting in the way that it extends garbage collection.  I 
really feel like a sudden convert.  I've known about GC for years, of 
course, but I didn't have a whole lot of experience working with it 
(other than with perl, where it's so utterly transparent at most levels 
that you don't even think about it).  Basically, I finally noticed that 
I was spending as much time messing about with memory as I was 
programming a lot of the time.  I decided that I didn't want to do it 
anymore, almost to the degree to where I'm thin
Oct 31 2002
next sibling parent "chris jones" <flak clara.co.uk> writes:
I was converted once i read alot of the posts on this newsgroup.

chris

"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:aprnqm$1b56$1 digitaldaemon.com...

 Walter wrote:

 Your ideas and reasoning are right on target. (I, too, used to think




badly

 of gc, until I implemented a gc for Java.)



 Been reading up a lot on GC lately, and this paper
 (http://cs.anu.edu.au/~Steve.Blackburn/pubs/papers/beltway-pldi-2002.pdf)
 looks interesting in the way that it extends garbage collection.  I
 really feel like a sudden convert.  I've known about GC for years, of
 course, but I didn't have a whole lot of experience working with it
 (other than with perl, where it's so utterly transparent at most levels
 that you don't even think about it).  Basically, I finally noticed that
 I was spending as much time messing about with memory as I was
 programming a lot of the time.  I decided that I didn't want to do it
 anymore, almost to the degree to where I'm thin
Oct 31 2002
prev sibling parent "Walter" <walter digitalmars.com> writes:
"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:aprnqm$1b56$1 digitaldaemon.com...

 Been reading up a lot on GC lately, and this paper
 (http://cs.anu.edu.au/~Steve.Blackburn/pubs/papers/beltway-pldi-2002.pdf)
 looks interesting in the way that it extends garbage collection.  I
 really feel like a sudden convert.  I've known about GC for years, of
 course, but I didn't have a whole lot of experience working with it
 (other than with perl, where it's so utterly transparent at most levels
 that you don't even think about it).  Basically, I finally noticed that
 I was spending as much time messing about with memory as I was
 programming a lot of the time.  I decided that I didn't want to do it
 anymore, almost to the degree to where I'm thin


I've noticed that my code just gets a lot simpler, cleaner, and less buggy
using a gc.
Nov 12 2002
prev sibling parent reply Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
ok, lets try that again, starting with the last sentance (mispost):

I decided that I didn't want to do it anymore, to the degree to where 
I'm thinking of going back to school to do some real research on the 
topic (implement GC as a kernel level serivce in linux and then start 
porting things to a D version that can use that service, and see what 
shakes out, performance wise, since I've seen very little research 
(beyond lispos) on GC as a system allocator).  Dunno.  I feel like its 
an idea who's time has definately come.

Evan
Oct 31 2002
parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
Why stop there, what about a whole OS written in D, everything an object and
collectable, including dynamicly linked libries/shared objects, applications
etc.
the GC would have to cope with a couple of extra things;
1) locking pages into ram so they don't get swapped for device drivers
2) the possibility that swap also needs to be gc'd
3) virtual <-> logical address translation and thread local addresses.
instead of each app running its own GC there will just be ONE system GC
(ideally a fully concurrent collector)

Mike.

"Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
news:apro38$1be5$1 digitaldaemon.com...

 ok, lets try that again, starting with the last sentance (mispost):

 I decided that I didn't want to do it anymore, to the degree to where
 I'm thinking of going back to school to do some real research on the
 topic (implement GC as a kernel level serivce in linux and then start
 porting things to a D version that can use that service, and see what
 shakes out, performance wise, since I've seen very little research
 (beyond lispos) on GC as a system allocator).  Dunno.  I feel like its
 an idea who's time has definately come.

 Evan
Oct 31 2002
next sibling parent "Andrew Edwards" <crxace13 comcast.net> writes:
You know, that's a very good idea.  I thought about that, but unfortunately
I'm not equipped to take on a challenge of such magnitude as yet.  I'd be
interested in seeing the product when you or anyone here decide to embark
on that journey.

Andrew

"Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
news:aprs2h$1fas$1 digitaldaemon.com...
| Why stop there, what about a whole OS written in D, everything an object
and
| collectable, including dynamicly linked libries/shared objects,
applications
| etc.
| the GC would have to cope with a couple of extra things;
| 1) locking pages into ram so they don't get swapped for device drivers
| 2) the possibility that swap also needs to be gc'd
| 3) virtual <-> logical address translation and thread local addresses.
| instead of each app running its own GC there will just be ONE system GC
| (ideally a fully concurrent collector)
|
| Mike.
|
| "Evan McClanahan" <evan dontSPAMaltarinteractive.com> wrote in message
| news:apro38$1be5$1 digitaldaemon.com...
| > ok, lets try that again, starting with the last sentance (mispost):
| >
| > I decided that I didn't want to do it anymore, to the degree to where
| > I'm thinking of going back to school to do some real research on the
| > topic (implement GC as a kernel level serivce in linux and then start
| > porting things to a D version that can use that service, and see what
| > shakes out, performance wise, since I've seen very little research
| > (beyond lispos) on GC as a system allocator).  Dunno.  I feel like its
| > an idea who's time has definately come.
| >
| > Evan
| >
|
|
Oct 31 2002
prev sibling next sibling parent Evan McClanahan <evan dontSPAMaltarinteractive.com> writes:
Mike Wynn wrote:

 Why stop there, what about a whole OS written in D, everything an object and
 collectable, including dynamicly linked libries/shared objects, applications
 etc.
 the GC would have to cope with a couple of extra things;
 1) locking pages into ram so they don't get swapped for device drivers
 2) the possibility that swap also needs to be gc'd
 3) virtual <-> logical address translation and thread local addresses.
 instead of each app running its own GC there will just be ONE system GC
 (ideally a fully concurrent collector)


My idea was something like that, but personally, I don't have the 
mental, education, or emotional resources to write a whole OS in a year. 
  I my idea, in more detail, was to add garbage collection as an option 
in addition to normal hand management of memory, which would be a 
globlal system that all programs would use, unless they choose to manage 
their own memory. While it would likely be simpler to write a very 
simple OS in D that has garbage collection only, I think that it's more 
practical and would be better at proving to people that GC has come of 
age for performance critical imperative apps if I worked it into some 
existing system, and then was able to objectively show (or not show) the 
benefits  of system level GC in terms of code size, speed, safety, and 
reliability on existing applications.  Of course, it's still at the idea 
stage, and the probability is low that I'll even be able to get into a 
graduate school with my undergrad grades, especially as I haven't even 
started applying yet, and my choice of schools is rather limited.  So it 
might not happen, but we'll see.


Evan
Nov 01 2002
prev sibling parent "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
news:aprs2h$1fas$1 digitaldaemon.com...

 Why stop there, what about a whole OS written in D, everything an object


and

 collectable, including dynamicly linked libries/shared objects,


applications

 etc.
 the GC would have to cope with a couple of extra things;
 1) locking pages into ram so they don't get swapped for device drivers
 2) the possibility that swap also needs to be gc'd
 3) virtual <-> logical address translation and thread local addresses.
 instead of each app running its own GC there will just be ONE system GC
 (ideally a fully concurrent collector)


I think that will come.
Nov 12 2002