www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - Delegate-object invariant bug on program exit.

reply pragma <pragma_member pathlink.com> writes:
DMD v100.00
OPTLINK v7.50B1
Win2k SP4

The following creates a general protection fault.  I've traced it down to
phobos\internal\invariant.d (_d_invariant).

import std.stdio;

class Foo{
public void notify(){
writefln("notified");
}
}
class Bar{
public void delegate() event;
~this(){
event();
}	
}
void main(){
Bar b = new Bar();
Foo f = new Foo();
b.event = &(f.notify);
}

It looks the instance of Foo is being collected before the instance of Bar, thus
invalidating the delegate.

- Pragma

[[ EricAnderton at (don't spam me anymore) yahoo.com ]]
Aug 24 2004
parent reply Ben Hinkle <bhinkle4 juno.com> writes:
pragma wrote:


 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4
 
 The following creates a general protection fault.  I've traced it down to
 phobos\internal\invariant.d (_d_invariant).
 
 import std.stdio;
 
 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }
 
 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.
 
 - Pragma
 
 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]


I think it will be quite hard getting this kind of thing to work. A
destructor should never try to reference another object - since as you
found out it might be a bogus reference. I'd like to see a sentance or two
in the D destructor doc to avoid referencing other objects - or anything
that belongs to the GC. See my post on the main newsgroup
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023

-Ben
Aug 24 2004
parent reply "antiAlias" <fu bar.com> writes:
"Ben Hinkle" <bhinkle4 juno.com> wrote in message
news:cggsr7$1i9o$1 digitaldaemon.com...

 pragma wrote:


 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4

 The following creates a general protection fault.  I've traced it down




to

 phobos\internal\invariant.d (_d_invariant).

 import std.stdio;

 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }

 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.

 - Pragma

 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]


 I think it will be quite hard getting this kind of thing to work. A
 destructor should never try to reference another object - since as you
 found out it might be a bogus reference. I'd like to see a sentance or two
 in the D destructor doc to avoid referencing other objects - or anything
 that belongs to the GC. See my post on the main newsgroup
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023

 -Ben



Yes, that would be helpful. But it's not /always/ practical to avoid
touching other objects during destruction, though perhaps 'desirable'.

And it's not an obvious error either, since one could hardly be blamed for
thinking the delegate holds a reference to its enclosing class instance
(it's a delegate; not a function). If so, then the instance should still be
live as long as the delegate is referenced ... yes? Apparently that's not
how it works. Sure looks like a bug.

Do you know what the delegate holds a reference to, Ben?
Aug 24 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
antiAlias wrote:


 "Ben Hinkle" <bhinkle4 juno.com> wrote in message
 news:cggsr7$1i9o$1 digitaldaemon.com...

 pragma wrote:


 DMD v100.00
 OPTLINK v7.50B1
 Win2k SP4

 The following creates a general protection fault.  I've traced it down




 to

 phobos\internal\invariant.d (_d_invariant).

 import std.stdio;

 class Foo{
 public void notify(){
 writefln("notified");
 }
 }
 class Bar{
 public void delegate() event;
 ~this(){
 event();
 }
 }
 void main(){
 Bar b = new Bar();
 Foo f = new Foo();
 b.event = &(f.notify);
 }

 It looks the instance of Foo is being collected before the instance of
 Bar, thus invalidating the delegate.

 - Pragma

 [[ EricAnderton at (don't spam me anymore) yahoo.com ]]


 I think it will be quite hard getting this kind of thing to work. A
 destructor should never try to reference another object - since as you
 found out it might be a bogus reference. I'd like to see a sentance or
 two in the D destructor doc to avoid referencing other objects - or
 anything that belongs to the GC. See my post on the main newsgroup
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/9023

 -Ben


 
 
 Yes, that would be helpful. But it's not /always/ practical to avoid
 touching other objects during destruction, though perhaps 'desirable'.


well... just as long as none of those objects need collecting at program
exit. Otherwise it's a game of chance which one gets GC'ed first.


 And it's not an obvious error either, since one could hardly be blamed for
 thinking the delegate holds a reference to its enclosing class instance
 (it's a delegate; not a function). If so, then the instance should still
 be live as long as the delegate is referenced ... yes? Apparently that's
 not how it works. Sure looks like a bug.


True - it should be considered a bug. Java (I think) allows other objects to
be referenced in finalizers. I don't know the details, though, since
finalizers are frowned upon I've never really gotten into it.


 Do you know what the delegate holds a reference to, Ben?


in the case above, f.
Aug 24 2004
parent reply pragma <pragma_member pathlink.com> writes:
 
 Yes, that would be helpful. But it's not /always/ practical to avoid
 touching other objects during destruction, though perhaps 'desirable'.


well... just as long as none of those objects need collecting at program
exit. Otherwise it's a game of chance which one gets GC'ed first.


It gets even more frustrating than that.  Similar to what you ran into with
streams, I have a .dll Library class that has delegate array (event) that
functions as a multicast callback to other interested parties when it is
unloaded.  This is a much needed runtime behavior, and is especially useful to
fire this event when the library is unloaded during finalization.


 And it's not an obvious error either, since one could hardly be blamed for
 thinking the delegate holds a reference to its enclosing class instance
 (it's a delegate; not a function). If so, then the instance should still
 be live as long as the delegate is referenced ... yes? Apparently that's
 not how it works. Sure looks like a bug.




Yep, that's basically where I was when I filed the report.  The delegate has an
outstanding reference, yet it's object was destroyed.


True - it should be considered a bug. Java (I think) allows other objects to
be referenced in finalizers. I don't know the details, though, since
finalizers are frowned upon I've never really gotten into it.


It creates a lot of problems, especially when you're relying on the GC to
perform collections during the execution of the program.  Its nice to know when
something is completely destroyed, and releases its resources:






Something like this becomes a problem with the current revision of D: it works
great during runtime and then (very likely) fails miserably once you exit the
program.  The problem also gets worse when expecting to be able to free a
resource on termination, but can't since some random destructor causes a GPF.  

Honestly, my expectation was that during the final collection (before program
exit) all the finalizers would be run *first* before any memory is freed.
















I get the impression that it's doing something like this instead:








- Pragma
[[ EricAnderton at (pounding away on code) yahoo.com ]]
Aug 25 2004
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 Honestly, my expectation was that during the final collection (before


program

 exit) all the finalizers would be run *first* before any memory is freed.


That makes sense - though it would cause two loops over the garbage instead
of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data. What if one of
the destructors assigns a reference to some garbage to a static variable and
thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was brought
to life. That would be a nasty performance hit.

It occured to me that this problem of having destructors reference other
objects isn't limited to program exit. It can happen any time there is a
collection. The reason it shows up at exit often is that D *always* collects
at exit and our programs that have the problem don't run long enough to
force collections during normal run-time (or the object being referenced
isn't garbage during runtime).

Or we can keep the current architecture and just document the fact that
destructors can't reference any other GC managed resources (or anything that
has managed GC resources). Nasty but it lets the GC do whatever it wants.
Aug 25 2004
parent reply pragma <pragma_member pathlink.com> writes:
In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...

 Honestly, my expectation was that during the final collection (before


program

 exit) all the finalizers would be run *first* before any memory is freed.


That makes sense - though it would cause two loops over the garbage instead
of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data. 


I see what you mean.  Granted, there's nothing keeping a destructor from
creating *more* data on the heap in a growing spiral of allocations.  But I
really don't see that as any more a problem as infinite recursion or
mutually-referenced functions (like in the invariant 'bug' menioned earlier in
this NG).  But I strongly feel that reducing destructors to playing with only
scalar types (as we've discovered that object references are all "weak" in the
present D implementation during the final collect) reduces their capability to
near uselessness.  

I also just noticed that my pseudocode on the previous post was also a little
flawed, and hence misleading.  If one were to loop over a /copy/ of the current
set of finalizers, after clearing out the old list, the outer loop stands a good
chance of actually terminating.


What if one of
the destructors assigns a reference to some garbage to a static variable and
thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was brought
to life. That would be a nasty performance hit.


Well, "ressurecting" an object properly would require the GC to register that
object's finalizer, in which case it looks like a normal object.  That /would/
thwart the present behavior as well as my informal proposal in my previous post.

Personally, I'd rather have all my objects a /chance/ of cleaning up propertly
on program exit rather than have unpredictable behavior inside of destructors.
I know what I'm asking for is deterministic destruction of objects, in all
cases, which has been something of a holy war on the D NG.

I think its possible to finialize in the proposed method and abort to a full
free of the heap if the loop runs for too long, or if there's no change in the
number of finalizers for a number of iterations.  At least that way, you have
given the program a fighting chance of being well behaved on exit without
bending over backwards in your code.

Another way to go would be to allow the GC to track a set of privileged
references that are guaranteed to be finalized before a dealloc on exit.  These
would be explicitly set in the client code and would be understood to delay
program termination by ever so much.

At the very least, it'd be nice to have a way to check if a reference is still
valid, and hasn't been collected.  That way one could treat object references as
'weak' if you're using code inside of a destructor.

-Pragma
[[ EricAnderton at (let my objects go) yahoo.com ]]
Aug 25 2004
next sibling parent reply "antiAlias" <fu bar.com> writes:
If this behavior is a trade-off between correct program-execution and
"performance" during program termination, then there's something very
seriously wrong with the foundations of this language.

As Pragma points out, if you can't reliably release resources during a
destructor then their value is diminished to the level of worthless.
Actually better not to have destructors at all. Now where would that leave
us?

What's the point of tracking references and dependencies if they're then
simply ignored during Object destruction? Utter lunacy.


"pragma" <pragma_member pathlink.com> wrote in message
news:cgif07$29ev$1 digitaldaemon.com...

 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...

 Honestly, my expectation was that during the final collection (before


program

 exit) all the finalizers would be run *first* before any memory is






freed.

That makes sense - though it would cause two loops over the garbage




instead

of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.


 I see what you mean.  Granted, there's nothing keeping a destructor from
 creating *more* data on the heap in a growing spiral of allocations.  But


I

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned


earlier in

 this NG).  But I strongly feel that reducing destructors to playing with


only

 scalar types (as we've discovered that object references are all "weak" in


the

 present D implementation during the final collect) reduces their


capability to

 near uselessness.

 I also just noticed that my pseudocode on the previous post was also a


little

 flawed, and hence misleading.  If one were to loop over a /copy/ of the


current

 set of finalizers, after clearing out the old list, the outer loop stands


a good

 chance of actually terminating.


What if one of
the destructors assigns a reference to some garbage to a static variable




and

thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was




brought

to life. That would be a nasty performance hit.


 Well, "ressurecting" an object properly would require the GC to register


that

 object's finalizer, in which case it looks like a normal object.  That


/would/

 thwart the present behavior as well as my informal proposal in my previous


post.

 Personally, I'd rather have all my objects a /chance/ of cleaning up


propertly

 on program exit rather than have unpredictable behavior inside of


destructors.

 I know what I'm asking for is deterministic destruction of objects, in all
 cases, which has been something of a holy war on the D NG.

 I think its possible to finialize in the proposed method and abort to a


full

 free of the heap if the loop runs for too long, or if there's no change in


the

 number of finalizers for a number of iterations.  At least that way, you


have

 given the program a fighting chance of being well behaved on exit without
 bending over backwards in your code.

 Another way to go would be to allow the GC to track a set of privileged
 references that are guaranteed to be finalized before a dealloc on exit.


These

 would be explicitly set in the client code and would be understood to


delay

 program termination by ever so much.

 At the very least, it'd be nice to have a way to check if a reference is


still

 valid, and hasn't been collected.  That way one could treat object


references as

 'weak' if you're using code inside of a destructor.

 -Pragma
 [[ EricAnderton at (let my objects go) yahoo.com ]]
Aug 25 2004
parent reply pragma <pragma_member pathlink.com> writes:
In article <cgiiff$2bio$1 digitaldaemon.com>, antiAlias says...

If this behavior is a trade-off between correct program-execution and
"performance" during program termination, then there's something very
seriously wrong with the foundations of this language.

As Pragma points out, if you can't reliably release resources during a
destructor then their value is diminished to the level of worthless.
Actually better not to have destructors at all. Now where would that leave
us?


I'll stay off my soapbox on this one.  However, I'll be happy to provide any
support to the cause should there be a weighing-in of pros and cons.

All I want is to write good software in D, and through that have D succeed.


What's the point of tracking references and dependencies if they're then
simply ignored during Object destruction? Utter lunacy.


Well let's not jump to conclusions... I would like to contend that this seems
like a mistake in implementation and is still likely a "bug" until proven
otherwise. :)

(Now, if it's been said before that the current behavior is quite deliberate,
then I'll stand corrected.)

-Pragma
[[ EricAnderton at (making D better) yahoo.com ]]
Aug 25 2004
parent reply "antiAlias" <fu bar.com> writes:
 Well let's not jump to conclusions... I would like to contend that this


seems

 like a mistake in implementation and is still likely a "bug" until proven
 otherwise. :)


Fair point Eric. Having come-a-cropper over so many fundamental "no ...
surely not!" type issues, I'm afraid my resilience against more of same has
been frayed to tatters. Part of the whole "library development issues: what
to do?" thread.


 All I want is to write good software in D, and through that have D


succeed.

Hear hear. There's no other reason why I've spent four months 80-hours a
week on Mango. Unfortunately, it sometimes feels as though the language, and
/particularly/ the maturation 'process' behind it, is actually working
against that goal (note that the "what to do?" thread all but disappeared
down the black-hole). More's the pity.
Aug 25 2004
parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"antiAlias" <fu bar.com> wrote in message
news:cgipm8$2f1j$1 digitaldaemon.com...

 Well let's not jump to conclusions... I would like to contend that this


 seems

 like a mistake in implementation and is still likely a "bug" until proven
 otherwise. :)


 Fair point Eric. Having come-a-cropper over so many fundamental "no ...
 surely not!" type issues, I'm afraid my resilience against more of same has
 been frayed to tatters. Part of the whole "library development issues: what
 to do?" thread.


LOL!

Shared, pain, old buddy. But fear not, you have several weeks while Phoenix
founders before you must face reality.

Or, optimistically, it might actually succeed, in which case we'll all be
happy. :)


 All I want is to write good software in D, and through that have D


 succeed.

 Hear hear. There's no other reason why I've spent four months 80-hours a
 week on Mango. Unfortunately, it sometimes feels as though the language, and
 /particularly/ the maturation 'process' behind it, is actually working
 against that goal (note that the "what to do?" thread all but disappeared
 down the black-hole). More's the pity.
Aug 26 2004
prev sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"pragma" <pragma_member pathlink.com> wrote in message
news:cgif07$29ev$1 digitaldaemon.com...

 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...

 Honestly, my expectation was that during the final collection (before


program

 exit) all the finalizers would be run *first* before any memory is






freed.

That makes sense - though it would cause two loops over the garbage




instead

of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.


 I see what you mean.  Granted, there's nothing keeping a destructor from
 creating *more* data on the heap in a growing spiral of allocations.  But


I

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned


earlier in

 this NG).  But I strongly feel that reducing destructors to playing with


only

 scalar types (as we've discovered that object references are all "weak" in


the

 present D implementation during the final collect) reduces their


capability to

 near uselessness.


Destructors should only be used to release external resources - like file
handles, malloc'ed memory etc. To quote from the D documentation on the
section about destructors:
<quote>
When the garbage collector calls a destructor for an object of a class that
has members that are references to garbage collected objects, those
references are no longer valid. This means that destructors cannot reference
sub objects. This rule does not apply to auto objects or objects deleted
with the DeleteExpression.
The garbage collector is not guaranteed to run the destructor for all
unreferenced objects. Furthermore, the order in which the garbage collector
calls destructors for unreference objects is not specified.
<\quote>

It would be nice if we lived in a world where the GC could be fast and run
destructors "nicely" but given the choice I would go for GC performance.
Destructors should be a very weak construct. You never know when or if they
will run so most of the time relying on them is a bug in disguise. Google
around for web pages about Finalizers and Java and you'll get an idea how
everyone in the Java world says not to use them. Remember all this
destructor running happens on every GC pass, not just program exit.


 I also just noticed that my pseudocode on the previous post was also a


little

 flawed, and hence misleading.  If one were to loop over a /copy/ of the


current

 set of finalizers, after clearing out the old list, the outer loop stands


a good

 chance of actually terminating.


What if one of
the destructors assigns a reference to some garbage to a static variable




and

thus brings it back to life? The GC needs to check all the program's
references after the destructors are run to make sure no garbage was




brought

to life. That would be a nasty performance hit.


 Well, "ressurecting" an object properly would require the GC to register


that

 object's finalizer, in which case it looks like a normal object.  That


/would/

 thwart the present behavior as well as my informal proposal in my previous


post.

 Personally, I'd rather have all my objects a /chance/ of cleaning up


propertly

 on program exit rather than have unpredictable behavior inside of


destructors.

 I know what I'm asking for is deterministic destruction of objects, in all
 cases, which has been something of a holy war on the D NG.

 I think its possible to finialize in the proposed method and abort to a


full

 free of the heap if the loop runs for too long, or if there's no change in


the

 number of finalizers for a number of iterations.  At least that way, you


have

 given the program a fighting chance of being well behaved on exit without
 bending over backwards in your code.

 Another way to go would be to allow the GC to track a set of privileged
 references that are guaranteed to be finalized before a dealloc on exit.


These

 would be explicitly set in the client code and would be understood to


delay

 program termination by ever so much.

 At the very least, it'd be nice to have a way to check if a reference is


still

 valid, and hasn't been collected.  That way one could treat object


references as

 'weak' if you're using code inside of a destructor.

 -Pragma
 [[ EricAnderton at (let my objects go) yahoo.com ]]
Aug 25 2004
next sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <cgisdb$2g83$1 digitaldaemon.com>, Ben Hinkle says...

Destructors should only be used to release external resources - like file
handles, malloc'ed memory etc. To quote from the D documentation on the
section about destructors:
<quote>
When the garbage collector calls a destructor for an object of a class that
has members that are references to garbage collected objects, those
references are no longer valid. This means that destructors cannot reference
sub objects. This rule does not apply to auto objects or objects deleted
with the DeleteExpression.
The garbage collector is not guaranteed to run the destructor for all
unreferenced objects. Furthermore, the order in which the garbage collector
calls destructors for unreference objects is not specified.
<\quote>


Yikes.  All apologies to Walter, who has very kindly saved me the embarassment
of being told this from himself.

I guess that says it.  Its not a bug, just not at all what I expected.  All the
same, I'd like to request that this be seriously reconsidered for V2.0.

So I suppose the proper "D" way to proceed is to use an auto object on the
"main" thread, that manages critical resources for me?


It would be nice if we lived in a world where the GC could be fast and run
destructors "nicely" but given the choice I would go for GC performance.
Destructors should be a very weak construct. You never know when or if they
will run so most of the time relying on them is a bug in disguise. Google
around for web pages about Finalizers and Java and you'll get an idea how
everyone in the Java world says not to use them. Remember all this
destructor running happens on every GC pass, not just program exit.


I see what you mean.  Having developers code destructors in this fashion can
have the consequence of increased performance.  I still contend that D
supporting "strong" constructors changes little but D's flexibility.

Thanks everybody (Ben, Kris, Sean) for their help here on this issue.  Again,
all apologies to Walter for my having wasted his time with something that was in
the documentation.

- Pragma
[[ EricAnderton at (auto + deamon = fix?) yahoo.com ]]
Aug 25 2004
next sibling parent Ben Hinkle <bhinkle4 juno.com> writes:
 Thanks everybody (Ben, Kris, Sean) for their help here on this issue. 
 Again, all apologies to Walter for my having wasted his time with
 something that was in the documentation.


Actually I hadn't read that part of the doc until today either. My eariler
post when I was messing with std.stream was trying to do too much in
destructors so you're definitely not alone in writing ambitious
destructors. Maybe if that part of the doc was in bold and blinking or
something... It's very easy to just think destructors are just like C++
destructors only you don't have to worry about freeing memory. oh boy is
that dangerous.
Aug 25 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...


The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.




Please see THIS POST from Walter, and the discussion before and after it:
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/3488

In particular, Walter said: "I must have misspoke, because when the gc collects
an object, it *does* run the destructor." (Walter's emphasis on *does*).

I have security apps planned which rely on Walter's statement being true. Where
is this "not guaranteed" quote in the manual? I couldn't find it. Basically -
help! Who's right? Walter, or the manual? Walter, please could you clarify
(again), and fix the manual if it's wrong.

Jill
Aug 26 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Arcane Jill wrote:


 In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...
 

The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.




 
 Please see THIS POST from Walter, and the discussion before and after it:
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/3488
 
 In particular, Walter said: "I must have misspoke, because when the gc
 collects an object, it *does* run the destructor." (Walter's emphasis on
 *does*).
 
 I have security apps planned which rely on Walter's statement being true.
 Where is this "not guaranteed" quote in the manual? I couldn't find it.
 Basically - help! Who's right? Walter, or the manual? Walter, please could
 you clarify (again), and fix the manual if it's wrong.
 
 Jill


The manual and Walter's post are consistent. The GC runs the destructor when
it collects an object but it isn't guaranteed to collect all unreferenced
objects (due to ambiguous references etc).
Aug 26 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgkkeq$925$1 digitaldaemon.com>, Ben Hinkle says...


The manual and Walter's post are consistent. The GC runs the destructor when
it collects an object but it isn't guaranteed to collect all unreferenced
objects (due to ambiguous references etc). 


Okay, but presumably it *is* guaranteed to collect everything not yet collected
(and to call all remaining destructors) at program exit. Right?
Aug 26 2004
parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Arcane Jill wrote:


 In article <cgkkeq$925$1 digitaldaemon.com>, Ben Hinkle says...
 

The manual and Walter's post are consistent. The GC runs the destructor
when it collects an object but it isn't guaranteed to collect all
unreferenced objects (due to ambiguous references etc).


 
 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?


nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.
Aug 26 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...


 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?





nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.


So if I have a static variable:



and its value happens by coincidence to be the run-time address of an instance
of some class or other, then the destructor of that instance will /never/ get
called? Not even at program exit?

This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).

Jill
Aug 26 2004
next sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...

This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).


As distressing as this inconsistency with destructors is, I managed to compose a
decent workaround last night in a fit of inspiration.

http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d

I'm basically eating my own dog food here: I suggested some time ago that we be
given a way to flag objects for explicit deletion on exit if nothing else. :)

There are other bits in that directory as well all are welcome to critique.
These are common pieces that have fallen out of DSP that are useful outside the
scope of the project.

The idea is that the SystemFinalizer is a singleton that is instantiated and
controlled via a separate AutoSystemFinalizer in main() (or your daemon thread
if you prefer).  This way you have a single point of access from which to mark
objects for deletion once the AutoSystemFinalizer goes out of scope.

Simply call SystemFinalizer.register(obj) anywhere in your app to flag an
object, and off you go.  A second register() method exists that allows for a
custom finalizer function, just in case the first form isn't flexible enough.

-Pragma
[[ EricAnderton at (yummy!) yahoo.com ]]
Aug 26 2004
next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
nifty. you could also use module constructors and destructors instead of
auto class ctors and dtors. Module ctors and dtors will always run. It
probably will be about the same - I guess the difference is that user code
doesn't have to worry about doing anything in main().

"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...

 In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...

This is worrying. I need a workaround, and I'm going to have to think




hard to

come up with it. (Fortunately it's not urgent).


 As distressing as this inconsistency with destructors is, I managed to


compose a

 decent workaround last night in a fit of inspiration.

 http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d

 I'm basically eating my own dog food here: I suggested some time ago that


we be

 given a way to flag objects for explicit deletion on exit if nothing else.


:)

 There are other bits in that directory as well all are welcome to


critique.

 These are common pieces that have fallen out of DSP that are useful


outside the

 scope of the project.

 The idea is that the SystemFinalizer is a singleton that is instantiated


and

 controlled via a separate AutoSystemFinalizer in main() (or your daemon


thread

 if you prefer).  This way you have a single point of access from which to


mark

 objects for deletion once the AutoSystemFinalizer goes out of scope.

 Simply call SystemFinalizer.register(obj) anywhere in your app to flag an
 object, and off you go.  A second register() method exists that allows for


a

 custom finalizer function, just in case the first form isn't flexible


enough.

 -Pragma
 [[ EricAnderton at (yummy!) yahoo.com ]]
Aug 26 2004
parent pragma <pragma_member pathlink.com> writes:
In article <cgkvmr$ec5$1 digitaldaemon.com>, Ben Hinkle says...

nifty. you could also use module constructors and destructors instead of
auto class ctors and dtors. Module ctors and dtors will always run. It
probably will be about the same - I guess the difference is that user code
doesn't have to worry about doing anything in main().


Well, the main advantage here is not /if/ the destructor will run, but rather
/how/.  In this case, use of a destructor registry (like SystemFinalizer)
guarantees that a registered destructor (via a call to 'delete') will run while
all other referenced objects are still alive.  This is in sharp contrast to how
D may-or-may-not run all outsitanding finalizers and delete their corresponding
objects one-by-one per finalizer.

Another way to look at it is like an event callback mechanism.  SystemFinalizer
is analagous to a "System Shutdown Event" that warns registered listeners to
clean up.

In retrospect, the latter would probably be more useful.

-Pragma
[[ EricAnderton at (D needs events) yahoo.com ]]
Aug 26 2004
prev sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...

 In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...

This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).


 As distressing as this inconsistency with destructors is, I managed to compose
a
 decent workaround last night in a fit of inspiration.

 http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d


Why not just have the instance in a static initialiser? (Need to force linking
via explicit mention of class??)



 I'm basically eating my own dog food here: I suggested some time ago that we be
 given a way to flag objects for explicit deletion on exit if nothing else. :)

 There are other bits in that directory as well all are welcome to critique.
 These are common pieces that have fallen out of DSP that are useful outside the
 scope of the project.

 The idea is that the SystemFinalizer is a singleton that is instantiated and
 controlled via a separate AutoSystemFinalizer in main() (or your daemon thread
 if you prefer).  This way you have a single point of access from which to mark
 objects for deletion once the AutoSystemFinalizer goes out of scope.

 Simply call SystemFinalizer.register(obj) anywhere in your app to flag an
 object, and off you go.  A second register() method exists that allows for a
 custom finalizer function, just in case the first form isn't flexible enough.

 -Pragma
 [[ EricAnderton at (yummy!) yahoo.com ]]
Aug 26 2004
parent pragma <pragma_member pathlink.com> writes:
In article <cglhec$nn3$1 digitaldaemon.com>, Matthew says...


"pragma" <pragma_member pathlink.com> wrote in message
news:cgku7g$dhv$1 digitaldaemon.com...

 http://svn.dsource.org/svn/projects/dsp/trunk/misc/systemFinalizer.d


Why not just have the instance in a static initialiser? (Need to force linking
via explicit mention of class??)







You're right.  That would make a better singleton wouldn't it?


-Pragma
[[ EricAnderton at (_instance) yahoo dot com ]]
Aug 26 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgkpfo$b62$1 digitaldaemon.com>, Arcane Jill says...

In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...


 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?





nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.


So if I have a static variable:



and its value happens by coincidence to be the run-time address of an instance
of some class or other, then the destructor of that instance will /never/ get
called? Not even at program exit?

This is worrying. I need a workaround, and I'm going to have to think hard to
come up with it. (Fortunately it's not urgent).


Does the final GC pass occur after module destructors are run?  If so, a
slightly lame workaround would be to have such destructors default initialize
all statics in the module.  ie.

static ~this() { n = n.init; }

Basically saying that every module is effectively responsible for picking up its
toys when it's done playing with them.


Sean
Aug 26 2004
prev sibling parent reply Nick <Nick_member pathlink.com> writes:
In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...

Arcane Jill wrote:

 [...]
 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit. Right?


nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.


Are you sure about this? Why does it scan static data but not dynamic?

Nick
Aug 29 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
Nick wrote:


 In article <cgknav$ab6$1 digitaldaemon.com>, Ben Hinkle says...

Arcane Jill wrote:

 [...]
 Okay, but presumably it *is* guaranteed to collect everything not yet
 collected (and to call all remaining destructors) at program exit.
 Right?


nope - at program exit it scans static data for roots so any object
reference stored in a static variable will never get collected. I don't
know if removing static data from the final scan would cause problems.


 
 Are you sure about this? Why does it scan static data but not dynamic?
 
 Nick


The static data is scanned for roots into the dynamic data. Once a root is
found it continues to mark anything reachable from that point. Normally it
looks for root in the static data and on all stacks, but since the program
is exiting the stacks are skipped. Users can add arbitrary chunks of memory
to list of things to scan for roots, too. Once it is done marking all
reachable dynamic data it garbage collects everything else. 
Does that help?
-Ben
Aug 29 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgkcsq$5fi$1 digitaldaemon.com>, Arcane Jill says...

In article <cgj8qb$2ler$1 digitaldaemon.com>, pragma says...


The garbage collector is not guaranteed to run the destructor for all
unreferenced objects.




Please see THIS POST from Walter, and the discussion before and after it:
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/3488

In particular, Walter said: "I must have misspoke, because when the gc collects
an object, it *does* run the destructor." (Walter's emphasis on *does*).

I have security apps planned which rely on Walter's statement being true. Where
is this "not guaranteed" quote in the manual? I couldn't find it. Basically -
help! Who's right? Walter, or the manual? Walter, please could you clarify
(again), and fix the manual if it's wrong.


IIRC this came up in the original discussion and that was Walter's reply.  The
spec should likely be corrected.


Sean
Aug 26 2004
prev sibling parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Ben Hinkle" <bhinkle mathworks.com> wrote in message
news:cgisdb$2g83$1 digitaldaemon.com...

 "pragma" <pragma_member pathlink.com> wrote in message
 news:cgif07$29ev$1 digitaldaemon.com...

 In article <cgia4c$273m$1 digitaldaemon.com>, Ben Hinkle says...

 Honestly, my expectation was that during the final collection (before


program

 exit) all the finalizers would be run *first* before any memory is






 freed.

That makes sense - though it would cause two loops over the garbage




 instead

of one. Also no-one has brought up the cases when garbage can be brought
back to life. That's one reason why Java's finalizers are frowned upon -
they can bring back dead references and generally cause the GC to jump
through hoops to make sure it isn't cleaning up live data.


 I see what you mean.  Granted, there's nothing keeping a destructor from
 creating *more* data on the heap in a growing spiral of allocations.  But


 I

 really don't see that as any more a problem as infinite recursion or
 mutually-referenced functions (like in the invariant 'bug' menioned


 earlier in

 this NG).  But I strongly feel that reducing destructors to playing with


 only

 scalar types (as we've discovered that object references are all "weak" in


 the

 present D implementation during the final collect) reduces their


 capability to

 near uselessness.


 Destructors should only be used to release external resources - like file
 handles, malloc'ed memory etc. To quote from the D documentation on the
 section about destructors:
 <quote>
 When the garbage collector calls a destructor for an object of a class that
 has members that are references to garbage collected objects, those
 references are no longer valid. This means that destructors cannot reference
 sub objects. This rule does not apply to auto objects or objects deleted
 with the DeleteExpression.
 The garbage collector is not guaranteed to run the destructor for all
 unreferenced objects. Furthermore, the order in which the garbage collector
 calls destructors for unreference objects is not specified.
 <\quote>


That does preclude the use of any kind of resource-management object being used
in composition, does it not? In which
case, D's OO is seriously hamstrung.

Now if D would allow embedding of auto members (of auto or non-auto classes),
then we'd have an answer. (Sounds
suspiciously like C++, of course, but then this is something C++ got very
right.)


 It would be nice if we lived in a world where the GC could be fast and run
 destructors "nicely" but given the choice I would go for GC performance.
 Destructors should be a very weak construct. You never know when or if they
 will run so most of the time relying on them is a bug in disguise. Google
 around for web pages about Finalizers and Java and you'll get an idea how
 everyone in the Java world says not to use them. Remember all this
 destructor running happens on every GC pass, not just program exit.


I agree with all that you say, but just want to mention that I think your
emphasis is a little off. They're a deprecated
technique for correctness rather than efficiency, surely?
Aug 26 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cgguse$1jab$1 digitaldaemon.com>, antiAlias says...

Yes, that would be helpful. But it's not /always/ practical to avoid
touching other objects during destruction, though perhaps 'desirable'.


I bet this works:

void main(){
auto Bar b = new Bar();
auto Foo f = new Foo();
b.event = &(f.notify);
}


And it's not an obvious error either, since one could hardly be blamed for
thinking the delegate holds a reference to its enclosing class instance
(it's a delegate; not a function). If so, then the instance should still be
live as long as the delegate is referenced ... yes? Apparently that's not
how it works. Sure looks like a bug.


The problem is that once execution has left main(), all bets are off.  So what's
happening here is that the references to Foo and Bar are being lost when main()
exits and then the gc is run.  I imagine more common problems with this might
possibly be addressed by having the gc make multiple passes on application exit,
but I have a feeling that this would be shaky at best.  I have no problem with
objects referencing other objects on destruction, but the user should be aware
that this is unreliable in this particular situation.


Sean
Aug 25 2004