www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - GC object finalization not guaranteed

reply Leandro Lucarella <llucax gmail.com> writes:
I've just found out[1] this[2]:

	The garbage collector is not guaranteed to run the destructor for
	all unreferenced objects.

Is there any reason why D can't guarantee that all finalizers will be
called, at least when the program ends?

[1] http://proj.llucax.com.ar/blog/dgc/blog/post/-43101db1
[2] http://www.digitalmars.com/d/1.0/class.html#destructors

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
<original> [Penis Uptime: 2days 13hrs 59mins 35secs]
<Yapa> viagra? :)
<original> yea, 20 pills
Apr 18 2009
next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com>  
wrote:

 I've just found out[1] this[2]:

 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.

 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?


Well, a couple of quick tests show that under normal situations (i.e.  
normal program termination and termination from an exception) the  
finalizers do run (D2). However, if a finalizer throws an exception, then  
the rest of the finalizers aren't called. Also, if you call  
std.c.stdlib.exit, the finalizers won't run.
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

I've just found out[1] this[2]:

	The garbage collector is not guaranteed to run the destructor for
	all unreferenced objects.

Is there any reason why D can't guarantee that all finalizers will be
called, at least when the program ends?


 
 Well, a couple of quick tests show that under normal situations (i.e.
 normal program termination and termination from an exception) the
 finalizers do run (D2). However, if a finalizer throws an exception,
 then the rest of the finalizers aren't called. Also, if you call
 std.c.stdlib.exit, the finalizers won't run.


Well, I'm not talking about experimentation, I'm talking about source code
;)

The current GC implementation don't call finalizers for data that's still
reference when the program *ended* (this is allowed by the specs, so it's
fine, the question is why it's allowed by the specs).

Here is a simple example that demonstrate the case:

$ cat finalize.d

version (Tango)
	import tango.stdc.stdio;
else
	import std.c.stdio;

class A
{
	~this() { printf("term\n"); }
}

A a;

static this() { a = new A; }

void main()
{
	version (FREE)
		a = null;
}

$ for compiler in dmd dmd2 gdmd; do for version in '' '-version=FREE'; do
$compiler $version finalize.d; echo $compiler $version; ./finalize; echo; done;
done
dmd

dmd -version=FREE
term

dmd2

dmd2 -version=FREE
term

gdmd

gdmd -version=FREE
term

(I have my ldc copy broken right now, that's why I didn't test that
compiler ;)


This can happen behind your back too if some nice non-pointer value in the
stack or static data unfortunately is a valid pointer.

This example shows that:

$ cat finalize2.d 

version (Tango)
	import tango.stdc.stdio;
else
	import std.c.stdio;

class A
{
	~this() { printf("term\n"); }
}

size_t x;

void main()
{
	A a = new A;
	x = cast(size_t) cast(void*) a;
	version (FREE)
		x = 0;
}

(the results are the same as the previous example)


-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
- Que hacés, ratita?
- Espero un ratito...
Apr 18 2009
next sibling parent reply Don <nospam nospam.com> writes:
Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

 I've just found out[1] this[2]:

 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.

 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?


 Well, a couple of quick tests show that under normal situations (i.e.
 normal program termination and termination from an exception) the
 finalizers do run (D2). However, if a finalizer throws an exception,
 then the rest of the finalizers aren't called. Also, if you call
 std.c.stdlib.exit, the finalizers won't run.


 
 Well, I'm not talking about experimentation, I'm talking about source code
 ;)
 
 The current GC implementation don't call finalizers for data that's still
 reference when the program *ended* (this is allowed by the specs, so it's
 fine, the question is why it's allowed by the specs).


I don't understand why D even has finalizers. Can they do anything useful?
Apr 18 2009
next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Don, el 18 de abril a las 22:25 me escribiste:

The current GC implementation don't call finalizers for data that's still
reference when the program *ended* (this is allowed by the specs, so it's
fine, the question is why it's allowed by the specs).


 
 I don't understand why D even has finalizers. Can they do anything useful?


Close a connection gracefully for example, I guess (I mean, send a "bye"
packed, not just close the socket abruptly). Same for closing files
writing some mark or something. They can be risky when finalization is not
deterministic though.

Of course that can be done manually if you are well organized when
programming, but then, you can do that with memory too, and the whole GC
loose its usefulness, so I guess is a valid argument for other things too
=)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Vaporeso sostenía a rajacincha la teoría del No-Water, la cual le
pertenecía y versaba lo siguiente: "Para darle la otra mejilla al fuego,
éste debe ser apagado con alpargatas apenas húmedas".
Apr 18 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is not
 deterministic though.


Scoped objects should be used for that, not gc.
Apr 18 2009
next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Walter Bright, el 18 de abril a las 14:34 me escribiste:

 Leandro Lucarella wrote:

Close a connection gracefully for example, I guess (I mean, send a "bye"
packed, not just close the socket abruptly). Same for closing files
writing some mark or something. They can be risky when finalization is not
deterministic though.


 
 Scoped objects should be used for that, not gc.


Then, why allowing finalizers in non-scope objects? It's a little
confusing...

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
The biggest lie you can tell yourself is
When I get what I want I will be happy
Apr 18 2009
prev sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Walter Bright wrote:

 Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is
 not
 deterministic though.


 
 Scoped objects should be used for that, not gc.


But you can't tell in a dtor whether you're being destroyed
deterministically or not.  The only safe assumption is that you aren't,
thus rendering dtors worse than useless.

All we'd need is a bool passed to a dtor to tell it whether it's being
destroyed deterministically or not and all will be rainbows and sunshine.

... except for the opDotExpr discussion.  Anyone who posts to that
thread is kuh-razy!

  -- Daniel
Apr 19 2009
next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Daniel Keep wrote:

 
 Walter Bright wrote:

 Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is
 not
 deterministic though.


 Scoped objects should be used for that, not gc.


 
 But you can't tell in a dtor whether you're being destroyed
 deterministically or not.  The only safe assumption is that you aren't,
 thus rendering dtors worse than useless.
 
 All we'd need is a bool passed to a dtor to tell it whether it's being
 destroyed deterministically or not and all will be rainbows and sunshine.
 
 ... except for the opDotExpr discussion.  Anyone who posts to that
 thread is kuh-razy!
 
   -- Daniel


That should be a relatively easy change, actually, and it's a pretty 
good idea.
Apr 19 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-04-19 09:10:12 -0400, Daniel Keep <daniel.keep.lists gmail.com> said:


 Walter Bright wrote:

 Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is
 not
 deterministic though.


 
 Scoped objects should be used for that, not gc.


 
 But you can't tell in a dtor whether you're being destroyed
 deterministically or not.  The only safe assumption is that you aren't,
 thus rendering dtors worse than useless.


Actually you could declare your class as scope, that'd make sure every 
instance is scope and gets destructed in a timely manner.

Hum, perhaps destructors should only be allowed on scope classes. :-)


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 25 2009
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Michel Fortin wrote:

 On 2009-04-19 09:10:12 -0400, Daniel Keep <daniel.keep.lists gmail.com>
 said:
 

 Walter Bright wrote:

 Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a
 "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is
 not
 deterministic though.


 Scoped objects should be used for that, not gc.


 But you can't tell in a dtor whether you're being destroyed
 deterministically or not.  The only safe assumption is that you aren't,
 thus rendering dtors worse than useless.


 
 Actually you could declare your class as scope, that'd make sure every
 instance is scope and gets destructed in a timely manner.


Except that's a pain in the butt to actually use.  You can't store them
in other classes or structs and you have to create all objects at the
highest scope they're used.  I proposed changes to let scope instances
to be returned from functions and stored in other scope classes to make
them more useful, but that never went anywhere.  :P


 Hum, perhaps destructors should only be allowed on scope classes. :-)


Probably, but only assuming scope objects are made more usable.

  -- Daniel
Apr 25 2009
prev sibling next sibling parent reply Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-04-18 22:25:32 +0200, Don <nospam nospam.com> said:


 Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

 I've just found out[1] this[2]:
 
 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.
 
 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?








when the main thread ends other threads might be running, and removing 
memory they are using is not necessarily a good idea.
Exceptions might delay things indefinitely and allocate more memory.

Actually with tango you can tweak what will happen with 
gc_getTermCleanupLevel (in gc.d), but I don't think that there is a 
satisfying solution for everybody. I think that the default in tango is 
quite reasonable (normal collect).


 Well, a couple of quick tests show that under normal situations (i.e.
 normal program termination and termination from an exception) the
 finalizers do run (D2). However, if a finalizer throws an exception,
 then the rest of the finalizers aren't called. Also, if you call
 std.c.stdlib.exit, the finalizers won't run.


 
 Well, I'm not talking about experimentation, I'm talking about source code
 ;)
 
 The current GC implementation don't call finalizers for data that's still
 reference when the program *ended* (this is allowed by the specs, so it's
 fine, the question is why it's allowed by the specs).


 
 I don't understand why D even has finalizers. Can they do anything useful?


Two things that I acn think of, there might be more:
- Release external resources. Well probably it is better not to rely on 
them if the resource is finite (basically always), but it can be useful 
at times
- weak refs
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Fawzi Mohamed, el 18 de abril a las 22:48 me escribiste:

 On 2009-04-18 22:25:32 +0200, Don <nospam nospam.com> said:
 

Leandro Lucarella wrote:

Robert Jacques, el 18 de abril a las 11:56 me escribiste:

On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

I've just found out[1] this[2]:
	The garbage collector is not guaranteed to run the destructor for
	all unreferenced objects.
Is there any reason why D can't guarantee that all finalizers will be
called, at least when the program ends?








 
 when the main thread ends other threads might be running, and removing
 memory they are using is not necessarily a good idea.


Then I guess gc_term() should be invoked when all threads have terminated.
Is there any technical difficulty to do that?


 Exceptions might delay things indefinitely and allocate more memory.


Then I guess gc_term() should be invoked when all exceptions were
processed. Is there any technical difficulty to do that?

I really didn't take a look at the compiler runtime stuff, so I don't
really know how all that works.


 Actually with tango you can tweak what will happen with
 gc_getTermCleanupLevel (in gc.d), but I don't think that there is
 a satisfying solution for everybody.


Yes, I saw that (I commented about that in the blog post), but there is
still no option to run the finalizers to all the live objects.

Anyways, I wonder why it's not part of the GC specs in druntime.


 I think that the default in tango is quite reasonable (normal collect).


Yes... With the current specs at least, I guess.

I think that if finalization is not guarantee, at least at
program exit, there shouldn't be there. It's useless.

Finalization should be possible only for scope classes if no finalization
guarantee can be provided by the GC. It's useless and error prone
(introducing a bug very difficult to track). Who wants randomly executed
destructors?

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Mi infancia fue en un loft, bien al costado del río
Cazabamos correcaminos y lo azabamos en el fogón
Después? Después me vine grande y la ciudad me deslumbró
Jugando al tejo en Lavalle me hice amigo del bongó
Apr 18 2009
next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Leandro Lucarella, el 18 de abril a las 18:03 me escribiste:

 Fawzi Mohamed, el 18 de abril a las 22:48 me escribiste:

 On 2009-04-18 22:25:32 +0200, Don <nospam nospam.com> said:
 

Leandro Lucarella wrote:

Robert Jacques, el 18 de abril a las 11:56 me escribiste:

On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

I've just found out[1] this[2]:
	The garbage collector is not guaranteed to run the destructor for
	all unreferenced objects.
Is there any reason why D can't guarantee that all finalizers will be
called, at least when the program ends?








 
 when the main thread ends other threads might be running, and removing
 memory they are using is not necessarily a good idea.


 
 Then I guess gc_term() should be invoked when all threads have terminated.
 Is there any technical difficulty to do that?
 

 Exceptions might delay things indefinitely and allocate more memory.


 
 Then I guess gc_term() should be invoked when all exceptions were
 processed. Is there any technical difficulty to do that?
 
 I really didn't take a look at the compiler runtime stuff, so I don't
 really know how all that works.


Ok, this is the code that handle the D main (Tango runtime
compiler/ldc/rt/dmain2.d, main() function, around line 260):

    void runMain()
    {
        debug(PRINTF) printf("main runMain\n");
        result = main(args);
    }

    void runAll()
    {
        debug(PRINTF) printf("main runAll\n");
        gc_init();
        _moduleCtor();
        if (runModuleUnitTests())
            tryExec(&runMain);
        thread_joinAll();
        _d_isHalting = true;
        _moduleDtor();
        gc_term();
    }

    tryExec(&runAll);

    debug(PRINTF) printf("main dtor\n");
    _STD_critical_term();
    _STD_monitor_staticdtor();

    return result;

If thread_joinAll() is called before gc_term(), I can't see how a thread
could be still running when gc_term() is called. So, in terms of threads,
I don't see a problem here.

About exceptions, I think it could be solved too. One option is to require
gc_term() to be "nonthrow" (I mean, for D1 and D2). I think the specs
doesn't say anything about destructors throwing exceptions. I think they
shouldn't but let's suppose they could. Since there is no way the use can
catch a destructor exception thrown in the GC (well, not in an useful way,
one could try/catch the new call or something but that doesn't work for
concurrent collectors that can sweep in a separate thread anyway, so
I think it's a bad idea), I guess it would be acceptable to:
a) Ignore the exception
b) Let the user provide a callback to handle that kind of exceptions

Then, gc_term() ca be moved outside the tryExec(&runAll) call and
gc_term() can safely call all the live objects finalizers and provide
guaranteed finalization.


PS: I have not read all the runtime, just this small part, so maybe I'm
    missing the big picture =)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
PADRES DENUNCIAN QUE SU HIJA SE ESCAPO CON UN PARAGUAYITO
	-- Crónica TV
Apr 18 2009
parent reply Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-04-18 23:22:21 +0200, Leandro Lucarella <llucax gmail.com> said:


 Leandro Lucarella, el 18 de abril a las 18:03 me escribiste:

 Fawzi Mohamed, el 18 de abril a las 22:48 me escribiste:

 On 2009-04-18 22:25:32 +0200, Don <nospam nospam.com> said:
 

 Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

 I've just found out[1] this[2]:
 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.
 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?








 
 when the main thread ends other threads might be running, and removing
 memory they are using is not necessarily a good idea.


 
 Then I guess gc_term() should be invoked when all threads have terminated.
 Is there any technical difficulty to do that?
 

 Exceptions might delay things indefinitely and allocate more memory.


 
 Then I guess gc_term() should be invoked when all exceptions were
 processed. Is there any technical difficulty to do that?
 
 I really didn't take a look at the compiler runtime stuff, so I don't
 really know how all that works.


 
 Ok, this is the code that handle the D main (Tango runtime
 compiler/ldc/rt/dmain2.d, main() function, around line 260):
 
     void runMain()
     {
         debug(PRINTF) printf("main runMain\n");
         result = main(args);
     }
 
     void runAll()
     {
         debug(PRINTF) printf("main runAll\n");
         gc_init();
         _moduleCtor();
         if (runModuleUnitTests())
             tryExec(&runMain);
         thread_joinAll();
         _d_isHalting = true;
         _moduleDtor();
         gc_term();
     }
 
     tryExec(&runAll);
 
     debug(PRINTF) printf("main dtor\n");
     _STD_critical_term();
     _STD_monitor_staticdtor();
 
     return result;
 
 If thread_joinAll() is called before gc_term(), I can't see how a thread
 could be still running when gc_term() is called. So, in terms of threads,
 I don't see a problem here.


daemon theads are not joined


 About exceptions, I think it could be solved too. One option is to require
 gc_term() to be "nonthrow" (I mean, for D1 and D2). I think the specs
 doesn't say anything about destructors throwing exceptions. I think they
 shouldn't but let's suppose they could. Since there is no way the use can
 catch a destructor exception thrown in the GC (well, not in an useful way,
 one could try/catch the new call or something but that doesn't work for
 concurrent collectors that can sweep in a separate thread anyway, so
 I think it's a bad idea), I guess it would be acceptable to:
 a) Ignore the exception
 b) Let the user provide a callback to handle that kind of exceptions


finalizer should not throw already now, if they do an exception is raised.


 Then, gc_term() ca be moved outside the tryExec(&runAll) call and
 gc_term() can safely call all the live objects finalizers and provide
 guaranteed finalization.
 
 
 PS: I have not read all the runtime, just this small part, so maybe I'm
     missing the big picture =)
Apr 18 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Fawzi Mohamed, el 18 de abril a las 23:43 me escribiste:

If thread_joinAll() is called before gc_term(), I can't see how a thread
could be still running when gc_term() is called. So, in terms of threads,
I don't see a problem here.


 
 daemon theads are not joined


So, you can keep threads going on even when you have terminated the GC?
That's odd! I think that's an accident waiting to happen =)

Anyways, if you do have a thread live when the GC is terminate, a lot of
care have to be taken, I don't see why "do not use GC allocated memory" in
threads that live longer than the GC should not be added to the
limitations. It seems pretty reasonable.


About exceptions, I think it could be solved too. One option is to require
gc_term() to be "nonthrow" (I mean, for D1 and D2). I think the specs
doesn't say anything about destructors throwing exceptions. I think they
shouldn't but let's suppose they could. Since there is no way the use can
catch a destructor exception thrown in the GC (well, not in an useful way,
one could try/catch the new call or something but that doesn't work for
concurrent collectors that can sweep in a separate thread anyway, so
I think it's a bad idea), I guess it would be acceptable to:
a) Ignore the exception
b) Let the user provide a callback to handle that kind of exceptions


 
 finalizer should not throw already now, if they do an exception is raised.


Ok, so then exceptions thrown by destructors are not a problem either.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
... los cuales son susceptibles a una creciente variedad de ataques previsibles,
tales como desbordamiento del tampón, falsificación de parámetros, ...
	-- Stealth - ISS LLC - Seguridad de IT
Apr 18 2009
prev sibling parent reply Fawzi Mohamed <fmohamed mac.com> writes:
On 2009-04-18 23:03:11 +0200, Leandro Lucarella <llucax gmail.com> said:


 Fawzi Mohamed, el 18 de abril a las 22:48 me escribiste:

 On 2009-04-18 22:25:32 +0200, Don <nospam nospam.com> said:
 

 Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella <llucax gmail.com> wrote:

 I've just found out[1] this[2]:
 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.
 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?








 
 when the main thread ends other threads might be running, and removing
 memory they are using is not necessarily a good idea.


 
 Then I guess gc_term() should be invoked when all threads have terminated.
 Is there any technical difficulty to do that?


well killing a daemon thread might not be so easy, the cleanest thing 
would be to simulate an exception into it and unwind the stack.
If you are not careful you might kill a thread when it did acquire a 
lock that might deadlock your own thread.


 Exceptions might delay things indefinitely and allocate more memory.


 
 Then I guess gc_term() should be invoked when all exceptions were
 processed. Is there any technical difficulty to do that?


this should be sort of doable (without having looked at the details)


 I really didn't take a look at the compiler runtime stuff, so I don't
 really know how all that works.
 

 Actually with tango you can tweak what will happen with
 gc_getTermCleanupLevel (in gc.d), but I don't think that there is
 a satisfying solution for everybody.


 
 Yes, I saw that (I commented about that in the blog post), but there is
 still no option to run the finalizers to all the live objects.


well you could try to ignore all roots but the gc ones (you don't want 
to deallocate it before you have finished using it :).


 Anyways, I wonder why it's not part of the GC specs in druntime.
 

 I think that the default in tango is quite reasonable (normal collect).


 
 Yes... With the current specs at least, I guess.
 
 I think that if finalization is not guarantee, at least at
 program exit, there shouldn't be there. It's useless.


not true, there are still cases in which it is useful (weak, ref, and 
what Robert Jaques said.
Actually one of the first comment when I added gc_setTermCleanupLevel, 
was nice so now my program can finish faster, not everybody wants to 
wait too long after the program has finished doing useful stuff for it 
to die.


 Finalization should be possible only for scope classes if no finalization
 guarantee can be provided by the GC. It's useless and error prone
 (introducing a bug very difficult to track). Who wants randomly executed
 destructors?


destructors are executed in a randmized order (so you might consume an 
external resource if it is limited). That they might not be executed at 
all, is only a small extra problem. Indeed they are less useful than 
(for example) C++, but can still have some uses.

Fawzi
Apr 18 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Fawzi Mohamed, el 18 de abril a las 23:41 me escribiste:

when the main thread ends other threads might be running, and removing
memory they are using is not necessarily a good idea.


Then I guess gc_term() should be invoked when all threads have terminated.
Is there any technical difficulty to do that?


 
 well killing a daemon thread might not be so easy, the cleanest thing
 would be to simulate an exception into it and unwind the stack.  If you
 are not careful you might kill a thread when it did acquire a lock that
 might deadlock your own thread.


But don't you agree that if a thread live longer than the GC, there are
a lot of problems to take care of? I see that scenario as a very rare
special case, not as a reasonable default. I guess some mechanism (like
gc_setTermCleanupLevel()) can be provided to tell the GC not to call
finalizers can be good enough for that special cases. And in that cases,
it's completely acceptable that the finalization guarantee is not provided
(but just because you asked not to be provided).


Exceptions might delay things indefinitely and allocate more memory.


Then I guess gc_term() should be invoked when all exceptions were
processed. Is there any technical difficulty to do that?


 
 this should be sort of doable (without having looked at the details)


I don't see a problem with this either.


I really didn't take a look at the compiler runtime stuff, so I don't
really know how all that works.

Actually with tango you can tweak what will happen with
gc_getTermCleanupLevel (in gc.d), but I don't think that there is
a satisfying solution for everybody.


Yes, I saw that (I commented about that in the blog post), but there is
still no option to run the finalizers to all the live objects.


 
 well you could try to ignore all roots but the gc ones (you don't want
 to deallocate it before you have finished using it :).


If the program ends, if the GC ends, you shouldn't be using any GC
allocated memory in the first place, don't you?


Anyways, I wonder why it's not part of the GC specs in druntime.

I think that the default in tango is quite reasonable (normal collect).


Yes... With the current specs at least, I guess.
I think that if finalization is not guarantee, at least at
program exit, there shouldn't be there. It's useless.


 
 not true, there are still cases in which it is useful (weak, ref, and
 what Robert Jaques said.


Ok, it's useful for memory related bookkeeping. That's true. But it's too
easy to introduce bizarre bugs for managing other resources.

And I think providing finalization guarantee is not that hard either.


 Actually one of the first comment when I added gc_setTermCleanupLevel,
 was nice so now my program can finish faster, not everybody wants to
 wait too long after the program has finished doing useful stuff for it
 to die.


Again, I'm not talking about collection. I really don't think the programs
should do a collection before it ends, it has no point, who am I freeing
memory to? What I think it's it necessary is to call the finalizers of the
live objects. Another way to see things is the program has not "finished
doing useful stuff" until all finalizers are called, because finalizers
are supposed to be useful =)


Finalization should be possible only for scope classes if no finalization
guarantee can be provided by the GC. It's useless and error prone
(introducing a bug very difficult to track). Who wants randomly executed
destructors?


 
 destructors are executed in a randmized order (so you might consume an
 external resource if it is limited). That they might not be executed at
 all, is only a small extra problem. Indeed they are less useful than
 (for example) C++, but can still have some uses.


I'm not talking about order when saying "randomly", I'm talking about the
destructor being executed at all =)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Y ya no tengo colores, sólo gris y negro
Aquí donde el amor es de hierro
Los días pasan y moriremos contando el tiempo
Apr 18 2009
prev sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 18 Apr 2009 16:25:32 -0400, Don <nospam nospam.com> wrote:


 Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella  
 <llucax gmail.com> wrote:

 I've just found out[1] this[2]:

 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.

 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?


 Well, a couple of quick tests show that under normal situations (i.e.
 normal program termination and termination from an exception) the
 finalizers do run (D2). However, if a finalizer throws an exception,
 then the rest of the finalizers aren't called. Also, if you call
 std.c.stdlib.exit, the finalizers won't run.


  Well, I'm not talking about experimentation, I'm talking about source  
 code
 ;)
  The current GC implementation don't call finalizers for data that's  
 still
 reference when the program *ended* (this is allowed by the specs, so  
 it's
 fine, the question is why it's allowed by the specs).


 I don't understand why D even has finalizers. Can they do anything  
 useful?


Yes. I use them to manage GPU memory/resources. Another are wrapper  
classes around manually allocated arrays (to avoid the false pointer  
problem with very large arrays). Both of these examples don't require  
finalization at program termination (unlike file handles/sockets, etc)
Apr 18 2009
parent reply Don <nospam nospam.com> writes:
Robert Jacques wrote:

 On Sat, 18 Apr 2009 16:25:32 -0400, Don <nospam nospam.com> wrote:
 

 Leandro Lucarella wrote:

 Robert Jacques, el 18 de abril a las 11:56 me escribiste:

 On Sat, 18 Apr 2009 11:24:14 -0400, Leandro Lucarella 
 <llucax gmail.com> wrote:

 I've just found out[1] this[2]:

     The garbage collector is not guaranteed to run the destructor for
     all unreferenced objects.

 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?


 Well, a couple of quick tests show that under normal situations (i.e.
 normal program termination and termination from an exception) the
 finalizers do run (D2). However, if a finalizer throws an exception,
 then the rest of the finalizers aren't called. Also, if you call
 std.c.stdlib.exit, the finalizers won't run.


  Well, I'm not talking about experimentation, I'm talking about 
 source code
 ;)
  The current GC implementation don't call finalizers for data that's 
 still
 reference when the program *ended* (this is allowed by the specs, so 
 it's
 fine, the question is why it's allowed by the specs).


 I don't understand why D even has finalizers. Can they do anything 
 useful?


 
 Yes. I use them to manage GPU memory/resources. Another are wrapper 
 classes around manually allocated arrays (to avoid the false pointer 
 problem with very large arrays). Both of these examples don't require 
 finalization at program termination (unlike file handles/sockets, etc)


OK, those both make perfect sense -- they are both memory issues. It 
still seems that gc is perfect for memory, but no good for anything else.
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Don, el 18 de abril a las 23:43 me escribiste:

I don't understand why D even has finalizers. Can they do anything useful?


Yes. I use them to manage GPU memory/resources. Another are wrapper classes
around manually allocated arrays (to avoid the false pointer problem with very 
large arrays). Both of these examples don't require finalization at program
termination (unlike file handles/sockets, etc)


 
 OK, those both make perfect sense -- they are both memory issues. It still
seems that gc is perfect for memory, but no good for anything else.


Well, you mean for resources the OS free for you at program end, right?
For other resources is not useful because the not guaranteed finalization
(if you don't have the OS to cover your ass, the GC won't either with this
limitation).

I think shared memory is an example of memory resource that's not freed by
the OS on program exit.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Ah, se va en 1981? Pero por qué? ... Ah, porque ya había sido determinado,
entonces quiere decir que pronto vamos a elegir presidente nuevo nosot...
Ah, nosotros no? Ah, lo van a elegir en la ... Ah! Quiere que le diga? Muy
bien pensado, porque cada vez que lo elegimos nosotros no duran nada!
	-- Tato vs. Tato (1980, Gobierno de Videla)
Apr 18 2009
parent reply Don <nospam nospam.com> writes:
Leandro Lucarella wrote:

 Don, el 18 de abril a las 23:43 me escribiste:

 I don't understand why D even has finalizers. Can they do anything useful?


 Yes. I use them to manage GPU memory/resources. Another are wrapper classes
around manually allocated arrays (to avoid the false pointer problem with very 
 large arrays). Both of these examples don't require finalization at program
termination (unlike file handles/sockets, etc)


 OK, those both make perfect sense -- they are both memory issues. It still
seems that gc is perfect for memory, but no good for anything else.


 
 Well, you mean for resources the OS free for you at program end, right?
 For other resources is not useful because the not guaranteed finalization
 (if you don't have the OS to cover your ass, the GC won't either with this
 limitation).


No, that's not what I mean at all. I actually think it's a good thing 
that the GC doesn't call finalizers at the end -- making it obvious that 
you shouldn't be using it for resource management.

 
 I think shared memory is an example of memory resource that's not freed by
 the OS on program exit.


Really? That sounds like an OS memory leak. Security issue, too: run 
your program, allocated shared memory, then exit. Repeat until all 
memory is exhausted.
Apr 18 2009
next sibling parent Robin KAY <komadori gekkou.co.uk> writes:
Don wrote:
[snip]

 I think shared memory is an example of memory resource that's not 
 freed by
 the OS on program exit.


 
 Really? That sounds like an OS memory leak. Security issue, too: run 
 your program, allocated shared memory, then exit. Repeat until all 
 memory is exhausted.


Run your program, create a file, then exit. Repeat until all disk space 
is exhausted.

SYSV IPC objects have a persistent identifier through which they can be 
accessed by multiple processes.

-- 
Robin KAY
Apr 19 2009
prev sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Don, el 19 de abril a las 07:35 me escribiste:

 Leandro Lucarella wrote:

I think shared memory is an example of memory resource that's not freed by
the OS on program exit.


 
 Really? That sounds like an OS memory leak. Security issue, too: run
 your program, allocated shared memory, then exit. Repeat until all
 memory is exhausted.


man 3 shm_open

Anyways. Again, the discussion is diverging very quickly. The point is,
according to the specs, the GC is allowed to not call finalizers at all.
So any current D program relying on the GC calling a destructor *ever* is
broken according to the specs, even when destructors are used for weak
pointers, for freeing malloc'ed memory or anything.

I think finalizers should be removed or fixed (I vote for the latter,
because it's easy and efficient to fix it).

Even more, I'm considering this a bug in the specs even for D1, so I'll
file a bug for this.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Mi mami llevo a tu papi con el fuquete! Menéalo! Mi mami llevo a tu
papi con el fuquete!
	-- Sidharta Kiwi
Apr 19 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Leandro Lucarella, el 19 de abril a las 13:08 me escribiste:

 Don, el 19 de abril a las 07:35 me escribiste:

 Leandro Lucarella wrote:

I think shared memory is an example of memory resource that's not freed by
the OS on program exit.


 
 Really? That sounds like an OS memory leak. Security issue, too: run
 your program, allocated shared memory, then exit. Repeat until all
 memory is exhausted.


 
 man 3 shm_open
 
 Anyways. Again, the discussion is diverging very quickly. The point is,
 according to the specs, the GC is allowed to not call finalizers at all.
 So any current D program relying on the GC calling a destructor *ever* is
 broken according to the specs, even when destructors are used for weak
 pointers, for freeing malloc'ed memory or anything.
 
 I think finalizers should be removed or fixed (I vote for the latter,
 because it's easy and efficient to fix it).
 
 Even more, I'm considering this a bug in the specs even for D1, so I'll
 file a bug for this.


http://d.puremagic.com/issues/show_bug.cgi?id=2858

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Aprendan de la primavera, que antecede al verano, precede al inverno y
no lo anda diciendo por ahí.
	-- Ricardo Vaporeso. Llanos de Luzuriaga, 1914.
Apr 19 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Leandro Lucarella wrote:

 The current GC implementation don't call finalizers for data that's still
 reference when the program *ended* (this is allowed by the specs, so it's
 fine, the question is why it's allowed by the specs).


The why is because of speed. What's the point of running a gc pause on 
program exit? The OS recovers all the memory anyway.
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Walter Bright, el 18 de abril a las 14:33 me escribiste:

 Leandro Lucarella wrote:

The current GC implementation don't call finalizers for data that's still
reference when the program *ended* (this is allowed by the specs, so it's
fine, the question is why it's allowed by the specs).


 
 The why is because of speed. What's the point of running a gc pause on
 program exit? The OS recovers all the memory anyway.


You missed the point. I'm not talking about freeing the memory. I'm
talking about finalizers. A finalizer could send a "bye" packet throgh the
net. That can't be handled by the OS.

And I'm not talking about doing a collection at program exit either. I'm
talking about just calling the finalizers.

What I say is to do this (in some kind of pseudocode, which is actually
valid code in my example naive GC implementation =):

void gc_term()
{
	foreach (cell; this.live_list)
		rt_finalize(cell.ptr);
}

No collection, no free.

What do you think about that? And about finalizers being completely
useless if the excecution is not guarateed? I would really like to know
(Sean's opinion would be appreciated too, being in charge of druntime).

Maybe this can be an improvement for D2 (I don't think you'll like to
change the specs for D1 =).

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
A veces quisiera ser un auto,
para chocar como choco siendo humano,
para romperme en mil pedazos.
Apr 18 2009
next sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
Leandro Lucarella wrote:

 You missed the point. I'm not talking about freeing the memory. I'm
 talking about finalizers. A finalizer could send a "bye" packet throgh the
 net. That can't be handled by the OS.


It can't be handled by the GC either, because:
  - This would require a high-level wrapper that knows about the "bye"
packet around a low-level socket that doesn't know.  By the time the
high-level wrapper is finalized, the low-level socket may already have
been collected.
  - It is bad form to wait for the next garbage-collection cycle before
cleanly terminating connections.

What you need is RAII, not garbage collection.


-- 
Rainer Deyke - rainerd eldwood.com
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Rainer Deyke, el 18 de abril a las 16:18 me escribiste:

 Leandro Lucarella wrote:

 You missed the point. I'm not talking about freeing the memory. I'm
 talking about finalizers. A finalizer could send a "bye" packet throgh the
 net. That can't be handled by the OS.


 
 It can't be handled by the GC either, because:
   - This would require a high-level wrapper that knows about the "bye"
 packet around a low-level socket that doesn't know.  By the time the
 high-level wrapper is finalized, the low-level socket may already have
 been collected.


I don't know what you are talking about, I'm talking about this:

class X {

	//...

	~this()
	{
		socket.send(bye_packet);
		socket.close();
	}


   - It is bad form to wait for the next garbage-collection cycle before
 cleanly terminating connections.
 
 What you need is RAII, not garbage collection.


I'm talking about long lived resources for example.

I don't understand people defending non-guaranteed finalization over
guaranteed finalization, even when it's:
a) Possible
b) Easy to do
c) Probably more efficient to terminate the program than the "safe"
   fullcollect()

I really don't get it. Seriously.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Hey you, out there on your own
Sitting naked by the phone
Would you touch me?
Apr 18 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Leandro Lucarella, el 18 de abril a las 19:36 me escribiste:

 Rainer Deyke, el 18 de abril a las 16:18 me escribiste:

 Leandro Lucarella wrote:

 You missed the point. I'm not talking about freeing the memory. I'm
 talking about finalizers. A finalizer could send a "bye" packet throgh the
 net. That can't be handled by the OS.


 
 It can't be handled by the GC either, because:
   - This would require a high-level wrapper that knows about the "bye"
 packet around a low-level socket that doesn't know.  By the time the
 high-level wrapper is finalized, the low-level socket may already have
 been collected.


 
 I don't know what you are talking about, I'm talking about this:
 
 class X {
 
 	//...
 
 	~this()
 	{
 		socket.send(bye_packet);
 		socket.close();
 	}


Ok, socket can be collected when the destructor of X is called. Now I get
you point.

I still think having guaranteed finalization make the language much more
robust, though. And it's easily doable, I don't see why it's not done.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
DONAN UN PENE EN NICARAGUA
	-- Crónica TV
Apr 18 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Leandro Lucarella wrote:

 You missed the point. I'm not talking about freeing the memory. I'm
 talking about finalizers. A finalizer could send a "bye" packet throgh the
 net. That can't be handled by the OS.


That shouldn't be handled by a finalizer. A "bye" packet can be handled 
by a static destructor.


 And I'm not talking about doing a collection at program exit either. I'm
 talking about just calling the finalizers.


The finalizers are found by doing a collection scan to see what's left, 
and then running the finalizers.



 What I say is to do this (in some kind of pseudocode, which is actually
 valid code in my example naive GC implementation =):
 
 void gc_term()
 {
 	foreach (cell; this.live_list)
 		rt_finalize(cell.ptr);
 }
 
 No collection, no free.
 
 What do you think about that? And about finalizers being completely
 useless if the excecution is not guarateed? I would really like to know
 (Sean's opinion would be appreciated too, being in charge of druntime).


Finalizers are fairly useless for gc. But they are a lot more useful as 
RAII destructors.


 Maybe this can be an improvement for D2 (I don't think you'll like to
 change the specs for D1 =).
Apr 18 2009
parent reply Christopher Wright <dhasenan gmail.com> writes:
Walter Bright wrote:

 Leandro Lucarella wrote:

 You missed the point. I'm not talking about freeing the memory. I'm
 talking about finalizers. A finalizer could send a "bye" packet throgh 
 the
 net. That can't be handled by the OS.


 
 That shouldn't be handled by a finalizer. A "bye" packet can be handled 
 by a static destructor.


That requires more work -- you have to keep track of a bunch of 
instances that are active, and then the static destructor goes through 
all of them and nukes them each in turn. It isn't a whole lot of work, I 
admit. But it is -- or at least, it is perceived as -- an idiom to make 
up for an inadequacy of the garbage collector.


 Finalizers are fairly useless for gc. But they are a lot more useful as 
 RAII destructors.


Then maybe finalizers should be divorced entirely from garbage collection.
Apr 18 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Christopher Wright, el 18 de abril a las 22:06 me escribiste:

 Walter Bright wrote:

Leandro Lucarella wrote:

You missed the point. I'm not talking about freeing the memory. I'm
talking about finalizers. A finalizer could send a "bye" packet throgh the
net. That can't be handled by the OS.


That shouldn't be handled by a finalizer. A "bye" packet can be handled by a
static destructor.


 
 That requires more work -- you have to keep track of a bunch of
 instances that are active, and then the static destructor goes through
 all of them and nukes them each in turn. It isn't a whole lot of work,
 I admit. But it is -- or at least, it is perceived as -- an idiom to
 make up for an inadequacy of the garbage collector.
 

Finalizers are fairly useless for gc. But they are a lot more useful as RAII
destructors.


 
 Then maybe finalizers should be divorced entirely from garbage collection.


Tha's an option too, but I think it's more useful to provide guaranteed
finalization, which is very doable too.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
When I was a child I had a fever
My hands felt just like two balloons.
Now I've got that feeling once again
I can't explain you would not understand
This is not how I am.
I have become comfortably numb.
Apr 19 2009
prev sibling next sibling parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
The simple solution is this:

1. If your class object only involves memory, freed OS handles, etc., it 
should be used as-is.  This is most class objects.  Destructors are 
needed to clamp resource use (see File class.)

2. If your class object involves hardware handles, transactional 
assurance, or data integrity, it must be scoped.  This is the same as it 
was in C, except D has better constucts for it (see scope.)

I don't see the problem.  For the majority of objects that only involve 
memory, life is easier.  For the rest, life is still easier (just not as 
much.)

-[Unknown]


Leandro Lucarella wrote:

 I've just found out[1] this[2]:
 
 	The garbage collector is not guaranteed to run the destructor for
 	all unreferenced objects.
 
 Is there any reason why D can't guarantee that all finalizers will be
 called, at least when the program ends?
 
 [1] http://proj.llucax.com.ar/blog/dgc/blog/post/-43101db1
 [2] http://www.digitalmars.com/d/1.0/class.html#destructors
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Unknown W. Brackets, el 18 de abril a las 16:16 me escribiste:

 The simple solution is this:
 
 1. If your class object only involves memory, freed OS handles, etc., it
 should be used as-is.  This is most class objects.  Destructors are
 needed to clamp resource use (see File class.)


A File class implementation, for instance, can loose data if they use
buffers. For example:

class File
{
	//...

	~this()
	{
		this.flush(); // write remaining buffered data
		close(handle);
	}
}

In this case, the OS will close the handle, but the data will not be
written.

So I guess this kind of things has to go in your 2nd group.

And this is a good example for a valid use that will break if guaranteed
finalization is not provided. A log file is a good example of an object
that lasts for all the program lifetime and needs guaranteed finalization.


 2. If your class object involves hardware handles, transactional
 assurance, or data integrity, it must be scoped.  This is the same as it
 was in C, except D has better constucts for it (see scope.)
 
 I don't see the problem.  For the majority of objects that only involve
 memory, life is easier.  For the rest, life is still easier (just not as
 much.)


Ok, I agree. Do you see any problems with support to guaranteed
finalization?

My proposal is:
a) Add to the specs that finalization is guaranteed, in the worse case at
   program termination.
b) Standarize the gc_setTermHandlerDepth() (or whatever the name is)
c) Use a "finalize all the live data" at program exit as a default for
   the TermHandlerDepth (without running a collection, but this is
   completely implementation dependant).

Comments?

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Soy como una mosca, parada en el agua.
Y vos sos un transatlántico, querés nadar a mi lado.
Y me estás ahogando.
Apr 18 2009
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Well, I guess it would be doable to guarantee destruction, but *only* if 
order of destruction was not guaranteed.

In other words, at the end, the GC would not scan, it would just 
destroy.  Scanning is a problem, due to cycles and various other things, 
but destroying all roots would be potentially doable.

I disagree that a log file needs guaranteed finalization, however.  It's 
when you think that way that you ignore the possibility of hardware 
failure.  I'm a server guy - and all the daemons I use log immediately 
to the filesystem, without buffering.  Anything else would be unacceptable.

I may be wrong, but I'm pretty sure std.stream's files do not work as 
you suggest.  BufferedFile may, but I do not think File does (aside from 
OS buffers and journals which are fine; an fsync is not mandatory.)

-[Unknown]


Leandro Lucarella wrote:

 Unknown W. Brackets, el 18 de abril a las 16:16 me escribiste:

 The simple solution is this:

 1. If your class object only involves memory, freed OS handles, etc., it
 should be used as-is.  This is most class objects.  Destructors are
 needed to clamp resource use (see File class.)


 
 A File class implementation, for instance, can loose data if they use
 buffers. For example:
 
 class File
 {
 	//...
 
 	~this()
 	{
 		this.flush(); // write remaining buffered data
 		close(handle);
 	}
 }
 
 In this case, the OS will close the handle, but the data will not be
 written.
 
 So I guess this kind of things has to go in your 2nd group.
 
 And this is a good example for a valid use that will break if guaranteed
 finalization is not provided. A log file is a good example of an object
 that lasts for all the program lifetime and needs guaranteed finalization.
 

 2. If your class object involves hardware handles, transactional
 assurance, or data integrity, it must be scoped.  This is the same as it
 was in C, except D has better constucts for it (see scope.)

 I don't see the problem.  For the majority of objects that only involve
 memory, life is easier.  For the rest, life is still easier (just not as
 much.)


 
 Ok, I agree. Do you see any problems with support to guaranteed
 finalization?
 
 My proposal is:
 a) Add to the specs that finalization is guaranteed, in the worse case at
    program termination.
 b) Standarize the gc_setTermHandlerDepth() (or whatever the name is)
 c) Use a "finalize all the live data" at program exit as a default for
    the TermHandlerDepth (without running a collection, but this is
    completely implementation dependant).
 
 Comments?
Apr 18 2009
parent reply Leandro Lucarella <llucax gmail.com> writes:
Unknown W. Brackets, el 18 de abril a las 16:51 me escribiste:

 Well, I guess it would be doable to guarantee destruction, but *only* if
 order of destruction was not guaranteed.


Yes, of course, order *can't* be guaranteed (unless you add read/write
barriers and a lot of overhead at least =)


 In other words, at the end, the GC would not scan, it would just
 destroy.


Yes, that's the proposal.


 Scanning is a problem, due to cycles and various other things,


I don't see any problems with scanning, it's just time-consuming and
provides no benefits. Even more, the actual implementation do
a fullcollect() (which marks, scans and sweeps).


 but destroying all roots would be potentially doable.


My proposal is to destroy everything live. There is no need to follow the
roots, all memory known as live should be finalized. Again, there is no
need on scanning the roots.


 I disagree that a log file needs guaranteed finalization, however.  It's
 when you think that way that you ignore the possibility of hardware
 failure.  I'm a server guy - and all the daemons I use log immediately
 to the filesystem, without buffering.  Anything else would be
 unacceptable.


Maybe you don't need that kind of security, but you would like your
program to behave as expected in normal situations (i.e. when no hardware
failure is involved).

Anyway the problem is no because of some example (I think it was a big
mistake from my part to take the discussion into the examples field). The
problem is the specs says now that "The garbage collector is not
guaranteed to run the destructor for all unreferenced objects.", it
doesn't say this is only applicable for destruction at the end of the
program. As far as the specs go, a GC that don't call finalizers during
collection *ever* is conforming.


-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Hay manos capaces de fabricar herramientas
con las que se hacen máquinas para hacer ordenadores
que a su vez diseñan máquinas que hacen herramientas
para que las use la mano
Apr 19 2009
parent "Unknown W. Brackets" <unknown simplemachines.org> writes:
Personally, I usually use destructors to clean up or to decrement use 
counts, bleach secure data, etc. but it should be guaranteed that if 
data is no longer allocated, the destructor is called.  That may be 
elsewhere in the spec, though.

Also, hardware failure should always be expected:

Webservers like Apache handle it.
Browsers like Firefox handle it.
Databases like MySQL handle it.
Operating systems like Windows handle it.
Productivity apps like Word handle it.
FTP clients like SmartFTP handle it.
Communication apps like Trillian handle it.

No one should esteem to write crapware.

-[Unknown]


Leandro Lucarella wrote:

 Unknown W. Brackets, el 18 de abril a las 16:51 me escribiste:

 Well, I guess it would be doable to guarantee destruction, but *only* if
 order of destruction was not guaranteed.


 
 Yes, of course, order *can't* be guaranteed (unless you add read/write
 barriers and a lot of overhead at least =)
 

 In other words, at the end, the GC would not scan, it would just
 destroy.


 
 Yes, that's the proposal.
 

 Scanning is a problem, due to cycles and various other things,


 
 I don't see any problems with scanning, it's just time-consuming and
 provides no benefits. Even more, the actual implementation do
 a fullcollect() (which marks, scans and sweeps).
 

 but destroying all roots would be potentially doable.


 
 My proposal is to destroy everything live. There is no need to follow the
 roots, all memory known as live should be finalized. Again, there is no
 need on scanning the roots.
 

 I disagree that a log file needs guaranteed finalization, however.  It's
 when you think that way that you ignore the possibility of hardware
 failure.  I'm a server guy - and all the daemons I use log immediately
 to the filesystem, without buffering.  Anything else would be
 unacceptable.


 
 Maybe you don't need that kind of security, but you would like your
 program to behave as expected in normal situations (i.e. when no hardware
 failure is involved).
 
 Anyway the problem is no because of some example (I think it was a big
 mistake from my part to take the discussion into the examples field). The
 problem is the specs says now that "The garbage collector is not
 guaranteed to run the destructor for all unreferenced objects.", it
 doesn't say this is only applicable for destruction at the end of the
 program. As far as the specs go, a GC that don't call finalizers during
 collection *ever* is conforming.
Apr 19 2009
prev sibling parent Jason House <jason.james.house gmail.com> writes:
Walter Bright Wrote:


 Leandro Lucarella wrote:

 Close a connection gracefully for example, I guess (I mean, send a "bye"
 packed, not just close the socket abruptly). Same for closing files
 writing some mark or something. They can be risky when finalization is not
 deterministic though.


 
 Scoped objects should be used for that, not gc.


What about member variables of scope objects? IIRC classes can't have scope
member variables which can be frustrating when creating wrappers.

Can scope global variables exist?
Apr 18 2009