www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Destructor semantics

reply foo <foo bar.com> writes:
In light on recent discussions of clear() and the distructor it seems to me
that we are going backwards from one of D's great improvements over C++ - the
difference in semantics between structs and classes.

IMO, instead of enhancing class desteructors they should be completely removed
and only allowed on structs with deterministic semantics and all uses cases of
class desteructors should be replaced with structs. 
Examples:
class SocketConnection : Connection {
// struct instance allocated inline  
SocketHandle handle; 
...
}

OR:

class SocketConnection : Connection {
struct {
   this()  { acquireHandle(); }
  ~this() { releaseHandle(); }
} handle;
...
}

The suggested semantics of the above code would be that creating a 
SocketConnection object would also construct a SocketHandle as part of the
object's memory and in turn that would call the struct's ctor. 
On destruction of the object, the struct member would be also destructed and
it's d-tor is called. This is safe since the struct is part of the same memory
as the object. 

in short, struct instances should be treated just like built-in types.
Aug 10 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:


 In light on recent discussions of clear() and the distructor it seems to  
 me that we are going backwards from one of D's great improvements over  
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely  
 removed and only allowed on structs with deterministic semantics and all  
 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of  
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also destructed  
 and it's d-tor is called. This is safe since the struct is part of the  
 same memory as the object.

 in short, struct instances should be treated just like built-in types.


That doesn't help.  deterministic destruction is not a struct-vs-class  
problem, its a GC-vs-manual-memory problem.  A struct on the heap that is  
finalized by the GC has the same issues as a class destructor.  In fact,  
struct destructors are not currently called when they are heap-allocated  
because the GC has no idea what is stored in those memory locations.

-Steve
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:
 

 In light on recent discussions of clear() and the distructor it seems to  
 me that we are going backwards from one of D's great improvements over  
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely  
 removed and only allowed on structs with deterministic semantics and all  
 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of  
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also destructed  
 and it's d-tor is called. This is safe since the struct is part of the  
 same memory as the object.

 in short, struct instances should be treated just like built-in types.


 
 That doesn't help.  deterministic destruction is not a struct-vs-class  
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that is  
 finalized by the GC has the same issues as a class destructor.  In fact,  
 struct destructors are not currently called when they are heap-allocated  
 because the GC has no idea what is stored in those memory locations.
 
 -Steve


Let me add to the above, that the GC should NOT manage structs allocated on the
heap. structs should only provide deterministic semantics.
Aug 10 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 17:23:39 -0400, foobar <foo bar.com> said:


 Steven Schveighoffer Wrote:
 

 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.
 
 -Steve


 
 Let me add to the above, that the GC should NOT manage structs 
 allocated on the heap. structs should only provide deterministic 
 semantics.


That's an interesting idea. By allowing classes only on the 
garbage-collected heap, and structs only in non-garbage-collected 
situations, we do indeed simplify the destructor problem. A struct 
destructor is always deterministic and a class destructor is not.

Unfortunately, that's not exactly where things are headed.

I'm not sure what's best, but I'm starting to believe that without this 
simple rule we'll have to add the complexity of having two kind of 
destructors in the language... would this make sense:

	class Test {

		~this()
		{
			// disposer destructor
			// called by deterministic destruction
			// can access GC-managed members safely
		}

		~~this()
		{
			// finalizer destructor
			// called during garbage collection
			// cannot access GC-managed members (might be dangling pointers)
			// unavailable in SafeD because of the dangling pointers
		}

	}


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Michel Fortin wrote:


 On 2010-08-10 17:23:39 -0400, foobar <foo bar.com> said:
 

 Steven Schveighoffer Wrote:
 

 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.
 
 -Steve


 
 Let me add to the above, that the GC should NOT manage structs
 allocated on the heap. structs should only provide deterministic
 semantics.


 
 That's an interesting idea. By allowing classes only on the
 garbage-collected heap, and structs only in non-garbage-collected
 situations, we do indeed simplify the destructor problem. A struct
 destructor is always deterministic and a class destructor is not.
 
 Unfortunately, that's not exactly where things are headed.
 
 I'm not sure what's best, but I'm starting to believe that without this
 simple rule we'll have to add the complexity of having two kind of
 destructors in the language... would this make sense:
 
 class Test {
 
 ~this()
 {
 // disposer destructor
 // called by deterministic destruction
 // can access GC-managed members safely
 }
 
 ~~this()
 {
 // finalizer destructor
 // called during garbage collection
 // cannot access GC-managed members (might be dangling pointers)
 // unavailable in SafeD because of the dangling pointers
 }
 
 }
 


This is what I proposed, but the other way around and the semantics of ~~this 
implemented by an interface. If we keep ~this as the finalizer, the language 
need not to change so it will not impact existing code too much. Especially 
because delete will go away. It is not too bad, we don't need to do everything 
with it like .NET has to. You can always wrap a class in a struct that will 
destroy it, or use reference counting like File does. I think that should be 
preferred.
Aug 10 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 18:51:24 -0400, Lutger <lutger.blijdestijn gmail.com> said:


 This is what I proposed, but the other way around and the semantics of 
 ~~this implemented by an interface. If we keep ~this as the finalizer, 
 the language need not to change so it will not impact existing code too 
 much.


Perhaps you're right, but I wonder...

Structs can be on the GC heap too, so structs also need two kinds of 
destructors. And I believe right now much more structs have a 
destructor than classes have one.

So the question becomes: are most struct destructors currently intended 
to be run during collection cycle or they are expected to be called in 
a more deterministic way? I don't have the answer to that 
unfortunately. The only thing I can say is that if a destructor 
accesses the GC-heap through one of its members then it is not suitable 
to be run during a collection cycle. (And most classes, strings, or 
arrays are in GC heap.)


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 17:23:39 -0400, foobar <foo bar.com> wrote:


 Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:


 In light on recent discussions of clear() and the distructor it seems  


 to

 me that we are going backwards from one of D's great improvements over
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely
 removed and only allowed on structs with deterministic semantics and  


 all

 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also  


 destructed

 and it's d-tor is called. This is safe since the struct is part of the
 same memory as the object.

 in short, struct instances should be treated just like built-in types.


 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that  
 is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.

 -Steve


 Let me add to the above, that the GC should NOT manage structs allocated  
 on the heap. structs should only provide deterministic semantics.


So either you are saying that structs that are in classes are never  
destroyed, and you have a resource leak, or every class has an  
auto-generated destructor that calls the struct destructors, and we have  
the same determinism problem you purport to solve.  If a struct is in a  
class, it's on the heap.  You have not solved the problem.

-Steve
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 17:23:39 -0400, foobar <foo bar.com> wrote:
 

 Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 16:33:08 -0400, foo <foo bar.com> wrote:


 In light on recent discussions of clear() and the distructor it seems  


 to

 me that we are going backwards from one of D's great improvements over
 C++ - the difference in semantics between structs and classes.

 IMO, instead of enhancing class desteructors they should be completely
 removed and only allowed on structs with deterministic semantics and  


 all

 uses cases of class desteructors should be replaced with structs.
 Examples:
 class SocketConnection : Connection {
 // struct instance allocated inline
 SocketHandle handle;
 ...
 }

 OR:

 class SocketConnection : Connection {
 struct {
    this()  { acquireHandle(); }
   ~this() { releaseHandle(); }
 } handle;
 ...
 }

 The suggested semantics of the above code would be that creating a
 SocketConnection object would also construct a SocketHandle as part of
 the object's memory and in turn that would call the struct's ctor.
 On destruction of the object, the struct member would be also  


 destructed

 and it's d-tor is called. This is safe since the struct is part of the
 same memory as the object.

 in short, struct instances should be treated just like built-in types.


 That doesn't help.  deterministic destruction is not a struct-vs-class
 problem, its a GC-vs-manual-memory problem.  A struct on the heap that  
 is
 finalized by the GC has the same issues as a class destructor.  In fact,
 struct destructors are not currently called when they are heap-allocated
 because the GC has no idea what is stored in those memory locations.

 -Steve


 Let me add to the above, that the GC should NOT manage structs allocated  
 on the heap. structs should only provide deterministic semantics.


 
 So either you are saying that structs that are in classes are never  
 destroyed, and you have a resource leak, or every class has an  
 auto-generated destructor that calls the struct destructors, and we have  
 the same determinism problem you purport to solve.  If a struct is in a  
 class, it's on the heap.  You have not solved the problem.
 
 -Steve


Sorry for the confusion, my explanation wasn't good enough. Let me try again:

We have 4 different cases:

case 1,  class contains a struct:
a.  the sturct is not an independent block of memory. Instead, it is part of
the same memory block of the class instance's memory. 
b. every class has an auto-generated destructor that calls the struct
destructors. This is safe because of the point a. 

case 2, class contains a class:
this has same non-deterministic semantics as today. The containing class cannot
call the dtor for the contained class.

case 3, struct contains a class:
struct dtor calls deterministically the auto generated dtor for the class
(clean any member structs of the class)

case 4, struct contains a struct
c++ deterministic dtor sementics. 

if you allocate a struct instance on the heap you have to deallocate it (NOT
GCed) and the explicit delete would trigger the deterministic dtor behavior.

example:

struct A { ~this(); }
class B { A a; }
struct C {B b; ~this(); };
auto foo = new C();
...
delete foo; // [*]

when we reach [*], this is what will happen:
C::~this() is called
  B::~this{} is called // this is ALWAYS auto generated
    A::~this() is called

in memory we have two separate memory blocks; 
the first contains an instance of C (this includes a ref to a B)
and the second contains an instance of B which also contains _inline_ an
instance of A.
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 18:09:53 -0400, foobar <foo bar.com> wrote:


 Steven Schveighoffer Wrote:


 So either you are saying that structs that are in classes are never
 destroyed, and you have a resource leak, or every class has an
 auto-generated destructor that calls the struct destructors, and we have
 the same determinism problem you purport to solve.  If a struct is in a
 class, it's on the heap.  You have not solved the problem.

 -Steve


 Sorry for the confusion, my explanation wasn't good enough. Let me try  
 again:

 We have 4 different cases:

 case 1,  class contains a struct:
 a.  the sturct is not an independent block of memory. Instead, it is  
 part of the same memory block of the class instance's memory.
 b. every class has an auto-generated destructor that calls the struct  
 destructors. This is safe because of the point a.


It's only safe if the struct destructor does not deallocate any heap data.

Here is a simple counter case

Class contains struct, which in turn contains another class. The struct  
destructor cannot run because the class it points to may already have been  
cleaned up.

Example:

class A {}

struct B { A a; ~this() {delete a;} }

class C { B b; }

what happens when GC destroys a C?

C::~this(); // auto generated
    B::~this(); // so good so far
       A::~this(); // oops!  the a is gone, program vomits bits all over  
itself and chokes to death.

-Steve
Aug 10 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 18:39:03 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 It's only safe if the struct destructor does not deallocate any heap data.


To be even clearer, it's only safe if the struct destructor doesn't 
access any heap data through one of its members.

And accessing an array or a string could potentially count as accessing 
heap data too (unless you can be sure it wasn't from the GC-heap).

So destructors called during the collection cycle are a very tricky 
thing to handle.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Michel Fortin Wrote:


 On 2010-08-10 18:39:03 -0400, "Steven Schveighoffer" 
 <schveiguy yahoo.com> said:
 

 It's only safe if the struct destructor does not deallocate any heap data.


 
 To be even clearer, it's only safe if the struct destructor doesn't 
 access any heap data through one of its members.
 
 And accessing an array or a string could potentially count as accessing 
 heap data too (unless you can be sure it wasn't from the GC-heap).
 
 So destructors called during the collection cycle are a very tricky 
 thing to handle.
 
 -- 
 Michel Fortin
 michel.fortin michelf.com
 http://michelf.com/
 


I agree :)

In general, the rule is that destructors can only access value types. This can
be enforced at compile time. 
Also, I was using structs and classes in my examples but this should be
understood as value types vs. reference types. 

For example, when a dtor for a fixed-sized array is called (value type) it will
in turn call dtors for its elements
Aug 10 2010
parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Tuesday, August 10, 2010 16:15:33 foobar wrote:

 I agree :)
 
 In general, the rule is that destructors can only access value types. This
 can be enforced at compile time. Also, I was using structs and classes in
 my examples but this should be understood as value types vs. reference
 types.
 
 For example, when a dtor for a fixed-sized array is called (value type) it
 will in turn call dtors for its elements


If attempts to use any reference types in destructors were a compile-time error 
with a clear error message, that could go a long way in stopping people from 
trying to misuse destructors. As it is, there _are_ going to be plenty of D 
programmers who write destructors which access references to GC-allocated data 
and won't understand the weird bugs that they're getting.

- Jonathan M Davis
Aug 10 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 19:58:08 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:


 On Tuesday, August 10, 2010 16:15:33 foobar wrote:

 I agree :)
 
 In general, the rule is that destructors can only access value types. This
 can be enforced at compile time. Also, I was using structs and classes in
 my examples but this should be understood as value types vs. reference
 types.
 
 For example, when a dtor for a fixed-sized array is called (value type) it
 will in turn call dtors for its elements


 
 If attempts to use any reference types in destructors were a compile-time error
 with a clear error message, that could go a long way in stopping people from
 trying to misuse destructors. As it is, there _are_ going to be plenty of D
 programmers who write destructors which access references to GC-allocated data
 and won't understand the weird bugs that they're getting.


Indeed.

But the compiler has no way to know at compile time whether a pointer, 
an array, or a reference points to GC-heap or somewhere else (global 
data or manually allocated). That's something only the programmer may 
know.

Perhaps this could be done for  safe mode however: disallow anything 
that could dereference a member (assuming it might be on the GC-heap).

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
prev sibling parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 18:09:53 -0400, foobar <foo bar.com> wrote:
 

 Steven Schveighoffer Wrote:


 So either you are saying that structs that are in classes are never
 destroyed, and you have a resource leak, or every class has an
 auto-generated destructor that calls the struct destructors, and we have
 the same determinism problem you purport to solve.  If a struct is in a
 class, it's on the heap.  You have not solved the problem.

 -Steve


 Sorry for the confusion, my explanation wasn't good enough. Let me try  
 again:

 We have 4 different cases:

 case 1,  class contains a struct:
 a.  the sturct is not an independent block of memory. Instead, it is  
 part of the same memory block of the class instance's memory.
 b. every class has an auto-generated destructor that calls the struct  
 destructors. This is safe because of the point a.


 
 It's only safe if the struct destructor does not deallocate any heap data.
 
 Here is a simple counter case
 
 Class contains struct, which in turn contains another class. The struct  
 destructor cannot run because the class it points to may already have been  
 cleaned up.
 
 Example:
 
 class A {}
 
 struct B { A a; ~this() {delete a;} }
 
 class C { B b; }
 
 what happens when GC destroys a C?
 
 C::~this(); // auto generated
     B::~this(); // so good so far
        A::~this(); // oops!  the a is gone, program vomits bits all over  
 itself and chokes to death.
 
 -Steve


This can only happen if you use delete on a class instance. My understanding
was that this is going to be removed from D2.

Without this (mis-)feature, the GC cannot remove an instance of A as long as
the matching instance of C which (indirectly) contains a reference to it is
still alive.
Aug 10 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 8/10/2010 16:59, foobar wrote:

 Steven Schveighoffer Wrote:

 what happens when GC destroys a C?
 
 C::~this(); // auto generated
   B::~this(); // so good so far 
     A::~this(); // oops!  the a is gone, program vomits bits all over itself
and
 chokes to death.
 
 -Steve


 
 This can only happen if you use delete on a class instance. My
 understanding was that this is going to be removed from D2.


Same problem without 'delete':

class A {
  void dispose();
}

struct B {
  A a;
  ~this() { a.dispose(); }
}

class C {
  B b;
}

C::~this(); // auto generated
  B::~this(); // so good so far
    A::dispose(); // oops!


-- 
Rainer Deyke - rainerd eldwood.com
Aug 10 2010
parent reply foobar <foo bar.com> writes:
Rainer Deyke Wrote:


 On 8/10/2010 16:59, foobar wrote:

 Steven Schveighoffer Wrote:

 what happens when GC destroys a C?
 
 C::~this(); // auto generated
   B::~this(); // so good so far 
     A::~this(); // oops!  the a is gone, program vomits bits all over itself
and
 chokes to death.
 
 -Steve


 
 This can only happen if you use delete on a class instance. My
 understanding was that this is going to be removed from D2.


 
 Same problem without 'delete':
 
 class A {
   void dispose();
 }
 
 struct B {
   A a;
   ~this() { a.dispose(); }
 }
 
 class C {
   B b;
 }
 
 C::~this(); // auto generated
   B::~this(); // so good so far
     A::dispose(); // oops!
 
 
 -- 
 Rainer Deyke - rainerd eldwood.com


I was posing late at night and hence made a mistake in my suggestion. 
Here's a better example of the problem:
class A{}
struct B{ A a; ~this(); this(ref A); }

auto obj = new A();
auto first = new B(obj);
auto second = new B(obj);

both first and second reference the same instance of A.

The correct semantics for the case of a struct containing a class (more
generally, value type contains a reference type):

the struct's dtor does NOT call the class dtor. 

The class dtor would be called either by the GC when it is collected or when it
is de-allocated by the user when its memory is managed by a different memory
scheme. 

Sorry for this confusion, it'll teach me not to post at 2AM..
Aug 11 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 03:12:57 -0400, foobar <foo bar.com> wrote:


 Rainer Deyke Wrote:


 On 8/10/2010 16:59, foobar wrote:

 Steven Schveighoffer Wrote:

 what happens when GC destroys a C?

 C::~this(); // auto generated
   B::~this(); // so good so far
     A::~this(); // oops!  the a is gone, program vomits bits all  




 over itself and

 chokes to death.

 -Steve


 This can only happen if you use delete on a class instance. My
 understanding was that this is going to be removed from D2.


 Same problem without 'delete':

 class A {
   void dispose();
 }

 struct B {
   A a;
   ~this() { a.dispose(); }
 }

 class C {
   B b;
 }

 C::~this(); // auto generated
   B::~this(); // so good so far
     A::dispose(); // oops!


 --
 Rainer Deyke - rainerd eldwood.com


 I was posing late at night and hence made a mistake in my suggestion.
 Here's a better example of the problem:
 class A{}
 struct B{ A a; ~this(); this(ref A); }

 auto obj = new A();
 auto first = new B(obj);
 auto second = new B(obj);

 both first and second reference the same instance of A.

 The correct semantics for the case of a struct containing a class (more  
 generally, value type contains a reference type):

 the struct's dtor does NOT call the class dtor.

 The class dtor would be called either by the GC when it is collected or  
 when it is de-allocated by the user when its memory is managed by a  
 different memory scheme.

 Sorry for this confusion, it'll teach me not to post at 2AM..


So if a struct has a class reference, it cannot clean it up?  What if the  
class contains a struct that has an open file reference, and you want to  
clean up that file immediately?  What if that class is private and the  
struct knows that there are no other references to it?

Your "solution" doesn't cover any new ground, we already have the issue  
that you cannot clean up or even access heap-allocated references in  
destructors.  The problem is, you don't know from the type system that  
they are heap-allocated, and the compiler cannot know what is owned and  
what is not owned, so it can't make the call to restrict you.

What we need is a way to determine whether we can access those resources  
or not in the destructor, and it has nothing to do with struct vs. class,  
and everything to do with deterministic vs GC.  Making an artificial  
distinction on class/struct lines doesn't help.  We have the same problem,  
only worse restrictions (now I can't destroy a handle in a class on  
destruction, I need to put in a struct layer around it).

-Steve
Aug 11 2010
next sibling parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:



 So if a struct has a class reference, it cannot clean it up?  What if the  
 class contains a struct that has an open file reference, and you want to  
 clean up that file immediately?  What if that class is private and the  
 struct knows that there are no other references to it?
 
 Your "solution" doesn't cover any new ground, we already have the issue  
 that you cannot clean up or even access heap-allocated references in  
 destructors.  The problem is, you don't know from the type system that  
 they are heap-allocated, and the compiler cannot know what is owned and  
 what is not owned, so it can't make the call to restrict you.
 
 What we need is a way to determine whether we can access those resources  
 or not in the destructor, and it has nothing to do with struct vs. class,  
 and everything to do with deterministic vs GC.  Making an artificial  
 distinction on class/struct lines doesn't help.  We have the same problem,  
 only worse restrictions (now I can't destroy a handle in a class on  
 destruction, I need to put in a struct layer around it).
 
 -Steve


I see your point above. I feel that my approach is more structured. I think
that at minimum the compiler should prevent (at compile time) any access to
reference types from a class dtor. 
I can't imagine what can be your use case for allowing such access. It seems to
me that any time you feel the need for one class instance to own a different
class instance you should use a struct instead to convey that owned
relationship.

Wouldn't you need an annotation system like the one suggested by Bartoz in
order to properly implement the owned relationship for reference types?
Aug 11 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 09:53:24 -0400, foobar <foo bar.com> wrote:


 Steven Schveighoffer Wrote:



 So if a struct has a class reference, it cannot clean it up?  What if  
 the
 class contains a struct that has an open file reference, and you want to
 clean up that file immediately?  What if that class is private and the
 struct knows that there are no other references to it?

 Your "solution" doesn't cover any new ground, we already have the issue
 that you cannot clean up or even access heap-allocated references in
 destructors.  The problem is, you don't know from the type system that
 they are heap-allocated, and the compiler cannot know what is owned and
 what is not owned, so it can't make the call to restrict you.

 What we need is a way to determine whether we can access those resources
 or not in the destructor, and it has nothing to do with struct vs.  
 class,
 and everything to do with deterministic vs GC.  Making an artificial
 distinction on class/struct lines doesn't help.  We have the same  
 problem,
 only worse restrictions (now I can't destroy a handle in a class on
 destruction, I need to put in a struct layer around it).

 -Steve


 I see your point above. I feel that my approach is more structured. I  
 think that at minimum the compiler should prevent (at compile time) any  
 access to reference types from a class dtor.
 I can't imagine what can be your use case for allowing such access. It  
 seems to me that any time you feel the need for one class instance to  
 own a different class instance you should use a struct instead to convey  
 that owned relationship.

 Wouldn't you need an annotation system like the one suggested by Bartoz  
 in order to properly implement the owned relationship for reference  
 types?


The point you are still not getting is that reference type != heap.  The  
issue is heap data vs. non-heap data.  Heap data is invalid (except for  
data contained in your current memory block), non-heap data is not.  The  
compiler cannot tell whether a reference type is referencing heap data, so  
it cannot have a say in what to restrict.  That is my whole point.  All it  
can do is tell the destructor if heap references are valid or not (which  
it does not do now), and then it's up to the programmer who knows where  
the data lives (or doesn't know and assumes it could be heap data) to  
determine whether to access it or not.

I agree with what Michael suggested that destructors should be restricted  
to unsafe D.

-Steve
Aug 11 2010
parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 07:08:27 Steven Schveighoffer wrote:

 I agree with what Michael suggested that destructors should be restricted
 to unsafe D.


The more this whole issue gets discussed, the less it seems like I know about 
the issue. It's just too complicated to be immediately understandable. 
Personally, I think that the very fact that it's possible to access references 
which are no longer valid in a class destructor (and not only possible but 
_easily_ as well) makes it so that class destructors should be disallowed in 
SafeD. They just don't sound safe. If you really know what you're doing, you
can 
use them, but other than that, forget it. That sounds precisely like why SafeD 
exists in the first place.

Not to mention, it's looking more and more to me like if you want any kind of 
reasonable destruction going on, you need to be using a struct on the stack 
anyway. I'd suggest not only disallowing class destructors in SafeD but also 
disallowing structs with destructors on the heap in SafeD. It's just too messy.

- Jonathan M Davis
Aug 11 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 13:10:08 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:


 The more this whole issue gets discussed, the less it seems like I know about
 the issue. It's just too complicated to be immediately understandable.
 Personally, I think that the very fact that it's possible to access references
 which are no longer valid in a class destructor (and not only possible but
 _easily_ as well) makes it so that class destructors should be disallowed in
 SafeD. They just don't sound safe. If you really know what you're 
 doing, you can
 use them, but other than that, forget it. That sounds precisely like why SafeD
 exists in the first place.


I've made a bug report about destructors and SafeD, if anyone wants to 
add to it.
<http://d.puremagic.com/issues/show_bug.cgi?id=4621>



 Not to mention, it's looking more and more to me like if you want any kind of
 reasonable destruction going on, you need to be using a struct on the stack
 anyway. I'd suggest not only disallowing class destructors in SafeD but also
 disallowing structs with destructors on the heap in SafeD. It's just too messy.


I'm not too sure that'll work very well. I think a better solution 
would be to have a way to distinguish between a struct that can be put 
on the GC heap and one that cannot. A struct that cannot go on the GC 
heap make it safe to access GC-managed members in its destructor, and 
thus can have a  safe destructor.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.


But couldn't the fact that a struct has a destructor make it so that it can't
be 
declared anywhere but on the heap? The destructor itself could be what 
distinguishes them. I don't see a need for any other  attributes or whatnot to 
distinguish them.

- Jonathan M Davis
Aug 11 2010
next sibling parent Joe Greer <jgreer doubletake.com> writes:
Jonathan M Davis <jmdavisprog gmail.com> wrote in
news:mailman.260.1281553632.13841.digitalmars-d puremagic.com: 


 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be
 put on the GC heap and one that cannot. A struct that cannot go on
 the GC heap make it safe to access GC-managed members in its
 destructor, and thus can have a  safe destructor.


 
 But couldn't the fact that a struct has a destructor make it so that
 it can't be declared anywhere but on the heap? The destructor itself
 could be what distinguishes them. I don't see a need for any other
  attributes or whatnot to distinguish them.
 
 - Jonathan M Davis
 


Pardon me for sticking my nose in here.  I don't use D as of yet, but I 

ways of handling this sort of problem is to split destruction and 
finalization.  That is, the destructor is never invoked by the GC, only 
explicitly or when an object on the stack goes out of scope.  This means 
that everything referenced by the object is valid because it couldn't 
have been collected yet.  A finalizer can be provided to be invoked by 
the GC for limited cleanup.  Normally, this is never used, but sometimes 
is desirable.  Obviously, in the finalizer you can't count on anything 
allocated by the GC to still exist, however you could still free malloced 
memory here if you wanted or more likely complain that the object wasn't 
destructed when it should have been. 

I just haven't seen this idea tossed about yet, so I thought I would 
throw it out.

joe
Aug 11 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:


 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.


 
 But couldn't the fact that a struct has a destructor make it so that it 
 can't be
 declared anywhere but on the heap? The destructor itself could be what
 distinguishes them. I don't see a need for any other  attributes or whatnot to
 distinguish them.


Sure, and now you can't use std.containers.Array as a member in a 
Class, neither std.stdio.File, etc. Is there something left classes can 
do?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
next sibling parent reply Don <nospam nospam.com> writes:
Michel Fortin wrote:

 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> 
 said:
 

 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.


 But couldn't the fact that a struct has a destructor make it so that 
 it can't be
 declared anywhere but on the heap? The destructor itself could be what
 distinguishes them. I don't see a need for any other  attributes or 
 whatnot to
 distinguish them.


 
 Sure, and now you can't use std.containers.Array as a member in a Class, 
 neither std.stdio.File, etc. Is there something left classes can do?


As far as I can tell, the use cases for finalizers are very limited. To 
me, destructors don't make any sense unless they are deterministic.
For example, I think having a File class is probably a bug. You may be 
on a system which has no limit on the number of file handles, but 
generally, if you need resource management, you need a guarantee that 
the destructor will be called.

If a file handle is stored as part of a class, with a destructor that 
closes the file, the file might never get closed. So I don't think it's 
unreasonable to say that a struct with a destructor cannot be a member 
of a class.

Personally I think destructors should be restricted to structs and scope 
classes. I suspect that some form of registration with the gc could do 
the job of finalizers.

Destructor <=> deterministic destruction.
Aug 11 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:

 As far as I can tell, the use cases for finalizers are very limited. To 
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be 
 on a system which has no limit on the number of file handles, but 
 generally, if you need resource management, you need a guarantee that 
 the destructor will be called.


This seems similar to what I have written before:

http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=115028
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=115057

But if the purpose of a destructor is just to help/speedup the deallocation of
a RAM resource (like to nullify the links of a tree to speed up the job of the
GC) then in my opinion it is acceptable for this destructor to not run
deterministically.

Bye,
bearophile
Aug 11 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 16:26:58 -0400, bearophile <bearophileHUGS lycos.com> said:


 But if the purpose of a destructor is just to help/speedup the 
 deallocation of a RAM resource (like to nullify the links of a tree to 
 speed up the job of the GC) then in my opinion it is acceptable for 
 this destructor to not run deterministically.


Indeed. So it makes sense to have both a destructor and a finalizer, 
because the destructor can sometime speed up things for the GC.

I've made a proposal on the related bug report that reads like this:

---
For instance, instead of having just destructors, we could have destructors
(~this) and finalizers (~~this). A struct with neither can go anywhere, a
struct with a destructor but no finalizer cannot go on the GC-heap,  a struct
with only a finalizer can go anywhere (the finalizer is used as the
destructor), and a struct with both can go anywhere. The finalizer cannot be
made  safe.

Doing this with structs would probably mean allowing only finalizers (~~this)
for classes, which according to my syntax suggestion would break existing code
for class destructors. Perhaps the syntax should be different.
---
<http://d.puremagic.com/issues/show_bug.cgi?id=4621#c6>

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 13:14:56 Don wrote:

 As far as I can tell, the use cases for finalizers are very limited. To
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be
 on a system which has no limit on the number of file handles, but
 generally, if you need resource management, you need a guarantee that
 the destructor will be called.
 
 If a file handle is stored as part of a class, with a destructor that
 closes the file, the file might never get closed. So I don't think it's
 unreasonable to say that a struct with a destructor cannot be a member
 of a class.
 
 Personally I think destructors should be restricted to structs and scope
 classes. I suspect that some form of registration with the gc could do
 the job of finalizers.
 
 Destructor <=> deterministic destruction.


It would certainly make destructors more straightforward if they were
guaranteed 
to be deterministic. And making it so that they're deterministic in some cases 
and not in others seems like it would make them bug-prone, since doing
something 
with a determinstic destructor could be perfectly valid and acceptable while it 
could be a problem in a nondeterministic one.

- Jonathan M Davis
Aug 11 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 13:14:56 Don wrote:

 As far as I can tell, the use cases for finalizers are very limited. To
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be
 on a system which has no limit on the number of file handles, but
 generally, if you need resource management, you need a guarantee that
 the destructor will be called.
 
 If a file handle is stored as part of a class, with a destructor that
 closes the file, the file might never get closed. So I don't think it's
 unreasonable to say that a struct with a destructor cannot be a member
 of a class.
 
 Personally I think destructors should be restricted to structs and scope
 classes. I suspect that some form of registration with the gc could do
 the job of finalizers.
 
 Destructor <=> deterministic destruction.


It would certainly make destructors more straightforward if they were
guaranteed 
to be deterministic. And making it so that they're deterministic in some cases 
and not in others seems like it would make them bug-prone, since doing
something 
with a determinstic destructor could be perfectly valid and acceptable while it 
could be a problem in a nondeterministic one.

- Jonathan M Davis
Aug 11 2010
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 16:14:56 -0400, Don <nospam nospam.com> said:


 Michel Fortin wrote:

 Sure, and now you can't use std.containers.Array as a member in a 
 Class, neither std.stdio.File, etc. Is there something left classes can 
 do?


 
 As far as I can tell, the use cases for finalizers are very limited. To 
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be 
 on a system which has no limit on the number of file handles, but 
 generally, if you need resource management, you need a guarantee that 
 the destructor will be called.
 
 If a file handle is stored as part of a class, with a destructor that 
 closes the file, the file might never get closed. So I don't think it's 
 unreasonable to say that a struct with a destructor cannot be a member 
 of a class.


Yes, this might make sense for a file handle. But not all struct 
destructors handle such kind of resource. The other example was 
std.containers.Array. Should its use be disallowed in a class?



 Personally I think destructors should be restricted to structs and 
 scope classes. I suspect that some form of registration with the gc 
 could do the job of finalizers.
 
 Destructor <=> deterministic destruction.


Having a clear distinction between destructors and finalizers certainly 
makes sense.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 16:14:56 -0400, Don <nospam nospam.com> wrote:


 Michel Fortin wrote:

 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com>  
 said:


 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.


 But couldn't the fact that a struct has a destructor make it so that  
 it can't be
 declared anywhere but on the heap? The destructor itself could be what
 distinguishes them. I don't see a need for any other  attributes or  
 whatnot to
 distinguish them.


  Sure, and now you can't use std.containers.Array as a member in a  
 Class, neither std.stdio.File, etc. Is there something left classes can  
 do?


 As far as I can tell, the use cases for finalizers are very limited. To  
 me, destructors don't make any sense unless they are deterministic.
 For example, I think having a File class is probably a bug. You may be  
 on a system which has no limit on the number of file handles, but  
 generally, if you need resource management, you need a guarantee that  
 the destructor will be called.

 If a file handle is stored as part of a class, with a destructor that  
 closes the file, the file might never get closed. So I don't think it's  
 unreasonable to say that a struct with a destructor cannot be a member  
 of a class.


So classes are not allowed to have open files?  That's too limited.   
Deterministic closing of the file is still possible, just not guaranteed.

Here's the thing, we shouldn't be preventing smart people from writing  
useful code because we can't think of a way to make sure all idiot  
programmers close all their resources.  Note also that a resource leak is  
not as damaging as memory corruption, which we have somewhat of an  
obligation to try and prevent idiots from doing.


 Personally I think destructors should be restricted to structs and scope  
 classes. I suspect that some form of registration with the gc could do  
 the job of finalizers.

 Destructor <=> deterministic destruction.


Destructors as they are now are too limited, because they cannot be sure  
they are being called by the GC or not, they must assume so.  So in one  
sense I agree with you.  But in another sense, we *still* need a way to  
clean up non-GC resources from GC-allocated items.  Preventing GC  
allocated items from holding non-GC resources is a step in a very wrong  
direction.

I think any one of the proposals here that separates finalizers from  
destructors should be adequate.  My backwards-compatible one is to pass a  
parameter to the destructor indicating whether it's being called  
deterministically or not, but that doesn't lend itself to preventing  safe  
finalizers.  Michael also has one for using ~~this() and ~this().  We  
could designate a method that clear uses, like dispose() that  
deterministically disposes all resources.  I don't really like the  
interface solution, because looking up an interface is expensive, plus  
clear is a template so it doesn't need to use interfaces.

Whatever gets decided, I don't think anything should prevent them from  
being GC allocated, it's just too limiting for useful code.  The one  
initiative the compiler could take is to prevent writing a finalizer in  
 safe code, since that can lead to memory corruption.

-Steve
Aug 12 2010
parent reply Joe Greer <jgreer doubletake.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in
news:op.vhbpkcaieav7ka localhost.localdomain: 


 
 Destructors as they are now are too limited, because they cannot be
 sure  they are being called by the GC or not, they must assume so.  So
 in one  sense I agree with you.  But in another sense, we *still* need
 a way to  clean up non-GC resources from GC-allocated items. 
 Preventing GC  allocated items from holding non-GC resources is a step
 in a very wrong  direction.
 
 I think any one of the proposals here that separates finalizers from  
 destructors should be adequate.  My backwards-compatible one is to
 pass a  parameter to the destructor indicating whether it's being
 called  deterministically or not, but that doesn't lend itself to
 preventing  safe  finalizers.  Michael also has one for using ~~this()
 and ~this().  We  could designate a method that clear uses, like
 dispose() that  deterministically disposes all resources.  I don't
 really like the  interface solution, because looking up an interface
 is expensive, plus  clear is a template so it doesn't need to use
 interfaces. 
 
 Whatever gets decided, I don't think anything should prevent them from
  being GC allocated, it's just too limiting for useful code.  The one 
 initiative the compiler could take is to prevent writing a finalizer
 in   safe code, since that can lead to memory corruption.
 
 -Steve
 


Logically speaking if an object isn't destructed, then it lives forever 
and if it continues to hold it's resource, then we have a programming 
error.  The GC is for reclaiming memory, not files.  It can take a long 
time for a GC to reclaim an object and you surely don't want a file 
locked for that long anymore than you want it held open forever.  My 
point is, that it is a programming error to expect the GC be involved in 
reclaiming anything but memory.  IMO, the best use of a finalizer is to 
error out if an object holding a resource hasn't been destructed, because 
there is obviously a programming error here and something has leaked.  
GCs aren't there to support sloppy programming.  They are there to make 
your life easier and safer.

joe
Aug 12 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 08:59:31 -0400, Joe Greer <jgreer doubletake.com>  
wrote:


 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbpkcaieav7ka localhost.localdomain:


 Destructors as they are now are too limited, because they cannot be
 sure  they are being called by the GC or not, they must assume so.  So
 in one  sense I agree with you.  But in another sense, we *still* need
 a way to  clean up non-GC resources from GC-allocated items.
 Preventing GC  allocated items from holding non-GC resources is a step
 in a very wrong  direction.

 I think any one of the proposals here that separates finalizers from
 destructors should be adequate.  My backwards-compatible one is to
 pass a  parameter to the destructor indicating whether it's being
 called  deterministically or not, but that doesn't lend itself to
 preventing  safe  finalizers.  Michael also has one for using ~~this()
 and ~this().  We  could designate a method that clear uses, like
 dispose() that  deterministically disposes all resources.  I don't
 really like the  interface solution, because looking up an interface
 is expensive, plus  clear is a template so it doesn't need to use
 interfaces.

 Whatever gets decided, I don't think anything should prevent them from
  being GC allocated, it's just too limiting for useful code.  The one
 initiative the compiler could take is to prevent writing a finalizer
 in   safe code, since that can lead to memory corruption.

 -Steve


 Logically speaking if an object isn't destructed, then it lives forever
 and if it continues to hold it's resource, then we have a programming
 error.  The GC is for reclaiming memory, not files.  It can take a long
 time for a GC to reclaim an object and you surely don't want a file
 locked for that long anymore than you want it held open forever.  My
 point is, that it is a programming error to expect the GC be involved in
 reclaiming anything but memory.  IMO, the best use of a finalizer is to
 error out if an object holding a resource hasn't been destructed, because
 there is obviously a programming error here and something has leaked.
 GCs aren't there to support sloppy programming.  They are there to make
 your life easier and safer.


An open file maybe, but why should the compiler decide the severity of not  
closing the resource?  What if the resource is just some C-malloc'd memory?

Note, you are free to write your finalizer to do exactly what you want  
(print some error if it's called and the file's still open).  I just don't  
think the compiler should be involved in the decision, because it's too  
ill-equipped to make one.

-Steve
Aug 12 2010
parent reply Joe Greer <jgreer doubletake.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in
news:op.vhbtwjhoeav7ka localhost.localdomain: 


 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer. 


 
 An open file maybe, but why should the compiler decide the severity of
 not  closing the resource?  What if the resource is just some
 C-malloc'd memory? 
 
 Note, you are free to write your finalizer to do exactly what you want
  (print some error if it's called and the file's still open).  I just
 don't  think the compiler should be involved in the decision, because
 it's too  ill-equipped to make one.
 
 -Steve


I think you misunderstand what I was saying.  The compiler doesn't decide 
any of that.  The programmer conserned with correctness has the option of 
making his finalizer complain about unfreed resources.  I language 
shouldn't be your nanny, but it should encourage and enable you to do the 
right thing.  There is just no substitute for knowing how to program.

joe
Aug 12 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  
wrote:


 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:


 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.


 An open file maybe, but why should the compiler decide the severity of
 not  closing the resource?  What if the resource is just some
 C-malloc'd memory?

 Note, you are free to write your finalizer to do exactly what you want
  (print some error if it's called and the file's still open).  I just
 don't  think the compiler should be involved in the decision, because
 it's too  ill-equipped to make one.

 -Steve


 I think you misunderstand what I was saying.  The compiler doesn't decide
 any of that.  The programmer conserned with correctness has the option of
 making his finalizer complain about unfreed resources.  I language
 shouldn't be your nanny, but it should encourage and enable you to do the
 right thing.  There is just no substitute for knowing how to program.


So you mark a struct something like:

 noheap struct File
{...}

That's a possible solution.  I just don't like the blanket assumptions  
being made.

-Steve
Aug 12 2010
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-12 15:18:33 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  wrote:
 

 I think you misunderstand what I was saying.  The compiler doesn't decide
 any of that.  The programmer conserned with correctness has the option of
 making his finalizer complain about unfreed resources.  I language
 shouldn't be your nanny, but it should encourage and enable you to do the
 right thing.  There is just no substitute for knowing how to program.


 
 So you mark a struct something like:
 
  noheap struct File
 {...}
 
 That's a possible solution.  I just don't like the blanket assumptions  
 being made.


I don't like this assumption either. But short of a type system that 
track the owner of each memory block, all our choices will rely on the 
assumption that the programmer has done the finalizer right. Separating 
the concept of destructor and finalizer should help people realize the 
difference, but finalizers will still be a very tricky thing.

By the way, there's currently a couple of structs in Phobos that aren't 
GC-friendly -- they basically have races when their destructor is 
called during the collection cycle (and File is one of them). I'm more 
and more of the opinion that the language itself should just block you 
from dereferencing members in the finalizer (with some sort of cast to 
bypass this if needed, but then you'll think twice about what you're 
doing). See bug 4624:
<http://d.puremagic.com/issues/show_bug.cgi?id=4624>


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 12 2010
prev sibling parent reply Don <nospam nospam.com> writes:
Steven Schveighoffer wrote:

 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com> 
 wrote:
 

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:


 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.


 An open file maybe, but why should the compiler decide the severity of
 not  closing the resource?  What if the resource is just some
 C-malloc'd memory?






That's the only example of an nearly unlimited resource which I've heard 
thus far, but that raises the question, why the hell would you be using 
malloc if you're not going to free it, when you have a language with a 
gc? Effectively, the gc is freeing your malloced memory.

I can think of a couple of reasons for using malloc, but finalisers 
aren't a good solution to any of them.


 That's a possible solution.  I just don't like the blanket assumptions 
 being made.


Actually it's the absence of a use case.

Hypothesis: if a finalizer is run, where it actually DOES something (as 
opposed to, for example, running a pile of asserts), there's always a bug.

It's an extreme hypothesis, so it should be really easy to disprove.
Can you come up with a counterexample?
Aug 12 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-12 16:08:17 -0400, Don <nospam nospam.com> said:


 That's the only example of an nearly unlimited resource which I've 
 heard thus far, but that raises the question, why the hell would you be 
 using malloc if you're not going to free it, when you have a language 
 with a gc? Effectively, the gc is freeing your malloced memory.
 
 I can think of a couple of reasons for using malloc, but finalisers 
 aren't a good solution to any of them.


For one thing, you could ask Andrei why he based std.containers.Array 
on malloc/free with a reference counter. Not being able to use Array as 
a member of a class would be quite drastic.



 Hypothesis: if a finalizer is run, where it actually DOES something (as 
 opposed to, for example, running a pile of asserts), there's always a 
 bug.
 
 It's an extreme hypothesis, so it should be really easy to disprove.
 Can you come up with a counterexample?


Here is a real-world example: the D/Objective-C bridge use a class 
destructor/finalizer to release the Objective-C counterpart of a 
wrapper (by calling the release method on the Objective-C object). The 
Objective-C object is not something I can allocate with the D GC, and 
the D counterpart wouldn't be very usable if it was not managed by the 
D GC. So finalization does the cleanup, and that works just fine.

I wonder what QtD does, but it's probably something similar too.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 12 2010
parent Max Samukha <spambox d-coding.com> writes:
On 08/13/2010 12:07 AM, Michel Fortin wrote:


 Here is a real-world example: the D/Objective-C bridge use a class
 destructor/finalizer to release the Objective-C counterpart of a wrapper
 (by calling the release method on the Objective-C object). The
 Objective-C object is not something I can allocate with the D GC, and
 the D counterpart wouldn't be very usable if it was not managed by the D
 GC. So finalization does the cleanup, and that works just fine.

 I wonder what QtD does, but it's probably something similar too.


It does the same. A Qt object will be destroyed when the wrapper is GCed 
(if the Qt object is owned by D and does not have references in C++)

We tried to impose management of all QtD objects on the user but that 
proved to be a bad idea. The exception is QObject. In Qt QObjects are 
arranged in trees. When the root node is destroyed all its children are 
destroyed as well. GC is disabled for QObjects.
Aug 13 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 16:08:17 -0400, Don <nospam nospam.com> wrote:


 Steven Schveighoffer wrote:

 On Thu, 12 Aug 2010 13:05:53 -0400, Joe Greer <jgreer doubletake.com>  
 wrote:


 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in
 news:op.vhbtwjhoeav7ka localhost.localdomain:


 Logically speaking if an object isn't destructed, then it lives
 forever and if it continues to hold it's resource, then we have a
 programming error.  The GC is for reclaiming memory, not files.  It
 can take a long time for a GC to reclaim an object and you surely
 don't want a file locked for that long anymore than you want it held
 open forever.  My point is, that it is a programming error to expect
 the GC be involved in reclaiming anything but memory.  IMO, the best
 use of a finalizer is to error out if an object holding a resource
 hasn't been destructed, because there is obviously a programming
 error here and something has leaked. GCs aren't there to support
 sloppy programming.  They are there to make your life easier and
 safer.


 An open file maybe, but why should the compiler decide the severity of
 not  closing the resource?  What if the resource is just some
 C-malloc'd memory?






 That's the only example of an nearly unlimited resource which I've heard  
 thus far, but that raises the question, why the hell would you be using  
 malloc if you're not going to free it, when you have a language with a  
 gc? Effectively, the gc is freeing your malloced memory.


malloc/free is a lot more efficient than the GC.  Partly on account of it  
being so mature, and partly because it doesn't have to deal with GC-like  
problems.

Kris of Tango made an allocator for tango.container that is extremely fast  
based on malloc/free, but you are required to use it only for pure value  
types (no references).

Another reason is to be able to use it in the finalizer, since malloc'd  
memory will be valid.  If, for instance, you registered some object with a  
name, you need to have a malloc'd copy of the name to be able to  
unregister it on finalization.

Finally, if the 3rd party library your using gives you malloc'd data, what  
do you do then?


 That's a possible solution.  I just don't like the blanket assumptions  
 being made.


 Actually it's the absence of a use case.

 Hypothesis: if a finalizer is run, where it actually DOES something (as  
 opposed to, for example, running a pile of asserts), there's always a  
 bug.

 It's an extreme hypothesis, so it should be really easy to disprove.
 Can you come up with a counterexample?


Isn't it a bug to rely on finalization to tell you something is wrong with  
your program?  Essentially, if an object is finalized, and it must close  
resources, you have two options: close the resources and continue silently  
or loudly alert the user that there is a programming bug.  Considering  
that finalization is not guaranteed, that raises significantly the  
potential of bugs escaping into shipping code because the "bad" case just  
didn't get triggered.  I'd say in most cases, the bad case isn't so bad.

What if a finalizer asserted things were closed, and then closed them if  
they weren't.  That way, you get asserts when not compiling in release  
mode, and in release mode, the program soldiers on as best it can without  
throwing seemingly random errors.  It's just a thought.  I don't like the  
idea of throwing errors in the finalizer, because the error report can  
possibly be completey decoupled from the source.

For an example, Tango manages all I/O with classes, and I've written code  
that could run indefinitely opening and closing files continuously.  I  
never ran into resource problems.  If all of a sudden it started  
complaining that I wasn't closing the files, I'd question why someone put  
in that change, since it was working fine before.  I agree that the GC is  
not the best place to rely on closing files, but it doesn't *hurt* to  
close resources when you realize the last reference to that resource is  
about to be eliminated.

-Steve
Aug 13 2010
parent Adam Ruppe <destructionator gmail.com> writes:
Perhaps relevant to this discussion is the Old New Thing posts this
week. This last one has a good sentence:

http://blogs.msdn.com/b/oldnewthing/archive/2010/08/13/10049634.aspx

"If I ruled the world, I would decree that the only thing you can do
in a finalizer is perform some tests to ensure that all the associated
external resources have already been explicitly released, and if not,
raise a fatal exception: System.Exception.Resource=ADLeak."
Aug 13 2010
prev sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 8/12/2010 06:59, Joe Greer wrote:

 Logically speaking if an object isn't destructed, then it lives forever 
 and if it continues to hold it's resource, then we have a programming 
 error.  The GC is for reclaiming memory, not files.  It can take a long 
 time for a GC to reclaim an object and you surely don't want a file 
 locked for that long anymore than you want it held open forever.


Furthermore, the GC is conservative, so it isn't guaranteed to collect
any particular object at all, even if manually invoked.  (This also
means that any D program that relies in the GC potentially leaks memory.)


-- 
Rainer Deyke - rainerd eldwood.com
Aug 12 2010
prev sibling parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Wednesday, August 11, 2010 12:43:10 Michel Fortin wrote:

 On 2010-08-11 15:09:45 -0400, Jonathan M Davis <jmdavisprog gmail.com> said:

 On Wednesday, August 11, 2010 11:33:54 Michel Fortin wrote:

 I'm not too sure that'll work very well. I think a better solution
 would be to have a way to distinguish between a struct that can be put
 on the GC heap and one that cannot. A struct that cannot go on the GC
 heap make it safe to access GC-managed members in its destructor, and
 thus can have a  safe destructor.


 
 But couldn't the fact that a struct has a destructor make it so that it
 can't be
 declared anywhere but on the heap? The destructor itself could be what
 distinguishes them. I don't see a need for any other  attributes or
 whatnot to distinguish them.


 
 Sure, and now you can't use std.containers.Array as a member in a
 Class, neither std.stdio.File, etc. Is there something left classes can
 do?


This mess is just too complicated, and I'm getting more and more confused the 
more it gets discussed. I'll obviously need to study it closely, if I want to
be 
able to understand it properly.

Ideally, I'd be able to use structs on the heap and structs in classes with 
impunity. Their destructors get called when they leave scope or the class that 
their in gets destroyed (which may or may not be - though likely is - when it's 
garbage collected). Structs put directly on the heap obviously have trouble
with 
how things are right now, though you'd think that they'd be able to have their 
destructors run properly as well.

This needs to be sorted out in a manner that minimizes how much pain it causes 
for the typical programmer in their typical programming tasks. Destructors 
should not be this much trouble.

- Jonathan M Davis
Aug 11 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Jonathan M Davis wrote:

 This needs to be sorted out in a manner that minimizes how much pain it causes 
 for the typical programmer in their typical programming tasks. Destructors 
 should not be this much trouble.


I agree. Having too many interacting options just winds up confusing the heck 
out of people.
Aug 11 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 17:08:49 -0400, Walter Bright <newshound2 digitalmars.com> said:


 Jonathan M Davis wrote:

 This needs to be sorted out in a manner that minimizes how much pain it 
 causes for the typical programmer in their typical programming tasks. 
 Destructors should not be this much trouble.


 
 I agree. Having too many interacting options just winds up confusing 
 the heck out of people.


Typically, a destructor is easy to write; it's the finalizer that is 
tricky. Separating these two concepts would help a lot.

I also filled this bug against Phobos today:
<http://d.puremagic.com/issues/show_bug.cgi?id=4624>

It shows how easily this kind of bug can slip in.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-11 09:03:53 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 The problem is, you don't know from the type system that  they are 
 heap-allocated, and the compiler cannot know what is owned and  what is 
 not owned, so it can't make the call to restrict you.


Indeed, that's a real problem.

And I'm beginning to be amazed by the number of issues just begging for 
including owner information in the type system.

1. Destructor and GC-allocated data
2. Synchronized classes members with one or more levels of indirection
3. Safe creation of immutable data through a unique reference
4. Safe moving of mutable data between threads through a unique reference

Am I missing any other?

In all these cases (except the first), the compiler makes the 
conservative assumption and when that assumption is too conservative 
the "solution" is always to cast. I've always found that "solution" 
ridiculous because, unless it's going to be very rare, adding casts 
everywhere is just going to make things worse. And at this point, given 
all the use cases that needs casts, I don't think anyone is going to 
say it's going to be rare.

So I'm not really sure what to do about destructors. If we want it to 
be consistent with the rest, the compiler should take the conservative 
approach and disallow dereferencing members (and also calling any 
function that might dereference a member, ouch!). But once again, this 
is a fake solution to the problem: if we require casts everywhere, 
it'll be worse. So where do we draw the line?

Well, at the very least SafeD should take the conservative route.

The only "correct" solution in the end is to have owner information in 
the type system. But this has other issues...

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 11 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 If attempts to use any reference types in destructors were a compile-time
error 
 with a clear error message, that could go a long way in stopping people from 
 trying to misuse destructors.


This sounds like a positive idea, maybe fit for an enhancement request.

Bye,
bearophile
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 20:28:32 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Jonathan M Davis:

 If attempts to use any reference types in destructors were a  
 compile-time error
 with a clear error message, that could go a long way in stopping people  
 from
 trying to misuse destructors.


 This sounds like a positive idea, maybe fit for an enhancement request.


No, reference types are not necessarily heap allocated.  Guys, the  
distinction is heap vs. manual, not reference vs. value.  Value types can  
be on the heap, and references can refer to non-heap data.

Adding artificial restrictions that force casting are not going to help at  
all.

-Steve
Aug 11 2010
parent reply foobar <foo bar.com> writes:
Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 20:28:32 -0400, bearophile <bearophileHUGS lycos.com>  
 wrote:
 

 Jonathan M Davis:

 If attempts to use any reference types in destructors were a  
 compile-time error
 with a clear error message, that could go a long way in stopping people  
 from
 trying to misuse destructors.


 This sounds like a positive idea, maybe fit for an enhancement request.


 
 No, reference types are not necessarily heap allocated.  Guys, the  
 distinction is heap vs. manual, not reference vs. value.  Value types can  
 be on the heap, and references can refer to non-heap data.
 
 Adding artificial restrictions that force casting are not going to help at  
 all.
 
 -Steve


I disagree. The distinction should be owned vs. not-owned and NOT heap vs.
manual. Hence values vs. references. 

Simply put:
case 1)
  struct S;
  class C { S s;} // a C instance *owns* a S instance

case 2)
  class B;
  class C { B b; } // a C instance does *not* own a B instance

I believe that anything more complicated would require Bartosz' ownership
system.
Aug 11 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Aug 2010 10:03:41 -0400, foobar <foo bar.com> wrote:


 Steven Schveighoffer Wrote:


 On Tue, 10 Aug 2010 20:28:32 -0400, bearophile  
 <bearophileHUGS lycos.com>
 wrote:


 Jonathan M Davis:

 If attempts to use any reference types in destructors were a
 compile-time error
 with a clear error message, that could go a long way in stopping  




 people

 from
 trying to misuse destructors.


 This sounds like a positive idea, maybe fit for an enhancement  


 request.

 No, reference types are not necessarily heap allocated.  Guys, the
 distinction is heap vs. manual, not reference vs. value.  Value types  
 can
 be on the heap, and references can refer to non-heap data.

 Adding artificial restrictions that force casting are not going to help  
 at
 all.

 -Steve


 I disagree. The distinction should be owned vs. not-owned and NOT heap  
 vs. manual. Hence values vs. references.

 Simply put:
 case 1)
   struct S;
   class C { S s;} // a C instance *owns* a S instance


Yes


 case 2)
   class B;
   class C { B b; } // a C instance does *not* own a B instance


No, the ownership of b is not clear because we don't know what b is for.   
A class is allowed to own referenced data, ownership is an abstract  
concept.  Whether C is responsible for cleaning up b depends on whether 1)  
it is GC allocated or not, and 2) whether the user specifically requested  
the destruction of C.

Here is a better example:

class C { private FILE *fp; }

C is responsible for cleaning up fp, because fp is a *NON-GC-ALLOCATED  
REFERENCE*.

I can also find ways to allocate a B so it is a non-gc-allocated  
reference.  If I have a restriction that I can't access B, then the  
compiler is disallowing valid code, and that is unacceptable.

But that is besides the point.  If C owns a B, and C is being destroyed  
with its b reference intact, it can clean up b immediately knowing that  
nobody is referencing its B.  Your solution doesn't allow that, all it  
allows is what we have now, but with worse requirements.

Again, restricting access to references both introduces artificial  
problems and does not solve the original problem.  It's a lose-lose.


 I believe that anything more complicated would require Bartosz'  
 ownership system.


In order for the user to convey to the compiler ownership, so the compiler  
could possibly make some restrictions as to what is owned and what is not,  
yes.  But that's not all, the GC would have to obey that relationship by  
not cleaning up owned resources.  We don't have that, so the only  
reasonable choice is to make *NO* restrictions.

Another solution that has been mentioned before is to declare a member  
class instance as scope, which could mean that the actual data for the  
class is allocated in the same block as the class that owns the  
reference.  Then the owner destructor can choose to deallocate  
deterministically the owned object.  But that *still* doesn't solve the  
problem of being able to determine whether heap references are valid or  
not (it does provide a workaround).

-Steve
Aug 11 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Max Samukha:

 We tried to impose management of all QtD objects on the user but that 
 proved to be a bad idea. The exception is QObject. In Qt QObjects are 
 arranged in trees. When the root node is destroyed all its children are 
 destroyed as well. GC is disabled for QObjects.


I think Phobos2 can enjoy a semi-manual overarching memory allocator, like
halloc.

Bye,
bearophile
Aug 15 2010
parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Sunday 15 August 2010 10:10:29 bearophile wrote:

 Max Samukha:

 We tried to impose management of all QtD objects on the user but that
 proved to be a bad idea. The exception is QObject. In Qt QObjects are
 arranged in trees. When the root node is destroyed all its children are
 destroyed as well. GC is disabled for QObjects.


 
 I think Phobos2 can enjoy a semi-manual overarching memory allocator, like
 halloc.
 
 Bye,
 bearophile


Cry halloc! and let slip the dogs of war.

- Jonathan M Davis


Okay, I honestly have no idea what halloc is, but I just had to say that.  I 
_have_ been accused from time to time of loving puns too much though...
Aug 15 2010