www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TDPL: Manual invocation of destructor

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On Page 187 it states (shortened here for convenience):
"By calling clear(b), you invoke b's destructor, obliterate the object's state
with Buffer.init, and call Buffer's default constructor"

On the next page there's a unittest, and the entire example is this:

import core.stdc.stdlib;

class Buffer
{
    private void* data;
    
    // constructor
    this()
    {
        data = malloc(1024);
    }
    
    // destructor
    ~this()
    {
        free(data);
    }
}

unittest
{
    auto b = new Buffer;
    auto b1 = b;    // extra alias for b
    clear(b);
    assert(b1.data is null);
}

void main() { }

The assert fails, regardless if there is another reference to the object or not
(b1 in this case).

I've added some writeln()'s (not shown here), and just like the book said,
after clear(b), b's destructor gets called, the fields get initialized to
.init, and then b's constructor gets called. But the constructor will allocate
memory for b1.data again, which means data is not null anymore. 

So I'm guessing the assert code is wrong in the example?
Aug 08 2010
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
One more thing:

Is it possible for the compiler to detect missing constructors in a class?

E.g.:

class A
{
    static A a1;
    static A a2;

    static ~this()
    {
        clear(a1);
    }

    static ~this()
    {
        clear(a2);
    }

}

void main() { }


This will cause an object access violation in runtime.

On Sun, Aug 8, 2010 at 8:09 PM, Andrej Mitrovic
<andrej.mitrovich gmail.com>wrote:


 On Page 187 it states (shortened here for convenience):
 "By calling clear(b), you invoke b's destructor, obliterate the object's
 state with Buffer.init, and call Buffer's default constructor"

 On the next page there's a unittest, and the entire example is this:

 import core.stdc.stdlib;

 class Buffer
 {
    private void* data;

    // constructor
    this()
    {
        data = malloc(1024);
    }

    // destructor
    ~this()
    {
        free(data);
    }
 }

 unittest
 {
    auto b = new Buffer;
    auto b1 = b;    // extra alias for b
    clear(b);
    assert(b1.data is null);
 }

 void main() { }

 The assert fails, regardless if there is another reference to the object or
 not (b1 in this case).

 I've added some writeln()'s (not shown here), and just like the book said,
 after clear(b), b's destructor gets called, the fields get initialized to
 .init, and then b's constructor gets called. But the constructor will
 allocate memory for b1.data again, which means data is not null anymore.

 So I'm guessing the assert code is wrong in the example?
Aug 08 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 08/08/2010 01:09 PM, Andrej Mitrovic wrote:

 On Page 187 it states (shortened here for convenience):
 "By calling clear(b), you invoke b's destructor, obliterate the object's state
with Buffer.init, and call Buffer's default constructor"

 On the next page there's a unittest, and the entire example is this:

 import core.stdc.stdlib;

 class Buffer
 {
      private void* data;

      // constructor
      this()
      {
          data = malloc(1024);
      }

      // destructor
      ~this()
      {
          free(data);
      }
 }

 unittest
 {
      auto b = new Buffer;
      auto b1 = b;    // extra alias for b
      clear(b);
      assert(b1.data is null);
 }

 void main() { }

 The assert fails, regardless if there is another reference to the object or
not (b1 in this case).

 I've added some writeln()'s (not shown here), and just like the book said,
after clear(b), b's destructor gets called, the fields get initialized to
.init, and then b's constructor gets called. But the constructor will allocate
memory for b1.data again, which means data is not null anymore.

 So I'm guessing the assert code is wrong in the example?


Yes, the assert is in error.

Andrei
Aug 08 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 08 Aug 2010 23:16:48 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 08/08/2010 01:09 PM, Andrej Mitrovic wrote:

 On Page 187 it states (shortened here for convenience):
 "By calling clear(b), you invoke b's destructor, obliterate the  
 object's state with Buffer.init, and call Buffer's default constructor"

 On the next page there's a unittest, and the entire example is this:

 import core.stdc.stdlib;

 class Buffer
 {
      private void* data;

      // constructor
      this()
      {
          data = malloc(1024);
      }

      // destructor
      ~this()
      {
          free(data);
      }
 }

 unittest
 {
      auto b = new Buffer;
      auto b1 = b;    // extra alias for b
      clear(b);
      assert(b1.data is null);
 }

 void main() { }

 The assert fails, regardless if there is another reference to the  
 object or not (b1 in this case).

 I've added some writeln()'s (not shown here), and just like the book  
 said, after clear(b), b's destructor gets called, the fields get  
 initialized to .init, and then b's constructor gets called. But the  
 constructor will allocate memory for b1.data again, which means data is  
 not null anymore.

 So I'm guessing the assert code is wrong in the example?


 Yes, the assert is in error.


Wait wait, when destroying an object manually, the runtime will re-call  
the constructor?  I thought it was just going to assign the TypeInfo's  
initial value to the class?

Re-calling the constructor on destruction seems like a horrible  
mis-feature.  Nobody will ever use it.

-Steve
Aug 09 2010
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
It's rather perplexing, isn't it? It states in TDPL:

"After you invoke clear, the object is still alive and well, but its
destructor has been called and the object is now carrying its
default-constructed stated. During the next garbage collection, the
destructor is called again, because the garbage collector has no idea in
what state you have left the object."

But in what situation would you want to manipulate an object that was
already cleared and ready for garbage collection?

On Mon, Aug 9, 2010 at 2:16 PM, Steven Schveighoffer <schveiguy yahoo.com>wrote:


 On Sun, 08 Aug 2010 23:16:48 -0400, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

  On 08/08/2010 01:09 PM, Andrej Mitrovic wrote:

 On Page 187 it states (shortened here for convenience):
 "By calling clear(b), you invoke b's destructor, obliterate the object's
 state with Buffer.init, and call Buffer's default constructor"

 On the next page there's a unittest, and the entire example is this:

 import core.stdc.stdlib;

 class Buffer
 {
     private void* data;

     // constructor
     this()
     {
         data = malloc(1024);
     }

     // destructor
     ~this()
     {
         free(data);
     }
 }

 unittest
 {
     auto b = new Buffer;
     auto b1 = b;    // extra alias for b
     clear(b);
     assert(b1.data is null);
 }

 void main() { }

 The assert fails, regardless if there is another reference to the object
 or not (b1 in this case).

 I've added some writeln()'s (not shown here), and just like the book
 said, after clear(b), b's destructor gets called, the fields get initialized
 to .init, and then b's constructor gets called. But the constructor will
 allocate memory for b1.data again, which means data is not null anymore.

 So I'm guessing the assert code is wrong in the example?


 Yes, the assert is in error.


 Wait wait, when destroying an object manually, the runtime will re-call the
 constructor?  I thought it was just going to assign the TypeInfo's initial
 value to the class?

 Re-calling the constructor on destruction seems like a horrible
 mis-feature.  Nobody will ever use it.

 -Steve
Aug 09 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


This seems totally wrong, what if an object has no default constructor?   
The spec used to say (maybe it still does) that a destructor is guaranteed  
to only ever be called once.

I honestly thought the point of clear was to simply leave the memory in  
place as a kind of "zombie" object until the GC could collect it, to avoid  
having the block get recycled into a new object, and then use an old  
reference to it (via delete).  I didn't know someone would ever  
purposefully use it.  What is the point of calling clear then, if clear  
doesn't get rid of the object and leave it uninitialized?


 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


None.  That's the point.  clear is saying "I don't want to use this object  
any more".  The runtime (I thought) was just being conservative and  
leaving the memory in place until it could verify there were no other  
pointers to it.

I'm starting to climb the fence into the "leave delete in the language"  
camp...

-Steve
Aug 09 2010
next sibling parent reply Max Samukha <spambox d-coding.com> writes:
On 08/09/2010 03:40 PM, Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed to only ever be called once.

 I honestly thought the point of clear was to simply leave the memory in
 place as a kind of "zombie" object until the GC could collect it, to
 avoid having the block get recycled into a new object, and then use an
 old reference to it (via delete). I didn't know someone would ever
 purposefully use it. What is the point of calling clear then, if clear
 doesn't get rid of the object and leave it uninitialized?


 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


 None. That's the point. clear is saying "I don't want to use this object
 any more". The runtime (I thought) was just being conservative and
 leaving the memory in place until it could verify there were no other
 pointers to it.

 I'm starting to climb the fence into the "leave delete in the language"
 camp...

 -Steve


I agree. The whole purpose of clearing a GC-allocated object is 
deterministically freeing the resources the object may have acquired. 
Otherwise, it could as well be left alive until the next garbage 
collection cycle. Reconstructing the object is not a good idea since the 
object may, for example, acquire an expensive resource in its 
constructor etc.


(IDisposable.Dispose) from finalizer. Let's learn something from it.
Aug 09 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Max Samukha wrote:

 On 08/09/2010 03:40 PM, Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed to only ever be called once.

 I honestly thought the point of clear was to simply leave the memory in
 place as a kind of "zombie" object until the GC could collect it, to
 avoid having the block get recycled into a new object, and then use an
 old reference to it (via delete). I didn't know someone would ever
 purposefully use it. What is the point of calling clear then, if clear
 doesn't get rid of the object and leave it uninitialized?


 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


 None. That's the point. clear is saying "I don't want to use this object
 any more". The runtime (I thought) was just being conservative and
 leaving the memory in place until it could verify there were no other
 pointers to it.

 I'm starting to climb the fence into the "leave delete in the language"
 camp...

 -Steve


 
 I agree. The whole purpose of clearing a GC-allocated object is 
 deterministically freeing the resources the object may have acquired. 
 Otherwise, it could as well be left alive until the next garbage 
 collection cycle. Reconstructing the object is not a good idea since the 
 object may, for example, acquire an expensive resource in its 
 constructor etc.
 

 (IDisposable.Dispose) from finalizer. Let's learn something from it.


I thought we did!

Andrei
Aug 09 2010
parent reply Max Samukha <spambox d-coding.com> writes:
On 08/09/2010 10:41 PM, Andrei Alexandrescu wrote:

 I thought we did!



reasonably compose objects that hold deterministically manageable 
resources. Imagine an object that owns another object that owns a resource:

class Resource
{
     private int handle;
     this() { handle = acquireResource(); }
     ~this { releaseResource(handle); }
}

class ResourceOwner
{
     private Resource resource;
     this() { resource = new Resource; }
}



class Resource : IDisposable
{
     private int handle;
     this() { handle = acquireResource(); }
     void dispose()
     {
         if (handle)
         {
             releaseResource(handle);
             handle = 0;
         }
     }

     ~this { dispose(); }
}

class ResourceOwner : IDisposable
{
     private Resource resource;
     this() { resource = new Resource; }
     void dispose()
     {
         resource.dispose();
     }
}

The above enables me to:

1. Not care about releasing the resource (some consider this a 
disadvantage):

auto owner = new ResourceOwner;

The resource and memory will be eventually freed by the GC.

2. Free the resource deterministically:

auto owner = new ResourceOwner;
owner.dispose();

3. RAII:

using (auto owner = new ResourceOwner)
{
     ...
}

The resource will be freed at the end of the "using" block.

Is it possible to realize 2-3 in D using clear()/scoped()?
Aug 10 2010
parent Lutger <lutger.blijdestijn gmail.com> writes:
Max Samukha wrote:
...

 2. Free the resource deterministically:
 
 auto owner = new ResourceOwner;
 owner.dispose();
 
 3. RAII:
 
 using (auto owner = new ResourceOwner)
 {
      ...
 }
 
 The resource will be freed at the end of the "using" block.
 
 Is it possible to realize 2-3 in D using clear()/scoped()?


I don't think it is possible, because owner.resource could be garbage collected 
before owner itself is. That would conflict with the first case. If you can 
forgot about that case it is easy to do with scope(exit) and leaving the 
destructor out, if you rewrite it as a struct even better.
Aug 10 2010
prev sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Max Samukha wrote:


 On 08/09/2010 03:40 PM, Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed to only ever be called once.

 I honestly thought the point of clear was to simply leave the memory in
 place as a kind of "zombie" object until the GC could collect it, to
 avoid having the block get recycled into a new object, and then use an
 old reference to it (via delete). I didn't know someone would ever
 purposefully use it. What is the point of calling clear then, if clear
 doesn't get rid of the object and leave it uninitialized?


 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


 None. That's the point. clear is saying "I don't want to use this object
 any more". The runtime (I thought) was just being conservative and
 leaving the memory in place until it could verify there were no other
 pointers to it.

 I'm starting to climb the fence into the "leave delete in the language"
 camp...

 -Steve


 
 I agree. The whole purpose of clearing a GC-allocated object is
 deterministically freeing the resources the object may have acquired.
 Otherwise, it could as well be left alive until the next garbage
 collection cycle. Reconstructing the object is not a good idea since the
 object may, for example, acquire an expensive resource in its
 constructor etc.
 

 (IDisposable.Dispose) from finalizer. Let's learn something from it.


IDisposable has quite a few problems and gotcha's of its own:

http://www.codeproject.com/KB/dotnet/idisposable.aspx 

Especially these:
- Finalizers are called if constructors throw an exception
- Microsoft recommends that every type implementing IDisposable will check in 
every method and property accessor to see if it has been disposed, and throw an 
exception if it has
Aug 09 2010
parent Max Samukha <spambox d-coding.com> writes:
On 08/10/2010 01:09 AM, Lutger wrote:

 http://www.codeproject.com/KB/dotnet/idisposable.aspx

 Especially these:
 - Finalizers are called if constructors throw an exception
 - Microsoft recommends that every type implementing IDisposable will check in
 every method and property accessor to see if it has been disposed, and throw an
 exception if it has


Nice reading! I was aware of some IDisposable gotchas but there are a 
few more. For example, who would know that 
GC.KeepAlive/GC.SuppressFinalize are not required in most cases?
Aug 10 2010
prev sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:
 

 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 
 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is guaranteed
 to only ever be called once.


The spec still does, it is not updated since it describes delete, not clear. 

If you omit the default constructor, no constructor will be called. Also not
for 
the base classes even if they have a default constructor. This looks like a
bug. 

Confusingly, if an object has a default constructor but is constructed from 
anything else, clear will still call the default constructor. I reckon it is 
also surprising if you later insert a previously omitted default constructor 
that the behavior can change a lot, especially when base classes are involved. 
 

 I honestly thought the point of clear was to simply leave the memory in
 place as a kind of "zombie" object until the GC could collect it, to avoid
 having the block get recycled into a new object, and then use an old
 reference to it (via delete).  I didn't know someone would ever
 purposefully use it.  What is the point of calling clear then, if clear
 doesn't get rid of the object and leave it uninitialized?
 

 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


 
 None.  That's the point.  clear is saying "I don't want to use this object
 any more".  The runtime (I thought) was just being conservative and
 leaving the memory in place until it could verify there were no other
 pointers to it.
 
 I'm starting to climb the fence into the "leave delete in the language"
 camp...
 
 -Steve
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Lutger wrote:

 Steven Schveighoffer wrote:
 

 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is guaranteed
 to only ever be called once.


 
 The spec still does, it is not updated since it describes delete, not clear. 
 
 If you omit the default constructor, no constructor will be called. Also not
for 
 the base classes even if they have a default constructor. This looks like a
bug. 


Yes, not calling the constructors of base classes is an implementation 
bug. If a class does not define any constructor at all, it has a de 
facto default constructor. If a class does define some constructor but 
not a parameterless one, it is a bug in the implementation if clear() 
compiles.


 Confusingly, if an object has a default constructor but is constructed from 
 anything else, clear will still call the default constructor.


I think that's reasonable. Otherwise the object would have to remember 
in a hidden state variable which constructor it was initialized from.


 I reckon it is 
 also surprising if you later insert a previously omitted default constructor 
 that the behavior can change a lot, especially when base classes are involved. 


That's a consequence of the implementation bugs above, I think.


Andrei
Aug 09 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea  
 in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is  
 guaranteed
 to only ever be called once.


  The spec still does, it is not updated since it describes delete, not  
 clear.  If you omit the default constructor, no constructor will be  
 called. Also not for the base classes even if they have a default  
 constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an implementation  
 bug. If a class does not define any constructor at all, it has a de  
 facto default constructor. If a class does define some constructor but  
 not a parameterless one, it is a bug in the implementation if clear()  
 compiles.


How can this be decided at compile time?

basic counter-example:

class C
{
    this(int x) {}
}

// how do you statically disable this?
void foo(Object o)
{
   clear(o);
}

void foo(C c)
{
    auto c = new C(1);
    foo(c);
}

I always thought clear would simply overwrite the object with it's default  
data as defined in the TypeInfo (i.e. before a constructor is normally  
called).  Calling the constructor is absolutely wrong.  We don't want to  
reset the object, we want to free it's resources.  Re-constructing it may  
reallocate resources *THAT WE JUST ASKED TO BE DESTROYED MANUALLY*.  To  
say clear as defined is a replacement for delete is complete fantasy.

-Steve
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no 
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is 
 guaranteed
 to only ever be called once.


  The spec still does, it is not updated since it describes delete, 
 not clear.  If you omit the default constructor, no constructor will 
 be called. Also not for the base classes even if they have a default 
 constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an implementation 
 bug. If a class does not define any constructor at all, it has a de 
 facto default constructor. If a class does define some constructor but 
 not a parameterless one, it is a bug in the implementation if clear() 
 compiles.


 
 How can this be decided at compile time?
 
 basic counter-example:
 
 class C
 {
    this(int x) {}
 }
 
 // how do you statically disable this?
 void foo(Object o)
 {
   clear(o);
 }
 
 void foo(C c)
 {
    auto c = new C(1);
    foo(c);
 }


Sorry, I got things mixed.


 I always thought clear would simply overwrite the object with it's 
 default data as defined in the TypeInfo (i.e. before a constructor is 
 normally called).  Calling the constructor is absolutely wrong.  We 
 don't want to reset the object, we want to free it's resources.  
 Re-constructing it may reallocate resources *THAT WE JUST ASKED TO BE 
 DESTROYED MANUALLY*.  To say clear as defined is a replacement for 
 delete is complete fantasy.


Well this is my view as well and the original intent of clear(). The 
problem is, if someone defined a default constructor, they presumably 
have a nontrivial invariant to satisfy. I'm unclear on what's the best 
path to take.

Andrei
Aug 09 2010
next sibling parent reply awishformore <awishformore nospam.plz> writes:
On 09/08/2010 22:31, Andrei Alexandrescu wrote:

 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default
 constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed
 to only ever be called once.


 The spec still does, it is not updated since it describes delete,
 not clear. If you omit the default constructor, no constructor will
 be called. Also not for the base classes even if they have a default
 constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an
 implementation bug. If a class does not define any constructor at
 all, it has a de facto default constructor. If a class does define
 some constructor but not a parameterless one, it is a bug in the
 implementation if clear() compiles.


 How can this be decided at compile time?

 basic counter-example:

 class C
 {
 this(int x) {}
 }

 // how do you statically disable this?
 void foo(Object o)
 {
 clear(o);
 }

 void foo(C c)
 {
 auto c = new C(1);
 foo(c);
 }


 Sorry, I got things mixed.


 I always thought clear would simply overwrite the object with it's
 default data as defined in the TypeInfo (i.e. before a constructor is
 normally called). Calling the constructor is absolutely wrong. We
 don't want to reset the object, we want to free it's resources.
 Re-constructing it may reallocate resources *THAT WE JUST ASKED TO BE
 DESTROYED MANUALLY*. To say clear as defined is a replacement for
 delete is complete fantasy.


 Well this is my view as well and the original intent of clear(). The
 problem is, if someone defined a default constructor, they presumably
 have a nontrivial invariant to satisfy. I'm unclear on what's the best
 path to take.

 Andrei


Quite frankly, I can't imagine any situation where I would ever want to 
use the clear the way you currently intend to implement it, and if 
you're unclear, you will probably agree that you don't really see a good 
way to implement it as things stand.

Rather than removing delete and implementing a completely useless clear, 
I would like to see an improved version of the GC that can correctly 
handle delete. Maybe you are approaching the issue from the wrong 
perspective.

/Max
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
awishformore wrote:

 Quite frankly, I can't imagine any situation where I would ever want to 
 use the clear the way you currently intend to implement it, and if 
 you're unclear, you will probably agree that you don't really see a good 
 way to implement it as things stand.
 
 Rather than removing delete and implementing a completely useless clear, 
 I would like to see an improved version of the GC that can correctly 
 handle delete. Maybe you are approaching the issue from the wrong 
 perspective.


If it's not easy to decide between two alternatives, choosing a third 
that's worse than either is probably not a good idea.

Regarding "correct" handling of delete by the GC - what does that mean? 
Once you define that, I'll be glad to put that behavior in clear() :o).


Andrei
Aug 09 2010
parent reply awishformore <awishformore nospam.plz> writes:
On 09/08/2010 22:52, Andrei Alexandrescu wrote:

 awishformore wrote:

 Quite frankly, I can't imagine any situation where I would ever want
 to use the clear the way you currently intend to implement it, and if
 you're unclear, you will probably agree that you don't really see a
 good way to implement it as things stand.

 Rather than removing delete and implementing a completely useless
 clear, I would like to see an improved version of the GC that can
 correctly handle delete. Maybe you are approaching the issue from the
 wrong perspective.


 If it's not easy to decide between two alternatives, choosing a third
 that's worse than either is probably not a good idea.

 Regarding "correct" handling of delete by the GC - what does that mean?
 Once you define that, I'll be glad to put that behavior in clear() :o).


 Andrei


In the sense of the word, I want "delete" to delete the object.

That means:

- all references to the object are invalid
- the memory is freed

As has been argued before, this obviously wouldn't belong in the SafeD 
subset of the language. As far the SafeD subset, there shouldn't be any 
such instrument at all.

/Max
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
awishformore wrote:

 On 09/08/2010 22:52, Andrei Alexandrescu wrote:

 awishformore wrote:

 Quite frankly, I can't imagine any situation where I would ever want
 to use the clear the way you currently intend to implement it, and if
 you're unclear, you will probably agree that you don't really see a
 good way to implement it as things stand.

 Rather than removing delete and implementing a completely useless
 clear, I would like to see an improved version of the GC that can
 correctly handle delete. Maybe you are approaching the issue from the
 wrong perspective.


 If it's not easy to decide between two alternatives, choosing a third
 that's worse than either is probably not a good idea.

 Regarding "correct" handling of delete by the GC - what does that mean?
 Once you define that, I'll be glad to put that behavior in clear() :o).


 Andrei


 
 In the sense of the word, I want "delete" to delete the object.
 
 That means:
 
 - all references to the object are invalid
 - the memory is freed
 
 As has been argued before, this obviously wouldn't belong in the SafeD 
 subset of the language. As far the SafeD subset, there shouldn't be any 
 such instrument at all.


First off, allocating a keyword for the unsafe subset of the language is 
wrong in more than one way. I think we should eliminate delete as a 
keyword as quickly as possible. It is an embarrassment.

Second, there is already a means to manually free memory in druntime's 
current GC: call GC.free() defined in memory.d. It is trivial to add a 
two-liner on top of it (call that nuke()) that invokes the destructor. 
clear() does not attempt to replace that function. This discussion is 
about clear(), so we shouldn't put it in competition with delete.

I personally do not condone adding nuke() to Phobos. It's not reasonable 
to expect that modern GCs support manual deallocation. This is a quirk 
of dmd's current GC, which I think we all agree is run-of-the-mill and 
leaves room for improvement. Adding improvements to the GC would most 
likely render GC.free() inoperant.

A much better possibility is to allow GC to focus on garbage collection 
and use manual deallocation schemes in conjunction with malloc() and free().


Andrei
Aug 09 2010
parent Jacob Carlborg <doob me.com> writes:
On 2010-08-09 23:14, Andrei Alexandrescu wrote:

 awishformore wrote:

 On 09/08/2010 22:52, Andrei Alexandrescu wrote:

 awishformore wrote:

 Quite frankly, I can't imagine any situation where I would ever want
 to use the clear the way you currently intend to implement it, and if
 you're unclear, you will probably agree that you don't really see a
 good way to implement it as things stand.

 Rather than removing delete and implementing a completely useless
 clear, I would like to see an improved version of the GC that can
 correctly handle delete. Maybe you are approaching the issue from the
 wrong perspective.


 If it's not easy to decide between two alternatives, choosing a third
 that's worse than either is probably not a good idea.

 Regarding "correct" handling of delete by the GC - what does that mean?
 Once you define that, I'll be glad to put that behavior in clear() :o).


 Andrei


 In the sense of the word, I want "delete" to delete the object.

 That means:

 - all references to the object are invalid
 - the memory is freed

 As has been argued before, this obviously wouldn't belong in the SafeD
 subset of the language. As far the SafeD subset, there shouldn't be
 any such instrument at all.


 First off, allocating a keyword for the unsafe subset of the language is
 wrong in more than one way. I think we should eliminate delete as a
 keyword as quickly as possible. It is an embarrassment.

 Second, there is already a means to manually free memory in druntime's
 current GC: call GC.free() defined in memory.d. It is trivial to add a
 two-liner on top of it (call that nuke()) that invokes the destructor.
 clear() does not attempt to replace that function. This discussion is
 about clear(), so we shouldn't put it in competition with delete.


The I suggest you start discussing about what should replace delete if 
clear isn't going to.


 I personally do not condone adding nuke() to Phobos. It's not reasonable
 to expect that modern GCs support manual deallocation. This is a quirk
 of dmd's current GC, which I think we all agree is run-of-the-mill and
 leaves room for improvement. Adding improvements to the GC would most
 likely render GC.free() inoperant.

 A much better possibility is to allow GC to focus on garbage collection
 and use manual deallocation schemes in conjunction with malloc() and
 free().


 Andrei



-- 
/Jacob Carlborg
Aug 10 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 16:31:39 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no  
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default  
 constructor?
 The spec used to say (maybe it still does) that a destructor is  
 guaranteed
 to only ever be called once.


  The spec still does, it is not updated since it describes delete,  
 not clear.  If you omit the default constructor, no constructor will  
 be called. Also not for the base classes even if they have a default  
 constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an implementation  
 bug. If a class does not define any constructor at all, it has a de  
 facto default constructor. If a class does define some constructor but  
 not a parameterless one, it is a bug in the implementation if clear()  
 compiles.


  How can this be decided at compile time?
  basic counter-example:
  class C
 {
    this(int x) {}
 }
  // how do you statically disable this?
 void foo(Object o)
 {
   clear(o);
 }
  void foo(C c)
 {
    auto c = new C(1);
    foo(c);
 }


 Sorry, I got things mixed.


 I always thought clear would simply overwrite the object with it's  
 default data as defined in the TypeInfo (i.e. before a constructor is  
 normally called).  Calling the constructor is absolutely wrong.  We  
 don't want to reset the object, we want to free it's resources.   
 Re-constructing it may reallocate resources *THAT WE JUST ASKED TO BE  
 DESTROYED MANUALLY*.  To say clear as defined is a replacement for  
 delete is complete fantasy.


 Well this is my view as well and the original intent of clear(). The  
 problem is, if someone defined a default constructor, they presumably  
 have a nontrivial invariant to satisfy. I'm unclear on what's the best  
 path to take.


Who cares about invariants when an object is destructed?  First,  
invariants are disabled in release mode.  Second, I find it rare that a  
class has an invariant.  Why take actions in release mode and that can run  
counter to what the programmer has requested (destruction of on object,  
not construction) to satisfy invariants that 99% of the time do not exist?

But we have to figure out how to make it so people can put in their  
invariants as desired, and also be able to use clear as desired.

A couple options I see:

1. object invariants only run when the destructor has not yet been  
called.  We probably can use some hidden bits to flag this.
2. make it known that defining an invariant on an object should pass  
before the constructor is called.

My preference is for 1, esp. since invariants are a huge drag on  
performance anyways, adding a little bit more isn't going to hurt.

I also strongly suggest the destructor only be called once.  Having a  
destructor called more than once means your destructor has to take into  
account that the object may be already destructed.  This would be helped  
by the same mechanism that would make sure invariants aren't run after  
destruction.

-Steve
Aug 09 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 16:31:39 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no 
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default 
 constructor?
 The spec used to say (maybe it still does) that a destructor is 
 guaranteed
 to only ever be called once.


  The spec still does, it is not updated since it describes delete, 
 not clear.  If you omit the default constructor, no constructor 
 will be called. Also not for the base classes even if they have a 
 default constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an 
 implementation bug. If a class does not define any constructor at 
 all, it has a de facto default constructor. If a class does define 
 some constructor but not a parameterless one, it is a bug in the 
 implementation if clear() compiles.


  How can this be decided at compile time?
  basic counter-example:
  class C
 {
    this(int x) {}
 }
  // how do you statically disable this?
 void foo(Object o)
 {
   clear(o);
 }
  void foo(C c)
 {
    auto c = new C(1);
    foo(c);
 }


 Sorry, I got things mixed.


 I always thought clear would simply overwrite the object with it's 
 default data as defined in the TypeInfo (i.e. before a constructor is 
 normally called).  Calling the constructor is absolutely wrong.  We 
 don't want to reset the object, we want to free it's resources.  
 Re-constructing it may reallocate resources *THAT WE JUST ASKED TO BE 
 DESTROYED MANUALLY*.  To say clear as defined is a replacement for 
 delete is complete fantasy.


 Well this is my view as well and the original intent of clear(). The 
 problem is, if someone defined a default constructor, they presumably 
 have a nontrivial invariant to satisfy. I'm unclear on what's the best 
 path to take.


 
 Who cares about invariants when an object is destructed?


The destructor itself.


 First, 
 invariants are disabled in release mode.


I was refering to invariant in-the-large, not D's invariant keyword and 
associated notion.


 I also strongly suggest the destructor only be called once.  Having a 
 destructor called more than once means your destructor has to take into 
 account that the object may be already destructed.  This would be helped 
 by the same mechanism that would make sure invariants aren't run after 
 destruction.


If some constructor has been called in between two calls to the 
destructor, there's shouldn't be any danger.


Andrei
Aug 09 2010
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Andrei Alexandrescu wrote:


 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 16:31:39 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 15:46:27 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default
 constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed
 to only ever be called once.


  The spec still does, it is not updated since it describes delete,
 not clear.  If you omit the default constructor, no constructor
 will be called. Also not for the base classes even if they have a
 default constructor. This looks like a bug.


 Yes, not calling the constructors of base classes is an
 implementation bug. If a class does not define any constructor at
 all, it has a de facto default constructor. If a class does define
 some constructor but not a parameterless one, it is a bug in the
 implementation if clear() compiles.


  How can this be decided at compile time?
  basic counter-example:
  class C
 {
    this(int x) {}
 }
  // how do you statically disable this?
 void foo(Object o)
 {
   clear(o);
 }
  void foo(C c)
 {
    auto c = new C(1);
    foo(c);
 }


 Sorry, I got things mixed.


 I always thought clear would simply overwrite the object with it's
 default data as defined in the TypeInfo (i.e. before a constructor is
 normally called).  Calling the constructor is absolutely wrong.  We
 don't want to reset the object, we want to free it's resources.
 Re-constructing it may reallocate resources *THAT WE JUST ASKED TO BE
 DESTROYED MANUALLY*.  To say clear as defined is a replacement for
 delete is complete fantasy.


 Well this is my view as well and the original intent of clear(). The
 problem is, if someone defined a default constructor, they presumably
 have a nontrivial invariant to satisfy. I'm unclear on what's the best
 path to take.


 
 Who cares about invariants when an object is destructed?


 
 The destructor itself.
 

 First,
 invariants are disabled in release mode.


 
 I was refering to invariant in-the-large, not D's invariant keyword and
 associated notion.
 

 I also strongly suggest the destructor only be called once.  Having a
 destructor called more than once means your destructor has to take into
 account that the object may be already destructed.  This would be helped
 by the same mechanism that would make sure invariants aren't run after
 destruction.


 
 If some constructor has been called in between two calls to the
 destructor, there's shouldn't be any danger.
 
 
 Andrei


Are there any problems with said 'some constructor' being a user-defined 
function, either through convention or interface? If a class needs to define a 
destructor, *and* requires a default constructor, I don't think it is too much 
to ask for a third 'reinitialize' function (or some other solution.)

Take a look at what IDisposable requires for comparison:

http://msdn.microsoft.com/en-us/library/system.idisposable.aspx
Aug 09 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 17:17:13 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 16:31:39 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 Well this is my view as well and the original intent of clear(). The  
 problem is, if someone defined a default constructor, they presumably  
 have a nontrivial invariant to satisfy. I'm unclear on what's the best  
 path to take.


  Who cares about invariants when an object is destructed?


 The destructor itself.


 First, invariants are disabled in release mode.


 I was refering to invariant in-the-large, not D's invariant keyword and  
 associated notion.


Care to post an example?  I thought you mean D invariants.  Now, I don't  
know what you are talking about.


 I also strongly suggest the destructor only be called once.  Having a  
 destructor called more than once means your destructor has to take into  
 account that the object may be already destructed.  This would be  
 helped by the same mechanism that would make sure invariants aren't run  
 after destruction.


 If some constructor has been called in between two calls to the  
 destructor, there's shouldn't be any danger.


No, but let's work past that.  Here are two hard requirements for clear:

1. it calls the target's destructor
2. it does not call a target's constructor.

If you can't do 2, nobody will use it.  I can just as well define a  
reset() function on the object to do 1 without 2 (and probably do it more  
efficiently).  I don't need any special library help for that.  The user  
is expecting some magic stuff to happen in clear, akin to freeing the  
object.  They are not expecting to reconstruct the object.

Here is a class which should be destructable.

class C
{
    private static C[int] instances;
    private static nextid = 0;
    private int id;
    this()
    {
       id = nextid++;
       instances[id] = this;
    }

    ~this()
    {
       instances.remove(id);
    }
}

Can we make sure instances of this object will be destroyed?  Because if  
you keep calling the constructor, it never dies.

-Steve
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 17:17:13 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:
 

 Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 16:31:39 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 Well this is my view as well and the original intent of clear(). The 
 problem is, if someone defined a default constructor, they 
 presumably have a nontrivial invariant to satisfy. I'm unclear on 
 what's the best path to take.


  Who cares about invariants when an object is destructed?


 The destructor itself.


 First, invariants are disabled in release mode.


 I was refering to invariant in-the-large, not D's invariant keyword 
 and associated notion.


 
 Care to post an example?  I thought you mean D invariants.  Now, I don't 
 know what you are talking about.


class File {
     FILE * fp;
     this() { fp = fopen("/tmp/temporary"); }
     ~this() { fclose(fp); }
}

The destructor does not test fp because it assumes it was opened.

Interestingly enough, by TDPL the code above is in fact invalid already. 
TDPL mentions that an object's lifetime starts as soon as it's been 
branded, which is before default construction. As a direct consequence, 
the destructor should be able to deal with an object stamped with all 
init values for all fields - in this case a null fp.


 I also strongly suggest the destructor only be called once.  Having a 
 destructor called more than once means your destructor has to take 
 into account that the object may be already destructed.  This would 
 be helped by the same mechanism that would make sure invariants 
 aren't run after destruction.


 If some constructor has been called in between two calls to the 
 destructor, there's shouldn't be any danger.


 
 No, but let's work past that.  Here are two hard requirements for clear:
 
 1. it calls the target's destructor
 2. it does not call a target's constructor.
 
 If you can't do 2, nobody will use it.  I can just as well define a 
 reset() function on the object to do 1 without 2 (and probably do it 
 more efficiently).  I don't need any special library help for that.  The 
 user is expecting some magic stuff to happen in clear, akin to freeing 
 the object.  They are not expecting to reconstruct the object.
 
 Here is a class which should be destructable.
 
 class C
 {
    private static C[int] instances;
    private static nextid = 0;
    private int id;
    this()
    {
       id = nextid++;
       instances[id] = this;
    }
 
    ~this()
    {
       instances.remove(id);
    }
 }
 
 Can we make sure instances of this object will be destroyed?  Because if 
 you keep calling the constructor, it never dies.


In wake of the above, it looks like that's possible. The oddity will be 
that occasionally the id==0 will be removed more than one time.


Andrei
Aug 09 2010
next sibling parent reply Jonathan M Davis <jmdavisprog gmail.com> writes:
On Monday, August 09, 2010 16:40:23 Andrei Alexandrescu wrote:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }
 
 The destructor does not test fp because it assumes it was opened.
 
 Interestingly enough, by TDPL the code above is in fact invalid already.
 TDPL mentions that an object's lifetime starts as soon as it's been
 branded, which is before default construction. As a direct consequence,
 the destructor should be able to deal with an object stamped with all
 init values for all fields - in this case a null fp.


I would take that as an argument for making clear() set the object in the state 
prior to the constructor call. That state is supposed to be completely valid 
from a memory standpoint. True, it goes against class invariants, but it would 
make clear() function in a manner closer to the delete that some folks are 
looking for. Truth be told, I'd sooner argue for removing clear() entirely, 
forcing people to either call a specific method on the class in question to
free 
up whatever resources that they're so interested in freeing up, or forcing them 
to just give up on trying to free them and let the garbage collector do its 
thing. But if there's enough demand for a function which destroys an object and 
puts it in some sort of state which won't have undefined behavior, then making 
clear() put the class in a pre-constructor state seems the best to me. It's at 
least closer to what the folks who want delete are looking for. Of course, they 
probably won't be entirely happy until they have the nuke() function that you 
mentioned earlier, but at some point you have to face the fact that you're 
dealing with a garbage collected langugage and let it do its thing unless you 
really need to manually manage memory (at which point you should manually
manage 
memory rather than try to contort the garbage collector).

- Jonathan M Davis
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jonathan M Davis wrote:

 On Monday, August 09, 2010 16:40:23 Andrei Alexandrescu wrote:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }

 The destructor does not test fp because it assumes it was opened.

 Interestingly enough, by TDPL the code above is in fact invalid already.
 TDPL mentions that an object's lifetime starts as soon as it's been
 branded, which is before default construction. As a direct consequence,
 the destructor should be able to deal with an object stamped with all
 init values for all fields - in this case a null fp.


 
 I would take that as an argument for making clear() set the object in the
state 
 prior to the constructor call.


[snip]

I agree. Do others?

Andrei
Aug 09 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-09 20:53:16 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Jonathan M Davis wrote:

 I would take that as an argument for making clear() set the object in 
 the state prior to the constructor call.


 [snip]
 
 I agree. Do others?


Well, allowing clear on any class is dangerous for a program's 
integrity as a whole, whether you call a destructor or not.

So my opinion is that it should either not exist, if it exists it 
should be buried somewhere with other functions that subvert the type 
system, and if it's not buried enough there should be a way to disable 
it for certain classes. That last option is quite like disabling the 
copy constructor in C++; I'd much prefer an opt-in than an opt-out 
option.

Beyond that I could not care less whether it does one or the other. In 
fact, why not create two variants of the function? There are valid use 
cases for both behaviours in my opinion.

	destroy(); // call destructor and wipe memory to init state
	reconstruct(); // call destroy() then call constructor

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:

 On 2010-08-09 20:53:16 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 

 Jonathan M Davis wrote:

 I would take that as an argument for making clear() set the object in 
 the state prior to the constructor call.


 [snip]

 I agree. Do others?


 
 Well, allowing clear on any class is dangerous for a program's integrity 
 as a whole, whether you call a destructor or not.
 
 So my opinion is that it should either not exist, if it exists it should 
 be buried somewhere with other functions that subvert the type system,


clear() would not subvert the type system.

Andrei
Aug 09 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-09 21:20:31 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 clear() would not subvert the type system.


Oh sure it does!

1. You acknowledged yourself one of my argument a few days ago that if 
the class has an immutable member it'll be wiped out. Someone else 
somewhere could have a reference to that member, and it's value will 
change which could cause bad things...

2. If someone somewhere has a reference to an object, clearing that 
object is basically the same as replacing that reference with a 
reference to a new, uninitialized object. This could break invariants 
of any code still using that reference. In normal times, those 
invariants could be protected with judicious usage of 'private', 
'protected' or 'package' in the object (or something more 
sophisticated), but all that gets subverted by clear().

So clear() can have long-reaching effects (break program invariants) if 
used at the wrong place, and it's difficult to protect against people 
using it at the wrong place.

I know for one thing that clearing any object used by the D/Objective-C 
bridge is a potential crasher (if you call a function on a cleared 
wrapper). I would suspect the same for QtD.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 09 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:

 On 2010-08-09 21:20:31 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 

 clear() would not subvert the type system.


 
 Oh sure it does!
 
 1. You acknowledged yourself one of my argument a few days ago that if 
 the class has an immutable member it'll be wiped out. Someone else 
 somewhere could have a reference to that member, and it's value will 
 change which could cause bad things...


I forgot that argument again :o). I acknowledge being destroyed (which 
is strictly worse than cleared...)


Andrei
Aug 09 2010
parent Max Samukha <spambox d-coding.com> writes:
On 08/10/2010 06:22 AM, Andrei Alexandrescu wrote:

 Michel Fortin wrote:

 On 2010-08-09 21:20:31 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:


 clear() would not subvert the type system.


 Oh sure it does!

 1. You acknowledged yourself one of my argument a few days ago that if
 the class has an immutable member it'll be wiped out. Someone else
 somewhere could have a reference to that member, and it's value will
 change which could cause bad things...


 I forgot that argument again :o). I acknowledge being destroyed (which
 is strictly worse than cleared...)


Is that argument valid? An immutable non-static member is still part of 
the object state. You shouldn't expect references to the object state 
remain valid after the object has been destroyed.
Aug 10 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 21:52:10 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2010-08-09 21:20:31 -0400, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> said:


 clear() would not subvert the type system.


 Oh sure it does!

 1. You acknowledged yourself one of my argument a few days ago that if  
 the class has an immutable member it'll be wiped out. Someone else  
 somewhere could have a reference to that member, and it's value will  
 change which could cause bad things..


This is a good point.  But here is the thing -- when you are calling  
clear, you are saying "Nothing in this program is using this data any  
more".  The point of not actually deleting the memory is to 1) not throw a  
monkey wrench into the natural rhythm of the GC, and 2) to ensure any  
dangling pointers do not corrupt memory.  Essentially, if you clear an  
object, and then use that object, the behavior should be undefined.

Note that delete has the same problem, but by the time you reference that  
dangling pointer, it may have been reallocated, so you might be corrupting  
memory in use.

I think it's fine to have clear break immutability, since to call clear,  
and then to access the data should be undefined.


 2. If someone somewhere has a reference to an object, clearing that  
 object is basically the same as replacing that reference with a  
 reference to a new, uninitialized object. This could break invariants of  
 any code still using that reference.


As any undefined behavior could.  If you don't want to cause this problem  
then either ensure your not using it, or don't use clear ;)


 In normal times, those invariants could be protected with judicious  
 usage of 'private', 'protected' or 'package' in the object (or something  
 more sophisticated), but all that gets subverted by clear().


clear is a replacement for delete.  Would you expect delete to keep  
invariants of dangling pointers alive?


 So clear() can have long-reaching effects (break program invariants) if  
 used at the wrong place, and it's difficult to protect against people  
 using it at the wrong place.

 I know for one thing that clearing any object used by the D/Objective-C  
 bridge is a potential crasher (if you call a function on a cleared  
 wrapper). I would suspect the same for QtD.


Undefined, undefined, undefined :)

-Steve
Aug 10 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 Undefined, undefined, undefined :)


So we agree on that. That's exactly what I was trying to prove to 
Andrei. Using clear() can break program invariants, break the type 
system (immutable members) and so on, even though I admit it can be 
useful at times.

**** So why give it a so innocuous-looking name such as "clear" !! ****

Call it destroy() at the very very least, and put it in a module where 
no one will find it by accident and think it is safe for general use. 
For instance, it could be made part of the GC API, allowing the GC to 
adjust the memory block to avoid calling the destructor a second time 
upon collection. GC.destroy() sounds like a good name for that kind 
functionality.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 10:11:21 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 Undefined, undefined, undefined :)


 So we agree on that. That's exactly what I was trying to prove to  
 Andrei. Using clear() can break program invariants, break the type  
 system (immutable members) and so on, even though I admit it can be  
 useful at times.

 **** So why give it a so innocuous-looking name such as "clear" !! ****


I think that book has shipped.


 Call it destroy() at the very very least, and put it in a module where  
 no one will find it by accident and think it is safe for general use.  
 For instance, it could be made part of the GC API, allowing the GC to  
 adjust the memory block to avoid calling the destructor a second time  
 upon collection. GC.destroy() sounds like a good name for that kind  
 functionality.


delete is currently available whenever you want it and is twice as  
dangerous.  It might as well be in object.di.  I think that's where clear  
belongs too.

Hiding it from users is not the right thing to do.  Once you find  
something, you aren't going to forget where it is.  Maybe we can move it  
to a random module on each release ;)

Placing danger warnings around it on all sides, with a legal disclaimer  
you must sign in order to use it, and make IDEs highlight it in red is  
probably all that is necessary.  Disallowing it in safeD might be a better  
deterrent than naming it a certain way or putting it in an obscure module.

-Steve
Aug 10 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 10:19:25 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Tue, 10 Aug 2010 10:11:21 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:
 

 Undefined, undefined, undefined :)


 
 So we agree on that. That's exactly what I was trying to prove to  
 Andrei. Using clear() can break program invariants, break the type  
 system (immutable members) and so on, even though I admit it can be  
 useful at times.
 
 **** So why give it a so innocuous-looking name such as "clear" !! ****


 
 I think that book has shipped.


That's not really an answer to the question. The answer I expected was 
more that it seemed innocuous at the time, even though now it appears 
more harmful. To me it's the C++ copy constructor all over again...

Can we really not fix it before every one start using it? In other 
words, which is worse: having something in the book deprecated just a 
few months after publication? or having hundreds of programers using 
clear() thinking it is innocuous?

At the very least I'd like to have a way to disable it for certain 
classes (by throwing an exception when you try).

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 10:48:06 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2010-08-10 10:19:25 -0400, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 On Tue, 10 Aug 2010 10:11:21 -0400, Michel Fortin   
 <michel.fortin michelf.com> wrote:


 On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer"   
 <schveiguy yahoo.com> said:


 Undefined, undefined, undefined :)


  So we agree on that. That's exactly what I was trying to prove to   
 Andrei. Using clear() can break program invariants, break the type   
 system (immutable members) and so on, even though I admit it can be   
 useful at times.
  **** So why give it a so innocuous-looking name such as "clear" !!  
 ****


  I think that book has shipped.


 That's not really an answer to the question. The answer I expected was  
 more that it seemed innocuous at the time, even though now it appears  
 more harmful. To me it's the C++ copy constructor all over again...

 Can we really not fix it before every one start using it? In other  
 words, which is worse: having something in the book deprecated just a  
 few months after publication? or having hundreds of programers using  
 clear() thinking it is innocuous?


I guess I don't agree that it's badly named, or I don't really care what  
it's named.  Clear sounds fine to me.  I use clear to clear out the data  
in a collection, seems about the same.


 At the very least I'd like to have a way to disable it for certain  
 classes (by throwing an exception when you try).


Hm... do you have a good use case?

A hook to indicate "hey object, clear is being called, not a GC collection  
cycle" may be useful for other purposes as well.

-Steve
Aug 10 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 11:12:32 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Tue, 10 Aug 2010 10:48:06 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 On 2010-08-10 10:19:25 -0400, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:
 

 On Tue, 10 Aug 2010 10:11:21 -0400, Michel Fortin   
 <michel.fortin michelf.com> wrote:
 

 On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer"   
 <schveiguy yahoo.com> said:
 

 Undefined, undefined, undefined :)


  So we agree on that. That's exactly what I was trying to prove to   
 Andrei. Using clear() can break program invariants, break the type   
 system (immutable members) and so on, even though I admit it can be   
 useful at times.
  **** So why give it a so innocuous-looking name such as "clear" !!  ****


  I think that book has shipped.


 
 That's not really an answer to the question. The answer I expected was  
 more that it seemed innocuous at the time, even though now it appears  
 more harmful. To me it's the C++ copy constructor all over again...
 
 Can we really not fix it before every one start using it? In other  
 words, which is worse: having something in the book deprecated just a  
 few months after publication? or having hundreds of programers using  
 clear() thinking it is innocuous?


 
 I guess I don't agree that it's badly named, or I don't really care 
 what  it's named.  Clear sounds fine to me.  I use clear to clear out 
 the data  in a collection, seems about the same.


But is using the collection after calling clear() undefined behaviour 
or not? Please make up your mind.

Seriously, if you're using "clear" to mean "empty that collection" at 
some place and using "clear" to mean "wipe this object's data, I assert 
no one will use it anymore" at others, then you've conflated two 
totally different concepts. The first one is something pretty safe to 
do, the later requires a lot more care, especially since it can break 
the type system and bypasses protection attributes (immutable and 
private members are wiped out too).



 At the very least I'd like to have a way to disable it for certain  
 classes (by throwing an exception when you try).


 
 Hm... do you have a good use case?


Catching bugs early. Calling clear() on any object you share through 
the D/Objective-C bridge will most likely result in a crash later if 
the Objective-C side still holds a reference to it somewhere, or if you 
try to use the object again. The bridge has some expectations about the 
lifetime of the objects it manages, you shouldn't be allowed to break 
those with an innocuous-looking function.

I would assume the same applies to QtD. In fact, any program that wants 
to protect invariants that go beyond the scope of a single object might 
want to disable clear().

So not all objects should be clearable.



 A hook to indicate "hey object, clear is being called, not a GC 
 collection  cycle" may be useful for other purposes as well.


Indeed.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 11:44:06 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2010-08-10 11:12:32 -0400, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 On Tue, 10 Aug 2010 10:48:06 -0400, Michel Fortin   
 <michel.fortin michelf.com> wrote:


 On 2010-08-10 10:19:25 -0400, "Steven Schveighoffer"   
 <schveiguy yahoo.com> said:


 On Tue, 10 Aug 2010 10:11:21 -0400, Michel Fortin    
 <michel.fortin michelf.com> wrote:


 On 2010-08-10 08:11:21 -0400, "Steven Schveighoffer"    
 <schveiguy yahoo.com> said:


 Undefined, undefined, undefined :)


  So we agree on that. That's exactly what I was trying to prove to    
 Andrei. Using clear() can break program invariants, break the type    
 system (immutable members) and so on, even though I admit it can  
 be   useful at times.
  **** So why give it a so innocuous-looking name such as "clear" !!   
 ****


  I think that book has shipped.


  That's not really an answer to the question. The answer I expected  
 was  more that it seemed innocuous at the time, even though now it  
 appears  more harmful. To me it's the C++ copy constructor all over  
 again...
  Can we really not fix it before every one start using it? In other   
 words, which is worse: having something in the book deprecated just a   
 few months after publication? or having hundreds of programers using   
 clear() thinking it is innocuous?


  I guess I don't agree that it's badly named, or I don't really care  
 what  it's named.  Clear sounds fine to me.  I use clear to clear out  
 the data  in a collection, seems about the same.


 But is using the collection after calling clear() undefined behaviour or  
 not? Please make up your mind.


It's not the same function, just the same name.  Though I can see how it  
might be confusing:

container.clear(); // ok to reuse container
clear(container); // not ok to reuse.

But hopefully syntax highlighting can help make this distinction.  It was  
kind of nice that delete was a keyword, at least you couldn't confuse it  
with anything else.


 Seriously, if you're using "clear" to mean "empty that collection" at  
 some place and using "clear" to mean "wipe this object's data, I assert  
 no one will use it anymore" at others, then you've conflated two totally  
 different concepts. The first one is something pretty safe to do, the  
 later requires a lot more care, especially since it can break the type  
 system and bypasses protection attributes (immutable and private members  
 are wiped out too).


Well, we have what we have.  I don't think clear is such a bad name for  
either.  The time to lobby for a different name is probably over, but  
that's up to Walter/Andrei.  I don't really feel as strongly about the  
name as you do.


 At the very least I'd like to have a way to disable it for certain   
 classes (by throwing an exception when you try).


  Hm... do you have a good use case?


 Catching bugs early. Calling clear() on any object you share through the  
 D/Objective-C bridge will most likely result in a crash later if the  
 Objective-C side still holds a reference to it somewhere, or if you try  
 to use the object again. The bridge has some expectations about the  
 lifetime of the objects it manages, you shouldn't be allowed to break  
 those with an innocuous-looking function.


Again, clear doesn't scream out "use me on everything!" It's a dangerous  
function and should be treated as such.  This seems solvable via  
documentation.  I'd rather have a compile-time solution than a runtime  
solution if at all.  Are Objective-C bindings structs or classes?  If they  
are structs, we may be able to have clear obey some sort of enum, although  
that seems like an ugly solution.

Anyone who's learned C and C++ knows that you use the destruction method  
that matches with the construction method.  This is kind of the same  
thing, you wouldn't expect someone to allocate an Objective-C resource via  
some non-D-standard function, and then use a D-standard function to  
deallocate.  I think the programmer has a responsibility to obey the rules  
of allocation/deallocation.


 I would assume the same applies to QtD. In fact, any program that wants  
 to protect invariants that go beyond the scope of a single object might  
 want to disable clear().

 So not all objects should be clearable.


Sure, but the question is, do we care?  Is it enough to just say "Don't  
clear these objects!" in the documentation?

-Steve
Aug 10 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 12:35:10 -0400, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 It's not the same function, just the same name.  Though I can see how 
 it  might be confusing:
 
 container.clear(); // ok to reuse container
 clear(container); // not ok to reuse.
 
 But hopefully syntax highlighting can help make this distinction.  It 
 was  kind of nice that delete was a keyword, at least you couldn't 
 confuse it  with anything else.


Syntax highlighting... You want an IDE flagging dangerous functions in 
red? Not a bad idea actually.



 Again, clear doesn't scream out "use me on everything!" It's a 
 dangerous  function and should be treated as such.  This seems solvable 
 via  documentation.


In my world, we add safeties to make dangerous things less dangerous. A 
typical microwave oven will refuse to start if you don't close the 
door. The documentation is there to tell you what not to do, but we 
don't rely solely on documentation.

People will always do stupid things, sometime by accident. It's 
especially easy to make the accident of writing clear(a) instead of 
a.clear() in my opinion. Think of this scenario:

	Employee:     Hey boss, how do I empty a container?
	Boss (busy):  clear()
	Employee:     Thanks!



 I'd rather have a compile-time solution than a runtime  solution if at all.


Me too.

And the solution is so simple: clearable classes and structs implement 
the clear() function. Non-clearable classes and structs do not. The 
only problem is that it goes against Andrei's book.



 Are Objective-C bindings structs or classes?  If they  are structs, we 
 may be able to have clear obey some sort of enum, although  that seems 
 like an ugly solution.


They're classes.



 Anyone who's learned C and C++ knows that you use the destruction 
 method  that matches with the construction method.  This is kind of the 
 same  thing, you wouldn't expect someone to allocate an Objective-C 
 resource via  some non-D-standard function, and then use a D-standard 
 function to  deallocate.  I think the programmer has a responsibility 
 to obey the rules  of allocation/deallocation.


You're misunderstanding the purpose of the bridge, which is to make 
things transparent for the user.

The D/Objective-C bridge in its current incarnation allow you to pass D 
objects on the Objective-C side as if they were Objective-C objects, 
and to treat Objective-C objects in D code as if they were D objects. 
The wrapping/unwrapping is done on the fly, making this mostly 
transparent to the user.

It's done that way so you can use the Cocoa framework, which wouldn't 
be very usable without an easy way to subclass Objective-C classes and 
move object instances around.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
prev sibling next sibling parent F. Almeida <francisco.m.almeida gmail.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s
article

 Jonathan M Davis wrote:

 On Monday, August 09, 2010 16:40:23 Andrei Alexandrescu wrote:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }

 The destructor does not test fp because it assumes it was






opened.

 Interestingly enough, by TDPL the code above is in fact invalid






already.

 TDPL mentions that an object's lifetime starts as soon as it's






been

 branded, which is before default construction. As a direct






consequence,

 the destructor should be able to deal with an object stamped






with all

 init values for all fields - in this case a null fp.


 I would take that as an argument for making clear() set the




object in the state

 prior to the constructor call.


 [snip]
 I agree. Do others?
 Andrei


A clear() that leaves the object in the initial (not constructed)
state is the most sensible choice.
This is good for two reasons: 1. it avoids useless resources
reallocation and 2. it leaves the destroyed object in a (somewhat)
safe and identifiable state.
One could easily make his own reset() function that calls clear()
and then the constructor, if needed.
Aug 09 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 20:53:16 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Jonathan M Davis wrote:

 On Monday, August 09, 2010 16:40:23 Andrei Alexandrescu wrote:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }

 The destructor does not test fp because it assumes it was opened.

 Interestingly enough, by TDPL the code above is in fact invalid  
 already.
 TDPL mentions that an object's lifetime starts as soon as it's been
 branded, which is before default construction. As a direct consequence,
 the destructor should be able to deal with an object stamped with all
 init values for all fields - in this case a null fp.


  I would take that as an argument for making clear() set the object in  
 the state prior to the constructor call.


 [snip]

 I agree. Do others?


Yes, this is exactly what I was saying.

-Steve
Aug 10 2010
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I don't have a lot of experience in this field, but my guess is people want
to help out the GC be more efficient in some regard.

"Hey, GC! I know you take the trash out every day, but I just want to let
you know that I've cleaned up my room and there's a whole bag o' dirt in
here ready to be thrown out that you should know about. See ya later!"

My assumption is that the programmer probably knows when and how he uses
memory more so than the GC itself. So my thinking is some kind of hybrid
approach would be a good choice between safety and performance here. But
correct me if I'm wrong, I'm walking in the dark really..

On Tue, Aug 10, 2010 at 1:57 AM, Jonathan M Davis <jmdavisprog gmail.com>wrote:


 On Monday, August 09, 2010 16:40:23 Andrei Alexandrescu wrote:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }

 The destructor does not test fp because it assumes it was opened.

 Interestingly enough, by TDPL the code above is in fact invalid already.
 TDPL mentions that an object's lifetime starts as soon as it's been
 branded, which is before default construction. As a direct consequence,
 the destructor should be able to deal with an object stamped with all
 init values for all fields - in this case a null fp.


 I would take that as an argument for making clear() set the object in the
 state
 prior to the constructor call. That state is supposed to be completely
 valid
 from a memory standpoint. True, it goes against class invariants, but it
 would
 make clear() function in a manner closer to the delete that some folks are
 looking for. Truth be told, I'd sooner argue for removing clear() entirely,
 forcing people to either call a specific method on the class in question to
 free
 up whatever resources that they're so interested in freeing up, or forcing
 them
 to just give up on trying to free them and let the garbage collector do its
 thing. But if there's enough demand for a function which destroys an object
 and
 puts it in some sort of state which won't have undefined behavior, then
 making
 clear() put the class in a pre-constructor state seems the best to me. It's
 at
 least closer to what the folks who want delete are looking for. Of course,
 they
 probably won't be entirely happy until they have the nuke() function that
 you
 mentioned earlier, but at some point you have to face the fact that you're
 dealing with a garbage collected langugage and let it do its thing unless
 you
 really need to manually manage memory (at which point you should manually
 manage
 memory rather than try to contort the garbage collector).

 - Jonathan M Davis
Aug 09 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 20:07:28 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 I don't have a lot of experience in this field, but my guess is people  
 want
 to help out the GC be more efficient in some regard.

 "Hey, GC! I know you take the trash out every day, but I just want to let
 you know that I've cleaned up my room and there's a whole bag o' dirt in
 here ready to be thrown out that you should know about. See ya later!"

 My assumption is that the programmer probably knows when and how he uses
 memory more so than the GC itself. So my thinking is some kind of hybrid
 approach would be a good choice between safety and performance here. But
 correct me if I'm wrong, I'm walking in the dark really..


That is delete.  Here is the problem:

class C {
int x;
}

void main()
{
   auto c = new C;
   c.x = 1;
   auto c2 = c;
   delete c;
   assert(c is null);
   assert(c2 !is null);
   c2.x = 5; // oops!  I just wrote to unallocated memory.
   auto c3 = new C; // might occupy space just freed
   c2.x = 5; // double oops! I may have just overwritten c3
}

compare this to clear.  If clear leaves the memory around until there are  
no references, then referring to that memory does not corrupt other  
unrelated parts of the code.  Memory corruption is the granddaddy of all  
horrible bugs, because the cause is usually a) long gone by the time you  
see any symptoms, b) can be completely decoupled from the symptom, and c)  
can cause *random* symptoms.  If D can avoid memory corruption, and it  
costs very little, it should do so.

The thing is, even with clear, the above code is undefined -- a cleared  
object is in no state to be used.  But the difference between delete and  
clear is -- delete ensures the reference you delete will cause an  
immediate error on use, but does nothing to prevent memory corruption to  
all the other references to the same memory.  Clear does the same for the  
reference you clear, but also prevents memory corruption for all the other  
references to the same memory.  So even though you are in undefined  
territory, the runtime makes a best effort to avoid you cutting off your  
head.  The error may still be silent, but it's not deadly.

It might even be a good idea to zero out the vtable to cause a loud error  
on the next dynamic cast or virtual function call :)

-Steve
Aug 10 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 The thing is, even with clear, the above code is undefined -- a cleared  
 object is in no state to be used.  But the difference between delete and  
 clear is -- delete ensures the reference you delete will cause an  
 immediate error on use, but does nothing to prevent memory corruption to  
 all the other references to the same memory.  Clear does the same for the  
 reference you clear, but also prevents memory corruption for all the other  
 references to the same memory.  So even though you are in undefined  
 territory, the runtime makes a best effort to avoid you cutting off your  
 head.  The error may still be silent, but it's not deadly.


If in nonrelease mode clear() sets a zombi-mode bit inside the object, the
language can then add asserts that stop the program if a zombi object is used
(field accessors too need such assert, it's an object invariant).

To reduce the runtime overhead of all those asserts you can do something like

objects tagged with something like  clearable, so all other objects don't need
those asserts.

This is a fully runtime situation and I don't like it a lot. But it's not so




 It might even be a good idea to zero out the vtable to cause a loud error  
 on the next dynamic cast or virtual function call :)


OK.

Bye,
bearophile
Aug 10 2010
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I see. I agree clear() is definitely safer in these situations. But if the
constructor gets called again automatically after clear(), that would mean
the other references could still use the object since it's in a valid state
again, right?

On Tue, Aug 10, 2010 at 2:25 PM, Steven Schveighoffer
<schveiguy yahoo.com>wrote:


 On Mon, 09 Aug 2010 20:07:28 -0400, Andrej Mitrovic <
 andrej.mitrovich gmail.com> wrote:

  I don't have a lot of experience in this field, but my guess is people

 want
 to help out the GC be more efficient in some regard.

 "Hey, GC! I know you take the trash out every day, but I just want to let
 you know that I've cleaned up my room and there's a whole bag o' dirt in
 here ready to be thrown out that you should know about. See ya later!"

 My assumption is that the programmer probably knows when and how he uses
 memory more so than the GC itself. So my thinking is some kind of hybrid
 approach would be a good choice between safety and performance here. But
 correct me if I'm wrong, I'm walking in the dark really..


 That is delete.  Here is the problem:

 class C {
 int x;
 }

 void main()
 {
  auto c = new C;
  c.x = 1;
  auto c2 = c;
  delete c;
  assert(c is null);
  assert(c2 !is null);
  c2.x = 5; // oops!  I just wrote to unallocated memory.
  auto c3 = new C; // might occupy space just freed
  c2.x = 5; // double oops! I may have just overwritten c3
 }

 compare this to clear.  If clear leaves the memory around until there are
 no references, then referring to that memory does not corrupt other
 unrelated parts of the code.  Memory corruption is the granddaddy of all
 horrible bugs, because the cause is usually a) long gone by the time you see
 any symptoms, b) can be completely decoupled from the symptom, and c) can
 cause *random* symptoms.  If D can avoid memory corruption, and it costs
 very little, it should do so.

 The thing is, even with clear, the above code is undefined -- a cleared
 object is in no state to be used.  But the difference between delete and
 clear is -- delete ensures the reference you delete will cause an immediate
 error on use, but does nothing to prevent memory corruption to all the other
 references to the same memory.  Clear does the same for the reference you
 clear, but also prevents memory corruption for all the other references to
 the same memory.  So even though you are in undefined territory, the runtime
 makes a best effort to avoid you cutting off your head.  The error may still
 be silent, but it's not deadly.

 It might even be a good idea to zero out the vtable to cause a loud error
 on the next dynamic cast or virtual function call :)

 -Steve
Aug 10 2010
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 09:21:30 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 I see. I agree clear() is definitely safer in these situations. But if  
 the
 constructor gets called again automatically after clear(), that would  
 mean
 the other references could still use the object since it's in a valid  
 state
 again, right?


Not really.  Who defines 'valid state'?  You might expect it to still  
contain the data you put in it through some member function.  An object  
goes through many states while alive, and arbitrarily picking one of those  
states and declaring it 'valid' is simply bad practice.  Better to pick a  
reasonable state that *all* objects can be set to, and call it invalid.

When you clear an object, you are saying "I no longer need this data from  
here or anywhere else"  Any access to that object after calling clear is a  
program error.  All clear does vs. delete is make it so an invalid access  
to that data doesn't cause your program to explode in unpredictable ways.

-Steve
Aug 10 2010
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-10 09:21:30 -0400, Andrej Mitrovic 
<andrej.mitrovich gmail.com> said:


 I see. I agree clear() is definitely safer in these situations. But if the
 constructor gets called again automatically after clear(), that would mean
 the other references could still use the object since it's in a valid state
 again, right?


The object you cleared might be in a valid state itself, but it might 
have broken a larger system relying on the state of that object. 
Normally you'd encapsulate this state with 'private' or other 
protection attributes to prevent accidental changes, but clear() 
bypasses all that.

For instance, let's say a class and a struct are used to keep track of 
the current number of 'users' for of each specific instance:

	module myobject;
	
	class MyObject {
	private: // only the struct User (in the same module) can access this
		int userCount;
		void addUser() { ++userCount; }
		void removeUser() { --userCount; }
	}

	struct User {
		MyObject object;
		this(MyObject o) { o.addUser(); object = o; }
		~this() { o.removeUser(); }
	}


	module main;
	import myobject;

	void main() {
		MyObject o = new MyObject;
		{
			User u1 = User(o); // o.userCount == 1
			User u2 = User(o); // o.userCount == 2
			clear(o);          // o.userCount == 0
		}
		// after u2.~this(): o.userCount == -1 !!
		// after u1.~this(): o.userCount == -2 !!
	}

Here you see clear() is playing havoc with userCount, and you can't 
rely on protection attributes to guard against it.

The same applies if you have immutable members in your class: someone 
might be relying on it, have a reference to it, but clear() destroys 
this type system guaranty.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 10 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 Second, I find it rare that a class has an invariant.


My stucts and classes sometimes have invariants, and I suggest you to add/use
them to your code :-)

Bye,
bearophile
Aug 09 2010
prev sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Andrei Alexandrescu wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:
 

 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is guaranteed
 to only ever be called once.


 
 The spec still does, it is not updated since it describes delete, not clear.
 
 If you omit the default constructor, no constructor will be called. Also not
 for the base classes even if they have a default constructor. This looks like
 a bug.


 
 Yes, not calling the constructors of base classes is an implementation
 bug. If a class does not define any constructor at all, it has a de
 facto default constructor. If a class does define some constructor but
 not a parameterless one, it is a bug in the implementation if clear()
 compiles.
 

 Confusingly, if an object has a default constructor but is constructed from
 anything else, clear will still call the default constructor.


 
 I think that's reasonable. Otherwise the object would have to remember
 in a hidden state variable which constructor it was initialized from.


The confusing part (to me) comes from the special role of the default 
constructor in the current scheme. You cannot use clear() to release a resource 
constructed with it because it is immediately acquired again and hold onto
until 
(if at all) the collector decides to collect it. Not to mention it is acquired 
twice. It seems to be unsuitable for acquiring an (expensive) resource and yet 
that is exactly what tdpl illustrates.  
 

 I reckon it is
 also surprising if you later insert a previously omitted default constructor
 that the behavior can change a lot, especially when base classes are
 involved.


 
 That's a consequence of the implementation bugs above, I think.
 
 
 Andrei


Thanks, that will help with the other points. Should I file bugs?
Aug 09 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Lutger wrote:

 Andrei Alexandrescu wrote:
 

 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default constructor?
 The spec used to say (maybe it still does) that a destructor is guaranteed
 to only ever be called once.


 The spec still does, it is not updated since it describes delete, not clear.

 If you omit the default constructor, no constructor will be called. Also not
 for the base classes even if they have a default constructor. This looks like
 a bug.


 Yes, not calling the constructors of base classes is an implementation
 bug. If a class does not define any constructor at all, it has a de
 facto default constructor. If a class does define some constructor but
 not a parameterless one, it is a bug in the implementation if clear()
 compiles.


 Confusingly, if an object has a default constructor but is constructed from
 anything else, clear will still call the default constructor.


 I think that's reasonable. Otherwise the object would have to remember
 in a hidden state variable which constructor it was initialized from.


 
 The confusing part (to me) comes from the special role of the default 
 constructor in the current scheme. You cannot use clear() to release a
resource 
 constructed with it because it is immediately acquired again and hold onto
until 
 (if at all) the collector decides to collect it. Not to mention it is acquired 
 twice. It seems to be unsuitable for acquiring an (expensive) resource and yet 
 that is exactly what tdpl illustrates.  


The default constructor for classes already has a special role, e.g. 
it's the only one known polymorphically and the only one used by the 
built-in object factory.

If someone acquires a resource in the default constructor, one can 
presume that all other constructors also allocate that resource so 
ownership of the resource is part of the object's invariant. 
Consequently, the destructor can assume ownership of the resource. If 
clear() initializes the object by bitblitting the initial values over 
the object's fields, then the later-invoked destructor will fail.


 I reckon it is
 also surprising if you later insert a previously omitted default constructor
 that the behavior can change a lot, especially when base classes are
 involved.


 That's a consequence of the implementation bugs above, I think.


 Andrei


 
 Thanks, that will help with the other points. Should I file bugs?


Yes please.


Andrei
Aug 09 2010
next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4609
Aug 09 2010
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2010-08-09 22:57, Andrei Alexandrescu wrote:

 Lutger wrote:

 Andrei Alexandrescu wrote:


 Lutger wrote:

 Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 08:28:38 -0400, Andrej Mitrovic
 <andrej.mitrovich gmail.com> wrote:


 It's rather perplexing, isn't it? It states in TDPL:

 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no
 idea in
 what state you have left the object."


 This seems totally wrong, what if an object has no default
 constructor?
 The spec used to say (maybe it still does) that a destructor is
 guaranteed
 to only ever be called once.


 The spec still does, it is not updated since it describes delete,
 not clear.

 If you omit the default constructor, no constructor will be called.
 Also not
 for the base classes even if they have a default constructor. This
 looks like
 a bug.


 Yes, not calling the constructors of base classes is an implementation
 bug. If a class does not define any constructor at all, it has a de
 facto default constructor. If a class does define some constructor but
 not a parameterless one, it is a bug in the implementation if clear()
 compiles.


 Confusingly, if an object has a default constructor but is
 constructed from
 anything else, clear will still call the default constructor.


 I think that's reasonable. Otherwise the object would have to remember
 in a hidden state variable which constructor it was initialized from.


 The confusing part (to me) comes from the special role of the default
 constructor in the current scheme. You cannot use clear() to release a
 resource constructed with it because it is immediately acquired again
 and hold onto until (if at all) the collector decides to collect it.
 Not to mention it is acquired twice. It seems to be unsuitable for
 acquiring an (expensive) resource and yet that is exactly what tdpl
 illustrates.


 The default constructor for classes already has a special role, e.g.
 it's the only one known polymorphically and the only one used by the
 built-in object factory.


Why doesn't the object factory look something like this ?
factory(ARGS...)(string classname, ARGS args);


 If someone acquires a resource in the default constructor, one can
 presume that all other constructors also allocate that resource so
 ownership of the resource is part of the object's invariant.
 Consequently, the destructor can assume ownership of the resource. If
 clear() initializes the object by bitblitting the initial values over
 the object's fields, then the later-invoked destructor will fail.


 I reckon it is
 also surprising if you later insert a previously omitted default
 constructor
 that the behavior can change a lot, especially when base classes are
 involved.


 That's a consequence of the implementation bugs above, I think.


 Andrei


 Thanks, that will help with the other points. Should I file bugs?


 Yes please.


 Andrei



-- 
/Jacob Carlborg
Aug 10 2010
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-08-09 08:28:38 -0400, Andrej Mitrovic 
<andrej.mitrovich gmail.com> said:


 "After you invoke clear, the object is still alive and well, but its
 destructor has been called and the object is now carrying its
 default-constructed stated. During the next garbage collection, the
 destructor is called again, because the garbage collector has no idea in
 what state you have left the object."
 
 But in what situation would you want to manipulate an object that was
 already cleared and ready for garbage collection?


To me, 'clear' looks like a way to implement a 'removeAll' function on 
containers but at a lower level that kind of work anywhere. This has 
the benefit that you don't have to implement clear on every container. 
But in the general case I find it rather pointless, and quite dangerous 
too as it can break some invariants, not invariant in the class itself 
but in the program as a whole.

The more I think about it, the more it looks like a misfeature in the 
same league as copy constructors that everyone has to disable in C++. 
But is there any way to disable 'clear' for a given class? I guess not, 
since all classes can be casted back to Object. Though perhaps omitting 
the default constructor would work.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 09 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Monday, August 09, 2010 17:07:28 Andrej Mitrovic wrote:

 I don't have a lot of experience in this field, but my guess is people want
 to help out the GC be more efficient in some regard.
 
 "Hey, GC! I know you take the trash out every day, but I just want to let
 you know that I've cleaned up my room and there's a whole bag o' dirt in
 here ready to be thrown out that you should know about. See ya later!"
 
 My assumption is that the programmer probably knows when and how he uses
 memory more so than the GC itself. So my thinking is some kind of hybrid
 approach would be a good choice between safety and performance here. But
 correct me if I'm wrong, I'm walking in the dark really..


A GC is tuned with the idea that it will run periodically and free memory which 
is freeable at that point rather than frequently freeing it at the programmer's 
request like happens with manually managed memory. And since freeing memory can 
be expensive, it can be argued that it's actually more efficient to just let
the 
memory not be freed until the garbage collector does its thing whenever that
is. 
I've heard that there have been papers showing that that's more efficient, but
I 
haven't read them. The issue is if the time that the garbage collector decides 
to run is when you're trying to do something time or resource-sensitive, and
the 
garbage collector picks a bad time to run. But that's generally a fact of life 
with garbage collectors, and if it's an efficient garbage collector, then 
hopefully the hit is minimal.

Also, it's not like the garbage collector is likely to be really freeing your 
memory anyway, unless it decides that it just has way too much allocated. So, 
telling it that you want it to free something really is only getting the 
destructor called for you, which likely only matters if you have another 
resource of some kind (like file handles or something) which you need freed.
And 
if that's what you need, you could just as easily call a function on the class 
to free that up as go and completely destroy it. The one thing that I can think 
of that clear() might get for you otherwise is helping the GC know that it any 
other references that that class holds aren't needed anymore, but if you no 
longer have any references to the object in question, the GC should already
have 
that information.

Really, in the general case, it's going to be most efficient to let the GC do
what 
it does. If it's well-written, it should do it efficiently. Trying to tell it
that 
you know best is not likely to work well (in the general case, at least). If
you 
really want to be freeing things as soon as you're done with them, you need to 
do manually memory management. Besides' s soon as you're keeping track of when 
it would be okay to clear() an object, that's pretty much what you're doing 
anyway.

- Jonathan M Davis
Aug 09 2010
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Thanks for the lengthy reply. I need to start studying GC's before I assume
more things like that. :)

On Tue, Aug 10, 2010 at 3:11 AM, Jonathan M Davis <jmdavisprog gmail.com>wrote:


 On Monday, August 09, 2010 17:07:28 Andrej Mitrovic wrote:

 I don't have a lot of experience in this field, but my guess is people


 want

 to help out the GC be more efficient in some regard.

 "Hey, GC! I know you take the trash out every day, but I just want to let
 you know that I've cleaned up my room and there's a whole bag o' dirt in
 here ready to be thrown out that you should know about. See ya later!"

 My assumption is that the programmer probably knows when and how he uses
 memory more so than the GC itself. So my thinking is some kind of hybrid
 approach would be a good choice between safety and performance here. But
 correct me if I'm wrong, I'm walking in the dark really..


 A GC is tuned with the idea that it will run periodically and free memory
 which
 is freeable at that point rather than frequently freeing it at the
 programmer's
 request like happens with manually managed memory. And since freeing memory
 can
 be expensive, it can be argued that it's actually more efficient to just
 let the
 memory not be freed until the garbage collector does its thing whenever
 that is.
 I've heard that there have been papers showing that that's more efficient,
 but I
 haven't read them. The issue is if the time that the garbage collector
 decides
 to run is when you're trying to do something time or resource-sensitive,
 and the
 garbage collector picks a bad time to run. But that's generally a fact of
 life
 with garbage collectors, and if it's an efficient garbage collector, then
 hopefully the hit is minimal.

 Also, it's not like the garbage collector is likely to be really freeing
 your
 memory anyway, unless it decides that it just has way too much allocated.
 So,
 telling it that you want it to free something really is only getting the
 destructor called for you, which likely only matters if you have another
 resource of some kind (like file handles or something) which you need
 freed. And
 if that's what you need, you could just as easily call a function on the
 class
 to free that up as go and completely destroy it. The one thing that I can
 think
 of that clear() might get for you otherwise is helping the GC know that it
 any
 other references that that class holds aren't needed anymore, but if you no
 longer have any references to the object in question, the GC should already
 have
 that information.

 Really, in the general case, it's going to be most efficient to let the GC
 do what
 it does. If it's well-written, it should do it efficiently. Trying to tell
 it that
 you know best is not likely to work well (in the general case, at least).
 If you
 really want to be freeing things as soon as you're done with them, you need
 to
 do manually memory management. Besides' s soon as you're keeping track of
 when
 it would be okay to clear() an object, that's pretty much what you're doing
 anyway.

 - Jonathan M Davis
Aug 09 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }


I am not expert enough about all this topic, so I try to keep myself out of
this discussion. Recently I have read this:


issues with finalizers. The main problem with finalizers and garbage collection
is that finalizers generally try to reclaim a non-memory resource (such as a
specific file, a file handle, or a network socket) based purely on a memory
reclamation policy (generally triggered by later memory allocations). Generally
those other resource exhaust well before memory does, then you're out of the
resource stuck waiting for garbage collection to notice a particular finalizer
needs to be run. A better general rule of thumb is "don't ever use finalizers".
Instead always call close inside a finally.<


This means that putting something like a fclose(fp) inside the ~this() is
wrong, you need to add a specific method to that File class to ask for the
closing of the file, because you generally can't rely on the GC for this,
because a GC is free to even never collect objects, if there is enough RAM.

In my opinion it's correct to put something like a fclose(fp) inside the
~this() only if you are sure this struct/class will be always allocated on the
stack, so its destructor will always be called deterministically when the scope
ends (a reference counting strategy too seems acceptable, because it is
deterministic).

In a GC-based language you can't assume your destructors are run, so your
destructors usually need to be empty, and you need to add other methods to free
explicitly and manually (or with a scope(exit)) all the resources that aren't
RAM.

Please take this cum grano salis, I am not an expert on this.

Bye,
bearophile
Aug 10 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 10 Aug 2010 08:27:19 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Andrei Alexandrescu:

 class File {
      FILE * fp;
      this() { fp = fopen("/tmp/temporary"); }
      ~this() { fclose(fp); }
 }


 I am not expert enough about all this topic, so I try to keep myself out  
 of this discussion. Recently I have read this:


 issues with finalizers. The main problem with finalizers and garbage  
 collection is that finalizers generally try to reclaim a non-memory  
 resource (such as a specific file, a file handle, or a network socket)  
 based purely on a memory reclamation policy (generally triggered by  
 later memory allocations). Generally those other resource exhaust well  
 before memory does, then you're out of the resource stuck waiting for  
 garbage collection to notice a particular finalizer needs to be run. A  
 better general rule of thumb is "don't ever use finalizers". Instead  
 always call close inside a finally.<


 This means that putting something like a fclose(fp) inside the ~this()  
 is wrong, you need to add a specific method to that File class to ask  
 for the closing of the file, because you generally can't rely on the GC  
 for this, because a GC is free to even never collect objects, if there  
 is enough RAM.

 In my opinion it's correct to put something like a fclose(fp) inside the  
 ~this() only if you are sure this struct/class will be always allocated  
 on the stack, so its destructor will always be called deterministically  
 when the scope ends (a reference counting strategy too seems acceptable,  
 because it is deterministic).

 In a GC-based language you can't assume your destructors are run, so  
 your destructors usually need to be empty, and you need to add other  
 methods to free explicitly and manually (or with a scope(exit)) all the  
 resources that aren't RAM.

 Please take this cum grano salis, I am not an expert on this.


destructors are for the purpose of clearing out resources that are not  
provided by the GC.  Not having it in the destructor is a bad thing.

The thing is, if you *don't* call it in the destructor, and require  
someone to manually call your explicit method, then someone will forget to  
make that call, and your resource stays open forever.  It also means you  
have to call your method when going out of scope.

By clearing that resource in the destructor, and allowing a way to  
manually call that destructor, the class works in all three situations:   
manual resource management (via clear), scoped destruction, and GC  
destruction.

I think in time, the GC may be associated with it's own thread, and may  
run on a schedule vs. having to wait for a memory allocation to run.  When  
this happens, it's more likely you can rely on the GC to free the  
resources in a reasonable amount of time.

-Steve
Aug 10 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 By clearing that resource in the destructor, and allowing a way to  
 manually call that destructor, the class works in all three situations:   
 manual resource management (via clear), scoped destruction, and GC  
 destruction.


From what I have seen so far:
1) the semantics of clear() is a mess
2) objects don't get deallocated deterministically when they go out of scope
3) and the GC is not deterministic, you can only partially hope your destructor
will be called at program end, in some random order. And sometimes your object
destructors will not be called even at the end of the program.

The only thing I see good here is to use a scope(exit) to close the file:
auto f = new File(...);
scope(exit) f.close();
...

While clear, scoped destruction of the object, and GC destruction don't seem
enough in this situation, or any other situation where there you have a
resource that is not RAM.



 I think in time, the GC may be associated with it's own thread, and may  
 run on a schedule vs. having to wait for a memory allocation to run.  When  
 this happens, it's more likely you can rely on the GC to free the  
 resources in a reasonable amount of time.


I'll believe that only when I see it. Even one of the most advanced GC around,
the one in the Sun JVM, doesn't act as deterministically as you say (maybe the
G1 now does, I am not sure).

Instead of a clear() function it can be added to classes and structs an
optional standard method (like .clear()) that when called performs the cleaning
of the object, frees all resources (but not the RAM used by the object
instance), puts references to null, etc. The deallocation of the RAM is fully
left to the GC later. An class that defines clear() keeps the extra bit of
state to tell apart the living or dead state of the object, plus an extra
assert in the class invariant. This is just an idea, probably not too much good
:-)

Bye,
bearophile
Aug 10 2010
parent reply Don <nospam nospam.com> writes:
bearophile wrote:

 Steven Schveighoffer:
 

 By clearing that resource in the destructor, and allowing a way to  
 manually call that destructor, the class works in all three situations:   
 manual resource management (via clear), scoped destruction, and GC  
 destruction.


 
 From what I have seen so far:
 1) the semantics of clear() is a mess
 2) objects don't get deallocated deterministically when they go out of scope
 3) and the GC is not deterministic, you can only partially hope your
destructor will be called at program end, in some random order. And sometimes
your object destructors will not be called even at the end of the program.
 
 The only thing I see good here is to use a scope(exit) to close the file:
 auto f = new File(...);
 scope(exit) f.close();
 ...
 
 While clear, scoped destruction of the object, and GC destruction don't seem
enough in this situation, or any other situation where there you have a
resource that is not RAM.


I completely agree. Everything I've read about finalizers indicates that 
it's a completely broken concept. It seems as though it was initially 

destructors. Apparently it took some time to realize that the value in 
destructors comes almost entirely from the deterministic lifetime.
Instead, finalizers seem to be equivalent to registering a callback with 
the runtime.

Their main legitimate use seems to be for contract programming (though 
I've never heard it described in that way): at the moment of garbage 
collection, check that the destructor has actually been run.




 
 

 I think in time, the GC may be associated with it's own thread, and may  
 run on a schedule vs. having to wait for a memory allocation to run.  When  
 this happens, it's more likely you can rely on the GC to free the  
 resources in a reasonable amount of time.


 
 I'll believe that only when I see it. Even one of the most advanced GC around,
the one in the Sun JVM, doesn't act as deterministically as you say (maybe the
G1 now does, I am not sure).
 
 Instead of a clear() function it can be added to classes and structs an
optional standard method (like .clear()) that when called performs the cleaning
of the object, frees all resources (but not the RAM used by the object
instance), puts references to null, etc. The deallocation of the RAM is fully
left to the GC later. An class that defines clear() keeps the extra bit of
state to tell apart the living or dead state of the object, plus an extra
assert in the class invariant. This is just an idea, probably not too much good
:-)
 
 Bye,
 bearophile
Aug 12 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 12 Aug 2010 14:46:00 -0400, Don <nospam nospam.com> wrote:


 bearophile wrote:

 Steven Schveighoffer:


 By clearing that resource in the destructor, and allowing a way to   
 manually call that destructor, the class works in all three  
 situations:   manual resource management (via clear), scoped  
 destruction, and GC  destruction.


  From what I have seen so far:
 1) the semantics of clear() is a mess
 2) objects don't get deallocated deterministically when they go out of  
 scope
 3) and the GC is not deterministic, you can only partially hope your  
 destructor will be called at program end, in some random order. And  
 sometimes your object destructors will not be called even at the end of  
 the program.
  The only thing I see good here is to use a scope(exit) to close the  
 file:
 auto f = new File(...);
 scope(exit) f.close();
 ...
  While clear, scoped destruction of the object, and GC destruction  
 don't seem enough in this situation, or any other situation where there  
 you have a resource that is not RAM.


 I completely agree. Everything I've read about finalizers indicates that  
 it's a completely broken concept. It seems as though it was initially  

 destructors. Apparently it took some time to realize that the value in  
 destructors comes almost entirely from the deterministic lifetime.
 Instead, finalizers seem to be equivalent to registering a callback with  
 the runtime.


No, it's purpose is to be a last-ditch effort to clean up unmanaged  
resources.  It's kind of like a safety net.

Essentially, if you are relying on the GC to destroy your object, and you  
haven't cleaned up all the resources in that object, you don't want those  
resources to leak.  Once the object is destroyed, the resource is leaked  
if it's not cleaned up.  Is allowing the destructor to clean up your  
resource a program error?  It's up to the developer to decide that.

D still lacks a standard way of deterministic destruction.  Clear doesn't  
work because it still calls the destructor, and the destructor must assume  
it's non-deterministic.


 Their main legitimate use seems to be for contract programming (though  
 I've never heard it described in that way): at the moment of garbage  
 collection, check that the destructor has actually been run.


It can be set up this way, just throw an exception if the resource isn't  
cleaned up instead of silently cleaning up the mess in your destructor.

-Steve
Aug 12 2010