www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Accessing memory after destroy

reply Johan Engelen <j j.nl> writes:
I'd like to check a bit of info I need for Address Sanitizer 
checking.

The spec says [1]:
Use the destroy function to finalize an object by calling its 
destructor. The memory of the object is not immediately 
deallocated, instead the GC will collect the memory of the object 
at an undetermined point after finalization:
```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
```

This tells me 2 things that I'd like to verify:
1. The destroyed memory is set to the type's `.init` value. (but 
the Ctor is not called)
2. It is _valid_ to access the memory after calling destroy.

Point 2 is worrying: what if there is a thread switch right after 
destroy, in which a GC collect happens?

Thanks,
   Johan

[1] https://dlang.org/spec/class.html#deallocators
Jul 29 2017
next sibling parent Eugene Wissner <belka caraus.de> writes:
On Saturday, 29 July 2017 at 20:44:30 UTC, Johan Engelen wrote:

 I'd like to check a bit of info I need for Address Sanitizer 
 checking.

 The spec says [1]:
 Use the destroy function to finalize an object by calling its 
 destructor. The memory of the object is not immediately 
 deallocated, instead the GC will collect the memory of the 
 object at an undetermined point after finalization:
 ```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
 ```

 This tells me 2 things that I'd like to verify:
 1. The destroyed memory is set to the type's `.init` value. 
 (but the Ctor is not called)
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right 
 after destroy, in which a GC collect happens?

 Thanks,
   Johan

 [1] https://dlang.org/spec/class.html#deallocators


For 1) look at rt_finalize2 in rt/lifetime.d*. It is the function 
called by destroy. It sets the class to its init value:

auto w = (*pc).initializer;
p[0 .. w.length] = w[];

So it will memcpy Foo.initializer into foo.

For 2) I think you're right, it seems to be unsafe to access the 
object after destroying.


* 
https://github.com/dlang/druntime/blob/3485ff859a29ba44e7949bc49e62d5dd3a2a9ff0/src/rt/lifetime.d#L1402
Jul 29 2017
prev sibling next sibling parent reply Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 29 July 2017 at 20:44:30 UTC, Johan Engelen wrote:

 [...]
 ```
   class Foo { int x; this() { x = 1; } }
   Foo foo = new Foo;
   destroy(foo);
   assert(foo.x == int.init);  // object is still accessible
 ```
 [...]
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right 
 after destroy, in which a GC collect happens?


D's GC uses (conservative) stop-the-world mark-and-sweep, i.e. as 
long as a memory chunk is reachable via a root it won't be 
collected (see [1]). Since in your example the (stack) variable 
`foo` is still referring to the (heap) memory location of the 
destroyed object, if a garbage collection cycle is started 
between `destroy(foo);` and `assert(foo.x == int.init);` by 
another thread then - barring bugs - the memory location `foo` 
refers to will be marked as alive and not collected.

[1] https://dlang.org/library/core/memory/gc.collect.html
Jul 29 2017
parent Johan Engelen <j j.nl> writes:
On Saturday, 29 July 2017 at 22:15:41 UTC, Moritz Maxeiner wrote:

  Since in your example the (stack) variable `foo` is still 
 referring to the (heap) memory location of the destroyed object


Ah yes of course, thanks!

- Johan
Jul 29 2017
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, July 29, 2017 20:44:30 Johan Engelen via Digitalmars-d wrote:

 I'd like to check a bit of info I need for Address Sanitizer
 checking.

 The spec says [1]:
 Use the destroy function to finalize an object by calling its
 destructor. The memory of the object is not immediately
 deallocated, instead the GC will collect the memory of the object
 at an undetermined point after finalization:
 ```
    class Foo { int x; this() { x = 1; } }
    Foo foo = new Foo;
    destroy(foo);
    assert(foo.x == int.init);  // object is still accessible
 ```

 This tells me 2 things that I'd like to verify:
 1. The destroyed memory is set to the type's `.init` value. (but
 the Ctor is not called)
 2. It is _valid_ to access the memory after calling destroy.

 Point 2 is worrying: what if there is a thread switch right after
 destroy, in which a GC collect happens?

 Thanks,
    Johan

 [1] https://dlang.org/spec/class.html#deallocators


If destroy has been called on a class object, then it is a bug to access it
at any point after that (IIRC, the expectation is that it will blow up in
your face, because the vtable is gone - TDPL talks about this, I believe,
but I don't know where my copy is at the moment, so I can't check). That
being said, the memory is still valid. And as Moritz pointed out, if the
memory is accessible, the GC won't free it. So, it's a bug to access the
object, but it should be memory safe to do so.

- Jonathan M Davis
Jul 29 2017
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis wrote:

 [1] https://dlang.org/spec/class.html#deallocators


 If destroy has been called on a class object, then it is a bug 
 to access it at any point after that (IIRC, the expectation is 
 that it will blow up in your face, because the vtable is gone - 
 TDPL talks about this, I believe, but I don't know where my 
 copy is at the moment, so I can't check). That being said, the 
 memory is still valid. And as Moritz pointed out, if the memory 
 is accessible, the GC won't free it. So, it's a bug to access 
 the object, but it should be memory safe to do so.


If this is the case, then we really need to improve and pin-down 
the spec on this point.

So `destroy` only calls the destructor and puts the object in 
`T.initializer` (non-constructed) state, and is otherwise not 
related to memory deallocation.
May the destructor be called again when the GC collects the 
memory?
Why is the object put in `T.initializer` state?

To make sure: My question is from a spec point of view. Not about 
what currently happens and what is "OK" with the current 
implementation.
Jul 30 2017
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
Johan Engelen wrote:


 If this is the case, then we really need to improve and pin-down the spec 
 on this point.

 So `destroy` only calls the destructor and puts the object in 
 `T.initializer` (non-constructed) state, and is otherwise not related to 
 memory deallocation.
 May the destructor be called again when the GC collects the memory?
 Why is the object put in `T.initializer` state?

 To make sure: My question is from a spec point of view. Not about what 
 currently happens and what is "OK" with the current implementation.


afair, somewhere in the spec there is a mention that dtor will be called 
at most once for each initialized object. and object state doesn't have any 
sense after calling dtor, but D still has to put something there, so 
`.init` looks like a reasonable choice. althru i'm not sure that anything 
in specs says that runtime *must* clear destroyed objects (they aren't 
really usable after calling dtor anyway, it is at least a logical bug to use 
object after destroying it).
Jul 30 2017
parent Temtaime <temtaime gmail.com> writes:
On Sunday, 30 July 2017 at 07:58:19 UTC, ketmar wrote:

 Johan Engelen wrote:


 [...]


 afair, somewhere in the spec there is a mention that dtor will 
 be called at most once for each initialized object. and object 
 state doesn't have any sense after calling dtor, but D still 
 has to put something there, so `.init` looks like a reasonable 
 choice. althru i'm not sure that anything in specs says that 
 runtime *must* clear destroyed objects (they aren't really 
 usable after calling dtor anyway, it is at least a logical bug 
 to use object after destroying it).


Seems that zeroing it out is better choice, maybe.
Jul 30 2017
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 30 July 2017 at 07:45:27 UTC, Johan Engelen wrote:

 On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis 
 wrote:

 [1] https://dlang.org/spec/class.html#deallocators


 If destroy has been called on a class object, then it is a bug 
 to access it at any point after that (IIRC, the expectation is 
 that it will blow up in your face, because the vtable is gone 
 - TDPL talks about this, I believe, but I don't know where my 
 copy is at the moment, so I can't check). That being said, the 
 memory is still valid. And as Moritz pointed out, if the 
 memory is accessible, the GC won't free it. So, it's a bug to 
 access the object, but it should be memory safe to do so.


 If this is the case, then we really need to improve and 
 pin-down the spec on this point.

 So `destroy` only calls the destructor and puts the object in 
 `T.initializer` (non-constructed) state, and is otherwise not 
 related to memory deallocation.
 May the destructor be called again when the GC collects the 
 memory?
 Why is the object put in `T.initializer` state?


rt_finalize2 zeroes the vptr out after assigning 
typeid(T).initializer, so calling the destructor more than once 
is not possible, unless someone manually saves the vptr and 
assigns it back to the object after the call to destroy / 
rt_finalize.


 To make sure: My question is from a spec point of view. Not 
 about what currently happens and what is "OK" with the current 
 implementation.


I guess the spec needs to be updated to state explicitly that a 
class destructor is called only once, if that's not already the 
case.
Jul 30 2017
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/30/17 9:17 AM, Petar Kirov [ZombineDev] wrote:

 On Sunday, 30 July 2017 at 07:45:27 UTC, Johan Engelen wrote:

 On Saturday, 29 July 2017 at 23:09:38 UTC, Jonathan M Davis wrote:

 [1] https://dlang.org/spec/class.html#deallocators


 If destroy has been called on a class object, then it is a bug to 
 access it at any point after that (IIRC, the expectation is that it 
 will blow up in your face, because the vtable is gone - TDPL talks 
 about this, I believe, but I don't know where my copy is at the 
 moment, so I can't check). That being said, the memory is still 
 valid. And as Moritz pointed out, if the memory is accessible, the GC 
 won't free it. So, it's a bug to access the object, but it should be 
 memory safe to do so.


 If this is the case, then we really need to improve and pin-down the 
 spec on this point.

 So `destroy` only calls the destructor and puts the object in 
 `T.initializer` (non-constructed) state, and is otherwise not related 
 to memory deallocation.
 May the destructor be called again when the GC collects the memory?
 Why is the object put in `T.initializer` state?


 
 rt_finalize2 zeroes the vptr out after assigning typeid(T).initializer, 
 so calling the destructor more than once is not possible, unless someone 
 manually saves the vptr and assigns it back to the object after the call 
 to destroy / rt_finalize.


Yes, this is on purpose. It's the way the runtime knows it has already 
been run. In fact, destroy used to NOT do this, and that caused problems.

A class instance is not usable after calling destroy. It is not intended 
to be usable, because classes require a constructor call to be valid, 
and there's no way to guarantee you can do this with destroy.

Structs are different.


 To make sure: My question is from a spec point of view. Not about what 
 currently happens and what is "OK" with the current implementation.


 
 I guess the spec needs to be updated to state explicitly that a class 
 destructor is called only once, if that's not already the case.


I thought this was in the spec, but I can't find it. This is definitely 
the case.

-Steve
Aug 01 2017