www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - InvalidMemoryOperationError when calling functions from destructors

reply "Volfram" <lonewolf325 gmail.com> writes:
I've run into a problem which I'd like to hope is a bug, but 
let's see if we can figure out if I'm doing something stupid 
first, eh?

When a destructor calls a function from another module, I get an 
InvalidMemoryOperationError.  When a destructor calls a function 
from another class in the same module, and that function calls 
the function I was trying to call initially, everything seems to 
go fine.  When a destructor calls a function in the same class, 
which calls a function in a different module, I get the 
InvalidMemoryOperationError again.

This may have to do with garbage collector subtleties I'm not 
familiar with.

Example:

File alpha.d
module alpha;

import beta;

class AlphaClass
{
     BetaClass bc;
     Alpha2Class a2c;

     void cleanup()
     {
         bc.cleanup();//if this is called from the destructor, 
expect an error.
     }

     public:
     this()
     {
         bc = new BetaClass();
         a2c = new Alpha2Class(bc);
     }
     ~this
     {
         bc.cleanup();//this will cause an error.
         a2c.cleanup();//this works fine
         cleanup();//this will cause an error.
     }
}

class Alpha2Class
{
     BetaClass bc;

     void cleanup()
     {
         bc.cleanup();
     }

     public:
     this(BetaClass initbc)
     {
         bc = initbc;
     }
}

File beta.d

module beta;

class BetaClass
{
     public:
     this()
     {
         //do something
     }
     void cleanup()
     {
         //clean up after the bosses
     }
}

Further info can be provided if necessary.
Apr 25 2013
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 25 April 2013 at 15:50:27 UTC, Volfram wrote:

 Further info can be provided if necessary.


Can you provide a full program that exhibits the behavior, or at 
least a stack trace?

You may need to recompile Phobos and Druntime with the -gs flag 
to enable correct stacktraces.
Apr 25 2013
parent "Volfram" <lonewolf325 gmail.com> writes:
On Thursday, 25 April 2013 at 16:00:31 UTC, Vladimir Panteleev 
wrote:

 On Thursday, 25 April 2013 at 15:50:27 UTC, Volfram wrote:

 Further info can be provided if necessary.


 Can you provide a full program that exhibits the behavior, or 
 at least a stack trace?

 You may need to recompile Phobos and Druntime with the -gs flag 
 to enable correct stacktraces.


Well I've got a 6000-line hackeneyed game engine in progress 
built using the kind of logic that usually only I can follow, but 
I somehow don't think that's what you're after.

Jacob Carlborg: I thought it might be something like that, which 
is part of the reason I didn't post this in "bug reports."  I'll 
figure out something else.  Thanks.
Apr 25 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-04-25 17:50, Volfram wrote:

 I've run into a problem which I'd like to hope is a bug, but let's see
 if we can figure out if I'm doing something stupid first, eh?

 When a destructor calls a function from another module, I get an
 InvalidMemoryOperationError.  When a destructor calls a function from
 another class in the same module, and that function calls the function I
 was trying to call initially, everything seems to go fine.  When a
 destructor calls a function in the same class, which calls a function in
 a different module, I get the InvalidMemoryOperationError again.

 This may have to do with garbage collector subtleties I'm not familiar
 with.

 Example:

 File alpha.d
 module alpha;

 import beta;

 class AlphaClass
 {
      BetaClass bc;
      Alpha2Class a2c;

      void cleanup()
      {
          bc.cleanup();//if this is called from the destructor, expect an
 error.
      }

      public:
      this()
      {
          bc = new BetaClass();
          a2c = new Alpha2Class(bc);
      }
      ~this
      {
          bc.cleanup();//this will cause an error.
          a2c.cleanup();//this works fine
          cleanup();//this will cause an error.
      }
 }


You cannot access GC controlled memory in class destructors. There's no 
guarantee in which order the destructors will be called. You don't know 
if the memory is still valid in a destructor.

-- 
/Jacob Carlborg
Apr 25 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 25 April 2013 at 16:17:57 UTC, Jacob Carlborg wrote:

 You cannot access GC controlled memory in class destructors.


Not true.


 There's no guarantee in which order the destructors will be 
 called. You don't know if the memory is still valid in a 
 destructor.


The memory is valid, it's just that referenced objects may 
already be finalized.

Regardless, the above (accessing objects in destructors) is not 
related to the InvalidMemoryOperationError. See the documentation 
for it:
Apr 25 2013
next sibling parent David <d dav1d.de> writes:
 Regardless, the above (accessing objects in destructors) is not related
 to the InvalidMemoryOperationError. See the documentation for it:



I had/have the same problem, it also occurs if you allocate in a
destructor during RT shutdown. Well other than that I couldn't reproduce
it with allocating in destructors or accessing objects owened by the gc
in the destructors.
Apr 25 2013
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-04-25 19:57, Vladimir Panteleev wrote:


 Regardless, the above (accessing objects in destructors) is not related
 to the InvalidMemoryOperationError. See the documentation for it:



Yeah, I though that was weird.

-- 
/Jacob Carlborg
Apr 25 2013
parent "Volfram" <lonewolf325 gmail.com> writes:
Further inspection suggests that the BetaClass object I was 
accessing was already finalized by the time the AlphaClass object 
got around to trying to mess with it.  Conclusion is that whether 
it's the in-destructor access or not, it's simply not safe to try 
to clean things up this way.  I think I have a safer alternative.

What was interesting is that even after the BetaClass object has 
been cleaned up, I can access any variables it may have 
contained.  If, for example, it has an integer value that's 
publicly visible, I can still read that.(haven't tried writing.)
Apr 26 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 25 Apr 2013 10:57:13 -0700, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:


 On Thursday, 25 April 2013 at 16:17:57 UTC, Jacob Carlborg wrote:

 You cannot access GC controlled memory in class destructors.


 Not true.


Can you be more specific?  Maybe the wording was too strong.  Should say  
"you shouldn't access GC controlled memory in class destructors."

Or are you thinking of some specific case?  Because the opposite of the  
above is DEFINITELY not true.  I just want to make that clear.

-Steve
Apr 26 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Saturday, 27 April 2013 at 06:12:03 UTC, Steven Schveighoffer 
wrote:

 On Thu, 25 Apr 2013 10:57:13 -0700, Vladimir Panteleev 
 <vladimir thecybershadow.net> wrote:


 On Thursday, 25 April 2013 at 16:17:57 UTC, Jacob Carlborg 
 wrote:

 You cannot access GC controlled memory in class destructors.


 Not true.


 Can you be more specific?  Maybe the wording was too strong.  
 Should say "you shouldn't access GC controlled memory in class 
 destructors."

 Or are you thinking of some specific case?  Because the 
 opposite of the above is DEFINITELY not true.  I just want to 
 make that clear.


Last time I checked, destructors were called separately from 
deallocation. Thus, referencing memory will not result in 
undefined behavior. The order of destruction is 
non-deterministic, but otherwise, it's the same as accessing a 
clear()'d object, which is perfectly safe (from a memory safety 
perspective).
Apr 27 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 27 Apr 2013 07:58:25 -0700, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:


 On Saturday, 27 April 2013 at 06:12:03 UTC, Steven Schveighoffer wrote:

 On Thu, 25 Apr 2013 10:57:13 -0700, Vladimir Panteleev  
 <vladimir thecybershadow.net> wrote:


 On Thursday, 25 April 2013 at 16:17:57 UTC, Jacob Carlborg wrote:

 You cannot access GC controlled memory in class destructors.


 Not true.


 Can you be more specific?  Maybe the wording was too strong.  Should  
 say "you shouldn't access GC controlled memory in class destructors."

 Or are you thinking of some specific case?  Because the opposite of the  
 above is DEFINITELY not true.  I just want to make that clear.


 Last time I checked, destructors were called separately from  
 deallocation. Thus, referencing memory will not result in undefined  
 behavior. The order of destruction is non-deterministic, but otherwise,  
 it's the same as accessing a clear()'d object, which is perfectly safe  
 (from a memory safety perspective).


No, destructors are called along with deallocation.  At the moment, the GC  
mutex is held while the collection is in progress, so it's not possible  
that a deallocated block could be reallocated before a dtor that  
references that block is called.  But that is an implementation detail.   
There is no requirement for the GC to behave that way.  In addition, there  
is no requirement for the GC to run in a specific thread, so if a  
destroyed object references a non-destroyed object, it's possible some  
thread is using the non-destroyed object while the dtor is using it in  
another thread, even if the object was only ever accessed from one thread!

-Steve
Apr 28 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 April 2013 at 14:44:32 UTC, Steven Schveighoffer 
wrote:

 On Sat, 27 Apr 2013 07:58:25 -0700, Vladimir Panteleev 
 <vladimir thecybershadow.net> wrote:


 On Saturday, 27 April 2013 at 06:12:03 UTC, Steven 
 Schveighoffer wrote:

 On Thu, 25 Apr 2013 10:57:13 -0700, Vladimir Panteleev 
 <vladimir thecybershadow.net> wrote:


 On Thursday, 25 April 2013 at 16:17:57 UTC, Jacob Carlborg 
 wrote:

 You cannot access GC controlled memory in class destructors.


 Not true.


 Can you be more specific?  Maybe the wording was too strong.  
 Should say "you shouldn't access GC controlled memory in 
 class destructors."

 Or are you thinking of some specific case?  Because the 
 opposite of the above is DEFINITELY not true.  I just want to 
 make that clear.


 Last time I checked, destructors were called separately from 
 deallocation. Thus, referencing memory will not result in 
 undefined behavior. The order of destruction is 
 non-deterministic, but otherwise, it's the same as accessing a 
 clear()'d object, which is perfectly safe (from a memory 
 safety perspective).


 No, destructors are called along with deallocation.  At the 
 moment, the GC mutex is held while the collection is in 
 progress, so it's not possible that a deallocated block could 
 be reallocated before a dtor that references that block is 
 called.


Right, so it makes no difference in practice.

What I meant was that IIRC early versions of the GC used to 
rebuild internal free lists, and clobber freed data, in-loop with 
calling destructors. Thus, when referencing something in a 
destructor, sometimes you'd get the same object (not yet 
finalized), and sometimes you'd get garbage. Now you get a 
finalized object instead of garbage.


 But that is an implementation detail.  There is no requirement 
 for the GC to behave that way.


I don't disagree with you, but do we have a spec for GCs?

I'd think preserving memory safety, even in a destructor, would 
be pretty important for a GC design.


 In addition, there is no requirement for the GC to run in a 
 specific thread, so if a destroyed object references a 
 non-destroyed object, it's possible some thread is using the 
 non-destroyed object while the dtor is using it in another 
 thread, even if the object was only ever accessed from one 
 thread!


Sorry, not following. What's the problem? How is this related?
Apr 28 2013
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 28 Apr 2013 16:25:06 -0700, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:


 On Sunday, 28 April 2013 at 14:44:32 UTC, Steven Schveighoffer wrote:



 No, destructors are called along with deallocation.  At the moment, the  
 GC mutex is held while the collection is in progress, so it's not  
 possible that a deallocated block could be reallocated before a dtor  
 that references that block is called.


 Right, so it makes no difference in practice.

 What I meant was that IIRC early versions of the GC used to rebuild  
 internal free lists, and clobber freed data, in-loop with calling  
 destructors. Thus, when referencing something in a destructor, sometimes  
 you'd get the same object (not yet finalized), and sometimes you'd get  
 garbage. Now you get a finalized object instead of garbage.


This was not a "bug," probably (can't be definite on it, since I'm unaware  
of how the GC used to work).


 But that is an implementation detail.  There is no requirement for the  
 GC to behave that way.


 I don't disagree with you, but do we have a spec for GCs?


AFAIK there is no spec, it's all based on the compiler implementation.   
For all I know, the GC on GDC or LDC may have a different API.


 I'd think preserving memory safety, even in a destructor, would be  
 pretty important for a GC design.


I wouldn't expect it.  code is code, not sure how you can guarantee memory  
safety in a dtor.


 In addition, there is no requirement for the GC to run in a specific  
 thread, so if a destroyed object references a non-destroyed object,  
 it's possible some thread is using the non-destroyed object while the  
 dtor is using it in another thread, even if the object was only ever  
 accessed from one thread!


 Sorry, not following. What's the problem? How is this related?


Like an unlocked reference counter, let's say.  It's only ever used in one  
thread.  But then an object with a reference becomes garbage.  Another  
thread then runs the garbage collector, enabling race conditions that  
would not normally be possible.

-Steve
Apr 28 2013