www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to debug memory errors

reply =?UTF-8?B?w5hpdmluZA==?= <oivind.loe gmail.com> writes:
Hi,

My app occasionally gives me a

*** Error in `./hauto-test': double free or corruption (fasttop): 
0x00007f504c002a60 ***

but gives me the following on every termination

core.exception.InvalidMemoryOperationError src/core/exception.d(693): Invalid
memory operation

How do I go about debugging and resolving these?

-Øivind
Nov 06 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Sunday, 6 November 2016 at 21:46:52 UTC, Øivind wrote:

 Hi,

 My app occasionally gives me a

 *** Error in `./hauto-test': double free or corruption 
 (fasttop): 0x00007f504c002a60 ***

 but gives me the following on every termination

 core.exception.InvalidMemoryOperationError src/core/exception.d(693): Invalid
memory operation

 How do I go about debugging and resolving these?

 -Øivind


I think that "Invalid Memory Operation" is the error the GC gives 
you when you try to allocate memory during the collection phase 
(i.e. inside a destructor). I suggest you avoid doing so.

I don't know if the double free problem is related to this. It 
may as well be a totally unrelated bug of some container you are 
using.
Nov 06 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 5:15 PM, Lodovico Giaretta wrote:

 On Sunday, 6 November 2016 at 21:46:52 UTC, Øivind wrote:

 Hi,

 My app occasionally gives me a

 *** Error in `./hauto-test': double free or corruption (fasttop):
 0x00007f504c002a60 ***

 but gives me the following on every termination

 core.exception.InvalidMemoryOperationError src/core/exception.d(693):
 Invalid memory operation

 How do I go about debugging and resolving these?

 -Øivind


 I think that "Invalid Memory Operation" is the error the GC gives you
 when you try to allocate memory during the collection phase (i.e. inside
 a destructor). I suggest you avoid doing so.

 I don't know if the double free problem is related to this. It may as
 well be a totally unrelated bug of some container you are using.


It absolutely could be related to this.

Imagine a resource wrapper like so:

class Foo
{
    int *mem;
    this() { mem = cast(int *)malloc(int.sizeof); }
    ~this() { .free(mem); }
}

Now, you have a problem if you do something like this:

class Bar
{
    Foo foo;
    ~this() { delete foo; }
}

Whenever Bar's dtor is called, it's going to throw the error when the 
delete call tries to free the block.

However, occasionally, the GC will have destroyed the foo instance 
BEFORE destroying the Bar instance that holds it. This will result in 
Foo's dtor being called twice for the same memory, resulting in the 
double-free call. Then of course, the memory free of the GC is not 
allowed, so we have the exception happening after.

OP: it's not legal to destroy or even access GC allocated members in a 
destructor. The GC may have already destroyed that data. I would 
recommend printing the stack trace when you get the exception, and 
figure out where the culprit is.

-Steve
Nov 06 2016
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:

 OP: it's not legal to destroy or even access GC allocated 
 members in a destructor. The GC may have already destroyed that 
 data. I would recommend printing the stack trace when you get 
 the exception, and figure out where the culprit is.


  Err.... that makes no sense... If that's the case why have a 
destructor at all?
Nov 06 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 10:57 PM, Era Scarecrow wrote:

 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:

 OP: it's not legal to destroy or even access GC allocated members in a
 destructor. The GC may have already destroyed that data. I would
 recommend printing the stack trace when you get the exception, and
 figure out where the culprit is.


  Err.... that makes no sense... If that's the case why have a destructor
 at all?


To free non-GC resources.

http://dlang.org/spec/class.html#destructors

"Furthermore, the order in which the garbage collector calls destructors 
for unreference objects is not specified. This means that when the 
garbage collector calls a destructor for an object of a class that has 
members that are references to garbage collected objects, those 
references may no longer be valid. This means that destructors cannot 
reference sub objects."

-Steve
Nov 07 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 03:27:32 UTC, Steven Schveighoffer 
wrote:

  Err.... that makes no sense... If that's the case why have a 
 destructor at all?


 To free non-GC resources.

 http://dlang.org/spec/class.html#destructors

 "Furthermore, the order in which the garbage collector calls 
 destructors for unreference objects is not specified. This 
 means that when the garbage collector calls a destructor for an 
 object of a class that has members that are references to 
 garbage collected objects, those references may no longer be 
 valid. This means that destructors cannot reference sub 
 objects."


  Hmmm.. I had the impression that if something was referenced by 
another object, then it couldn't be collected, so sub-objects 
shouldn't/couldn't be collected until the object holding them was 
dealt with (since it holds a reference).

  Although I suppose it's possible to rush in to the deepest 
levels and start collecting there first on objects presumed to be 
unneeded, but that just _feels_ wrong.
Nov 07 2016
parent reply thedeemon <dlang thedeemon.com> writes:
On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow wrote:


  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,


Another *live* object, i.e. reachable from globals and stack.
If you have a big tree and it becomes unreachable (you only had a 
pointer to its root and you nulled it), then this whole tree 
becomes garbage, and its nodes and leafs will be collected in 
unpredictable order, with destructors being run in unpredictable 
order, even when these dead nodes reference each other.
Nov 07 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 06:04:59 UTC, thedeemon wrote:

 On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow 
 wrote:


  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,


 Another *live* object, I.e. reachable from globals and stack. 
 If you have a big tree and it becomes unreachable (you only had 
 a pointer to its root and you nulled it), then this whole tree 
 becomes garbage, and its nodes and leafs will be collected in 
 unpredictable order, with destructors being run in 
 unpredictable order, even when these dead nodes reference each 
 other.


  And I can't help but hope it would start at the largest/base 
object and work it's way up. Or the largest object and then work 
it's way down. Alright...

  Shouldn't for warnings then in the destructor about accessing 
arrays, class objects or other allocated items that they might 
not even exist anymore?

  If I think of it like making a class/struct that does 
compression and the blob that manages tracking the compressed 
data uses simple array appending, and then the struct or class 
notices it's thrown away and it has an active connection to save 
it's contents to a file, as part of the destructor I'd probably 
write it so it would save and flush what data was left before 
deallocating everything or closing the file descriptors. With 
this you might have to manage your own memory to ensure the GC 
doesn't accidentally remove important data first...
Nov 07 2016
parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Tuesday, 8 November 2016 at 07:39:12 UTC, Era Scarecrow wrote:

 On Tuesday, 8 November 2016 at 06:04:59 UTC, thedeemon wrote:

 On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow 
 wrote:


  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,


 Another *live* object, I.e. reachable from globals and stack. 
 If you have a big tree and it becomes unreachable (you only 
 had a pointer to its root and you nulled it), then this whole 
 tree becomes garbage, and its nodes and leafs will be 
 collected in unpredictable order, with destructors being run 
 in unpredictable order, even when these dead nodes reference 
 each other.


  And I can't help but hope it would start at the largest/base 
 object and work it's way up. Or the largest object and then 
 work it's way down. Alright...


One of the reasons it is not specified is that very often the 
hierarchy is not a simple tree, but a graph with possibly many 
cycles. As a matter of fact, very often child nodes have pointers 
to parent nodes, so that what is logically a tree is practically 
a graph with lots of cycles. So it is not possible to identify a 
root object which does not have incoming dead pointers, and no 
guarantee can be provided.
Nov 08 2016
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:

 OP: it's not legal to destroy or even access GC allocated 
 members in a destructor. The GC may have already destroyed that 
 data.


Isn't destroy() fine there? It doesn't call destructors for 
already destroyed objects, so I guess it should be safe. 
(assuming the destructors don't do any allocations)
Nov 06 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 11:01 PM, thedeemon wrote:

 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:

 OP: it's not legal to destroy or even access GC allocated members in a
 destructor. The GC may have already destroyed that data.


 Isn't destroy() fine there? It doesn't call destructors for already
 destroyed objects, so I guess it should be safe. (assuming the
 destructors don't do any allocations)


The problem is that you don't know when the GC has destroyed that 
object. It may have been freed and already reallocated to something 
else. The time between when your object becomes garbage and when it's 
collected is not explicitly defined. Nor is the order of collection or 
if it will even be collected at all.

Another possibility is that your object destroys something that is 
pointed at by someone else. This is not a horrible condition, but could 
result in some unwanted segfaults.

Hence, just don't access members from the destructor. You'll be glad you 
didn't.

-Steve
Nov 07 2016
prev sibling parent reply bachmeier <no spam.net> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:

 Imagine a resource wrapper like so:

 class Foo
 {
    int *mem;
    this() { mem = cast(int *)malloc(int.sizeof); }
    ~this() { .free(mem); }
 }

 Now, you have a problem if you do something like this:

 class Bar
 {
    Foo foo;
    ~this() { delete foo; }
 }


Is there a valid use case for something like this? Why would you 
want to do anything inside ~this with GC memory?
Nov 08 2016
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 11:26:55 UTC, bachmeier wrote:

 Is there a valid use case for something like this? Why would 
 you want to do anything inside ~this with GC memory?


  If we assume it's a C++ attachment/library/object using 
different memory allocation?
Nov 08 2016
parent bachmeier <no spam.net> writes:
On Tuesday, 8 November 2016 at 11:53:37 UTC, Era Scarecrow wrote:

 On Tuesday, 8 November 2016 at 11:26:55 UTC, bachmeier wrote:

 Is there a valid use case for something like this? Why would 
 you want to do anything inside ~this with GC memory?


  If we assume it's a C++ attachment/library/object using 
 different memory allocation?


I'm not sure what "it's" is referring to in your statement. In 
the example, Foo wraps manually allocated memory that is freed 
when Foo goes out of scope. If the goal is to free the memory in 
Foo when Bar goes out of scope, that can be done without error by 
rewriting Foo. Therefore I'm not understanding how this situation 
can arise in real world code (I'm sure it can, though, which is 
why I'm asking).
Nov 08 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/8/16 6:26 AM, bachmeier wrote:

 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:

 Imagine a resource wrapper like so:

 class Foo
 {
    int *mem;
    this() { mem = cast(int *)malloc(int.sizeof); }
    ~this() { .free(mem); }
 }

 Now, you have a problem if you do something like this:

 class Bar
 {
    Foo foo;
    ~this() { delete foo; }
 }


 Is there a valid use case for something like this? Why would you want to
 do anything inside ~this with GC memory?


Indeed, you should not. I'm saying this type of error can explain the 
observed behavior.

The original post I responded to said "I don't know if the double free 
problem is related to this."

-Steve
Nov 08 2016
parent bachmeier <no spam.net> writes:
On Tuesday, 8 November 2016 at 14:55:53 UTC, Steven Schveighoffer 
wrote:


 Indeed, you should not. I'm saying this type of error can 
 explain the observed behavior.

 The original post I responded to said "I don't know if the 
 double free problem is related to this."

 -Steve


Okay. I thought maybe there was a gotcha that I wasn't aware of.
Nov 08 2016