www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to debug memory errors

reply =?UTF-8?B?w5hpdmluZA==?= <oivind.loe gmail.com> writes:
Hi,

My app occasionally gives me a

*** Error in `./hauto-test': double free or corruption (fasttop): 
0x00007f504c002a60 ***

but gives me the following on every termination

core.exception.InvalidMemoryOperationError src/core/exception.d(693): Invalid
memory operation

How do I go about debugging and resolving these?

-Øivind
Nov 06 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Sunday, 6 November 2016 at 21:46:52 UTC, Øivind wrote:
 Hi,

 My app occasionally gives me a

 *** Error in `./hauto-test': double free or corruption 
 (fasttop): 0x00007f504c002a60 ***

 but gives me the following on every termination

 core.exception.InvalidMemoryOperationError src/core/exception.d(693): Invalid
memory operation

 How do I go about debugging and resolving these?

 -Øivind
I think that "Invalid Memory Operation" is the error the GC gives you when you try to allocate memory during the collection phase (i.e. inside a destructor). I suggest you avoid doing so. I don't know if the double free problem is related to this. It may as well be a totally unrelated bug of some container you are using.
Nov 06 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 5:15 PM, Lodovico Giaretta wrote:
 On Sunday, 6 November 2016 at 21:46:52 UTC, Øivind wrote:
 Hi,

 My app occasionally gives me a

 *** Error in `./hauto-test': double free or corruption (fasttop):
 0x00007f504c002a60 ***

 but gives me the following on every termination

 core.exception.InvalidMemoryOperationError src/core/exception.d(693):
 Invalid memory operation

 How do I go about debugging and resolving these?

 -Øivind
I think that "Invalid Memory Operation" is the error the GC gives you when you try to allocate memory during the collection phase (i.e. inside a destructor). I suggest you avoid doing so. I don't know if the double free problem is related to this. It may as well be a totally unrelated bug of some container you are using.
It absolutely could be related to this. Imagine a resource wrapper like so: class Foo { int *mem; this() { mem = cast(int *)malloc(int.sizeof); } ~this() { .free(mem); } } Now, you have a problem if you do something like this: class Bar { Foo foo; ~this() { delete foo; } } Whenever Bar's dtor is called, it's going to throw the error when the delete call tries to free the block. However, occasionally, the GC will have destroyed the foo instance BEFORE destroying the Bar instance that holds it. This will result in Foo's dtor being called twice for the same memory, resulting in the double-free call. Then of course, the memory free of the GC is not allowed, so we have the exception happening after. OP: it's not legal to destroy or even access GC allocated members in a destructor. The GC may have already destroyed that data. I would recommend printing the stack trace when you get the exception, and figure out where the culprit is. -Steve
Nov 06 2016
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:
 OP: it's not legal to destroy or even access GC allocated 
 members in a destructor. The GC may have already destroyed that 
 data. I would recommend printing the stack trace when you get 
 the exception, and figure out where the culprit is.
Err.... that makes no sense... If that's the case why have a destructor at all?
Nov 06 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 10:57 PM, Era Scarecrow wrote:
 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:
 OP: it's not legal to destroy or even access GC allocated members in a
 destructor. The GC may have already destroyed that data. I would
 recommend printing the stack trace when you get the exception, and
 figure out where the culprit is.
Err.... that makes no sense... If that's the case why have a destructor at all?
To free non-GC resources. http://dlang.org/spec/class.html#destructors "Furthermore, the order in which the garbage collector calls destructors for unreference objects is not specified. This means that when the garbage collector calls a destructor for an object of a class that has members that are references to garbage collected objects, those references may no longer be valid. This means that destructors cannot reference sub objects." -Steve
Nov 07 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 03:27:32 UTC, Steven Schveighoffer 
wrote:
  Err.... that makes no sense... If that's the case why have a 
 destructor at all?
To free non-GC resources. http://dlang.org/spec/class.html#destructors "Furthermore, the order in which the garbage collector calls destructors for unreference objects is not specified. This means that when the garbage collector calls a destructor for an object of a class that has members that are references to garbage collected objects, those references may no longer be valid. This means that destructors cannot reference sub objects."
Hmmm.. I had the impression that if something was referenced by another object, then it couldn't be collected, so sub-objects shouldn't/couldn't be collected until the object holding them was dealt with (since it holds a reference). Although I suppose it's possible to rush in to the deepest levels and start collecting there first on objects presumed to be unneeded, but that just _feels_ wrong.
Nov 07 2016
parent reply thedeemon <dlang thedeemon.com> writes:
On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow wrote:

  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,
Another *live* object, i.e. reachable from globals and stack. If you have a big tree and it becomes unreachable (you only had a pointer to its root and you nulled it), then this whole tree becomes garbage, and its nodes and leafs will be collected in unpredictable order, with destructors being run in unpredictable order, even when these dead nodes reference each other.
Nov 07 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 06:04:59 UTC, thedeemon wrote:
 On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow 
 wrote:

  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,
Another *live* object, I.e. reachable from globals and stack. If you have a big tree and it becomes unreachable (you only had a pointer to its root and you nulled it), then this whole tree becomes garbage, and its nodes and leafs will be collected in unpredictable order, with destructors being run in unpredictable order, even when these dead nodes reference each other.
And I can't help but hope it would start at the largest/base object and work it's way up. Or the largest object and then work it's way down. Alright... Shouldn't for warnings then in the destructor about accessing arrays, class objects or other allocated items that they might not even exist anymore? If I think of it like making a class/struct that does compression and the blob that manages tracking the compressed data uses simple array appending, and then the struct or class notices it's thrown away and it has an active connection to save it's contents to a file, as part of the destructor I'd probably write it so it would save and flush what data was left before deallocating everything or closing the file descriptors. With this you might have to manage your own memory to ensure the GC doesn't accidentally remove important data first...
Nov 07 2016
parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Tuesday, 8 November 2016 at 07:39:12 UTC, Era Scarecrow wrote:
 On Tuesday, 8 November 2016 at 06:04:59 UTC, thedeemon wrote:
 On Tuesday, 8 November 2016 at 05:36:22 UTC, Era Scarecrow 
 wrote:

  Hmmm.. I had the impression that if something was referenced 
 by another object, then it couldn't be collected,
Another *live* object, I.e. reachable from globals and stack. If you have a big tree and it becomes unreachable (you only had a pointer to its root and you nulled it), then this whole tree becomes garbage, and its nodes and leafs will be collected in unpredictable order, with destructors being run in unpredictable order, even when these dead nodes reference each other.
And I can't help but hope it would start at the largest/base object and work it's way up. Or the largest object and then work it's way down. Alright...
One of the reasons it is not specified is that very often the hierarchy is not a simple tree, but a graph with possibly many cycles. As a matter of fact, very often child nodes have pointers to parent nodes, so that what is logically a tree is practically a graph with lots of cycles. So it is not possible to identify a root object which does not have incoming dead pointers, and no guarantee can be provided.
Nov 08 2016
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:
 OP: it's not legal to destroy or even access GC allocated 
 members in a destructor. The GC may have already destroyed that 
 data.
Isn't destroy() fine there? It doesn't call destructors for already destroyed objects, so I guess it should be safe. (assuming the destructors don't do any allocations)
Nov 06 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/6/16 11:01 PM, thedeemon wrote:
 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:
 OP: it's not legal to destroy or even access GC allocated members in a
 destructor. The GC may have already destroyed that data.
Isn't destroy() fine there? It doesn't call destructors for already destroyed objects, so I guess it should be safe. (assuming the destructors don't do any allocations)
The problem is that you don't know when the GC has destroyed that object. It may have been freed and already reallocated to something else. The time between when your object becomes garbage and when it's collected is not explicitly defined. Nor is the order of collection or if it will even be collected at all. Another possibility is that your object destroys something that is pointed at by someone else. This is not a horrible condition, but could result in some unwanted segfaults. Hence, just don't access members from the destructor. You'll be glad you didn't. -Steve
Nov 07 2016
prev sibling parent reply bachmeier <no spam.net> writes:
On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer 
wrote:
 Imagine a resource wrapper like so:

 class Foo
 {
    int *mem;
    this() { mem = cast(int *)malloc(int.sizeof); }
    ~this() { .free(mem); }
 }

 Now, you have a problem if you do something like this:

 class Bar
 {
    Foo foo;
    ~this() { delete foo; }
 }
Is there a valid use case for something like this? Why would you want to do anything inside ~this with GC memory?
Nov 08 2016
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 8 November 2016 at 11:26:55 UTC, bachmeier wrote:
 Is there a valid use case for something like this? Why would 
 you want to do anything inside ~this with GC memory?
If we assume it's a C++ attachment/library/object using different memory allocation?
Nov 08 2016
parent bachmeier <no spam.net> writes:
On Tuesday, 8 November 2016 at 11:53:37 UTC, Era Scarecrow wrote:
 On Tuesday, 8 November 2016 at 11:26:55 UTC, bachmeier wrote:
 Is there a valid use case for something like this? Why would 
 you want to do anything inside ~this with GC memory?
If we assume it's a C++ attachment/library/object using different memory allocation?
I'm not sure what "it's" is referring to in your statement. In the example, Foo wraps manually allocated memory that is freed when Foo goes out of scope. If the goal is to free the memory in Foo when Bar goes out of scope, that can be done without error by rewriting Foo. Therefore I'm not understanding how this situation can arise in real world code (I'm sure it can, though, which is why I'm asking).
Nov 08 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/8/16 6:26 AM, bachmeier wrote:
 On Monday, 7 November 2016 at 02:22:35 UTC, Steven Schveighoffer wrote:
 Imagine a resource wrapper like so:

 class Foo
 {
    int *mem;
    this() { mem = cast(int *)malloc(int.sizeof); }
    ~this() { .free(mem); }
 }

 Now, you have a problem if you do something like this:

 class Bar
 {
    Foo foo;
    ~this() { delete foo; }
 }
Is there a valid use case for something like this? Why would you want to do anything inside ~this with GC memory?
Indeed, you should not. I'm saying this type of error can explain the observed behavior. The original post I responded to said "I don't know if the double free problem is related to this." -Steve
Nov 08 2016
parent bachmeier <no spam.net> writes:
On Tuesday, 8 November 2016 at 14:55:53 UTC, Steven Schveighoffer 
wrote:

 Indeed, you should not. I'm saying this type of error can 
 explain the observed behavior.

 The original post I responded to said "I don't know if the 
 double free problem is related to this."

 -Steve
Okay. I thought maybe there was a gotcha that I wasn't aware of.
Nov 08 2016