www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - using GC needs particular skills?

reply Alexandr Druzhinin <drug2004 bk.ru> writes:
I make simple learning project and encounter the problem that when my 
application exits I get access violations (yes, I use windows and 
moreover - XP version :) ). I don't know the reason of errors, I'm sure 
that exceptions are thrown by runtime, not my code directly - its occurs 
right behind/after my code execution. Where can I learn about using of 
GC ( I think my code don't support gc or how to say it )? Or may be it 
is a common case, I'm not alone and there is some simple and clear solution?

Thanks in advance
Jul 15 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 15-07-2012 17:31, Alexandr Druzhinin wrote:

 I make simple learning project and encounter the problem that when my
 application exits I get access violations (yes, I use windows and
 moreover - XP version :) ). I don't know the reason of errors, I'm sure
 that exceptions are thrown by runtime, not my code directly - its occurs
 right behind/after my code execution. Where can I learn about using of
 GC ( I think my code don't support gc or how to say it )? Or may be it
 is a common case, I'm not alone and there is some simple and clear
 solution?

 Thanks in advance


It's really, really hard to say what's wrong from this information. You 
need to post some reduced test case demonstrating the problem.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Jul 15 2012
next sibling parent Alexandr Druzhinin <drug2004 bk.ru> writes:
15.07.2012 22:33, Alex Rønne Petersen пишет:

 It's really, really hard to say what's wrong from this information. You
 need to post some reduced test case demonstrating the problem.


You're right, it's really hard. But I'd like to learn about using GC only.
Of course I'll try to reduce my project to get simple test demonstrating 
the problem, but it'll take the time. Although may be I'll solve the 
problem by myself trying to reduce my project, as it often happens. :)
Jul 15 2012
prev sibling parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
15.07.2012 22:33, Alex Rønne Petersen пишет:
test case:

class A {
}
__gshared A a;

  void main(string[] args) {	
  	a = new A;
}

every time after finishing application I get 
core.exception.InvalidMemoryOperationError
I suspect the problem is misusing __gshared
Jul 15 2012
next sibling parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
15.07.2012 22:56, Alexandr Druzhinin пишет:

 15.07.2012 22:33, Alex Rønne Petersen пишет:
 test case:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }

 every time after finishing application I get
 core.exception.InvalidMemoryOperationError
 I suspect the problem is misusing __gshared


sorry for my hurry - I've localized the problem in the linked libraries, 
not in my code. Will find further... what the h*ll
Sorry again
Jul 15 2012
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 7/16/2012 1:01 AM, Alexandr Druzhinin wrote:

 15.07.2012 22:56, Alexandr Druzhinin пишет:

 15.07.2012 22:33, Alex Rønne Petersen пишет:
 test case:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }

 every time after finishing application I get
 core.exception.InvalidMemoryOperationError
 I suspect the problem is misusing __gshared


 sorry for my hurry - I've localized the problem in the linked libraries,
 not in my code. Will find further... what the h*ll
 Sorry again


Which libraries are you linking with?
Jul 15 2012
parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
16.07.2012 9:46, Mike Parker пишет:

 On 7/16/2012 1:01 AM, Alexandr Druzhinin wrote:

 15.07.2012 22:56, Alexandr Druzhinin пишет:

 15.07.2012 22:33, Alex Rønne Petersen пишет:
 test case:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }

 every time after finishing application I get
 core.exception.InvalidMemoryOperationError
 I suspect the problem is misusing __gshared


 sorry for my hurry - I've localized the problem in the linked libraries,
 not in my code. Will find further... what the h*ll
 Sorry again


 Which libraries are you linking with?


The reason were bindings to GeographicLib C++ library written in 
analogue to Derelict bindings (I used it in my projects too).
Now I'm trying to make simple test cases for my trouble.
Jul 16 2012
parent reply Mike Parker <aldacron gmail.com> writes:
On 7/17/2012 1:27 AM, Alexandr Druzhinin wrote:

 16.07.2012 9:46, Mike Parker пишет:

 On 7/16/2012 1:01 AM, Alexandr Druzhinin wrote:

 15.07.2012 22:56, Alexandr Druzhinin пишет:

 15.07.2012 22:33, Alex Rønne Petersen пишет:
 test case:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }

 every time after finishing application I get
 core.exception.InvalidMemoryOperationError
 I suspect the problem is misusing __gshared


 sorry for my hurry - I've localized the problem in the linked libraries,
 not in my code. Will find further... what the h*ll
 Sorry again


 Which libraries are you linking with?


 The reason were bindings to GeographicLib C++ library written in
 analogue to Derelict bindings (I used it in my projects too).
 Now I'm trying to make simple test cases for my trouble.


Bindings based on Derelict will release the shared libraries in a static 
module destructor. So if you are calling any bound functions from inside 
class destructors and letting your objects be cleaned up by the GC, then 
you are guaranteed to get a segfault at exit.

That's usually the cause of the problem you're seeing. And if that is 
indeed the root of your problem, you should never rely on class 
destructors to clean up system resources. You cannot control when they 
will be called.
Jul 17 2012
parent reply Alexandr Druzhinin <drug2004 bk.ru> writes:
17.07.2012 18:34, Mike Parker пишет:

 On 7/17/2012 1:27 AM, Alexandr Druzhinin wrote:

 The reason were bindings to GeographicLib C++ library written in
 analogue to Derelict bindings (I used it in my projects too).
 Now I'm trying to make simple test cases for my trouble.


 Bindings based on Derelict will release the shared libraries in a static
 module destructor. So if you are calling any bound functions from inside
 class destructors and letting your objects be cleaned up by the GC, then
 you are guaranteed to get a segfault at exit.

 That's usually the cause of the problem you're seeing. And if that is
 indeed the root of your problem, you should never rely on class
 destructors to clean up system resources. You cannot control when they
 will be called.


I think you hit the problem. When I removed resources releasing from 
destructors in my bindings the error disappeared too. But frankly I 
didn't understand why it happened :( and how I should free resources 
now. What is I can rely on to clean up system resources?
Definitly I need read the TDPL again...)
Is it because GC may mess calling object destructors and static module 
destructors? Don't GC make difference between them and don't call their 
desctructors in predefined order (module destructors before other 
destructors for example or vice versa)?
Jul 17 2012
parent reply Mike Parker <aldacron gmail.com> writes:
On 7/18/2012 12:45 AM, Alexandr Druzhinin wrote:

 17.07.2012 18:34, Mike Parker пишет:

 On 7/17/2012 1:27 AM, Alexandr Druzhinin wrote:

 The reason were bindings to GeographicLib C++ library written in
 analogue to Derelict bindings (I used it in my projects too).
 Now I'm trying to make simple test cases for my trouble.


 Bindings based on Derelict will release the shared libraries in a static
 module destructor. So if you are calling any bound functions from inside
 class destructors and letting your objects be cleaned up by the GC, then
 you are guaranteed to get a segfault at exit.

 That's usually the cause of the problem you're seeing. And if that is
 indeed the root of your problem, you should never rely on class
 destructors to clean up system resources. You cannot control when they
 will be called.


 I think you hit the problem. When I removed resources releasing from
 destructors in my bindings the error disappeared too. But frankly I
 didn't understand why it happened :( and how I should free resources
 now. What is I can rely on to clean up system resources?
 Definitly I need read the TDPL again...)
 Is it because GC may mess calling object destructors and static module
 destructors? Don't GC make difference between them and don't call their
 desctructors in predefined order (module destructors before other
 destructors for example or vice versa)?


Destructors are unreliable. There is no guarantee that a destructor will 
be called before the garbage collector is terminated. When the program 
exits, the runtime will call gc_term which will then call destructors on 
any objects that haven't yet been cleaned up. But the order in which 
those destructors are called is unpredictable. This is a recipe for all 
sorts of problems.

Static class destructors and module destructors are more reliable in 
that you know they will be called in a particular order. But, they are 
called before the gc is terminated.

Your particular problem is this. Derelict-style bindings load shared 
libraries dynamically via system calls. That means that every bound 
function is actually a function pointer. The shared library is then 
unloaded in a static module destructor. When DRuntime exits, it calls 
all the module destructors *before* calling gc_term. So what's happening is:

1. The module destructors are run
2. Derelict unloads the shared library, thereby causing all of the 
function pointers into that library to become invalid.
3. gc_term is run
4. The destructor of one of your objects is called and it tries to call 
a function from the Derelict binding, but since that function pointer is 
no longer valid, you get a segfault.

When cleaning up resources in D, you should generally not rely on class 
destructors to do so. You'll want to include some sort of process to 
clean up everything yourself. What I tend to do is something like this:

========
void term()
{
     // initiate cleanup here
}

void main()
{
     scope(exit) term();
     init();
     run();
}
========

The scope(exit) will ensure that the cleanup is run regardless of how 
the program exits. Every subsystem in my program will have term() 
function or method that substitutes for a destructor. This works fine 
and I have no problems with it.

Of course, you can still use destructors for scoped object instances in 
cases where you want RAII inside a particular scope.
Jul 17 2012
parent Alexandr Druzhinin <drug2004 bk.ru> writes:
18.07.2012 8:00, Mike Parker пишет:

 Destructors are unreliable. There is no guarantee that a destructor will
 be called before the garbage collector is terminated. When the program
 exits, the runtime will call gc_term which will then call destructors on
 any objects that haven't yet been cleaned up. But the order in which
 those destructors are called is unpredictable. This is a recipe for all
 sorts of problems.

 Static class destructors and module destructors are more reliable in
 that you know they will be called in a particular order. But, they are
 called before the gc is terminated.

 Your particular problem is this. Derelict-style bindings load shared
 libraries dynamically via system calls. That means that every bound
 function is actually a function pointer. The shared library is then
 unloaded in a static module destructor. When DRuntime exits, it calls
 all the module destructors *before* calling gc_term. So what's happening
 is:

 1. The module destructors are run
 2. Derelict unloads the shared library, thereby causing all of the
 function pointers into that library to become invalid.
 3. gc_term is run
 4. The destructor of one of your objects is called and it tries to call
 a function from the Derelict binding, but since that function pointer is
 no longer valid, you get a segfault.

 When cleaning up resources in D, you should generally not rely on class
 destructors to do so. You'll want to include some sort of process to
 clean up everything yourself. What I tend to do is something like this:

 ========
 void term()
 {
      // initiate cleanup here
 }

 void main()
 {
      scope(exit) term();
      init();
      run();
 }
 ========

 The scope(exit) will ensure that the cleanup is run regardless of how
 the program exits. Every subsystem in my program will have term()
 function or method that substitutes for a destructor. This works fine
 and I have no problems with it.

 Of course, you can still use destructors for scoped object instances in
 cases where you want RAII inside a particular scope.


Thank you very much for your help! Now I undestand it - I've never used 
gc before.
Jul 18 2012
prev sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/15/2012 09:01 AM, Alexandr Druzhinin wrote:

 15.07.2012 22:56, Alexandr Druzhinin пишет:

 15.07.2012 22:33, Alex Rønne Petersen пишет:
 test case:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }

 every time after finishing application I get
 core.exception.InvalidMemoryOperationError
 I suspect the problem is misusing __gshared


 sorry for my hurry - I've localized the problem in the linked libraries,
 not in my code. Will find further... what the h*ll
 Sorry again



I experienced this behavior when I was banging on pyd. Culprit was using 
gc allocated references inside a class destructor, using closures inside 
a class destructor, or asserting or throwing anything inside a class 
destructor.
Jul 15 2012
parent Alexandr Druzhinin <drug2004 bk.ru> writes:
16.07.2012 10:29, Ellery Newcomer пишет:

 On 07/15/2012 09:01 AM, Alexandr Druzhinin wrote:

 15.07.2012 22:56, Alexandr Druzhinin пишет:

 sorry for my hurry - I've localized the problem in the linked libraries,
 not in my code. Will find further... what the h*ll
 Sorry again



 I experienced this behavior when I was banging on pyd. Culprit was using
 gc allocated references inside a class destructor, using closures inside
 a class destructor, or asserting or throwing anything inside a class
 destructor.


does it means I should not use destructor for resource releasing at all 
while using GC for instatiating class object?
Jul 17 2012
prev sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 15-07-2012 17:56, Alexandr Druzhinin wrote:

 class A {
 }
 __gshared A a;

   void main(string[] args) {
       a = new A;
 }


Seems to work for me? (This use of __gshared is perfectly fine, btw.)

alexrp alexrp ~/Projects/tests $ cat test.d
class A {
}
__gshared A a;

  void main(string[] args) {
      a = new A;
}
alexrp alexrp ~/Projects/tests $ rdmd test.d
alexrp alexrp ~/Projects/tests $

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Jul 15 2012