www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - GC collecting "too much"..

reply simendsjo <simendsjo gmail.com> writes:
I'm doing some coding against a c library, and Ds GC keeps collecting c  
owned objects (I think - disabling the GC makes everything work)
But how can I figure out what the GC is (read: I am) fucking up?
I have some to!string(c_struct_field) and format("%s", c_struct_field) and  
field = c_fields[i] etc etc
Mar 25 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
 I'm doing some coding against a c library, and Ds GC keeps 
 collecting c owned objects (I think - disabling the GC makes 
 everything work)
Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
Mar 25 2012
parent reply simendsjo <simendsjo gmail.com> writes:
On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS lycos.com>  
wrote:

 On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
 I'm doing some coding against a c library, and Ds GC keeps collecting c  
 owned objects (I think - disabling the GC makes everything work)
Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
I've been able to find where the code fails, but now I don't understand what's happening at all. Is calling GC.collect() from an extern(C) function undefined? The following code just starts a mongoose web server and tries to run GC.collect in the handler. extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info) { GC.collect(); // segfault return null; } void main() { auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array(); mg_start(&cb, null, cast(const(char**))opts); GC.collect(); // no problem readln(); } If I collect memory from main(), it works as expected (removing the collect from cb of course) while(readln().chomp() != "q") { GC.collect(); } The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?
Mar 26 2012
parent reply simendsjo <simendsjo gmail.com> writes:
On Mon, 26 Mar 2012 10:13:35 +0200, simendsjo <simendsjo gmail.com> wrote:

 On Sun, 25 Mar 2012 22:18:02 +0200, bearophile  
 <bearophileHUGS lycos.com> wrote:

 On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:
 I'm doing some coding against a c library, and Ds GC keeps collecting  
 c owned objects (I think - disabling the GC makes everything work)
Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
I've been able to find where the code fails, but now I don't understand what's happening at all. Is calling GC.collect() from an extern(C) function undefined? The following code just starts a mongoose web server and tries to run GC.collect in the handler. extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info) { GC.collect(); // segfault return null; } void main() { auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array(); mg_start(&cb, null, cast(const(char**))opts); GC.collect(); // no problem readln(); } If I collect memory from main(), it works as expected (removing the collect from cb of course) while(readln().chomp() != "q") { GC.collect(); } The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?
It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?
Mar 26 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/26/2012 11:55 AM, simendsjo wrote:
 It seems threads created in the c library is totally unknown to D. How
 can I make D aware of these threads when there is no library support for
 it?
You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Mar 26 2012
parent reply simendsjo <simendsjo gmail.com> writes:
On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/26/2012 11:55 AM, simendsjo wrote:
 It seems threads created in the c library is totally unknown to D. How
 can I make D aware of these threads when there is no library support for
 it?
You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.
Mar 26 2012
parent reply simendsjo <simendsjo gmail.com> writes:
On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo gmail.com> wrote:

 On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/26/2012 11:55 AM, simendsjo wrote:
 It seems threads created in the c library is totally unknown to D. How
 can I make D aware of these threads when there is no library support  
 for
 it?
You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.
Some info from gdb: [New Thread 0xf7cfcb70 (LWP 18614)] [New Thread 0xf74fbb70 (LWP 18615)] Program received signal SIGUSR1, User defined signal 1. 0xf7fdd430 in __kernel_vsyscall () (gdb) info threads Id Target Id Frame 3 Thread 0xf74fbb70 (LWP 18615) "main" 0xf7fdd430 in __kernel_vsyscall () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () * 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 18615)] 0x080855b4 in gc.gcx.Gcx.mark() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 18615) "main" 0x080855b4 in gc.gcx.Gcx.mark() () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () ------------ And if I change the code to: extern(C) handler() { GC.collect(); } [New Thread 0xf7cfcb70 (LWP 21454)] [New Thread 0xf74fbb70 (LWP 21455)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 21455)] 0x0808970e in core.thread.thread_suspendAll() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 21455) "main" 0x0808970e in core.thread.thread_suspendAll() () 2 Thread 0xf7cfcb70 (LWP 21454) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 21451) "main" 0xf7fdd430 in __kernel_vsyscall () ----- Any idea where I should go from here?
Mar 29 2012
parent simendsjo <simendsjo gmail.com> writes:
On Thu, 29 Mar 2012 14:14:10 +0200, simendsjo <simendsjo gmail.com> wrote:

 On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo gmail.com>  
 wrote:

 On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 03/26/2012 11:55 AM, simendsjo wrote:
 It seems threads created in the c library is totally unknown to D. How
 can I make D aware of these threads when there is no library support  
 for
 it?
You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.
I compiled with a debug build of druntime, and it fails on dereferencing a pointer, pbot in gcx.d:2511: void mark(void *pbot, void *ptop, int nRecurse) void **p1 = cast(void **)pbot; (...) for (; p1 < p2; p1++) { auto p = cast(byte *)(*p1); // segfault! (...) gdb) print p1 $10 = (void **) 0xf74fc000 (gdb) print *p1 $11 = (void *) 0x0 Some more dbg info: (gdb) bt ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2511 pbot=0xf74f4df4) at src/gc/gcx.d:2494 p2=0xffffd190, p1=0xf74f4df4, type=<incomplete type>) at src/core/thread.d:2743 src/core/thread.d:2698 src/core/thread.d:2746 stackTop=0xf74f4df4) at src/gc/gcx.d:2776 src/gc/gcx.d:2662 src/gc/gcx.d:1372 request_info=0x80ebec0) at main.d:54 Backtrace stopped: Not enough registers or memory available to unwind further (gdb) l 2511 2506 uint changes = 0; 2507 2508 //printf("marking range: %p -> %p\n", pbot, ptop); 2509 for (; p1 < p2; p1++) 2510 { 2511 auto p = cast(byte *)(*p1); 2512 2513 //if (log) debug(PRINTF) printf("\tmark %p\n", p); 2514 if (p >= minAddr && p < maxAddr) 2515 {
 And if I change the code to:
 extern(C) handler()
 {
      GC.collect();
 }
(snip) Checking with debug build of druntime, and I see it tries to suspend(thread_getThis()). This gives a null reference error as thread_getThis() returns null when it's not attached.
Mar 29 2012