digitalmars.D.learn - GC collecting "too much"..
- simendsjo (5/5) Mar 25 2012 I'm doing some coding against a c library, and Ds GC keeps collecting c ...
- bearophile (10/13) Mar 25 2012 Three alternative solutions:
- simendsjo (25/36) Mar 26 2012 I've been able to find where the code fails, but now I don't understand ...
- simendsjo (3/42) Mar 26 2012 It seems threads created in the c library is totally unknown to D. How c...
- Timon Gehr (3/6) Mar 26 2012 You may be looking for this:
I'm doing some coding against a c library, and Ds GC keeps collecting c
owned objects (I think - disabling the GC makes everything work)
But how can I figure out what the GC is (read: I am) fucking up?
I have some to!string(c_struct_field) and format("%s", c_struct_field) and
field = c_fields[i] etc etc
Mar 25 2012
On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
Mar 25 2012
On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS lycos.com> wrote:On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:I've been able to find where the code fails, but now I don't understand what's happening at all. Is calling GC.collect() from an extern(C) function undefined? The following code just starts a mongoose web server and tries to run GC.collect in the handler. extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info) { GC.collect(); // segfault return null; } void main() { auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array(); mg_start(&cb, null, cast(const(char**))opts); GC.collect(); // no problem readln(); } If I collect memory from main(), it works as expected (removing the collect from cb of course) while(readln().chomp() != "q") { GC.collect(); } The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
Mar 26 2012
On Mon, 26 Mar 2012 10:13:35 +0200, simendsjo <simendsjo gmail.com> wrote:On Sun, 25 Mar 2012 22:18:02 +0200, bearophile <bearophileHUGS lycos.com> wrote:It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?On Sunday, 25 March 2012 at 19:15:05 UTC, simendsjo wrote:I've been able to find where the code fails, but now I don't understand what's happening at all. Is calling GC.collect() from an extern(C) function undefined? The following code just starts a mongoose web server and tries to run GC.collect in the handler. extern(C) void* cb(mg_event event, mg_connection* conn, mg_request_info* request_info) { GC.collect(); // segfault return null; } void main() { auto opts = ["listening_ports", "6969"].map!(toUTFz!(char*))().array(); mg_start(&cb, null, cast(const(char**))opts); GC.collect(); // no problem readln(); } If I collect memory from main(), it works as expected (removing the collect from cb of course) while(readln().chomp() != "q") { GC.collect(); } The documentation in core.memory is a bit sparse on how the GC works. Are there any articles on the D GC?I'm doing some coding against a c library, and Ds GC keeps collecting c owned objects (I think - disabling the GC makes everything work)Three alternative solutions: - Allocate from the C heap the memory that C will need to use, and free it manually or in a struct destructor (RAII) or with scope(exit). - Keep a pointer to the D-GC memory in the D code too. - In core.memory there are ways to disable scanning of a memory zone. Maybe it's usable for your purposes too. Bye, bearophile
Mar 26 2012
On 03/26/2012 11:55 AM, simendsjo wrote:It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Mar 26 2012
On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:On 03/26/2012 11:55 AM, simendsjo wrote:Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Mar 26 2012
On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo gmail.com> wrote:On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:Some info from gdb: [New Thread 0xf7cfcb70 (LWP 18614)] [New Thread 0xf74fbb70 (LWP 18615)] Program received signal SIGUSR1, User defined signal 1. 0xf7fdd430 in __kernel_vsyscall () (gdb) info threads Id Target Id Frame 3 Thread 0xf74fbb70 (LWP 18615) "main" 0xf7fdd430 in __kernel_vsyscall () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () * 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () (gdb) c Continuing. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 18615)] 0x080855b4 in gc.gcx.Gcx.mark() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 18615) "main" 0x080855b4 in gc.gcx.Gcx.mark() () 2 Thread 0xf7cfcb70 (LWP 18614) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 18611) "main" 0xf7fdd430 in __kernel_vsyscall () ------------ And if I change the code to: extern(C) handler() { GC.collect(); } [New Thread 0xf7cfcb70 (LWP 21454)] [New Thread 0xf74fbb70 (LWP 21455)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xf74fbb70 (LWP 21455)] 0x0808970e in core.thread.thread_suspendAll() () (gdb) info threads Id Target Id Frame * 3 Thread 0xf74fbb70 (LWP 21455) "main" 0x0808970e in core.thread.thread_suspendAll() () 2 Thread 0xf7cfcb70 (LWP 21454) "main" 0xf7fdd430 in __kernel_vsyscall () 1 Thread 0xf7dfd9c0 (LWP 21451) "main" 0xf7fdd430 in __kernel_vsyscall () ----- Any idea where I should go from here?On 03/26/2012 11:55 AM, simendsjo wrote:Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThis
Mar 29 2012
simendsjo <simendsjo gmail.com> On Thu, 29 Mar 2012 14:14:10 +0200, simendsjo <simendsjo gmail.com> wrote:On Mon, 26 Mar 2012 20:15:40 +0200, simendsjo <simendsjo gmail.com> wrote:I compiled with a debug build of druntime, and it fails on dereferencing a pointer, pbot in gcx.d:2511: void mark(void *pbot, void *ptop, int nRecurse) void **p1 = cast(void **)pbot; (...) for (; p1 < p2; p1++) { auto p = cast(byte *)(*p1); // segfault! (...) gdb) print p1 $10 = (void **) 0xf74fc000 (gdb) print *p1 $11 = (void *) 0x0 Some more dbg info: (gdb) bt ptop=0xffffd190, pbot=0xf74f4df4) at src/gc/gcx.d:2511 pbot=0xf74f4df4) at src/gc/gcx.d:2494 p2=0xffffd190, p1=0xf74f4df4, type=<incomplete type>) at src/core/thread.d:2743 src/core/thread.d:2698 src/core/thread.d:2746 stackTop=0xf74f4df4) at src/gc/gcx.d:2776 src/gc/gcx.d:2662 src/gc/gcx.d:1372 request_info=0x80ebec0) at main.d:54 Backtrace stopped: Not enough registers or memory available to unwind further (gdb) l 2511 2506 uint changes = 0; 2507 2508 //printf("marking range: %p -> %p\n", pbot, ptop); 2509 for (; p1 < p2; p1++) 2510 { 2511 auto p = cast(byte *)(*p1); 2512 2513 //if (log) debug(PRINTF) printf("\tmark %p\n", p); 2514 if (p >= minAddr && p < maxAddr) 2515 {On Mon, 26 Mar 2012 17:10:34 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:On 03/26/2012 11:55 AM, simendsjo wrote:Thanks, but I tried that too and couldn't get it to work. I added the following: extern(C) handler() { synchronized // needed here to avoid the GC to collect while attaching thread? { if(!Thread.getThis()) // thread unknown to D { thread_attachThis(); assert(Thread.getThis()); // now D knows about it } } GC.collect(); // still segfaults } Actually, using attachThis segfaults GC.collect() outside the thread handling code too.It seems threads created in the c library is totally unknown to D. How can I make D aware of these threads when there is no library support for it?You may be looking for this: http://dlang.org/phobos/core_thread.html#thread_attachThisAnd if I change the code to: extern(C) handler() { GC.collect(); }(snip) Checking with debug build of druntime, and I see it tries to suspend(thread_getThis()). This gives a null reference error as thread_getThis() returns null when it's not attached.
Mar 29 2012