www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Foreign threads in D code.

reply Igor Shirkalin <mathsoft inbox.ru> writes:
Hello!

I have written some D code that I need to link to :C++ huge 
project. Let it be just one function that uses GC. The question 
is: if C++ code creates several threads and runs this :D function 
simultaneously, will GC work correctly?

p.s. Of course the druntime is initialized before it.

Igor Shirkalin
Jul 10 2017
parent reply Biotronic <simen.kjaras gmail.com> writes:
On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:

 Hello!

 I have written some D code that I need to link to :C++ huge 
 project. Let it be just one function that uses GC. The question 
 is: if C++ code creates several threads and runs this :D 
 function simultaneously, will GC work correctly?

 p.s. Of course the druntime is initialized before it.

 Igor Shirkalin


If DRuntime is not made aware of the thread's existence, the 
thread will not be stopped by the GC, and the GC might collect 
memory that the thread is referencing on the stack or in non-GC 
memory. Anything allocated by the GC would still be scanned.

To inform DRuntime about your thread, you should call 
thread_attachThis:

https://dlang.org/phobos/core_thread.html#.thread_attachThis

As pointed out in the documentation of thread_attachThis, you 
might also want to call rt_moduleTlsCtor, to run thread local 
static constructors. Depending on your usage, this might not be 
necessary.

--
   Biotronic
Jul 10 2017
next sibling parent reply Igor Shirkalin <mathsoft inbox.ru> writes:
On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:

 On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:

 [...]


 If DRuntime is not made aware of the thread's existence, the 
 thread will not be stopped by the GC, and the GC might collect 
 memory that the thread is referencing on the stack or in non-GC 
 memory. Anything allocated by the GC would still be scanned.

 To inform DRuntime about your thread, you should call 
 thread_attachThis:

 https://dlang.org/phobos/core_thread.html#.thread_attachThis

 As pointed out in the documentation of thread_attachThis, you 
 might also want to call rt_moduleTlsCtor, to run thread local 
 static constructors. Depending on your usage, this might not be 
 necessary.

 --
   Biotronic


Thanks for very useful information!
Jul 11 2017
parent reply Guillaume Piolat <contact spam.com> writes:
On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:

 On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:

 On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:

 [...]


 If DRuntime is not made aware of the thread's existence, the 
 thread will not be stopped by the GC, and the GC might collect 
 memory that the thread is referencing on the stack or in 
 non-GC memory. Anything allocated by the GC would still be 
 scanned.

 To inform DRuntime about your thread, you should call 
 thread_attachThis:

 https://dlang.org/phobos/core_thread.html#.thread_attachThis

 As pointed out in the documentation of thread_attachThis, you 
 might also want to call rt_moduleTlsCtor, to run thread local 
 static constructors. Depending on your usage, this might not 
 be necessary.

 --
   Biotronic


 Thanks for very useful information!


Just one small note.
If you don't know the foreign thread lifetime, it's cleaner to 
detach it from the runtime upon exit.

Else may fall in the following scenario.
1. you register thread A
2. thread A is destroyed later on, in the C++ code
3. another thread B come into your callback and allocate. The GC 
triggers and try to pause a non-existing thread A.
Jul 12 2017
parent Igor Shirkalin <mathsoft inbox.ru> writes:
On Wednesday, 12 July 2017 at 09:49:32 UTC, Guillaume Piolat 
wrote:

 On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:

 [...]


 --
   Biotronic


 Thanks for very useful information!


 Just one small note.
 If you don't know the foreign thread lifetime, it's cleaner to 
 detach it from the runtime upon exit.

 Else may fall in the following scenario.
 1. you register thread A
 2. thread A is destroyed later on, in the C++ code
 3. another thread B come into your callback and allocate. The 
 GC triggers and try to pause a non-existing thread A.


This is important note. Yes, usually the lifetime of foreign 
thread is unknown.

You, guys, helped me a lot.
Jul 12 2017
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-11 08:18, Biotronic wrote:


 If DRuntime is not made aware of the thread's existence, the thread will 
 not be stopped by the GC, and the GC might collect memory that the 
 thread is referencing on the stack or in non-GC memory.


Are you sure? Wouldn't that make malloc or any other custom allocators 
completely useless and the D GC would completely break the C standard 
library because it could collect memory allocated by the C standard library?

 From "How Garbage Collection Works":

"5. Freeing all **GC** allocated memory that has no active pointers to 
it and do not need destructors to run" [1]. I added the emphasize on "GC".

 From "Interfacing Garbage Collected Objects With Foreign Code"

"If the only pointer to an object is held outside of these areas, then 
the collector will miss it and free the memory. To avoid this from 
happening, either
* reallocate and copy the object using the foreign code's storage 
allocator or using the C runtime library's malloc/free." [2].

[1] http://dlang.org/spec/garbage.html#how_gc_works
[2] http://dlang.org/spec/garbage.html#gc_foreign_obj

-- 
/Jacob Carlborg
Jul 12 2017
parent reply Biotronic <simen.kjaras gmail.com> writes:
On Wednesday, 12 July 2017 at 09:10:07 UTC, Jacob Carlborg wrote:

 On 2017-07-11 08:18, Biotronic wrote:


 If DRuntime is not made aware of the thread's existence, the 
 thread will not be stopped by the GC, and the GC might collect 
 memory that the thread is referencing on the stack or in 
 non-GC memory.


 Are you sure? Wouldn't that make malloc or any other custom 
 allocators completely useless and the D GC would completely 
 break the C standard library because it could collect memory 
 allocated by the C standard library?

 From "How Garbage Collection Works":

 "5. Freeing all **GC** allocated memory that has no active 
 pointers to it and do not need destructors to run" [1]. I added 
 the emphasize on "GC".

 From "Interfacing Garbage Collected Objects With Foreign Code"

 "If the only pointer to an object is held outside of these 
 areas, then the collector will miss it and free the memory. To 
 avoid this from happening, either
 * reallocate and copy the object using the foreign code's 
 storage allocator or using the C runtime library's 
 malloc/free." [2].

 [1] http://dlang.org/spec/garbage.html#how_gc_works
 [2] http://dlang.org/spec/garbage.html#gc_foreign_obj


That's basically what I tried to say - the GC may collect memory 
*it has allocated* if the only pointers to it are in memory the 
GC doesn't scan (i.e. on the stack of an unregistered thread or 
in memory not allocated via the GC).

It will not collect memory allocated by other means, but that 
Foo* you got from D and are using in C++ might point to a Bar 
soon after the GC runs.

--
   Biotronic
Jul 12 2017
parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-12 11:28, Biotronic wrote:


 That's basically what I tried to say 


It wasn't very clear to me at least.


 - the GC may collect memory *it has 
 allocated* if the only pointers to it are in memory the GC doesn't scan 
 (i.e. on the stack of an unregistered thread or in memory not allocated 
 via the GC).
 
 It will not collect memory allocated by other means, but that Foo* you 
 got from D and are using in C++ might point to a Bar soon after the GC 
 runs.


Yes, that can happen.

-- 
/Jacob Carlborg
Jul 12 2017
parent Biotronic <simen.kjaras gmail.com> writes:
On Wednesday, 12 July 2017 at 12:08:35 UTC, Jacob Carlborg wrote:

 On 2017-07-12 11:28, Biotronic wrote:


 That's basically what I tried to say


 It wasn't very clear to me at least.


Yeah, I see it in retrospect. "might collect memory that the 
thread is referencing on the stack or in non-GC memory" doesn't 
convey that the collected memory is only that which is allocated 
by the GC.

--
   Biotronic
Jul 12 2017