www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Merge pull request #593 from dawgfoto/dynamicLoading

reply Martin Nowak <code dawg.eu> writes:
We just merged the low-level implementation for full shared library 
support. There should be an article or some tutorial but I simply lack 
the time for that. So if you want to experiment with it you should read 
this and have a look at the test cases 
(https://github.com/D-Programming-Language/druntime/tree/master/test/shared).
I'd be thankful for every bug report, please add the keyword dll 
(http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---).


Just a repost of the github comment 
(https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789).

I updated and finished the low-level implementation.
This now supports almost all shared library scenarios merely automatically.
It is also tested pretty thoroughly.
Here are the main gotchas for experimenting.

There is an extra layer of thread local reference accounting over the 
global one in the runtime linker which has some implications on how to 
use shared libraries.

Loading libraries should only be done through 
Runtime.loadLibrary/Runtime.unloadLibrary
(or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local 
accounting doesn't
miss anything.
The recommended pattern to load a D library from a pure C program is to 
dlopen
phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter.

Loading a library only initializes it and it's linked dependencies for 
the current
thread.

Threads where a library is not initialize will not see the library's 
modules, nor the EH
tables and the GC will miss to scan the library's TLS memory.

A spawned thread (core.thread.Thread) will initialize the same libraries 
as it's parent
thread. There is an implicit dlopen call for every dynamically loaded 
library so to
prevent the parent thread from unloading. When a thread finishes it will 
drop/unload all
remaining references to loaded libraries.
The high-level implementation of the shared libraries support will 
provide a method to
iterate over all loaded libraries, thus it will be possible to 
explicitly unload unused
inherited libraries.

Finding the dependencies of a library is somewhat expensive so it is 
only performed for D libraries and non-recursively.

Because all dependencies are loaded before the library itself we do know 
the recursive dependencies. In case of loading a C library that depends 
on D libraries or for D->C->D dependency chains we will miss the 
recursive dependency.
This might possibly lead to uninitialized threads when loading such a 
library in more than one thread.
I will try to fix any arising bugs ASAP but I don't have much time left 
currently.
Please tag bugzilla entries with the dll keyword.

Work on the high-level interface will start around the end of next week.
Sep 07 2013
next sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 07.09.2013 19:09, schrieb Martin Nowak:
 We just merged the low-level implementation for full shared library
 support. There should be an article or some tutorial but I simply lack
 the time for that. So if you want to experiment with it you should read
 this and have a look at the test cases
 (https://github.com/D-Programming-Language/druntime/tree/master/test/shared).

 I'd be thankful for every bug report, please add the keyword dll
 (http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---).


 Just a repost of the github comment
 (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789).


 I updated and finished the low-level implementation.
 This now supports almost all shared library scenarios merely automatically.
 It is also tested pretty thoroughly.
 Here are the main gotchas for experimenting.

 There is an extra layer of thread local reference accounting over the
 global one in the runtime linker which has some implications on how to
 use shared libraries.

 Loading libraries should only be done through
 Runtime.loadLibrary/Runtime.unloadLibrary
 (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local
 accounting doesn't
 miss anything.
 The recommended pattern to load a D library from a pure C program is to
 dlopen
 phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter.

 Loading a library only initializes it and it's linked dependencies for
 the current
 thread.

 Threads where a library is not initialize will not see the library's
 modules, nor the EH
 tables and the GC will miss to scan the library's TLS memory.

 A spawned thread (core.thread.Thread) will initialize the same libraries
 as it's parent
 thread. There is an implicit dlopen call for every dynamically loaded
 library so to
 prevent the parent thread from unloading. When a thread finishes it will
 drop/unload all
 remaining references to loaded libraries.
 The high-level implementation of the shared libraries support will
 provide a method to
 iterate over all loaded libraries, thus it will be possible to
 explicitly unload unused
 inherited libraries.

 Finding the dependencies of a library is somewhat expensive so it is
 only performed for D libraries and non-recursively.

 Because all dependencies are loaded before the library itself we do know
 the recursive dependencies. In case of loading a C library that depends
 on D libraries or for D->C->D dependency chains we will miss the
 recursive dependency.
 This might possibly lead to uninitialized threads when loading such a
 library in more than one thread.
 I will try to fix any arising bugs ASAP but I don't have much time left
 currently.
 Please tag bugzilla entries with the dll keyword.

 Work on the high-level interface will start around the end of next week.
Which plattforms does this work on?
Sep 07 2013
parent Martin Nowak <code dawg.eu> writes:
On 09/07/2013 07:15 PM, Benjamin Thaut wrote:
 Which plattforms does this work on?
Linux only for now
Sep 07 2013
prev sibling next sibling parent "Zhouxuan" <pycerl qq.com> writes:
On Saturday, 7 September 2013 at 17:09:17 UTC, Martin Nowak wrote:
 We just merged the low-level implementation for full shared 
 library support. There should be an article or some tutorial 
 but I simply lack the time for that. So if you want to 
 experiment with it you should read this and have a look at the 
 test cases 
 (https://github.com/D-Programming-Language/druntime/tree/master/test/shared).
 I'd be thankful for every bug report, please add the keyword 
 dll 
 (http://d.puremagic.com/issues/buglist.cgi?keywords=dll&resolution=---).


 Just a repost of the github comment 
 (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789).

 I updated and finished the low-level implementation.
 This now supports almost all shared library scenarios merely 
 automatically.
 It is also tested pretty thoroughly.
 Here are the main gotchas for experimenting.

 There is an extra layer of thread local reference accounting 
 over the global one in the runtime linker which has some 
 implications on how to use shared libraries.

 Loading libraries should only be done through 
 Runtime.loadLibrary/Runtime.unloadLibrary
 (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread 
 local accounting doesn't
 miss anything.
 The recommended pattern to load a D library from a pure C 
 program is to dlopen
 phobos/druntime first and to use 
 rt_loadLibrary/rt_unloadLibrary hereafter.

 Loading a library only initializes it and it's linked 
 dependencies for the current
 thread.

 Threads where a library is not initialize will not see the 
 library's modules, nor the EH
 tables and the GC will miss to scan the library's TLS memory.

 A spawned thread (core.thread.Thread) will initialize the same 
 libraries as it's parent
 thread. There is an implicit dlopen call for every dynamically 
 loaded library so to
 prevent the parent thread from unloading. When a thread 
 finishes it will drop/unload all
 remaining references to loaded libraries.
 The high-level implementation of the shared libraries support 
 will provide a method to
 iterate over all loaded libraries, thus it will be possible to 
 explicitly unload unused
 inherited libraries.

 Finding the dependencies of a library is somewhat expensive so 
 it is only performed for D libraries and non-recursively.

 Because all dependencies are loaded before the library itself 
 we do know the recursive dependencies. In case of loading a C 
 library that depends on D libraries or for D->C->D dependency 
 chains we will miss the recursive dependency.
 This might possibly lead to uninitialized threads when loading 
 such a library in more than one thread.
 I will try to fix any arising bugs ASAP but I don't have much 
 time left currently.
 Please tag bugzilla entries with the dll keyword.

 Work on the high-level interface will start around the end of 
 next week.
Good job, Congrats!!
Sep 07 2013
prev sibling parent Robert Schadek <realburner gmx.de> writes:
Most awesome
Sep 08 2013