www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Status of Dynamically Loadable D Libraries

reply "Nick Ulle" <nim.junk gmail.com> writes:
What's the current status of dynamic loading (and/or linking) for 
D libraries?

I've been playing with this a bit in hopes of calling a D shared 
library from R through the C ABI, and noticed that on Windows it 
seems fine, but on Linux dmd won't build my library unless it 
includes main(), and it segfaults if I call writeln().

I've seen this thread:
http://forum.dlang.org/thread/hmhaldyfziejrplgzazt forum.dlang.org

And also this one:
http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com

What I gather is that right now, DLLs work on Windows, but on 
*nix, shared libraries cannot yet use anything from Phobos.

Am I correct that it's safe to dynamically load a D library on 
Windows? How out-of-date is the information in the tutorial for 
making DLLs on dlang?

Is supporting dynamically loadable libraries a priority for the 
developers, or something that might not happen for a while?
Feb 10 2013
next sibling parent reply "jerro" <a a.com> writes:
 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while?
Judging by this thread: http://forum.dlang.org/thread/kf1223$oo8$1 digitalmars.com It has just become a priority.
Feb 10 2013
parent "Nick Ulle" <nim.junk gmail.com> writes:
On Sunday, 10 February 2013 at 22:46:29 UTC, jerro wrote:
 Judging by this thread:

 http://forum.dlang.org/thread/kf1223$oo8$1 digitalmars.com

 It has just become a priority.
Thanks! This is great to hear. I didn't come across that thread while searching the forums before.
Feb 10 2013
prev sibling next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 10 February 2013 at 22:11:52 UTC, Nick Ulle wrote:
 What's the current status of dynamic loading (and/or linking) 
 for D libraries?

 I've been playing with this a bit in hopes of calling a D 
 shared library from R through the C ABI, and noticed that on 
 Windows it seems fine, but on Linux dmd won't build my library 
 unless it includes main(), and it segfaults if I call writeln().

 I've seen this thread:
 http://forum.dlang.org/thread/hmhaldyfziejrplgzazt forum.dlang.org

 And also this one:
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com

 What I gather is that right now, DLLs work on Windows, but on 
 *nix, shared libraries cannot yet use anything from Phobos.
 
 Am I correct that it's safe to dynamically load a D library on 
 Windows? How out-of-date is the information in the tutorial for 
 making DLLs on dlang?

 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while
For shared libraries (linux) I'd use ldc . It's just works.
Feb 10 2013
next sibling parent "Nick Ulle" <nim.junk gmail.com> writes:
On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:
 For shared libraries (linux) I'd use ldc . It's just works.
Not quite. Using ldc2 -shared -relocation-model=pic mylib.d Compiles and links, but on loading there are undefined symbols (_Dmain). Including a main() function gets me a loadable library, but much like the case with dmd, it segfaults if I call writeln.
Feb 10 2013
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:
 For shared libraries (linux) I'd use ldc . It's just works.
You are living a dangerous life. ;) It appears to work at first, but the test suite is known to segfault in random places with druntime/Phobos being built as a shared library, most probably because the GC/threading code doesn't really handle dynamic libraries at all. David
Feb 10 2013
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 10 February 2013 at 23:52:30 UTC, David Nadlinger 
wrote:
 On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:
 For shared libraries (linux) I'd use ldc . It's just works.
You are living a dangerous life. ;) It appears to work at first, but the test suite is known to segfault in random places with druntime/Phobos being built as a shared library, most probably because the GC/threading code doesn't really handle dynamic libraries at all. David
Does this pretty much completely invalidate projects like this: https://bitbucket.org/ariovistus/pyd ? I've had success using it, as well as loading d shared libraries from C/IDL, with the appropriate rt_init calls etc.
Feb 13 2013
prev sibling next sibling parent "Dave Wilson" <boff1984 hotmail.com> writes:
You can arrange to have d's runtime initialized when dlopen loads
the shared lib by adding a small shim that uses gcc attributes,
as follows:

// shim.c

__attribute__((constructor))
static void dinit() {
      rt_init();
}
__attribute__((destructor))
static void dfini() {
      rt_term();
}

// end of shim.c

Compile this to a .o with gcc, then include the .o in the build.
Note that I've only tried this trick with a simple "hello world"
shared lib, so I apologize in advance if it doesn't work for
real-world uses.
Feb 10 2013
prev sibling next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Sunday, 10 February 2013 at 22:11:52 UTC, Nick Ulle wrote:
 What's the current status of dynamic loading (and/or linking) 
 for D libraries?

 I've been playing with this a bit in hopes of calling a D 
 shared library from R through the C ABI, and noticed that on 
 Windows it seems fine, but on Linux dmd won't build my library 
 unless it includes main(), and it segfaults if I call writeln().
Dynamic linking works in linux (at least observed features work, see example http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2). Dynamic loading does not work.
 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while?
Yes, couple of days ago Walter announced that shared libraries is priority and he will get into this after releasing new dmd version.
Feb 10 2013
parent reply Lee Braiden <leebraid gmail.com> writes:
On Mon, 11 Feb 2013 07:25:55 +0100, Maxim Fomin wrote:
 Dynamic linking works in linux (at least observed features work, see
 example
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2).
 Dynamic loading does not work.
 
 [snip]
Yes, couple of days ago Walter announced that shared libraries is priority and he will get into this after releasing new dmd version.
I'm not sure what to hope for here. Does this mean we'll be able to, say, scan a directory, and dynamically load D plugins from it, with proper exception handling, threading, etc., across the app/library/plugin divide? And will it work the same way on different platforms, so you can build the (right binaries, obviously) for the platform, and load it the same way, with the underlying code taking care of the details? -- Lee
Feb 23 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/13 3:53 AM, Lee Braiden wrote:
 On Mon, 11 Feb 2013 07:25:55 +0100, Maxim Fomin wrote:
 Dynamic linking works in linux (at least observed features work, see
 example
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2).
 Dynamic loading does not work.

 [snip]
Yes, couple of days ago Walter announced that shared libraries is priority and he will get into this after releasing new dmd version.
I'm not sure what to hope for here. Does this mean we'll be able to, say, scan a directory, and dynamically load D plugins from it, with proper exception handling, threading, etc., across the app/library/plugin divide?
That's about right.
 And will it work the same way on different platforms, so you can build
 the (right binaries, obviously) for the platform, and load it the same
 way, with the underlying code taking care of the details?
Linux first. Andrei
Feb 23 2013
prev sibling parent Martin Nowak <code dawg.eu> writes:
On 02/10/2013 11:11 PM, Nick Ulle wrote:
 What's the current status of dynamic loading (and/or linking) for D
 libraries?
No dynamic loading in sight yet. We need to extend the GC, find semantics/techniques for TLS initialization, provide tools to load symbols ... What is on the way but will take some more weeks is link-time support for shared libraries. https://github.com/D-Programming-Language/druntime/pull/395
 I've been playing with this a bit in hopes of calling a D shared library
 from R through the C ABI, and noticed that on Windows it seems fine, but
 on Linux dmd won't build my library unless it includes main(), and it
 segfaults if I call writeln().
The reasons for your crashes are missing initialization. You could experiment with calling rt_init/rt_term which are exported from your shared library, but only if you're not threaded.
 What I gather is that right now, DLLs work on Windows, but on *nix,
 shared libraries cannot yet use anything from Phobos.
Yes, there is rudimentary support on Windows, but you'll inevitably run into ODR issues. http://d.puremagic.com/issues/show_bug.cgi?id=7020#c4
 Am I correct that it's safe to dynamically load a D library on Windows?
 How out-of-date is the information in the tutorial for making DLLs on
 dlang?
AFAIK it should work.
 Is supporting dynamically loadable libraries a priority for the
 developers, or something that might not happen for a while?
It's a huge priority but also a huge undertaking.
Feb 13 2013