www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Cross-platform dynamic libraries (GNU/Linux, Mac OS X, Windows,...)

reply Hugues De Keyzer <"digitalmars _NO_SPAM_ " hugues.info> writes:
Hi,

I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
and Windows) application in D. One of its key characteristics is that it 
will be based on modules (plug-ins). Ideally, functionalities more 
advanced than just normal shared libraries would be desirable.

I took a look at DDL, but it seems to be targeted only at Windows.

I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as long 
as the garbage collector isn't involved in the library. Instancing a 
class in the library works only if new() and delete() are defined and 
don't call the gc. When new() and delete() are not defined, instancing a 
class results in a bus error. gdb gives the following message:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x0005389a in _d_newclass ()

I also tried calling std.gc.setGCHandle(), but it also fails:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x00053435 in std.gc.addRange(void*, void*) ()

This may sound like a newbie question, but is there an easy way (or a 
way at all) to develop a cross-platform application in D with shared 
libraries?

Dynamic linking is a very useful feature for software development. 
Including this in the D specification (like threads) would be great.

Hugues De Keyzer

--
The only constant in life is change
    -- www.wikipedia.org
Sep 05 2007
parent reply Gregor Richards <Richards codu.org> writes:
Hugues De Keyzer wrote:

 Hi,
 
 I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
 and Windows) application in D. One of its key characteristics is that it 
 will be based on modules (plug-ins). Ideally, functionalities more 
 advanced than just normal shared libraries would be desirable.
 
 I took a look at DDL, but it seems to be targeted only at Windows.
 
 I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as long 
 as the garbage collector isn't involved in the library. Instancing a 
 class in the library works only if new() and delete() are defined and 
 don't call the gc. When new() and delete() are not defined, instancing a 
 class results in a bus error. gdb gives the following message:
 
 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x0005389a in _d_newclass ()
 
 I also tried calling std.gc.setGCHandle(), but it also fails:
 
 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x00053435 in std.gc.addRange(void*, void*) ()
 
 This may sound like a newbie question, but is there an easy way (or a 
 way at all) to develop a cross-platform application in D with shared 
 libraries?
 
 Dynamic linking is a very useful feature for software development. 
 Including this in the D specification (like threads) would be great.
 
 Hugues De Keyzer
 
 -- 
 The only constant in life is change
    -- www.wikipedia.org


GDC supports proper .so files on all platforms with .so files. Build the 
.so files without a standard library (-no-stdlib or something like that) 
and then build the host exporting dynamic symbols (default on OS X, 
-rdynamic on everything else). DSSS, of course, does all of this for you.

Windows DLL's will never be capable of full, proper support until phobos 
is in a .dll. Even then, if there was conflicting typeinfo, it could 
still fail. However, DDL ought to cover it there.

  - Gregor Richards
Sep 05 2007
next sibling parent reply "Vladimir Panteleev" <thecybershadow gmail.com> writes:
On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards codu.org> wrote:


 Windows DLL's will never be capable of full, proper support until phobos
 is in a .dll. Even then, if there was conflicting typeinfo, it could
 still fail. However, DDL ought to cover it there.


I wonder how hard it would be to develop a native-code cross-platform plug-in
architecture for the IA32 platform. The idea is that the "plugins" are compiled
to native code, either position-independent or with relocation information, and
don't contain any of the standard library. Instead, on initialization, the
main, platform-dependant application passes the plug-in a structure filled with
function/global var pointers, which have a cross-platform ABI, which will allow
the plugins to access the standard library and other host app functionality.

Sounds theoretically feasable? I think there are no problems with the idea as
long as there are no platform-dependant data types.

-- 
Best regards,
  Vladimir                          mailto:thecybershadow gmail.com
Sep 05 2007
parent reply Gregor Richards <Richards codu.org> writes:
Vladimir Panteleev wrote:

 On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards codu.org> wrote:
 

 Windows DLL's will never be capable of full, proper support until phobos
 is in a .dll. Even then, if there was conflicting typeinfo, it could
 still fail. However, DDL ought to cover it there.


 
 I wonder how hard it would be to develop a native-code cross-platform plug-in
architecture for the IA32 platform. The idea is that the "plugins" are compiled
to native code, either position-independent or with relocation information, and
don't contain any of the standard library. Instead, on initialization, the
main, platform-dependant application passes the plug-in a structure filled with
function/global var pointers, which have a cross-platform ABI, which will allow
the plugins to access the standard library and other host app functionality.
 
 Sounds theoretically feasable? I think there are no problems with the idea as
long as there are no platform-dependant data types.
 


So, ELF? I think AT&T developed that around 25 years ago.

  - Gregor Richards
Sep 05 2007
parent reply "Vladimir Panteleev" <thecybershadow gmail.com> writes:
On Wed, 05 Sep 2007 22:58:13 +0300, Gregor Richards <Richards codu.org> wrote:


 Vladimir Panteleev wrote:

 On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards codu.org> wrote:


 Windows DLL's will never be capable of full, proper support until phobos
 is in a .dll. Even then, if there was conflicting typeinfo, it could
 still fail. However, DDL ought to cover it there.


 I wonder how hard it would be to develop a native-code cross-platform plug-in
architecture for the IA32 platform. The idea is that the "plugins" are compiled
to native code, either position-independent or with relocation information, and
don't contain any of the standard library. Instead, on initialization, the
main, platform-dependant application passes the plug-in a structure filled with
function/global var pointers, which have a cross-platform ABI, which will allow
the plugins to access the standard library and other host app functionality.

 Sounds theoretically feasable? I think there are no problems with the idea as
long as there are no platform-dependant data types.


 So, ELF? I think AT&T developed that around 25 years ago.


Link to Windows implementation please - preferably something D-friendly :P

-- 
Best regards,
  Vladimir                          mailto:thecybershadow gmail.com
Sep 05 2007
parent Gregor Richards <Richards codu.org> writes:
Vladimir Panteleev wrote:

 On Wed, 05 Sep 2007 22:58:13 +0300, Gregor Richards <Richards codu.org> wrote:
 

 Vladimir Panteleev wrote:

 On Wed, 05 Sep 2007 20:55:14 +0300, Gregor Richards <Richards codu.org> wrote:


 Windows DLL's will never be capable of full, proper support until phobos
 is in a .dll. Even then, if there was conflicting typeinfo, it could
 still fail. However, DDL ought to cover it there.


 I wonder how hard it would be to develop a native-code cross-platform plug-in
architecture for the IA32 platform. The idea is that the "plugins" are compiled
to native code, either position-independent or with relocation information, and
don't contain any of the standard library. Instead, on initialization, the
main, platform-dependant application passes the plug-in a structure filled with
function/global var pointers, which have a cross-platform ABI, which will allow
the plugins to access the standard library and other host app functionality.

 Sounds theoretically feasable? I think there are no problems with the idea as
long as there are no platform-dependant data types.


 So, ELF? I think AT&T developed that around 25 years ago.


 
 Link to Windows implementation please - preferably something D-friendly :P
 


http://svn.berlios.de/svnroot/repos/crosslibc/other/winelf/ .

OK, no .so support yet (I'm no ELF expert), but when/if it gets .so 
support. there ya go :P

  - Gregor Richards
Sep 05 2007
prev sibling parent reply Hugues De Keyzer <"digitalmars _NO_SPAM_ " hugues.info> writes:
Gregor Richards wrote:

 Hugues De Keyzer wrote:

 Hi,

 I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
 and Windows) application in D. One of its key characteristics is that 
 it will be based on modules (plug-ins). Ideally, functionalities more 
 advanced than just normal shared libraries would be desirable.

 I took a look at DDL, but it seems to be targeted only at Windows.

 I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as 
 long as the garbage collector isn't involved in the library. 
 Instancing a class in the library works only if new() and delete() are 
 defined and don't call the gc. When new() and delete() are not 
 defined, instancing a class results in a bus error. gdb gives the 
 following message:

 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x0005389a in _d_newclass ()

 I also tried calling std.gc.setGCHandle(), but it also fails:

 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x00053435 in std.gc.addRange(void*, void*) ()

 This may sound like a newbie question, but is there an easy way (or a 
 way at all) to develop a cross-platform application in D with shared 
 libraries?

 Dynamic linking is a very useful feature for software development. 
 Including this in the D specification (like threads) would be great.

 Hugues De Keyzer

 -- 
 The only constant in life is change
    -- www.wikipedia.org


 
 GDC supports proper .so files on all platforms with .so files. Build the 
 ..so files without a standard library (-no-stdlib or something like 
 that) and then build the host exporting dynamic symbols (default on OS 
 X, -rdynamic on everything else). DSSS, of course, does all of this for 
 you.
 
 Windows DLL's will never be capable of full, proper support until phobos 
 is in a .dll. Even then, if there was conflicting typeinfo, it could 
 still fail. However, DDL ought to cover it there.
 
  - Gregor Richards


Thank you for this quick reply. I tested on Mac OS X with -nostdlib and 
had of course undefined references. After some more searching, I 
succeeded in building a dynamic library with the following options:

gdc -dynamiclib -nostdlib -undefined dynamic_lookup -o mylib.so mylib.d

This seems to work. The GC works fine.

Another question: is it safe to directly load a mangled D symbol like this:

void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");

for static void MyModyle.test()?

It works, but will it work an all platforms?

-- 
The only constant in life is change
    -- www.wikipedia.org
Sep 10 2007
parent reply Gregor Richards <Richards codu.org> writes:
Hugues De Keyzer wrote:

 Gregor Richards wrote:

 Hugues De Keyzer wrote:

 Hi,

 I'm planning to create a multi-platform (at least GNU/Linux, Mac OS X 
 and Windows) application in D. One of its key characteristics is that 
 it will be based on modules (plug-ins). Ideally, functionalities more 
 advanced than just normal shared libraries would be desirable.

 I took a look at DDL, but it seems to be targeted only at Windows.

 I made some dlopen() tests on Mac OS X 10.4 with gdc. It works, as 
 long as the garbage collector isn't involved in the library. 
 Instancing a class in the library works only if new() and delete() 
 are defined and don't call the gc. When new() and delete() are not 
 defined, instancing a class results in a bus error. gdb gives the 
 following message:

 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x0005389a in _d_newclass ()

 I also tried calling std.gc.setGCHandle(), but it also fails:

 Program received signal EXC_BAD_ACCESS, Could not access memory.
 Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
 0x00053435 in std.gc.addRange(void*, void*) ()

 This may sound like a newbie question, but is there an easy way (or a 
 way at all) to develop a cross-platform application in D with shared 
 libraries?

 Dynamic linking is a very useful feature for software development. 
 Including this in the D specification (like threads) would be great.

 Hugues De Keyzer

 -- 
 The only constant in life is change
    -- www.wikipedia.org


 GDC supports proper .so files on all platforms with .so files. Build 
 the ..so files without a standard library (-no-stdlib or something 
 like that) and then build the host exporting dynamic symbols (default 
 on OS X, -rdynamic on everything else). DSSS, of course, does all of 
 this for you.

 Windows DLL's will never be capable of full, proper support until 
 phobos is in a .dll. Even then, if there was conflicting typeinfo, it 
 could still fail. However, DDL ought to cover it there.

  - Gregor Richards


 
 Thank you for this quick reply. I tested on Mac OS X with -nostdlib and 
 had of course undefined references. After some more searching, I 
 succeeded in building a dynamic library with the following options:
 
 gdc -dynamiclib -nostdlib -undefined dynamic_lookup -o mylib.so mylib.d
 
 This seems to work. The GC works fine.
 
 Another question: is it safe to directly load a mangled D symbol like this:
 
 void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");
 
 for static void MyModyle.test()?
 
 It works, but will it work an all platforms?
 


The mangling is the same on all platforms, so that should work, but is 
bound to break eventually. Better would be to make an extern (C) symbol 
in the loaded library.

  - Gregor Richards
Sep 10 2007
parent reply Tomas Lindquist Olsen <tomas famolsen.dk> writes:
Gregor Richards wrote:


 Hugues De Keyzer wrote:

 Another question: is it safe to directly load a mangled D symbol like
 this:
 
 void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");
 
 for static void MyModyle.test()?
 
 It works, but will it work an all platforms?
 


 
 The mangling is the same on all platforms, so that should work, but is
 bound to break eventually. Better would be to make an extern (C) symbol
 in the loaded library.
 
   - Gregor Richards


There is also the .mangleof property.
Sep 10 2007
parent Gregor Richards <Richards codu.org> writes:
Tomas Lindquist Olsen wrote:

 Gregor Richards wrote:
 

 Hugues De Keyzer wrote:

 Another question: is it safe to directly load a mangled D symbol like
 this:

 void* test = dlsym(handle, "_D8mymodule8MyModule4testFZv");

 for static void MyModyle.test()?

 It works, but will it work an all platforms?


 The mangling is the same on all platforms, so that should work, but is
 bound to break eventually. Better would be to make an extern (C) symbol
 in the loaded library.

   - Gregor Richards


 
 There is also the .mangleof property.


Yes, that's the reliable cross-platform etc etc way to do it. I was 
assuming he didn't necessarily have the .d[i] file corresponding to the 
library, but I guess that doesn't make any sense :)

  - Gregor Richards
Sep 10 2007