www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Status D dynamic libs?

reply Frank Benoit <keinfarbton googlemail.com> writes:
If i want to put dwt into a dynamic lib, then i have those components

- D runtime (tango)
- dwt (either dwt-win or dwt-linux, depends on tango)
- dwt-addons (optionally, dependending on dwt-win/dwt-linux and tango)
- application code, depending on all above.

What i want to do is, to put all dwt and dwt-addons stuff into two libs.

I hope to get shorter build times and smaller executables.

Is that possible on win/linux?
What are the problems?

As far as I remember, Walter said at the conference that DMD /does/ 
generate pic (position independent code) and it /should/ work. If there 
are problems, he would need examples and/or bug reports. Are there 
problems? Are there reports?
Apr 19 2008
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Frank Benoit wrote:

 If i want to put dwt into a dynamic lib, then i have those components
 
 - D runtime (tango)
 - dwt (either dwt-win or dwt-linux, depends on tango)
 - dwt-addons (optionally, dependending on dwt-win/dwt-linux and tango)
 - application code, depending on all above.
 
 What i want to do is, to put all dwt and dwt-addons stuff into two libs.
 
 I hope to get shorter build times and smaller executables.
 
 Is that possible on win/linux?
 What are the problems?
 
 As far as I remember, Walter said at the conference that DMD /does/ 
 generate pic (position independent code) and it /should/ work. If there 
 are problems, he would need examples and/or bug reports. Are there 
 problems? Are there reports?
 


First of all, with Windows, you can already create DLLs and use them 
now. It also has nothing whatsoever to do with PIC. There is an example 
on doing Windows DLLs in the samples directory.

Shared libraries under Linux are completely different. The compiler does 
generate PIC code, so the compiler work is done, but nobody has sat down 
and figured out the details of making Phobos work as a shared library.
Apr 19 2008
next sibling parent e-t172 <e-t172 akegroup.org> writes:
Walter Bright a écrit :

 Shared libraries under Linux are completely different. The compiler does 
 generate PIC code, so the compiler work is done, but nobody has sat down 
 and figured out the details of making Phobos work as a shared library.


Frank Benoit does not need to compile Phobos as a shared library to do 
what he wants to do. So the answer is yes, it's possible.
Apr 19 2008
prev sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
I would like to collect as much information as possible.

Windows:
Looking at the DLL example, it looks like DLLs are fully supported.
- Has someone used this successfully?

into a DLL project?
- Why isn't it possible to call "new MyClass" instead of "getMyClass"?

D runtime:
- What are the problems, doing it into a dyn. lib?
Apr 19 2008
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Frank Benoit wrote:

 I would like to collect as much information as possible.
 
 Windows:
 Looking at the DLL example, it looks like DLLs are fully supported.
 - Has someone used this successfully?

 into a DLL project?


I don't know. Haven't investigated it.


 - Why isn't it possible to call "new MyClass" instead of "getMyClass"?


??


 
 D runtime:
 - What are the problems, doing it into a dyn. lib?
 


I don't know. It hasn't been investigated.
Apr 19 2008
parent reply Frank Benoit <keinfarbton googlemail.com> writes:
Walter Bright schrieb:

 Frank Benoit wrote:

 - Why isn't it possible to call "new MyClass" instead of "getMyClass"?


 
 ??


In the example a getMyClass is used. So i thought it might not be 
possible to call "new MyClass" directly?
Apr 19 2008
parent Walter Bright <newshound1 digitalmars.com> writes:
Frank Benoit wrote:

 Walter Bright schrieb:

 Frank Benoit wrote:

 - Why isn't it possible to call "new MyClass" instead of "getMyClass"?


 ??


 
 In the example a getMyClass is used. So i thought it might not be 
 possible to call "new MyClass" directly?


The idea is to be careful which heap MyClass is allocated on - the DLL's 
or the caller's?
Apr 19 2008
prev sibling next sibling parent reply e-t172 <e-t172 akegroup.org> writes:
Frank Benoit a écrit :

 If i want to put dwt into a dynamic lib, then i have those components
 
 - D runtime (tango)
 - dwt (either dwt-win or dwt-linux, depends on tango)
 - dwt-addons (optionally, dependending on dwt-win/dwt-linux and tango)
 - application code, depending on all above.
 
 What i want to do is, to put all dwt and dwt-addons stuff into two libs.
 
 I hope to get shorter build times and smaller executables.
 
 Is that possible on win/linux?
 What are the problems?


On Windows, I don't know. On Linux, I have already compiled a D shared 
library with no problems whatsoever. I'm using GDC, but if DMD supports 
PIC you should be just fine.

Just remember not to include Phobos/Tango when creating your shared 
library (it won't work, because Phobos/Tango is not compiled PIC). With 
GDC you can avoid linking to Phobos/Tango using the -nophoboslib switch. 
I don't know for DMD. Of course, you still have to link to Phobos/Tango 
when building the actual application.

However, when maintaining shared libraries, automatic function inlining 
might cause problems. See the "library standardization" thread for details.
Apr 19 2008
parent reply Frank Benoit <keinfarbton googlemail.com> writes:
e-t172 schrieb:

 On Windows, I don't know. On Linux, I have already compiled a D shared 
 library with no problems whatsoever. I'm using GDC, but if DMD supports 
 PIC you should be just fine.
 
 Just remember not to include Phobos/Tango when creating your shared 
 library (it won't work, because Phobos/Tango is not compiled PIC). With 
 GDC you can avoid linking to Phobos/Tango using the -nophoboslib switch. 
 I don't know for DMD. Of course, you still have to link to Phobos/Tango 
 when building the actual application.
 
 However, when maintaining shared libraries, automatic function inlining 
 might cause problems. See the "library standardization" thread for details.


Did you use the example code from the digitalmars windows example?
Can you post a minimized linux example?
Can "new MyClass" be called?


Not linking tango you mean just the tango-base-dmd.a, like supplying an 
empty dummy.a ?
Apr 19 2008
parent reply e-t172 <e-t172 akegroup.org> writes:
Frank Benoit a écrit :

 Did you use the example code from the digitalmars windows example?


No. I don't develop on Windows.


 Can you post a minimized linux example?


I don't see the point. As far as I know, any D code can be made into a 
shared library. Just compile it with PIC and create the library.


 Can "new MyClass" be called?


Yes, why not?





IIRC, yes. And it works.


 Not linking tango you mean just the tango-base-dmd.a, like supplying an 
 empty dummy.a ?


I don't use DMD, so I can't tell for sure. I don't see any reason why 
this wouldn't work, though.
Apr 19 2008
parent reply Frank Benoit <keinfarbton googlemail.com> writes:
e-t172 schrieb:

 Frank Benoit a écrit :

 Can "new MyClass" be called?


 
 Yes, why not?


In my understanding, if you have an object reference, a virtual call can 
be done with it. The reference points you to the vtable, and that to the 
methods address.

What about static, non-virtual and ctor calls?
- static & ctor: no obj ref, hence no access to vtable
- final & package methods: probably not in the vtable
- public static member vars: access over global address
I don't know, this is just a guess.

If the .dll/.so is statically linked the linker does the magic.

But if you load the .so/.dll dynamically, the address needs to be loaded 
via dlsym() or getProcAddress()?

This is why i think the ctor might be problematic.
Apr 19 2008
parent Robert Fraser <fraserofthenight gmail.com> writes:
Frank Benoit wrote:

 - final & package methods: probably not in the vtable


About "package"... I noticed this too and had to change stuff o public. 
What does "package" have to do with the vtable; it means that the symbol 
is only accessible from within the package. So if a method is 
package-protected, any other module in the same package should be able 
to override it.
Apr 19 2008
prev sibling next sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Frank Benoit wrote:

 [...]


I really wish DDL was still alive, or something similar. That had the 
right idea and would have solved many of the issues D has with 
traditional dynamic libraries.
Apr 19 2008
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Robert Fraser wrote:


 Frank Benoit wrote:

 [...]


 
 I really wish DDL was still alive, or something similar. That had the
 right idea and would have solved many of the issues D has with
 traditional dynamic libraries.


On Windows at least - turns out *nix .so's can do most of what was ever
planned for DDL in any case. Not that I think DDL is dead dead, just not
gaining features at a very quick pace.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango
Apr 19 2008
prev sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
Robert Fraser schrieb:

 Frank Benoit wrote:

 [...]


 
 I really wish DDL was still alive, or something similar. That had the 
 right idea and would have solved many of the issues D has with 
 traditional dynamic libraries.


I really wish, i would understand the problems (or how it works)
On windows, walter says it works.
On linux, e-t172 says it works.

Can you give an example for problems?
Apr 19 2008
parent Max Samukha <nospam nospam.com> writes:
On Sun, 20 Apr 2008 01:50:46 +0200, Frank Benoit
<keinfarbton googlemail.com> wrote:

There is a summary of issues with DLLs posted by the DDL's creator to
the wiki long time ago:
http://www.prowiki.org/wiki4d/wiki.cgi?BestPractices/DLL
AFAIK, all those issues still remain unresolved.
Apr 19 2008
prev sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
The problem with windows DLLs is that they don't support partial linking 
( or how it was called ). That is, you can't have undefined symbols in a 
DLL. What does it mean for an average D program? Each DLL will have to 
contain all symbols that it imports (uses). So multiple std-libs, 
multiple ClassInfos for the same classes, multiple globals. Unless these 
are somehow magically matched at runtime, you'll be in serious trouble. 
E.g. casting an object returned from a DLL to Object will return null. 
The ClassInfo in DLL and the host will be different objects, thus 
failing at the cast. This may be worked around (to some extent), e.g. by 
walking the rtti manually and comparing class names, doing casting thru 
void* pointers...

Still, exceptions bork for the same reason. Whatever is being thrown, 
it's converted to the type being caught, thus exception handling will 
fail really badly across binary boundaries. And IIRC, really badly == 
instant crash at an exception being thrown inside a DLL.

More info on DLLs and D: http://dsource.org/forums/viewtopic.php?t=677 
it's quite old but describes the problems and highlights why DDL was 
conceived.

DDL on the other hand, provides pretty much the functionality that .SO 
on Unix do (plus introspection of the loaded modules), thus allowing 
partial linking. Everything 'just works' since the undefined symbols 
will be taken from the host application, thus globals, typeinfo and 
whatnot will be shared between the host and the plugins.

DDL is not being actively maintained because Eric is currently busy with 
other stuff, mainly due to rand() real life events.
Last time I checked, DDL compiled with Tango and the basic examples worked.

DDL only works with DMD-win though. As for GDC on MSYS, someone might 
try binding EDLL: http://edll.sourceforge.net/, porting it to work with 
DDL's interface or helping Gregor's WinELF: 
http://svn.berlios.de/svnroot/repos/crosslibc/other/winelf/

Hope this info is useful :) Cheers!


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Apr 19 2008
next sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
Tom S schrieb:

 More info on DLLs and D: http://dsource.org/forums/viewtopic.php?t=677 
 it's quite old but describes the problems and highlights why DDL was 
 conceived.


I think you did mean this one?
http://dsource.org/forums/viewtopic.php?t=959
Apr 20 2008
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Frank Benoit wrote:

 Tom S schrieb:

 More info on DLLs and D: http://dsource.org/forums/viewtopic.php?t=677 
 it's quite old but describes the problems and highlights why DDL was 
 conceived.


 
 I think you did mean this one?
 http://dsource.org/forums/viewtopic.php?t=959


This one is a continuation of the other thread's DLL/DDL ramblings :>

-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Apr 20 2008
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Tom S wrote:

 
 DDL on the other hand, provides pretty much the functionality that .SO 
 on Unix do (plus introspection of the loaded modules), thus allowing 
 partial linking. Everything 'just works' since the undefined symbols 
 will be taken from the host application, thus globals, typeinfo and 
 whatnot will be shared between the host and the plugins.
 
 DDL is not being actively maintained because Eric is currently busy with 
 other stuff, mainly due to rand() real life events.
 Last time I checked, DDL compiled with Tango and the basic examples worked.
 


Yup. Due to the nature of DLL's, D-to-D shared libraries are never going 
to work right (unless one restricts the API to C-level constructs only, 
ie, no classes, no exceptions, etc.).
That's why a project such as DDL is so important, and it's a shame that 
Walter does not realize how beneficial such technology would be for D. 

languages) knows how comfortable it is not having to worry about any of 
such issues, and still have the full power of the language available 
across libraries.

-- 
Bruno Medeiros - Software Developer, MSc. in CS/E graduate
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Apr 25 2008
parent Frank Benoit <keinfarbton googlemail.com> writes:
Bruno Medeiros schrieb:

 Yup. Due to the nature of DLL's, D-to-D shared libraries are never going 
 to work right (unless one restricts the API to C-level constructs only, 
 ie, no classes, no exceptions, etc.).
 That's why a project such as DDL is so important, and it's a shame that 
 Walter does not realize how beneficial such technology would be for D. 

 languages) knows how comfortable it is not having to worry about any of 
 such issues, and still have the full power of the language available 
 across libraries.
 


yes.
And even if .so on linux can do most of the stuff DDL does, it would be 
good to have a toolchain that works on all plattforms in the same way.

That would also mean that the binary format is defined in the D language 
spec.
Apr 25 2008