www.digitalmars.com         C & C++   DMDScript  

D - Runtime linking question

reply "chris jones" <flak clara.co.uk> writes:
Hi, im not sure what the corect terminoly is but i have a question. Many
programs are utilising plugins, scanned for and linked at runtime, and the
ones i know of are all based on dlls. But dlls are limited to a C based
interface so if you want to use OO code then you have bodge it together at
either end. So i think it would be very usefull if there was some way to
have D object code that could link at runtime, or even a DDLL that could
have an object orientated interface. I guese that there would need to be a
compatible class declaration in both the host and the plugin and the
compatibilty would need to be check when th plugin is loaded, but that
shouldnt be to hard to do? Or is this posible already, if it is where should
i be looking for some info?

thanks,

chris
Sep 25 2002
parent reply Mac Reiter <Mac_member pathlink.com> writes:
In article <amsh1o$2oup$1 digitaldaemon.com>, chris jones says...

Hi, im not sure what the corect terminoly is but i have a question. Many
programs are utilising plugins, scanned for and linked at runtime, and the
ones i know of are all based on dlls. But dlls are limited to a C based
interface so if you want to use OO code then you have bodge it together at
either end. So i think it would be very usefull if there was some way to
have D object code that could link at runtime, or even a DDLL that could
have an object orientated interface. I guese that there would need to be a
compatible class declaration in both the host and the plugin and the
compatibilty would need to be check when th plugin is loaded, but that
shouldnt be to hard to do? Or is this posible already, if it is where should
i be looking for some info?

thanks,
chris


Not sure if this exactly fits your question, but thought I'd throw it out there
anyway.  Nothing about DLLs or other runtime linkable libraries precludes the
ability to have OO linkage.  The problem comes from non-standardized name
mangling.  ANSI did not specify any standard method of name mangling to support
overloaded functions and other C++ issues, so each compiler developer made up
their own.  Since linkage occurs by names, and since each compiler has its own
method of coming up with names, that means that the linkage between components
made by different compilers won't work.  In straight C, there is no name
mangling (or, more accurately, very minimal name mangling that is standardized
so that everyone does it the same way), so a MSVC .EXE can link in a Borland
DLL because the used the same names.

At least in theory (I have seen articles on this, but have never actually tried
it) it is possible to have classes in DLLs.  You just have to use the same
compiler for your client as you did for the DLL.  This potentially means not
only the same manufacturer, but the same version and maybe even the same patch
level.

The second issue involves truly dynamic loading.  If the plugin is being found
through a directory search or other runtime system, the LoadLibrary (Win32) and
GetProcAddress calls must be made, and I'm not sure exactly which procedures you
would have to get the address of to properly support classes.  I think all of
the articles I've seen on DLL based classes used the .LIB stub library approach,
which is more of a shared library than a dynamic library methodology.

Dunno if that provided any useful info, or if I just wasted a couple of K of
bandwidth...  Hopefully the former.
Mac
Sep 25 2002
parent reply "chris jones" <flak clara.co.uk> writes:
"Mac Reiter" <Mac_member pathlink.com> wrote in message
news:amsung$c85$1 digitaldaemon.com...

 In article <amsh1o$2oup$1 digitaldaemon.com>, chris jones says...


 Not sure if this exactly fits your question, but thought I'd throw it out


there

 anyway.  Nothing about DLLs or other runtime linkable libraries precludes


the

 ability to have OO linkage.  The problem comes from non-standardized name
 mangling.  ANSI did not specify any standard method of name mangling to


support

 overloaded functions and other C++ issues, so each compiler developer made


up

 their own.  Since linkage occurs by names, and since each compiler has its


own

 method of coming up with names, that means that the linkage between


components

 made by different compilers won't work.  In straight C, there is no name
 mangling (or, more accurately, very minimal name mangling that is


standardized

 so that everyone does it the same way), so a MSVC .EXE can link in a


Borland

 DLL because the used the same names.

 At least in theory (I have seen articles on this, but have never actually


tried

 it) it is possible to have classes in DLLs.  You just have to use the same
 compiler for your client as you did for the DLL.  This potentially means


not

 only the same manufacturer, but the same version and maybe even the same


patch

 level.


Its not an apraoch that would be suitable for an open format, i mean its ok
to do this if you are the programer of both host and plugin, but if lots of
diferant people are writing plugins it would be troublesome.

The way VST plugins work all the functions are routed to the object through
static functions via pointers. Its quite messy. Theres about twice as much
code just for c interface as there is for the class it is interfacing.


 The second issue involves truly dynamic loading.  If the plugin is being


found

 through a directory search or other runtime system, the LoadLibrary


(Win32) and

 GetProcAddress calls must be made, and I'm not sure exactly which


procedures you

 would have to get the address of to properly support classes.  I think all


of

 the articles I've seen on DLL based classes used the .LIB stub library


approach,

 which is more of a shared library than a dynamic library methodology.


I'm not sure what the .LIB stub apraoch is. I am thinking that a specifation
for dynamicly loadable code with an OO interface would be very valuable,
although i have little understanding of the complexities it would involve.
For it to be effective the actualy memory structure of the object would need
to defined indenticly in plugin and host. Which may be practical if the same
compiler is used for both. The more i think about the more complicated it
seems. Mabey its not even a D issue but an OS issue?

chris



 Sep 25 2002



  parent reply "Walter" <walter digitalmars.com>  writes:


D can access DLLs written that expose functions with a C interface. It can
also interface with COM objects, which is the more usual approach to putting
a class interface on a DLL.



 Sep 26 2002



  parent reply Burton Radons <loth users.sourceforge.net>  writes:


Walter wrote:

 D can access DLLs written that expose functions with a C interface. It can
 also interface with COM objects, which is the more usual approach to putting
 a class interface on a DLL.


Stonewheel handles classes in loaded objects.  I'm going to revisit it 
again someday.



 Sep 26 2002



  parent reply "chris jones" <flak clara.co.uk>  writes:


"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an0jho$ukv$1 digitaldaemon.com...

 Walter wrote:

 D can access DLLs written that expose functions with a C interface. It




can

 also interface with COM objects, which is the more usual approach to




putting

 a class interface on a DLL.


 Stonewheel handles classes in loaded objects.  I'm going to revisit it
 again someday.


Eh? Does that mean COM is very slow?

chris



 Sep 27 2002



 next sibling parent  "Walter" <walter digitalmars.com>  writes:


"chris jones" <flak clara.co.uk> wrote in message
news:an2roc$319j$1 digitaldaemon.com...

 Eh? Does that mean COM is very slow?


COM has its share of problems, but efficiency isn't one of them.



 Sep 28 2002



prev sibling  parent reply Burton Radons <loth users.sourceforge.net>  writes:


chris jones wrote:

 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an0jho$ukv$1 digitaldaemon.com...
 

Walter wrote:


D can access DLLs written that expose functions with a C interface. It




 
 can
 

also interface with COM objects, which is the more usual approach to




 
 putting
 

a class interface on a DLL.


Stonewheel handles classes in loaded objects.  I'm going to revisit it
again someday.


 
 Eh? Does that mean COM is very slow?


No - it goes through a normal vtable, so it's as fast as a completely 
virtual object, although it's twice as large on stack (if I understand 
the object model properly - interfaces aren't in my port yet).  Which I 
appear to not, looking at DMD.  Walter, is there a search involved in 
calling a method on any interface reference?  If there is, it'll be 
significantly slower.  If there isn't, it'll be slightly faster 
depending on how the cache is feeling.

But anyway.  Most classes aren't fully virtual; with a normal class, 
involving public fields and final and static methods, Stonewheel will be 
far faster, as it's just like linking the module into the executable.



 Sep 29 2002



 next sibling parent  "chris jones" <flak clara.co.uk>  writes:


"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an827i$2hc4$1 digitaldaemon.com...

 chris jones wrote:

 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an0jho$ukv$1 digitaldaemon.com...


 But anyway.  Most classes aren't fully virtual; with a normal class,
 involving public fields and final and static methods, Stonewheel will be
 far faster, as it's just like linking the module into the executable.


Its the 'Stonewheel' that has thown me, i thought it was a joke as in 'stone
wheels' would be very slow :o)

chris



 Sep 29 2002



prev sibling  parent reply "Walter" <walter digitalmars.com>  writes:


"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an827i$2hc4$1 digitaldaemon.com...

 No - it goes through a normal vtable, so it's as fast as a completely
 virtual object, although it's twice as large on stack (if I understand
 the object model properly - interfaces aren't in my port yet).  Which I
 appear to not, looking at DMD.  Walter, is there a search involved in
 calling a method on any interface reference?  If there is, it'll be
 significantly slower.  If there isn't, it'll be slightly faster
 depending on how the cache is feeling.


No, no search. It uses vtbl[]s and adjustor thunks, and so is as efficient
as MI dispatch in C++.



 Oct 05 2002



  parent reply Burton Radons <loth users.sourceforge.net>  writes:


Walter wrote:

 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an827i$2hc4$1 digitaldaemon.com...
 

No - it goes through a normal vtable, so it's as fast as a completely
virtual object, although it's twice as large on stack (if I understand
the object model properly - interfaces aren't in my port yet).  Which I
appear to not, looking at DMD.  Walter, is there a search involved in
calling a method on any interface reference?  If there is, it'll be
significantly slower.  If there isn't, it'll be slightly faster
depending on how the cache is feeling.


 
 No, no search. It uses vtbl[]s and adjustor thunks, and so is as efficient
 as MI dispatch in C++.


Know of a link describing memory behaviour here?  Glancing over C++'s 
spec, I don't see them describing it.  Although to be honest, I can't 
look too hard at it for fear of becoming blind or insane.



 Oct 05 2002



  parent reply "Walter" <walter digitalmars.com>  writes:


"Burton Radons" <loth users.sourceforge.net> wrote in message
news:annomt$1ujr$2 digitaldaemon.com...

 Walter wrote:

 No, no search. It uses vtbl[]s and adjustor thunks, and so is as




efficient

 as MI dispatch in C++.


 Know of a link describing memory behaviour here?  Glancing over C++'s
 spec, I don't see them describing it.  Although to be honest, I can't
 look too hard at it for fear of becoming blind or insane.


It wouldn't be described in the C++ spec, as it's an implementation detail.
I think Bjarne Stroustrup wrote an article on a scheme to do this long ago,
but I can't recall for sure. Check his web site.



 Oct 05 2002



 next sibling parent  Antti Sykari <jsykari cc.hut.fi>  writes:


"Walter" <walter digitalmars.com> writes:


 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:annomt$1ujr$2 digitaldaemon.com...

 Walter wrote:

 No, no search. It uses vtbl[]s and adjustor thunks, and so is as




 efficient

 as MI dispatch in C++.


 Know of a link describing memory behaviour here?  Glancing over C++'s
 spec, I don't see them describing it.  Although to be honest, I can't
 look too hard at it for fear of becoming blind or insane.


 It wouldn't be described in the C++ spec, as it's an implementation detail.
 I think Bjarne Stroustrup wrote an article on a scheme to do this long ago,
 but I can't recall for sure. Check his web site.


Stroustrup's paper "Multiple Inheritance in C++" is available at:

http://citeseer.nj.nec.com/stroustrup99multiple.html


Antti.



 Oct 12 2002



prev sibling  parent reply Antti Sykari <jsykari cc.hut.fi>  writes:


By the way,

here's another paper I bumped into. It claims to do faster MI virtual
function call dispatch than the traditional C++ compilers, also
enabling separate compilation of base classes:

http://citeseer.nj.nec.com/myers95bidirectional.html

The abstract:

  Bidirectional Object Layout for Separate Compilation

  Existing schemes for object layout and dispatch in the presence of
  multiple inheritance and separate compilation waste space and are
  slower than systems with single inheritance. This paper describes
  the bidirectional object layout, a new scheme for object layout that
  produces smaller objects and faster method invocations than existing
  schemes by automatically optimizing particular uses of multiple
  inheritance. The bidirectional object layout is used for the
  programming language Theta, and is applicable to languages like C++.
  This paper also demonstrates how to efficiently implement method
  dispatch when method signatures are allowed to change in subclasses.
  Most current statically compiled languages require identical
  signatures for efficiency.

A.



 Oct 12 2002



  parent  "Walter" <walter digitalmars.com>  writes:


Thanks!



 Oct 12 2002