www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.plugin ?

reply "FreeSlave" <freeslave93 gmail.com> writes:
Loading of dynamic libraries at runtime is common task in many 
applications. Even now some D-bindings to C libraries use this 
approach (see Derelict). Some C++ frameworks and libraries (for 
example Qt and Poco) have high level cross-platform abstractions 
for library loading. They also allow to export and load classes 
at runtime. What's about making such module for phobos in future 
when D will have good support of dynamic libraries?

I'm currenly working on such module now. Here it is 
https://bitbucket.org/FreeSlave/dido
Sep 30 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-09-30 22:53, FreeSlave wrote:

 Loading of dynamic libraries at runtime is common task in many
 applications. Even now some D-bindings to C libraries use this approach
 (see Derelict). Some C++ frameworks and libraries (for example Qt and
 Poco) have high level cross-platform abstractions for library loading.
 They also allow to export and load classes at runtime. What's about
 making such module for phobos in future when D will have good support of
 dynamic libraries?

 I'm currenly working on such module now. Here it is
 https://bitbucket.org/FreeSlave/dido


Class loading, how necessary is this? We already have ClassInfo.find and 
"create".

-- 
/Jacob Carlborg
Oct 01 2013
parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Tuesday, 1 October 2013 at 07:54:18 UTC, Jacob Carlborg wrote:

 Class loading, how necessary is this? We already have 
 ClassInfo.find and "create".


I just checked it on Linux. ClassInfo.find can't find the class 
from shared library. That's what I did:

//dmd rttitest.d baseclass.d -L-ldl
module rttitest;

import std.string;
import std.conv;
import std.stdio;
import std.c.linux.linux;

import baseclass;

int main()
{
     void* lib = dlopen(toStringz("./libdclass.so"), RTLD_LAZY);
     assert(lib, to!string(dlerror()));
     auto classInfo = ClassInfo.find("classimpl.MyClass");
     assert(classInfo, "Can't find ClassInfo");
     return 0;
}

module baseclass;

abstract class AbstractClass
{
     int doThing(int a, int b);
}

//dmd -shared -fPIC classimpl.d -I. -oflibdclass.so
module classimpl;
import baseclass;

class MyClass : AbstractClass
{
     override int doThing(int a, int b)
     {
         return a+b;
     }
}

I also tried to pass other flags to dlopen (RTLD_GLOBAL and 
RTLD_NOW) but it still does not work.
Oct 01 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-10-01 13:36, FreeSlave wrote:


 I just checked it on Linux. ClassInfo.find can't find the class from
 shared library. That's what I did:


Dynamic libraries are not completely implemented yet in D. I expect this 
to work when the implementation is complete.

BTW, which version are you using. git HEAD has some new functionality in 
this area.

-- 
/Jacob Carlborg
Oct 01 2013
next sibling parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Tuesday, 1 October 2013 at 18:00:34 UTC, Jacob Carlborg wrote:

 On 2013-10-01 13:36, FreeSlave wrote:


 I just checked it on Linux. ClassInfo.find can't find the 
 class from
 shared library. That's what I did:


 Dynamic libraries are not completely implemented yet in D. I 
 expect this to work when the implementation is complete.

 BTW, which version are you using. git HEAD has some new 
 functionality in this area.


Now I also did this on Windows using Runtime.loadLibrary instead 
of dlopen, but result is same. I have 2.062 on Linux and 2.063 on 
Windows. Ok, I need to update to check out the actual status.

Even if it would be possible to create instances of classes using 
ClassInfo, I still think phobos needs module to load functions at 
runtime. Especially it concerns functions with mangled names 
(i.e. all extern(D) functions). C++ frameworks usually deal only 
with extern "C" functions because C++ mangling is not 
standardized and therefore it depends on current compiler. D 
however can provide same rules for all compilers. In my project 
I've implemented 'mangle' function by hand but I have not 
properly test it yet. Well, D has .mangleof but it's compiletime 
only. We need official runtime version that allows to write 
things like:

auto func = lib.getDFunction!FunctionType 
("package_name.module_name.function_name");
Oct 01 2013
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 1 October 2013 at 19:17:11 UTC, FreeSlave wrote:

 Now I also did this on Windows using Runtime.loadLibrary 
 instead of dlopen, but result is same. I have 2.062 on Linux 
 and 2.063 on Windows. Ok, I need to update to check out the 
 actual status.


For experiments with dynamically loadable plugins I'd recommend 
to use only git master HEAD (future 2.064) on Linux, as it was 
announced priority for upcoming release and lot of changes have 
been made.
Oct 01 2013
prev sibling parent Martin Nowak <code dawg.eu> writes:
On 10/01/2013 09:17 PM, FreeSlave wrote:

 D however can provide same rules for all compilers. In my project I've
 implemented 'mangle' function by hand but I have not properly test it
 yet. Well, D has .mangleof but it's compiletime only. We need official
 runtime version that allows to write things like:

 auto func = lib.getDFunction!FunctionType
 ("package_name.module_name.function_name");


I'm sorry that the information about the current work didn't reach you 
in time.
Please have a look at 
https://github.com/D-Programming-Language/druntime/pull/617, be critical 
and contribute whatever you think is missing.
Oct 01 2013
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Oct 1, 2013, at 11:00 AM, Jacob Carlborg <doob me.com> wrote:


 On 2013-10-01 13:36, FreeSlave wrote:
=20

 I just checked it on Linux. ClassInfo.find can't find the class from
 shared library. That's what I did:


=20
 Dynamic libraries are not completely implemented yet in D. I expect =


this to work when the implementation is complete.

This.  loadLibrary doesn't yet graft in the ModuleInfo tree from the =
dynamic library, so ClassInfo.find will fail.=
Oct 01 2013
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On 10/01/2013 01:36 PM, FreeSlave wrote:

 I just checked it on Linux. ClassInfo.find can't find the class from
 shared library. That's what I did:


This will work on master but we still lack support for looking up 
classes within a specific library, after all you can have two different 
implementations with the same name in two different libraries (plugins).

There is another problem with your example, and it's somewhat nasty.
You need to put the base class in a separate library and link both the 
classimpl and the executable against it. The way you did it now there is 
a cyclic dependency between your executable and the shared library.
In fact the shared library can only be loaded if you build your 
executable with -export-dynamic.
Oct 01 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-10-02 03:39, Martin Nowak wrote:


 This will work on master but we still lack support for looking up
 classes within a specific library, after all you can have two different
 implementations with the same name in two different libraries (plugins).


What happens then, it just picks the first one?

-- 
/Jacob Carlborg
Oct 02 2013
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On 09/30/2013 10:53 PM, FreeSlave wrote:

 I'm currenly working on such module now. Here it is
 https://bitbucket.org/FreeSlave/dido


https://github.com/D-Programming-Language/druntime/pull/617
http://dconf.org/talks/nowak.html
Oct 01 2013
parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Wednesday, 2 October 2013 at 01:31:42 UTC, Martin Nowak wrote:

 On 09/30/2013 10:53 PM, FreeSlave wrote:

 I'm currenly working on such module now. Here it is
 https://bitbucket.org/FreeSlave/dido


 https://github.com/D-Programming-Language/druntime/pull/617
 http://dconf.org/talks/nowak.html


Yeah, that was video I inspired from. Well, seems my work was 
needless... Anyway where is it better to contribute - here or on 
github?

About the base class - user still needs to cast Object returned 
by ClassInfo to appropriate interface otherwise she can't get 
functionality of loaded class. Also I assume ClassInfo can't 
create instances of C++ classes. Though I'm not sure if standard 
D library should have this capability.

Ok, I looked your implementation on github. Are you sure we don't 
need to use exceptions instead of just returning null? I believe 
as library does not export required function it's definitely some 
kind of critical error and can't be omitted.
Oct 01 2013
parent Sean Kelly <sean invisibleduck.org> writes:
On Oct 1, 2013, at 7:36 PM, FreeSlave <freeslave93 gmail.com> wrote:


 On Wednesday, 2 October 2013 at 01:31:42 UTC, Martin Nowak wrote:

 On 09/30/2013 10:53 PM, FreeSlave wrote:

 I'm currenly working on such module now. Here it is
 https://bitbucket.org/FreeSlave/dido


=20
 https://github.com/D-Programming-Language/druntime/pull/617
 http://dconf.org/talks/nowak.html


=20
 Yeah, that was video I inspired from. Well, seems my work was =


needless... Anyway where is it better to contribute - here or on github?

If you're going to contribute, I suggest joining the appropriate mailing =
list.  You'll be less likely to duplicate effort if you're watching that =
discussion.=
Oct 02 2013