www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is it possible to dynamically load a safe function from a shared

reply wjoe <invalid example.com> writes:
I've got a plug-in which is a shared library. Like this

module plugin;

 safe int VersionOfAPI()
{
    return 1;
}

this is builds to plugin.so

in main.d I'm loading the plugin and bind the those functions 
like so:

module app;
 safe:

alias apiverfn = int function();
apiverfn apiVersion;

void main()
{
    LoadPlugin("plugin.so");
}

void LoadPlugin(string path)
{
    void* plugin = loadLibrary(path);

     trusted void* bindSymbol(void** pfn, const(char)* symbolName)
    {
        *pfn = dlsym(plugin, symbolName);
    }

    bindSymbol(&apiVersion, "VersionOfAPI");
}

The compiler now refuses to call bindSymbol:
   Error: function bindSymbol(void**, const(char)*) is not 
callable using argument types (int function()  safe*, string)  
cannot pass &apiVersion of type int function()  safe* to 
parameter void** pfn

It makes sense that the compiler refuses to assign a  system 
function* to a  safe one but dlsym is a  system function which 
returns a  system fn* and I know that the function is  safe.

Is it possible to convince the compiler to look the other way 
while binding  safe functions from the plugin ?
Mar 13 2020
parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:


    bindSymbol(&apiVersion, "VersionOfAPI");
 }



 Is it possible to convince the compiler to look the other way 
 while binding  safe functions from the plugin ?


It probably has nothing to do with  safe, but is because of the 
void**.

bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");
Mar 13 2020
parent reply wjoe <invalid example.com> writes:
On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:


    bindSymbol(&apiVersion, "VersionOfAPI");
 }



 Is it possible to convince the compiler to look the other way 
 while binding  safe functions from the plugin ?


 It probably has nothing to do with  safe, but is because of the 
 void**.

 bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");


Than works, thanks :)
But isn't apiVersion a function pointer ?
Mar 13 2020
parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:

 On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:


    bindSymbol(&apiVersion, "VersionOfAPI");
 }



 Is it possible to convince the compiler to look the other way 
 while binding  safe functions from the plugin ?


 It probably has nothing to do with  safe, but is because of 
 the void**.

 bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");


 Than works, thanks :)
 But isn't apiVersion a function pointer ?


Yes, but when you take the address of *any* kind of pointer, you 
can't assign it to void** without a cast.
Mar 13 2020
parent reply wjoe <invalid example.com> writes:
On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:

 On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote:


    bindSymbol(&apiVersion, "VersionOfAPI");
 }



 Is it possible to convince the compiler to look the other 
 way while binding  safe functions from the plugin ?


 It probably has nothing to do with  safe, but is because of 
 the void**.

 bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");


 Than works, thanks :)
 But isn't apiVersion a function pointer ?


 Yes, but when you take the address of *any* kind of pointer, 
 you can't assign it to void** without a cast.


Didn't know that. I'm curious as to why this is the case.
This also means that LoadPlugin() can't be  safe - or at least 
the call to bindSymbol.

Thanks for the reply.
Mar 13 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 13, 2020 at 06:11:01PM +0000, wjoe via Digitalmars-d-learn wrote:

 On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:

 On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote:






[...]

 bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");








[...]

 This also means that LoadPlugin() can't be  safe - or at least the
 call to bindSymbol.


[...]

Of course it cannot be  safe, because it depends on whether the symbol
defined in the library you loaded is actually  safe.  You cannot know
that for sure (for example, maybe it exports a symbol that happens to
coincide with the mangling of a  safe function, but isn't in fact
 safe).  Similarly, at least on Posix, shared libraries only export
symbol names, the actual type is not part of the shared library API
other than what is encoded in the mangled symbol.  So you don't know for
sure that you're actually casting to the correct type, for example; if
you make a mistake, you might get UB and memory corruption.

So essentially, you're trusting that the symbol you just looked up is
actually pointing to what you think it's pointing to.  Therefore, it
makes sense that such calls have to be  trusted.


T

-- 
If it breaks, you get to keep both pieces. -- Software disclaimer notice
Mar 13 2020
parent reply wjoe <invalid example.com> writes:
On Friday, 13 March 2020 at 18:30:51 UTC, H. S. Teoh wrote:

 On Fri, Mar 13, 2020 at 06:11:01PM +0000, wjoe via 
 Digitalmars-d-learn wrote:

 On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote:

 On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote:

 On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker 
 wrote:






 [...]

 bindSymbol(cast(void**)&apiVersion, "VersionOfAPI");








 [...]

 This also means that LoadPlugin() can't be  safe - or at least 
 the
 call to bindSymbol.


 [...]

 Of course it cannot be  safe, because it depends on whether the 
 symbol defined in the library you loaded is actually  safe.  
 You cannot know that for sure (for example, maybe it exports a 
 symbol that happens to coincide with the mangling of a  safe 
 function, but isn't in fact  safe).  Similarly, at least on 
 Posix, shared libraries only export symbol names, the actual 
 type is not part of the shared library API other than what is 
 encoded in the mangled symbol.  So you don't know for sure that 
 you're actually casting to the correct type, for example; if 
 you make a mistake, you might get UB and memory corruption.

 So essentially, you're trusting that the symbol you just looked 
 up is actually pointing to what you think it's pointing to.  
 Therefore, it makes sense that such calls have to be  trusted.


 T


I wasn't aware that pragma(mangle, ..) can practically name any 
function anything. So from what I understand, because, at least 
on Posix, since there's only a symbol name there's nothing I can 
do in my loader to verify that a function is or does what it 
claim to be/do.

This is kind of disappointing but well worth the lessons learned.

Thanks for your reply.
Mar 13 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/13/20 4:22 PM, wjoe wrote:


 I wasn't aware that pragma(mangle, ..) can practically name any function 
 anything. So from what I understand, because, at least on Posix, since 
 there's only a symbol name there's nothing I can do in my loader to 
 verify that a function is or does what it claim to be/do.


But that's the same as the linker anyway. It's possible to stub the 
function as  safe, but implement it as  system and just pragma(mangle) 
the thing. Therefore the compiler is no safer.

I would expect that something could be written to turn a signature 
string into a mangling and also provide the correct type upon return. 
Something like:

auto f = getFunction!( safe void function(int))("package.module.symbol");

and have it properly mangle the expected function name and pull it from 
the dynamic library.

-Steve
Mar 13 2020
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via
Digitalmars-d-learn wrote:
[...]

 I would expect that something could be written to turn a signature
 string into a mangling and also provide the correct type upon return.
 Something like:
 
 auto f = getFunction!( safe void function(int))("package.module.symbol");
 
 and have it properly mangle the expected function name and pull it
 from the dynamic library.


[...]

This would still have to be  trusted, of course, since there's no
telling what's actually inside the object file.

But this sort of facility totally should be in Phobos, or at least in
some dub package somewhere.  It will make working with dynamically
loaded libraries in D so much more convenient.


T

-- 
If you think you are too small to make a difference, try sleeping in a closed
room with a mosquito. -- Jan van Steenbergen
Mar 13 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/13/20 5:06 PM, H. S. Teoh wrote:

 On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 [...]

 I would expect that something could be written to turn a signature
 string into a mangling and also provide the correct type upon return.
 Something like:

 auto f = getFunction!( safe void function(int))("package.module.symbol");

 and have it properly mangle the expected function name and pull it
 from the dynamic library.


 [...]
 
 This would still have to be  trusted, of course, since there's no
 telling what's actually inside the object file.


Of course it has to be trusted. The underlying function uses void *.

But not because you can't trust the mangling.


 But this sort of facility totally should be in Phobos, or at least in
 some dub package somewhere.  It will make working with dynamically
 loaded libraries in D so much more convenient.


I'm not really motivated to make such a package, as I don't do any 
software with runtime loading of libraries. But I bet it would be either 
accepted into Phobos and/or wouldn't be that hard to add for a library.

-Steve
Mar 15 2020
prev sibling parent reply wjoe <invalid example.com> writes:
On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer 
wrote:

 On 3/13/20 4:22 PM, wjoe wrote:
 I would expect that something could be written to turn a 
 signature string into a mangling and also provide the correct 
 type upon return. Something like:

 auto f = getFunction!( safe void 
 function(int))("package.module.symbol");

 and have it properly mangle the expected function name and pull 
 it from the dynamic library.

 -Steve


Thanks for your reply.

core.demangle: mangle; comes to mind. And in fact, because the 
exports weren't extern(C), that's how I imported the symbols; 
prior to reading H. S. Teoh's reply. That's when I realized that 
I've got a problem.

Even though I know that the function being exported are all 
compiled  safe, that doesn't mean I can count on the fact.
Since a plugin is a separate file it can be swapped out with a 
version that exports all the same (forged) symbols names but with 
a  system implementation.
Forging these symbol names is as easy as something like 
mangle!(int function(int))("a.b")); (=_D1a1bPFiZi) and 
copy/pasting that into pragma(mangle, "_D1a1bPFiZi") and voila, 
the loader can nothing but trust that this is true.

Forging is maybe too hard a word but my vocabulary lacks a better 
one.
Mar 14 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/14/20 6:06 AM, wjoe wrote:

 On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer wrote:

 On 3/13/20 4:22 PM, wjoe wrote:
 I would expect that something could be written to turn a signature 
 string into a mangling and also provide the correct type upon return. 
 Something like:

 auto f = getFunction!( safe void function(int))("package.module.symbol");

 and have it properly mangle the expected function name and pull it 
 from the dynamic library.


 
 Thanks for your reply.
 
 core.demangle: mangle; comes to mind. And in fact, because the exports 
 weren't extern(C), that's how I imported the symbols; prior to reading 
 H. S. Teoh's reply. That's when I realized that I've got a problem.
 
 Even though I know that the function being exported are all compiled 
  safe, that doesn't mean I can count on the fact.
 Since a plugin is a separate file it can be swapped out with a version 
 that exports all the same (forged) symbols names but with a  system 
 implementation.
 Forging these symbol names is as easy as something like mangle!(int 
 function(int))("a.b")); (=_D1a1bPFiZi) and copy/pasting that into 
 pragma(mangle, "_D1a1bPFiZi") and voila, the loader can nothing but 
 trust that this is true.
 
 Forging is maybe too hard a word but my vocabulary lacks a better one.


My original point is that forging is possible without using a dynamic 
library. Anyone who has pragma(mangle)'d the symbol can "forge" anything 
they want, it's not that hard.

Even without using pragma(mangle, it's possible):

mod1.d:

__gshared someGlobal;
extern(C) void realImpl() // no mangling
{
    someGlobal = 5; // yay side effects!
    *(cast(ubyte*)0xdeadbeef) = 5; // yay no safety!
    throw new Exception("yay, I can throw!");
}

mod2.d:

extern(C) void realImpl()  safe pure nothrow;

void foo()  safe pure nothrow // mangled properly
{
    realImpl();
}

So why worry about the dynamic library case? I would say if you obey the 
mangling rules, the compiler should trust the mangling.

-Steve
Mar 15 2020
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 13, 2020 at 08:22:53PM +0000, wjoe via Digitalmars-d-learn wrote:
[...]

 So from what I understand, because, at least on Posix, since there's
 only a symbol name there's nothing I can do in my loader to verify
 that a function is or does what it claim to be/do.


[...]

As far as I know, it's the same thing on Windows PE format.

Basically, once your code is compiled down to the object file level,
there really isn't anything at the level of higher programming language
concepts anymore, like type signatures and stuff.  As far as object
files are concerned, it's just a bunch of opaque binary data with string
labels tacked on them, plus some extra information like relocation data
and other such OS-level concepts.  What's in that opaque binary data
isn't really the purview of the object file format; things like types
and function signatures are an interpretation laid upon the data by
higher-level application code. The OS doesn't know what it is, and
doesn't care (nor should it, that's not its job).

So given some arbitrary object file, there's really no real guarantee as
to what the contents are inside.  Anybody can craft an object file that
exports symbol names that look like the symbols generated by some
higher-level programming language, but the actual binary data the names
point to may do something completely different.  In fact, this is
exactly why pragma(mangle) is so useful: Adam Ruppe's jni.d, for
example, makes extensive use of this in order to make the D code inside
the object file appear like Java JNI symbols to the JVM.  D's C++
interop is also based on the same concept: export symbols that look like
C++ mangled symbols, but inside is actually D code, not C++ code.

Far from being "disappointing", I think this stuff is very powerful, and
lots of fun if you do it right.  But it does come with the caveat that
you're essentially meddling around under the hood, so it's your
responsibility not to do something that will cause the engine to blow
up.  Hence the  trusted tag.


T

-- 
Never step over a puddle, always step around it. Chances are that whatever made
it is still dripping.
Mar 13 2020