• Matthew (38/38) Dec 31 2003 Guys (and gals, if there are any yet on the D ng)
• Matthew (53/91) Dec 31 2003 Too late chaps and chappettes - I've finished it! (It shows how much wri...

"Matthew" <matthew.hat stlsoft.dot.org> writes:

Guys (and gals, if there are any yet on the D ng)

I've dug out the extent of my work on the D dynamic library loader, and I'm
keen to get some input wrt the overall design before implementing it for
Linux. I encountered these issues several years ago while designing on a
platform-independent component-based parser architecture  -
<modesty-break>which went on to become the world standard for the (>50%
global market share) client</modesty-break>.

Basically, on Win32 one can effect reference counting on dynamic libraries
via the LoadLibrary, FreeLibrary and GetModuleFileName functions.

 - you load the library via LoadLibrary
 - you "release" the library via FreeLibrary
 - you "increment" a reference via calls to GetModuleFileName and then
LoadLibrary

When working with dynamic architectures, it's sometimes really useful to be
able to have client-side (i.e. in your code, rather than just relying on the
system) reference-counted dynamic loading. Unfortunately, UNIX systems don't
provide the analogous functionality. The way I've worked around this in the
past (in C++ land) is to have a centralising API, which is the only port of
call for library loading and unloading, e.g.

  HXModule ExeModule_Load(char const *name);
  HXModule ExeModule_AddRef(HXModule hxmod);
  HXModule ExeModule_Release(HXModule hxmod);
  void            *ExeModule_GetSymbol(HXModule hxmod, char const *symbol);

In non-Win32 systems, the library maintains internal records that associate
the module handle with the underlying operating system handle, and either
implements an internal reference-count, or associates the module handle with
the module name. Naturally, we'd use associative arrays for speed, but even
linear management is not going to be troublesome given the reasonable
dynamic library limits of a given process, and the relative costs of the
underlying OS loading call. In fact, if we internalise the reference-count
and only do a single load/free, this'll probably be faster.

This is what I'd like to do for D, so that there's a single API for all
concerned. Naturally, being as how we're all supposed to be heart-and-soul
Object Obsessed Desparados these days, I will also provide a wrapping auto
class along with the function-based API.

I'd appreciate any feedback you may wish to give.

Matthew

P.S. Happy New Year to all, btw.

"Matthew" <matthew.hat stlsoft.dot.org> writes:

Too late chaps and chappettes - I've finished it! (It shows how much writing
a book can adle the mind, as this was my way of taking a morning off in
celebration of it being a new year. <g> )

It's fully platform-independent, and works with both the Win32 LoadLibrary
and Linux dlopen APIs (albeit that I tested it on Win32 with my UNIX
Emulation lib).

The module's std.loader, and contains an API, and a thin class over the top
of that.

    // Functions
    typedef void    *HXModule;

    public int ExeModule_Init(); // Called internally, in static
initialisation
    public void ExeModule_Uninit(); // Called internally, in static
un-initialisation

    public HXModule ExeModule_Load(in char[] moduleName);
    public HXModule ExeModule_AddRef(in HXModule hModule);
    public void ExeModule_Release(inout HXModule hModule);

    public void *ExeModule_GetSymbol(inout HXModule hModule, in char[]
symbolName);

    public char[] ExeModule_Error();


    // Class(es)

    class ExeModuleException;

    auto class ExeModule
    {
        this(inout HXModule hModule, boolean bTakeOwnership);
        this(char[] moduleName); // Throws ExeModuleException
        ~this();
        void Close();

        void *GetSymbol(in char[] symbolName); // Throws ExeModuleException
        void *FindSymbol(in char[] symbolName); // Returns NULL if not there

        HXModule Handle();
        char[] Path();
    }


I've just sent it to Walter, so hopefully it might make it into Phobos for
0.77. It's even got comments!

The one caveat is that it's not thread-safe in the Linux version, as there
are several threading aspects I'm hoping to get Walter to include in D in
the near future that will make most of the issues moot.

Now back to that damn book ...

Cheers

Matthew



"Matthew" <matthew.hat stlsoft.dot.org> wrote in message
news:bsvmh0$2gap$1 digitaldaemon.com...
 Guys (and gals, if there are any yet on the D ng)

 I've dug out the extent of my work on the D dynamic library loader, and

I'm
 keen to get some input wrt the overall design before implementing it for
 Linux. I encountered these issues several years ago while designing on a
 platform-independent component-based parser architecture  -
 <modesty-break>which went on to become the world standard for the (>50%
 global market share) client</modesty-break>.

 Basically, on Win32 one can effect reference counting on dynamic libraries
 via the LoadLibrary, FreeLibrary and GetModuleFileName functions.

  - you load the library via LoadLibrary
  - you "release" the library via FreeLibrary
  - you "increment" a reference via calls to GetModuleFileName and then
 LoadLibrary

 When working with dynamic architectures, it's sometimes really useful to

be
 able to have client-side (i.e. in your code, rather than just relying on

the
 system) reference-counted dynamic loading. Unfortunately, UNIX systems

don't
 provide the analogous functionality. The way I've worked around this in

the
 past (in C++ land) is to have a centralising API, which is the only port

of
 call for library loading and unloading, e.g.

   HXModule ExeModule_Load(char const *name);
   HXModule ExeModule_AddRef(HXModule hxmod);
   HXModule ExeModule_Release(HXModule hxmod);
   void            *ExeModule_GetSymbol(HXModule hxmod, char const

*symbol);
 In non-Win32 systems, the library maintains internal records that

associate
 the module handle with the underlying operating system handle, and either
 implements an internal reference-count, or associates the module handle

with
 the module name. Naturally, we'd use associative arrays for speed, but

even
 linear management is not going to be troublesome given the reasonable
 dynamic library limits of a given process, and the relative costs of the
 underlying OS loading call. In fact, if we internalise the reference-count
 and only do a single load/free, this'll probably be faster.

 This is what I'd like to do for D, so that there's a single API for all
 concerned. Naturally, being as how we're all supposed to be heart-and-soul
 Object Obsessed Desparados these days, I will also provide a wrapping auto
 class along with the function-based API.

 I'd appreciate any feedback you may wish to give.

 Matthew

 P.S. Happy New Year to all, btw.