www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Irritating shortcoming with modules and externs

reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
Maybe it's not so much a shortcoming as it is just dumb.

Say I want to create a library which has a "pluggable function."  By that I 
mean that I want the library to reference an external function, defined by 
the user of the library.  So, in my library module, I write:


module mod;

extern(D) int intFunc();

void spork()
{
    for(int x = intFunc(); x != 0; x = intFunc())
        writefln(x);
}


Something purposeless, but it shows that I want to be able to use this 
function within this module.

So I compile this to an obj/lib, and create my "host" program which uses the 
library.  I have this:


module main;

import mod;

int intFunc()
{
    static int[5] ints = [4, 3, 2, 1, 0];
    static int counter = 0;

    counter = (counter + 1) % 5;
    return ints[counter];
}

void main()
{
    spork();
}


The problem: even though I've defined my intFunc() to match the signature to 
be the same as the signature expected by the library, the linker won't 
resolve the reference because the function names are mangled with the module 
names as well.  Meaning that the linker is looking for mod.intFunc, but I've 
given it main.intFunc.  So I limit the ability to use the library by 
dictating that the user define the intFunc in a certain module (and it gets 
worse when multiple source levels come up).

One workaround is to put "extern(C)" before the reference in the library and 
before the definition, but I don't know if that interferes with the 
exception handling stuff.

I doubt there's any robust solution to this, but it's frustrating 
nonetheless.  This "feature" also dictates that import-only files (to go 
along with closed-source modules) must follow the exact directory structure 
as the original source so that the mangling matches up, instead of putting 
all the declarations into one big import file.  A fairly minor 
inconvenience, but still..
May 16 2006
parent reply Derek Parnell <derek psych.ward> writes:
On Tue, 16 May 2006 18:22:40 -0400, Jarrett Billingsley wrote:


 Maybe it's not so much a shortcoming as it is just dumb.
 
 Say I want to create a library which has a "pluggable function."  By that I 
 mean that I want the library to reference an external function, defined by 
 the user of the library.  So, in my library module, I write:
 
 module mod;
 
 extern(D) int intFunc();
 
 void spork()
 {
     for(int x = intFunc(); x != 0; x = intFunc())
         writefln(x);
 }
 
 Something purposeless, but it shows that I want to be able to use this 
 function within this module.
 
 So I compile this to an obj/lib, and create my "host" program which uses the 
 library.  I have this:
 
 module main;
 
 import mod;
 
 int intFunc()
 {
     static int[5] ints = [4, 3, 2, 1, 0];
     static int counter = 0;
 
     counter = (counter + 1) % 5;
     return ints[counter];
 }
 
 void main()
 {
     spork();
 }
 
 The problem: even though I've defined my intFunc() to match the signature to 
 be the same as the signature expected by the library, the linker won't 
 resolve the reference because the function names are mangled with the module 
 names as well.  Meaning that the linker is looking for mod.intFunc, but I've 
 given it main.intFunc.  So I limit the ability to use the library by 
 dictating that the user define the intFunc in a certain module (and it gets 
 worse when multiple source levels come up).
 
 One workaround is to put "extern(C)" before the reference in the library and 
 before the definition, but I don't know if that interferes with the 
 exception handling stuff.
 
 I doubt there's any robust solution to this, but it's frustrating 
 nonetheless.  


I've hit this limitation a few times too. The method I've used to get
around it (which might actually be the D-way) is to isolate the "pluggable"
function in its own module.

// --------- mod.d --------
module mod;
import std.stdio;
import oth;
void spork()
{
    for(int x = oth.intFunc(); x != 0; x = oth.intFunc())
        writefln(x);
}
//  -------- end mod.d -----------

// ---------- oth_r.d  ------------
// This is the source code for the library object.
module oth;
int intFunc()
{
    static int[5] ints = [4, 3, 2, 1, 0];
    static int counter = 0;

    counter = (counter + 1) % 5;
    return ints[counter];
}

// --------end of oth_r.d -----------

// ---------- oth_h.d  ------------
// This is the 'header' file for the library object.
module oth;
int intFunc();
// --------end of oth_h.d -----------

// -------- main. d ----------
module main;
import mod;
void main()
{
    mod.spork();
}
// ------- end of main.d -----------


To compile these, first I created the library...


 copy oth_r.d oth.d /Y
 build oth -full -Tmylib.lib


Then I created the executable ...


 copy oth_h.d oth.d /Y
 build main -full mylib.lib


Then I ran it ...


main


  3
  2
  1


-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocracy!"
17/05/2006 10:14:19 AM
May 16 2006
parent reply Brad Roberts <braddr puremagic.com> writes:
On Wed, 17 May 2006, Derek Parnell wrote:


 On Tue, 16 May 2006 18:22:40 -0400, Jarrett Billingsley wrote:
 

 Say I want to create a library which has a "pluggable function."  By 
 that I mean that I want the library to reference an external function, 
 defined by the user of the library.  So, in my library module, I 
 write:




<snip>


 The problem: even though I've defined my intFunc() to match the signature to 
 be the same as the signature expected by the library, the linker won't 
 resolve the reference because the function names are mangled with the module 
 names as well.  Meaning that the linker is looking for mod.intFunc, but I've 
 given it main.intFunc.  So I limit the ability to use the library by 
 dictating that the user define the intFunc in a certain module (and it gets 
 worse when multiple source levels come up).
 
 One workaround is to put "extern(C)" before the reference in the library and 
 before the definition, but I don't know if that interferes with the 
 exception handling stuff.
 
 I doubt there's any robust solution to this, but it's frustrating 
 nonetheless.  


 
 I've hit this limitation a few times too. The method I've used to get
 around it (which might actually be the D-way) is to isolate the "pluggable"
 function in its own module.


How about moving to a delegate model instead of a direct global symbol 
model?  It's much more obvious, imho.

Later,
Brad
May 16 2006
next sibling parent Derek Parnell <derek psych.ward> writes:
On Tue, 16 May 2006 18:44:39 -0700, Brad Roberts wrote:


 On Wed, 17 May 2006, Derek Parnell wrote:
 

 On Tue, 16 May 2006 18:22:40 -0400, Jarrett Billingsley wrote:
 

 Say I want to create a library which has a "pluggable function."  By 
 that I mean that I want the library to reference an external function, 
 defined by the user of the library.  So, in my library module, I 
 write:




 
 <snip>
 

 The problem: even though I've defined my intFunc() to match the signature to 
 be the same as the signature expected by the library, the linker won't 
 resolve the reference because the function names are mangled with the module 
 names as well.  Meaning that the linker is looking for mod.intFunc, but I've 
 given it main.intFunc.  So I limit the ability to use the library by 
 dictating that the user define the intFunc in a certain module (and it gets 
 worse when multiple source levels come up).
 
 One workaround is to put "extern(C)" before the reference in the library and 
 before the definition, but I don't know if that interferes with the 
 exception handling stuff.
 
 I doubt there's any robust solution to this, but it's frustrating 
 nonetheless.  


 
 I've hit this limitation a few times too. The method I've used to get
 around it (which might actually be the D-way) is to isolate the "pluggable"
 function in its own module.


 
 How about moving to a delegate model instead of a direct global symbol 
 model?  It's much more obvious, imho.
 
 Later,
 Brad


That is actually what I did in Build ;-)

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocracy!"
17/05/2006 1:01:44 PM
May 16 2006
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Brad Roberts" <braddr puremagic.com> wrote in message 
news:Pine.LNX.4.64.0605161842050.2422 bellevue.puremagic.com...


 How about moving to a delegate model instead of a direct global symbol
 model?  It's much more obvious, imho.


That's probably what I'll end up doing, although the function that I wanted 
to do this with was main (so basically I could "intercept" the call to main 
in my library, and call the user-defined "LibMain" or something to that 
effect).  Of course, seeing that D is an OOP language, I might as well make 
some kind of context class that wraps the entire program, ending you up with 
something like:

abstract class Context
{
...
}

class MyContext : Context
{
    // override all those stubbed-out methods
}

void main(char[][] args)
{
    MyContext c = new MyContext();
    c.main(args);
}
May 16 2006
parent reply Mike Parker <aldacron71 yahoo.com> writes:
Jarrett Billingsley wrote:

 

 How about moving to a delegate model instead of a direct global symbol
 model?  It's much more obvious, imho.


 
 That's probably what I'll end up doing, although the function that I wanted 
 to do this with was main (so basically I could "intercept" the call to main 
 in my library, and call the user-defined "LibMain" or something to that 
 effect).  Of course, seeing that D is an OOP language, I might as well make 
 some kind of context class that wraps the entire program, ending you up with 
 something like:
 
 abstract class Context
 {
 ...
 }
 
 class MyContext : Context
 {
     // override all those stubbed-out methods
 }
 
 void main(char[][] args)
 {
     MyContext c = new MyContext();
     c.main(args);
 } 
 
 


How about this:


// main.d
module main;

abstract class Context
{
public:
	abstract void run(char[][] args);
	
protected:
	this()
	{
		theContext = this;
	}
}

private Context theContext;

void main(char[][] args)
{
	theContext.run(args);
}

===========================================

// mygame.d
module mygame;

import main;

class MyContext : Context
{
public:
	override void run(char[][] args)
	{
		// blah
	}	
}

static this()
{
	new MyContext();
}


I did something similar with a GameTask system I was working on. Works 
like a charm.
May 16 2006
parent reply Mike Parker <aldacron71 yahoo.com> writes:
Mike Parker wrote:


 
 How about this:
 

 // main.d
 module main;
 
 abstract class Context
 {
 public:
     abstract void run(char[][] args);
     
 protected:
     this()
     {
         theContext = this;
     }
 }
 


Of course, there should be a null check in the constructor so that only 
one primary cotext is ever set:
	if(theConext is null)
		theContext = this;
May 16 2006
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Mike Parker" <aldacron71 yahoo.com> wrote in message 
news:e4ed7h$162h$2 digitaldaemon.com...

 Mike Parker wrote:


 How about this:


 // main.d
 module main;

 abstract class Context
 {
 public:
     abstract void run(char[][] args);
     protected:
     this()
     {
         theContext = this;
     }
 }


 Of course, there should be a null check in the constructor so that only 
 one primary cotext is ever set:
 if(theConext is null)
 theContext = this;


Oh that's really cool :)  I might go with that.
May 17 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Jarrett Billingsley wrote:

 "Mike Parker" <aldacron71 yahoo.com> wrote in message 
 news:e4ed7h$162h$2 digitaldaemon.com...

 Mike Parker wrote:


 How about this:


 // main.d
 module main;

 abstract class Context
 {
 public:
     abstract void run(char[][] args);
     protected:
     this()
     {
         theContext = this;
     }
 }


 Of course, there should be a null check in the constructor so that only 
 one primary cotext is ever set:
 if(theConext is null)
 theContext = this;


 
 Oh that's really cool :)  I might go with that. 
 
 


I do something very similar:

   int main(char[][] args)
   {
     // .. proccess args..
     // ..maybe load some libs (like Derelict)..
     App.appmain();
     return 0;
   }

However, in my case, App is a module. If you only have one Context, and 
it is allways active during the program lifetime, why bother creating a 
singleton instead of simply using a module?

-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
May 20 2006
next sibling parent Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Bruno Medeiros wrote:

 Jarrett Billingsley wrote:
 

 "Mike Parker" <aldacron71 yahoo.com> wrote in message 
 news:e4ed7h$162h$2 digitaldaemon.com...


 Mike Parker wrote:


 How about this:


 // main.d
 module main;

 abstract class Context
 {
 public:
     abstract void run(char[][] args);
     protected:
     this()
     {
         theContext = this;
     }
 }


 Of course, there should be a null check in the constructor so that 
 only one primary cotext is ever set:
 if(theConext is null)
 theContext = this;



 Oh that's really cool :)  I might go with that.


 
 I do something very similar:
 
   int main(char[][] args)
   {
     // .. proccess args..
     // ..maybe load some libs (like Derelict)..
     App.appmain();
     return 0;
   }
 
 However, in my case, App is a module. If you only have one Context, and 
 it is allways active during the program lifetime, why bother creating a 
 singleton instead of simply using a module?
 


The singleton pattern can be useful if you want to pass a reference to the
singleton to 
some function or method later that expects an arbitrary object (maybe
implementing a 
certain interface, for example).  Although, if you only have the one singleton,
and 
there's no possible reason to pass it around, your point is dead-on.

-- Chris Nicholson-Sauls
May 20 2006
prev sibling next sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Bruno Medeiros" <brunodomedeirosATgmail SPAM.com> wrote in message 
news:e4ngb0$24g4$1 digitaldaemon.com...


 I do something very similar:

   int main(char[][] args)
   {
     // .. proccess args..
     // ..maybe load some libs (like Derelict)..
     App.appmain();
     return 0;
   }

 However, in my case, App is a module. If you only have one Context, and it 
 is allways active during the program lifetime, why bother creating a 
 singleton instead of simply using a module?


At least in my case, the nice thing about having a context class is that I 
can derive from it.  I can create some other predefined contexts (i.e. which 
set some things up for you automatically, or which are designed for a 
specific type of program in mind), and by giving them some abstract methods 
meant to be overriden, I can create "framework" contexts which are 
ready-to-use and which can just be derived and have the extra parts filled 
in.  That, and a compiler error (an "unimplemented abstract method" error) 
is nicer than a linker error saying that it can't find an undefined, mangled 
symbol.
May 20 2006
prev sibling parent Mike Parker <aldacron71 yahoo.com> writes:
Bruno Medeiros wrote:


 
 However, in my case, App is a module. If you only have one Context, and 
 it is allways active during the program lifetime, why bother creating a 
 singleton instead of simply using a module?
 


It's not really a singleton that I am using. It's an abstract class that 
sets a specific variable once when the first instance is constructed. 
You can create as many instances as you want after that. In my case, it 
is used in conjunction with a stack-based task (or state) system

In a game, you have different game state transitions. The title screen, 
multiple menu screens, the game play screen, and so on. Encapsulating 
all of these into state instances and using a stack to manage them makes 
it easy to transition from one state to another and back.

The example I gave is not how I'm actually using it, but close enough. 
The first state instance that is instantiated is added to the state 
manager as the default state and is never removed from the stack. This 
is done in the constructor of the abstract base class, allowing the game 
engine to maintain control flow and do a lot of core setup (reading 
config files, initializing the display and audio systems, and so on) 
behind the scenes.

The primary state instance is created in a static module constructor. I 
pass that module to Build, which can then compile the entire game engine 
along with it if I don't precompile it in a library first. I can 
implement multiple games by implementing different modules with static 
constructors that instantiate a default state and pass those to Build 
instead. And, if I wanted, I could implement the entire game using one 
state and never bother implementing others to transition to. It's a 
pluggable system that abstracts away quite a bit.
May 20 2006