www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - pass a delegate to an API as a context pointer?

reply teo <teo.ubuntu yahoo.com> writes:
Hi *, is it possible to pass a delegate to an API function which takes a
pointer to a callback function and a context pointer? The code bellow describes
what I'm trying to achieve.

// C-Library
typedef void (*callback)(void *context);
void xyz(callback handler, void *context);

// D-Program
alias void function(void *context) callback;
extern(C) void xyz(callback handler, void *context);
alias int delegate() foo;

class A
{
	int abc() { return 1; }
}
class B
{
	static void handler(void *context)
	{
		foo f = cast(foo)context;
		int i = f();	// call A.abc();
	}
	void test(foo f)
	{
		xyz(cast(callback)handler, cast(void*)f);
	}
}
void main()
{
	A a = new A();
	B b = new B();
	b.test(&a.abc);
}

Thanks for any tips!
-- teo
Jul 02 2007
next sibling parent reply Russell Lewis <webmaster villagersonline.com> writes:
teo wrote:
 Hi *, is it possible to pass a delegate to an API function which takes a
pointer to a callback function and a context pointer? The code bellow describes
what I'm trying to achieve.
 
 // C-Library
 typedef void (*callback)(void *context);
 void xyz(callback handler, void *context);
 
 // D-Program
 alias void function(void *context) callback;
 extern(C) void xyz(callback handler, void *context);
 alias int delegate() foo;
Your code below shouldn't work because void* is 4 bytes (a single pointer) whereas a delegate is 8 bytes (two pointers). But you can do this: struct CallbackBouncer { int delegate() whatToCall; } extern(C) void CallbackBouncer_callback(void *ptr) { CallbackBouncer *cb = cast(CallbackBouncer*)ptr; ptr.whatToCall(); } void main() { A a = new A(); CallbackBouncer *cb = new CallbackBouncer[1]; cb.whatToCall = &a.abc; xyz( CallbackBouncer_callback, cast(void*)cb); } // A quibble: notice that the 'int' return code in // A.abc is being ignored b/c C expects a return code // of void.
 class A
 {
 	int abc() { return 1; }
 }
 class B
 {
 	static void handler(void *context)
 	{
 		foo f = cast(foo)context;
 		int i = f();	// call A.abc();
 	}
 	void test(foo f)
 	{
 		xyz(cast(callback)handler, cast(void*)f);
 	}
 }
 void main()
 {
 	A a = new A();
 	B b = new B();
 	b.test(&a.abc);
 }
Jul 02 2007
next sibling parent reply Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Russell Lewis wrote:
 teo wrote:
 
 Hi *, is it possible to pass a delegate to an API function which takes 
 a pointer to a callback function and a context pointer? The code 
 bellow describes what I'm trying to achieve.

 // C-Library
 typedef void (*callback)(void *context);
 void xyz(callback handler, void *context);

 // D-Program
 alias void function(void *context) callback;
 extern(C) void xyz(callback handler, void *context);
 alias int delegate() foo;
Your code below shouldn't work because void* is 4 bytes (a single pointer) whereas a delegate is 8 bytes (two pointers). But you can do this: struct CallbackBouncer { int delegate() whatToCall; } extern(C) void CallbackBouncer_callback(void *ptr) { CallbackBouncer *cb = cast(CallbackBouncer*)ptr; ptr.whatToCall(); } void main() { A a = new A(); CallbackBouncer *cb = new CallbackBouncer[1]; cb.whatToCall = &a.abc; xyz( CallbackBouncer_callback, cast(void*)cb); } // A quibble: notice that the 'int' return code in // A.abc is being ignored b/c C expects a return code // of void.
There is a serious problem with this. The garbage collector will not scan the C library for references. As far as it is concerned, that struct on the heap will have no references to it (assuming your program is more interesting than a single main() function, and this callback is being assigned in some function that returns), and it might be collected. Furthermore, if that delegate holds the only reference to that 'A' instance, it might be collected, too. You'll need to somehow keep a reference to that struct locally (that is, within your D code). Also, a minor quibble: You have to use & when getting a pointer to a function in D. -- Kirk McDonald http://kirkmcdonald.blogspot.com Pyd: Connecting D and Python http://pyd.dsource.org
Jul 02 2007
next sibling parent reply teo <teo.ubuntu yahoo.com> writes:
Kirk McDonald Wrote:
 There is a serious problem with this. The garbage collector will not 
 scan the C library for references. As far as it is concerned, that 
 struct on the heap will have no references to it (assuming your program 
 is more interesting than a single main() function, and this callback is 
 being assigned in some function that returns), and it might be 
 collected. Furthermore, if that delegate holds the only reference to 
 that 'A' instance, it might be collected, too.
 
 You'll need to somehow keep a reference to that struct locally (that is, 
 within your D code).
Is there any way to force the garbage collector to "cleanup", so I can experiment with it and see where my software breaks? Perhaps I do have some weak references.
Jul 03 2007
parent Kirk McDonald <kirklin.mcdonald gmail.com> writes:
teo wrote:
 Kirk McDonald Wrote:
 There is a serious problem with this. The garbage collector will not 
 scan the C library for references. As far as it is concerned, that 
 struct on the heap will have no references to it (assuming your program 
 is more interesting than a single main() function, and this callback is 
 being assigned in some function that returns), and it might be 
 collected. Furthermore, if that delegate holds the only reference to 
 that 'A' instance, it might be collected, too.

 You'll need to somehow keep a reference to that struct locally (that is, 
 within your D code).
Is there any way to force the garbage collector to "cleanup", so I can experiment with it and see where my software breaks? Perhaps I do have some weak references.
Yes: std.gc.fullCollect() -- Kirk McDonald http://kirkmcdonald.blogspot.com Pyd: Connecting D and Python http://pyd.dsource.org
Jul 03 2007
prev sibling parent Russell Lewis <webmaster villagersonline.com> writes:
Kirk McDonald wrote:
 Russell Lewis wrote:
 teo wrote:

 Hi *, is it possible to pass a delegate to an API function which 
 takes a pointer to a callback function and a context pointer? The 
 code bellow describes what I'm trying to achieve.

 // C-Library
 typedef void (*callback)(void *context);
 void xyz(callback handler, void *context);

 // D-Program
 alias void function(void *context) callback;
 extern(C) void xyz(callback handler, void *context);
 alias int delegate() foo;
Your code below shouldn't work because void* is 4 bytes (a single pointer) whereas a delegate is 8 bytes (two pointers). But you can do this: struct CallbackBouncer { int delegate() whatToCall; } extern(C) void CallbackBouncer_callback(void *ptr) { CallbackBouncer *cb = cast(CallbackBouncer*)ptr; ptr.whatToCall(); } void main() { A a = new A(); CallbackBouncer *cb = new CallbackBouncer[1]; cb.whatToCall = &a.abc; xyz( CallbackBouncer_callback, cast(void*)cb); } // A quibble: notice that the 'int' return code in // A.abc is being ignored b/c C expects a return code // of void.
There is a serious problem with this. The garbage collector will not scan the C library for references. As far as it is concerned, that struct on the heap will have no references to it (assuming your program is more interesting than a single main() function, and this callback is being assigned in some function that returns), and it might be collected. Furthermore, if that delegate holds the only reference to that 'A' instance, it might be collected, too. You'll need to somehow keep a reference to that struct locally (that is, within your D code).
Good catch! As another possible solution, I think that there is an interface into the GC to make something a "root"...that is, something which is inherently *not* garbage. So you could save the struct that way, too.
 Also, a minor quibble: You have to use & when getting a pointer to a 
 function in D.
True.
Jul 03 2007
prev sibling parent reply teo <teo.ubuntu yahoo.com> writes:
Russell Lewis Wrote:
 Your code below shouldn't work because void* is 4 bytes (a single 
 pointer) whereas a delegate is 8 bytes (two pointers).  But you can do this:
So, my only chance is if I wrap the delegate somehow. Thanks Russell.
Jul 03 2007
parent reply teo <teo.ubuntu yahoo.com> writes:
teo Wrote:

 Russell Lewis Wrote:
 Your code below shouldn't work because void* is 4 bytes (a single 
 pointer) whereas a delegate is 8 bytes (two pointers).  But you can do this:
So, my only chance is if I wrap the delegate somehow. Thanks Russell.
By the way, I tried to pass the address of the delegate like this: (for details see my first post in this thread) void test(foo f) { xyz(cast(callback)&handler, cast(void*)&f); } Unfortunately I wasn't able to convert it back to delegate later in the callback: static void handler(void *context) { foo f = *(cast(foo*)context); int i = f(); // call A.abc(); } It simply crashes. The context value is exactly the same as &f (in test). Any ideas why? Russell? Kirk?
Jul 03 2007
parent reply Jascha Wetzel <firstname mainia.de> writes:
teo wrote:
 teo Wrote:
 
 Russell Lewis Wrote:
 Your code below shouldn't work because void* is 4 bytes (a single 
 pointer) whereas a delegate is 8 bytes (two pointers).  But you can do this:
So, my only chance is if I wrap the delegate somehow. Thanks Russell.
By the way, I tried to pass the address of the delegate like this: (for details see my first post in this thread) void test(foo f) { xyz(cast(callback)&handler, cast(void*)&f); } Unfortunately I wasn't able to convert it back to delegate later in the callback: static void handler(void *context) { foo f = *(cast(foo*)context); int i = f(); // call A.abc(); } It simply crashes. The context value is exactly the same as &f (in test). Any ideas why? Russell? Kirk?
this is most likely because the pointer doesn't exist any more when the C library calls back. the &f from test() becomes invalid when test returns, because f goes out of scope. you need to put the delegate in a place that lasts longer. if you don't have a function scope that lives long enough, put it on the heap and call std.gc.addRoot on it to make sure it's not collected by the GC.
Jul 03 2007
next sibling parent reply teo <teo.ubuntu yahoo.com> writes:
Jascha Wetzel Wrote:

 teo wrote:
 teo Wrote:
 
 Russell Lewis Wrote:
 Your code below shouldn't work because void* is 4 bytes (a single 
 pointer) whereas a delegate is 8 bytes (two pointers).  But you can do this:
So, my only chance is if I wrap the delegate somehow. Thanks Russell.
By the way, I tried to pass the address of the delegate like this: (for details see my first post in this thread) void test(foo f) { xyz(cast(callback)&handler, cast(void*)&f); } Unfortunately I wasn't able to convert it back to delegate later in the callback: static void handler(void *context) { foo f = *(cast(foo*)context); int i = f(); // call A.abc(); } It simply crashes. The context value is exactly the same as &f (in test). Any ideas why? Russell? Kirk?
this is most likely because the pointer doesn't exist any more when the C library calls back. the &f from test() becomes invalid when test returns, because f goes out of scope. you need to put the delegate in a place that lasts longer. if you don't have a function scope that lives long enough, put it on the heap and call std.gc.addRoot on it to make sure it's not collected by the GC.
Good point, however the "a" object is still alive. main() doesn't exit after calling b.test(&a.abc). There's more stuff bellow. So, the delegate is there and its address should still be valid. Or maybe you mean that the delegate foo f in the context of void test(foo f) function and &a.abc isn't the same thing? All I want right now is to understand what happen.
Jul 03 2007
parent Jascha Wetzel <firstname mainia.de> writes:
teo wrote:
 Or maybe you mean that the delegate foo f in the context of void test(foo f)
function and &a.abc isn't the same thing?
yep, it's passed by value, therefore f and &a.abc are stored in different places. but also check the calling conventions, as frits pointed out.
 All I want right now is to understand what happen.
if you know some x86 assembler, just use obj2asm or ddbg to look "inside" the code.
Jul 03 2007
prev sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Jascha Wetzel wrote:
 teo wrote:
 teo Wrote:

 Russell Lewis Wrote:
 Your code below shouldn't work because void* is 4 bytes (a single 
 pointer) whereas a delegate is 8 bytes (two pointers).  But you can 
 do this:
So, my only chance is if I wrap the delegate somehow. Thanks Russell.
By the way, I tried to pass the address of the delegate like this: (for details see my first post in this thread)
From your original post: --- alias void function(void *context) callback; extern(C) void xyz(callback handler, void *context); --- That first line also needs an extern(C), or callback will be a pointer to a function with D calling convention instead of C calling convention.
 void test(foo f)
 {
     xyz(cast(callback)&handler, cast(void*)&f);
Those casts shouldn't be necessary. (And make sure the C function won't store the pointers somewhere, since the pointer to f will be invalid after test exits since it's a local variable)
 }

 Unfortunately I wasn't able to convert it back to delegate later in 
 the callback:

 static void handler(void *context)
 {
     foo f = *(cast(foo*)context);
     int i = f();    // call A.abc();
 }
This one also needs extern(C) to allow the C function to call it as a callback.
 It simply crashes. The context value is exactly the same as &f (in 
 test). Any ideas why? Russell? Kirk?
this is most likely because the pointer doesn't exist any more when the C library calls back. the &f from test() becomes invalid when test returns, because f goes out of scope.
I think it's probably just the mismatching calling convention. The handler function expects the argument in EAX (assuming DMD) while the C function puts it on the stack. This causes the D code to dereference whatever is left in EAX, which will cause a crash unless you're very "lucky".
Jul 03 2007
parent teo <teo.ubuntu yahoo.com> writes:
Frits van Bommel Wrote:
 void test(foo f)
 {
     xyz(cast(callback)&handler, cast(void*)&f);
Those casts shouldn't be necessary. (And make sure the C function won't store the pointers somewhere, since the pointer to f will be invalid after test exits since it's a local variable)
The pointer to f is stored. The C-library may call my callback say one hour later or every 5 sec.
 I think it's probably just the mismatching calling convention. The 
 handler function expects the argument in EAX (assuming DMD) while the C 
 function puts it on the stack. This causes the D code to dereference 
 whatever is left in EAX, which will cause a crash unless you're very 
 "lucky".
Good that I'm not that "lucky". Otherwise I would have thought that I've done my job well... I'll check the calling convention and report back. Right now I cannot test.
Jul 03 2007
prev sibling parent reply teo <teo.ubuntu yahoo.com> writes:
teo Wrote:

 Hi *, is it possible to pass a delegate to an API function which takes a
pointer to a callback function and a context pointer? The code bellow describes
what I'm trying to achieve.
 
<snip> I got it! Thanks to all of you Russell, Kirk, Jascha and Frits! Let me summarize: the idea is to keep the delegate in a member variable of the class and pass the address of that variable as a context data. This way the address remains valid when the callback is called from within the C-library. Perhaps it can be done in a more elegant way, but I'm satisfied with this solution: // C-Library typedef void (*callback)(void *context); void xyz(callback handler, void *context); // D-Program alias extern(C) void function(void *context) callback; extern(C) void xyz(callback handler, void *context); alias int delegate() foo; class A { int abc() { return 1; } } class B { foo _f = null; static extern(C) void handler(void *context) { foo f = *(cast(foo*)context); int i = f(); // call A.abc(); } void test(foo f) { _f = f; // this way the address remains valid xyz(&handler, &_f); } } void main() { A a = new A(); B b = new B(); b.test(&a.abc); // ... return; } By the way, I'm going to play with garbage collector, so I might have further questions.
Jul 03 2007
parent Russell Lewis <webmaster villagersonline.com> writes:
teo wrote:
 teo Wrote:
 
 Hi *, is it possible to pass a delegate to an API function which takes a
pointer to a callback function and a context pointer? The code bellow describes
what I'm trying to achieve.
<snip> I got it! Thanks to all of you Russell, Kirk, Jascha and Frits! Let me summarize: the idea is to keep the delegate in a member variable of the class and pass the address of that variable as a context data. This way the address remains valid when the callback is called from within the C-library. Perhaps it can be done in a more elegant way, but I'm satisfied with this solution: // C-Library typedef void (*callback)(void *context); void xyz(callback handler, void *context); // D-Program alias extern(C) void function(void *context) callback; extern(C) void xyz(callback handler, void *context); alias int delegate() foo; class A { int abc() { return 1; } } class B { foo _f = null; static extern(C) void handler(void *context) { foo f = *(cast(foo*)context); int i = f(); // call A.abc(); } void test(foo f) { _f = f; // this way the address remains valid xyz(&handler, &_f); } } void main() { A a = new A(); B b = new B(); b.test(&a.abc); // ... return; }
I don't see any problems with this implementation as it stands, but remember: this only works if xyz() calls your handler back synchronously. If it calls back asynchronously (that is, after main() exits), then the GC may have run and cleaned up the 'B b' variable you declared in main(). Obviously, that doesn't seem likely if with main()...but it is quite possible if the function is just some ordinary function in a larger program. Russ
Jul 05 2007