www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - function pointer bug?

reply "bitwise" <nothx gmail.com> writes:
I am trying to store a function pointer in a class in a generic 
way(works with all member/non-member/global functions).

In the main() function below, there are two cases(A, B).

If case B is commented out, this code complies/runs fine. 
Otherwise, I get these errors, and the compiler pointing at 
"dg.funcptr = &T[0];" in TestClas.invoke(void*).

Error: this for instanceMethod needs to be type TestClass not 
type main.FuncPtr!(instanceMethod).FuncPtr
Error: cannot implicitly convert expression 
(&(__error).instanceMethod) of type void delegate() to void 
function()
Error: template instance main.FuncPtr!(instanceMethod) error 
instantiating


Am I missing something here or is this a bug?

class TestClass {
     void instanceMethod() {
         writeln("Instance Method!");
     }

     static void staticMethod() {
         writeln("Static Method!");
     }
}

void GlobalMethod() {
     writeln("Global Method!");
}

void invokeFunction(T...)(void *instance){
     alias typeof(T[0])                       method_type;
     alias ReturnType!method_type             return_type;
     alias ParameterTypeTuple!method_type     param_types;
     alias return_type delegate(param_types)  delegate_type;

     delegate_type dg;
     dg.ptr = instance;
     dg.funcptr = &T[0];
     dg();
}

class FuncPtr(T...) {
     void invoke(void *instance) {
         alias typeof(T[0])                       method_type;
         alias ReturnType!method_type             return_type;
         alias ParameterTypeTuple!method_type     param_types;
         alias return_type delegate(param_types)  delegate_type;

         delegate_type dg;
         dg.ptr = instance;
         dg.funcptr = &T[0];
         dg();
     }
}

void main() {
     TestClass testClass = new TestClass();

     // case A
     
invokeFunction!(TestClass.instanceMethod)(cast(void*)testClass);
     invokeFunction!(TestClass.staticMethod)(null);
     invokeFunction!(GlobalMethod)(null);

     // case B
     auto fp1 = new FuncPtr!(TestClass.instanceMethod);
     auto fp2 = new FuncPtr!(TestClass.staticMethod);
     auto fp3 = new FuncPtr!(GlobalMethod);

     fp1.invoke(cast(void*)testClass);
     fp2.invoke(null);
     fp3.invoke(null);
}
Oct 25 2014
next sibling parent "Kagamin" <spam here.lot> writes:
Looks like &T[0] tries to take delegate to instanceMethod, hence 
complains about this type.
Oct 26 2014
prev sibling parent reply "Solomon E" <default avatar.org> writes:
It works after I added 'static' to the declaration of 'invoke()' 
(and import std.stdio, std.traits.)

I fiddled around with it for hours before I tried 'static' there, 
because I've only been studying D for a week, so only about half 
of this code and the error messages made any sense to me when I 
started on it.

Once it started passing all the tests, I still didn't quite get 
how it all worked, because of suspecting there was a typo in the 
code between writing TestClass and testClass. So I tested whether 
specific instances are called (instead of maybe just the first 
one constructed) by adding a static counter to TestClass, and 
storage of the counter by each instance. It looks all right.
Oct 27 2014
parent reply "bitwise" <nothx gmail.com> writes:
On Monday, 27 October 2014 at 16:08:26 UTC, Solomon E wrote:
 It works after I added 'static' to the declaration of 
 'invoke()' (and import std.stdio, std.traits.)

 I fiddled around with it for hours before I tried 'static' 
 there, because I've only been studying D for a week, so only 
 about half of this code and the error messages made any sense 
 to me when I started on it.

 Once it started passing all the tests, I still didn't quite get 
 how it all worked, because of suspecting there was a typo in 
 the code between writing TestClass and testClass. So I tested 
 whether specific instances are called (instead of maybe just 
 the first one constructed) by adding a static counter to 
 TestClass, and storage of the counter by each instance. It 
 looks all right.
I'm pretty sure this is a bug. I'm going to file a report. If one of those template works, both should. I have been trying to build a reflection system over the last few weeks, and noticed different incarnations of this problem several times: <b> &(__error).instanceMethod </b> In certain cases, the compiler seems to drop the enclosing type of the method, but I'm not sure why. This error seems like it may be related some how: enum index = __traits(getVirtualIndex, TestClass.instanceMethod); enum p = TestClass.classinfo.vtbl[index]; The above code will produce this error: Error: typeid(main.TestClass).vtbl is not yet implemented at compile time but if this is the problem, shouldn't Both of the test cases fail?
Oct 27 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 27 Oct 2014 22:17:24 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have been trying to build a reflection system over the last few
 weeks, and noticed different incarnations of this problem several
 times:
=20
 &(__error).instanceMethod
this means that something gone wrong in the process. '(__error)' is the "pseudotype" for failed CTFE/instantiation.
 This error seems like it may be related some how:
=20
 enum index =3D __traits(getVirtualIndex, TestClass.instanceMethod);
 enum p =3D TestClass.classinfo.vtbl[index];
=20
 The above code will produce this error:
 Error: typeid(main.TestClass).vtbl is not yet implemented at
 compile time
=20
 but if this is the problem, shouldn't Both of the test cases fail?
why? everything is ok here. you can get method index in compile-time, but there is no built VMT in compile time. what do you want to achieve with second enum?
Oct 27 2014
parent reply "bitwise" <nothx gmail.com> writes:
 this means that something gone wrong in the process. 
 '(__error)' is the
 "pseudotype" for failed CTFE/instantiation.
But why did the instantiation fail? and more importantly, why did it not have consistent behaviour between the two templates above?
what do you want to achieve
with second enum?
I'm just trying to come up with a test case which may offer some insight into what's going wrong with the code in the OP. for example, I am wondering how far this issue extends; meaning, is it only classinfo.vtbl that's not available at compile time? or is it the address of member functions as a whole that are unavailable? The code below suggests the latter, although it doesn't explicitly state it: static addr = &TestClass.instanceMethod; Error: non-constant expression & instanceMethod I may be missing a subtle difference, but in C++, this code works: class TestAddr { public: virtual void test() { cout << "test" << endl; } }; int main(int argc, const char * argv[]) { TestAddr test; static auto ptr = &TestAddr::test; (test.*ptr)(); return 0; } so why would it not in D?
Oct 27 2014
parent reply "bitwise" <nothx gmail.com> writes:
 quotes self
Here is a better example, showing that virtual function pointers are available at compile time in C++. Essentially, I would expect my D code to function similarly, but it won't compile. class TestAddr { public: virtual void test() { cout << "test" << endl; } }; template<void (TestAddr::*FN)()> class PtrWrapper { public: void invoke(TestAddr *instance) { (instance->*FN)(); } }; int main(int argc, const char * argv[]) { TestAddr test; PtrWrapper<&TestAddr::test> wrapper; wrapper.invoke(&test); return 0; }
Oct 27 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 27 Oct 2014 23:52:38 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 The code below suggests the latter, although it doesn't=20
 explicitly state it:
=20
 static addr =3D &TestClass.instanceMethod;
 Error: non-constant expression & instanceMethod
=20
 I may be missing a subtle difference, but in C++, this code works:
=20
 class TestAddr {
      public: virtual void test() { cout << "test" << endl; }
 };
=20
 int main(int argc, const char * argv[])
 {
      TestAddr test;
      static auto ptr =3D &TestAddr::test;
      (test.*ptr)();
      return 0;
 }
C++ compiler does some trickery behind the curtains. besides, you aren't supposed to make such hackish things easily in D. yet you can: class TestClass { int n =3D 42; void test() { writeln("test: ", n); } } void main () { auto test =3D new TestClass(); void delegate () a; { a.ptr =3D *cast(void**)&test; // protection from opCast() // this can be done as `cast(void*)test;` too enum idx =3D __traits(getVirtualIndex, TestClass.test); a.funcptr =3D cast(void function())TestClass.classinfo.vtbl[idx]; a(); // outputs 'test: 42' } } you need to manually create initialization code that C++ compilers creates behind the curtains. if you aren't in urgent need of that code, may i suggest you to read D books and D specs? D is not C++, and D delegates aren't C++ member function pointers (yet they works nearly the same). this is kind of advanced topic, and i don't think that dumping working source code at you will help without you grasp the low-level mechanics first. besides, you can use CTFE to build wrapper code. Adam Ruppe has that in his jsvar.d, and i have that in my cong.d (cmdcon-ng) too. not that i'm not willing to help you, but i can't see what you understand and what not, so i don't know where i should start explaining.
Oct 27 2014
parent reply "bitwise" <nothx gmail.com> writes:
 C++ compiler does some trickery behind the curtains. besides, 
 you
 aren't supposed to make such hackish things easily in D. yet 
 you can:
There is nothing hackish in the above code. It's a non-type template parameter and a member function pointer. If I was trying to access the (implementation dependant)vtable pointer in C++ to call the function manually, I may concede to calling it a hack, but there is nothing non-standard about the above code.
D is not C++, and D delegates aren't C++ member function 
pointers (yet they works nearly the same).
I was simply trying to show that what I wanted to do was possible in a similar language. I still believe it's a bug in the language. In the original code, both test cases should either both work, or both fail to compile. I have looked through the D docs online, and can't find anything supporting the argument that this is the intended behaviour of the compiler.
Oct 27 2014
parent reply "bitwise" <nothx gmail.com> writes:
 besides, you can use CTFE to build wrapper code. Adam Ruppe 
 has that in
 his jsvar.d, and i have that in my cong.d (cmdcon-ng) too.
I have actually found a work around as well, which was to wrap the actual retrieval of the function address in a lambda, and pass the lambda by template parameter instead: auto getMethod = function() { return &__traits(getMember, SCOPE, member); }; this is later assigned to a delegate when called: delegate_type dg; dg.ptr = instance; dg.funcptr = getMethod(); dg(); The above works for global or static functions too. This seems to me like an unnecessary workaround though.. my current project can be viewed here: https://github.com/bitwise-github/D-Reflection/blob/master/reflection.d#L796 It's been implemented based on Andrei's suggestions here: http://forum.dlang.org/thread/juf7sk$16rl$1 digitalmars.com as far as I could google, no one has attempted this yet. My prototype works pretty much as he has described, less a feature here and there.
Oct 27 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 28 Oct 2014 01:36:01 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have actually found a work around as well, which was to wrap=20
 the actual retrieval of the function address in a lambda, and=20
 pass the lambda by template parameter instead:
it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy. you can't get address of anything in compile time, 'cause there is no such address. but you can do it in runtime, of course. that's what you doing: postponing "&" to runtime.
 as far as I could google, no one has attempted this yet.
no one published it yet, not "no one attempted". i desperately need runtime reflection for my (still private) component builder project, so i'm slowly writing that mechanics. and my "command console" using similar technique to allow user inspect variables and classes, and call function/methods in runtime. yet the thing is still in post-alpha stage (but it works). i also found some compiler bugs while writing it, but not yet filled bugreports (i know, i know, i shouldn't be so lazy).
 My prototype works pretty much as he has described, less a feature=20
 here and there.
ah, that's good. i'm not aiming for full reflection mechanics, but i want to have it. ;-)
Oct 27 2014
next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Tuesday, 28 October 2014 at 02:34:14 UTC, ketmar via 
Digitalmars-d wrote:
 On Tue, 28 Oct 2014 01:36:01 +0000
 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have actually found a work around as well, which was to wrap 
 the actual retrieval of the function address in a lambda, and 
 pass the lambda by template parameter instead:
it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy. you can't get address of anything in compile time, 'cause there is no such address. but you can do it in runtime, of course. that's what you doing: postponing "&" to runtime.
 as far as I could google, no one has attempted this yet.
no one published it yet, not "no one attempted". i desperately need runtime reflection for my (still private) component builder project, so i'm slowly writing that mechanics. and my "command console" using similar technique to allow user inspect variables and classes, and call function/methods in runtime.
I do something possibly similar for generating console help text using runtime reflection. It could have used compile-time reflection, but it gets annoying to use. Using https://shardsoft.com/stash/projects/SHARD/repos/dap/browse/sou ce/dap/Standalone.d (with https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/Shar Tools/CommandLine.d and https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/ShardTools/Reflection.d?until=45ded3019f3f05d7b68e57 6d34da1de7433ccf6), it generates something like: D Asset Pipeline Converts assets into an intermediate post-processed format more efficiently loaded at runtime. Commands: [-h|--help]: Displays the help string. [-a|--add]: Adds the given raw asset to the asset store using the default processor and default settings. [-r|--remove]: Removes the asset with the specified qualified name from the asset store. [-l|--list]: Lists all assets currently stored. [-b|--build]: Builds all dirty assets using current settings. [-i|--inspect]: Shows all properties of the given asset. [-m|--modify]: Modifies a property of a processor on an asset, or the processor used to build the asset. Options: [--input-folder]: The folder that assets should be read from and settings stored in. Default: Content\Input [--output-folder]: The folder that generated assets should be saved to. Default: Content\Output [--log-level]: The minimum severity for a message to be logged. Default: info The command lines then invoke methods registered through runtime reflection, which allow the user to change individual properties on the set processor using --modify, again using runtime reflection to find the setting that the user wants and converting their input to the appropriate type.
Oct 28 2014
prev sibling parent reply "bitwise" <nothx gmail.com> writes:
On Tuesday, 28 October 2014 at 02:34:14 UTC, ketmar via
Digitalmars-d wrote:
 On Tue, 28 Oct 2014 01:36:01 +0000
 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 I have actually found a work around as well, which was to wrap 
 the actual retrieval of the function address in a lambda, and 
 pass the lambda by template parameter instead:
it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy.
Yea... I've tried several ways to clean this up.. the most favorable being this: MethodImpl!({ return &__traits(getMethod, SCOPE, m); })(...); But, although the above has worked for me in a trivial test case, it won't compile with my reflection implementation right now. Definitely on the list though.
 no one published it yet, not "no one attempted". i desperately
publish or perish! =)
 yet the thing is still in post-alpha stage (but it works).
Yeah.. mine too.. I have only really been testing the common cases. Not looking forward to unit-test time...
 i also found some compiler bugs while writing it, but not yet 
 filled bugreports
I've hit a few myself... but the compiler messages have been a bit obscure.. so I'm not even sure how to classify some of the bugs I've hit. I think should classify the one in the OP as something like: "inconsistent availability of function pointers at compile time"
Oct 28 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 29 Oct 2014 00:13:52 +0000
bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 no one published it yet, not "no one attempted". i desperately
publish or perish! =3D)
oh, i want it to be at least pre-beta before showing it to the world. ;-) what i'm really aiming at is a system like BlackBox Component Builder (it's alot of work, but at least it's fun).
 yet the thing is still in post-alpha stage (but it works).
Yeah.. mine too.. I have only really been testing the common cases. Not looking forward to unit-test time...
lucky me, my system doesn't need full-featured runtime reflection, only some easy cases for classes, structs and free functions. and i'm not event started to mess with shared libraries yet, which are the next key element of my project.
Oct 28 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/27/14 4:52 PM, bitwise wrote:
 quotes self
Here is a better example, showing that virtual function pointers are available at compile time in C++. Essentially, I would expect my D code to function similarly, but it won't compile. class TestAddr { public: virtual void test() { cout << "test" << endl; } }; template<void (TestAddr::*FN)()> class PtrWrapper { public: void invoke(TestAddr *instance) { (instance->*FN)(); } }; int main(int argc, const char * argv[]) { TestAddr test; PtrWrapper<&TestAddr::test> wrapper; wrapper.invoke(&test); return 0; }
That won't work in D because in D pointers to methods carry "this" with them, whereas in C++ they don't. -- Andrei
Oct 28 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 29 October 2014 at 01:22:43 UTC, Andrei 
Alexandrescu wrote:
 That won't work in D because in D pointers to methods carry 
 "this" with them, whereas in C++ they don't. -- Andrei
I have an idea ! We should call them delegates so people won't make the confusion !
Oct 28 2014
parent reply "Kagamin" <spam here.lot> writes:
Though, explicit reference to TestClass.instanceMethod is not a 
delegate. The delegate is constructed manually later in the code 
on attempt to call the method.
Oct 29 2014
parent "bitwise" <nothx gmail.com> writes:
I think I've got it figured out.

In my original example, I added the following line inside the 
function-template, and the class-template:

pragma(msg, typeof(T).stringof);

in both cases, the (correct)result was a tuple of a function 
(void())

But, when it came time to retrieve the address of the function, 
the behaviour was not the same inside the scope of the function, 
and the class.

this line behaved differently in both cases:

dg.funcptr = &T[0];

inside the function-template, the above line worked correctly, 
and the function's address was taken. However, inside the 
class-template, the compiler attempted to call T[0] which yielded 
this error:

Error: this for instanceMethod needs to be type TestClass not 
type main.FuncPtr!(instanceMethod).FuncPtr

So, at this point, I saw that the compiler was not parsing things 
in a consistent way, and came up with this template:

template addressOf(T...) {
     enum addressOf = &T[0];
}

Finally, my example worked as expected after making this change:

dg.funcptr = &T[0];
to
dg.funcptr = addressOf!(T[0]);


Onward..

Given that my current project is a reflection library, I 
proceeded to start swapping in __traits.

All of these worked correctly (class methods):

invokeFunction!(__traits(getMember, TestClass, 
"instanceMethod"))(cast(void*)testClass);
invokeFunction!(__traits(getMember, TestClass, 
"staticMethod"))(null);
auto fp1 = new FuncPtr!(__traits(getMember, TestClass, 
"instanceMethod"));
auto fp2 = new FuncPtr!(__traits(getMember, TestClass, 
"staticMethod"));

But these did not(global methods).

invokeFunction!(__traits(getMember, thisModule, 
"GlobalMethod"))(null);
auto fp3 = new FuncPtr!(__traits(getMember, thisModule, 
"GlobalMethod"));

Both of the above lines failed with the following error:

Error: function main.GlobalMethod () is not callable using 
argument types (void)



So finally, I would describe these bugs as follows:

1) the addressOf(T...) template should not be needed inside the 
class-template. Both cases in the original post should behave 
consistently. The problem is inconsistent handling of template 
arguments between function templates and class templates.

2) the return of __traits(getMember) is not treated the same way 
as the symbol itself when parsed. In the case of GlobalMethod, 
the compiler tries to call the method when it's returned from a 
__trait, but does not when it's referenced directly.

Does anyone concur with these diagnoses?

And is there any way in D to explicitly request that a function 
not be called without parens?
Oct 29 2014
prev sibling parent reply "Kapps" <opantm2+spam gmail.com> writes:
On Monday, 27 October 2014 at 22:17:25 UTC, bitwise wrote:

 This error seems like it may be related some how:

 enum index = __traits(getVirtualIndex, 
 TestClass.instanceMethod);
 enum p = TestClass.classinfo.vtbl[index];

 The above code will produce this error:
 Error: typeid(main.TestClass).vtbl is not yet implemented at
 compile time

 but if this is the problem, shouldn't Both of the test cases 
 fail?
You shouldn't need the vtbl at compile-time, only the index of the method. The method indices within the vtbl are computed, which is what getVirtualIndex gives you, but CTFE doesn't support directly accessing the vtbl yet. That being said, you only need to worry about any of this if you want to support virtual methods and have it invoke the actual overridden method, not the one you have saved through reflection. (For example, if Bar : Foo overrides foo, and you generated reflection info for Foo, it would call Foo.foo instead of Bar.foo even if passed in an instance of Bar.) It's hard to tell exactly what you're trying to do in your original code, but one thing to keep in mind is that something like Foo!(TestClass.TestInstance) may mean multiple things, as the method has overloads. At some point you may need to use __traits(getOverloads) (such as in https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/ShardTools/Reflection.d?until=45ded3019f3f05d7b68e5746d3 to get the actual methods which you can then get a function pointer for (again though, requires more effort for properly handling virtual methods).
Oct 28 2014
parent "bitwise" <nothx gmail.com> writes:
 That being said, you only need to worry about any of this if 
 you want to support virtual methods and have it invoke the 
 actual overridden method, not the one you have saved through 
 reflection. (For example, if Bar : Foo overrides foo, and you 
 generated reflection info for Foo, it would call Foo.foo 
 instead of Bar.foo even if passed in an instance of Bar.)
I understand what you mean, but I don't think I need to, or should support that based on my current design. Methods can be retrieved/invoked like this: getModule("test").getClass("Bar").getMethod("foo").invoke(cast(void*)new Bar); above, you would be explicitly asking for the version of 'foo' which is callable from an instance of 'Bar'. i.e. Each instance of MethodDeclImpl retrieved from my reflection system represents exactly one method. the type of the instance passed in will eventually be checked, and invoke() will throw an exception if the instance is of the wrong type. In the case of a base pointer, one could use the following code: Foo bar = new Bar; foreach(c; getModule("test").classes) { if(typeid(bar) == c.type) c.findMethod("foo").invoke(cast(void*)bar); } However, I would like to eventually have something more elegant than this.
 one thing to keep in mind is that something like 
 Foo!(TestClass.TestInstance) may mean multiple things, as the 
 method has overloads. At some point you may need to use 
 __traits(getOverloads) (such as in
This was next on the list =) I just pushed support for overloaded functions, and added tests for virtual functions.
Oct 28 2014