digitalmars.D - function pointer bug?
- bitwise (62/62) Oct 25 2014 I am trying to store a function pointer in a class in a generic
- Kagamin (2/2) Oct 26 2014 Looks like &T[0] tries to take delegate to instanceMethod, hence
- Solomon E (12/12) Oct 27 2014 It works after I added 'static' to the declaration of 'invoke()'
- bitwise (18/31) Oct 27 2014 I'm pretty sure this is a bug. I'm going to file a report. If one
- ketmar via Digitalmars-d (7/22) Oct 27 2014 this means that something gone wrong in the process. '(__error)' is the
- bitwise (24/29) Oct 27 2014 I'm just trying to come up with a test case which may offer some
- bitwise (19/20) Oct 27 2014 Here is a better example, showing that virtual function pointers
- ketmar via Digitalmars-d (32/51) Oct 27 2014 C++ compiler does some trickery behind the curtains. besides, you
- bitwise (12/18) Oct 27 2014 There is nothing hackish in the above code. It's a non-type
- bitwise (20/23) Oct 27 2014 I have actually found a work around as well, which was to wrap
- ketmar via Digitalmars-d (17/23) Oct 27 2014 it's not a "workaround", it's almost exactly what i did in my sample,
- Kapps (39/62) Oct 28 2014 I do something possibly similar for generating console help text
- bitwise (16/28) Oct 28 2014 Yea... I've tried several ways to clean this up.. the most
- ketmar via Digitalmars-d (9/14) Oct 28 2014 oh, i want it to be at least pre-beta before showing it to the
- Andrei Alexandrescu (3/23) Oct 28 2014 That won't work in D because in D pointers to methods carry "this" with
- Kapps (20/29) Oct 28 2014 You shouldn't need the vtbl at compile-time, only the index of
- bitwise (24/34) Oct 28 2014 I understand what you mean, but I don't think I need to, or
I am trying to store a function pointer in a class in a generic way(works with all member/non-member/global functions). In the main() function below, there are two cases(A, B). If case B is commented out, this code complies/runs fine. Otherwise, I get these errors, and the compiler pointing at "dg.funcptr = &T[0];" in TestClas.invoke(void*). Error: this for instanceMethod needs to be type TestClass not type main.FuncPtr!(instanceMethod).FuncPtr Error: cannot implicitly convert expression (&(__error).instanceMethod) of type void delegate() to void function() Error: template instance main.FuncPtr!(instanceMethod) error instantiating Am I missing something here or is this a bug? class TestClass { void instanceMethod() { writeln("Instance Method!"); } static void staticMethod() { writeln("Static Method!"); } } void GlobalMethod() { writeln("Global Method!"); } void invokeFunction(T...)(void *instance){ alias typeof(T[0]) method_type; alias ReturnType!method_type return_type; alias ParameterTypeTuple!method_type param_types; alias return_type delegate(param_types) delegate_type; delegate_type dg; dg.ptr = instance; dg.funcptr = &T[0]; dg(); } class FuncPtr(T...) { void invoke(void *instance) { alias typeof(T[0]) method_type; alias ReturnType!method_type return_type; alias ParameterTypeTuple!method_type param_types; alias return_type delegate(param_types) delegate_type; delegate_type dg; dg.ptr = instance; dg.funcptr = &T[0]; dg(); } } void main() { TestClass testClass = new TestClass(); // case A invokeFunction!(TestClass.instanceMethod)(cast(void*)testClass); invokeFunction!(TestClass.staticMethod)(null); invokeFunction!(GlobalMethod)(null); // case B auto fp1 = new FuncPtr!(TestClass.instanceMethod); auto fp2 = new FuncPtr!(TestClass.staticMethod); auto fp3 = new FuncPtr!(GlobalMethod); fp1.invoke(cast(void*)testClass); fp2.invoke(null); fp3.invoke(null); }
Oct 25 2014
Looks like &T[0] tries to take delegate to instanceMethod, hence complains about this type.
Oct 26 2014
It works after I added 'static' to the declaration of 'invoke()' (and import std.stdio, std.traits.) I fiddled around with it for hours before I tried 'static' there, because I've only been studying D for a week, so only about half of this code and the error messages made any sense to me when I started on it. Once it started passing all the tests, I still didn't quite get how it all worked, because of suspecting there was a typo in the code between writing TestClass and testClass. So I tested whether specific instances are called (instead of maybe just the first one constructed) by adding a static counter to TestClass, and storage of the counter by each instance. It looks all right.
Oct 27 2014
On Monday, 27 October 2014 at 16:08:26 UTC, Solomon E wrote:It works after I added 'static' to the declaration of 'invoke()' (and import std.stdio, std.traits.) I fiddled around with it for hours before I tried 'static' there, because I've only been studying D for a week, so only about half of this code and the error messages made any sense to me when I started on it. Once it started passing all the tests, I still didn't quite get how it all worked, because of suspecting there was a typo in the code between writing TestClass and testClass. So I tested whether specific instances are called (instead of maybe just the first one constructed) by adding a static counter to TestClass, and storage of the counter by each instance. It looks all right.I'm pretty sure this is a bug. I'm going to file a report. If one of those template works, both should. I have been trying to build a reflection system over the last few weeks, and noticed different incarnations of this problem several times: <b> &(__error).instanceMethod </b> In certain cases, the compiler seems to drop the enclosing type of the method, but I'm not sure why. This error seems like it may be related some how: enum index = __traits(getVirtualIndex, TestClass.instanceMethod); enum p = TestClass.classinfo.vtbl[index]; The above code will produce this error: Error: typeid(main.TestClass).vtbl is not yet implemented at compile time but if this is the problem, shouldn't Both of the test cases fail?
Oct 27 2014
On Mon, 27 Oct 2014 22:17:24 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:I have been trying to build a reflection system over the last few weeks, and noticed different incarnations of this problem several times: =20 &(__error).instanceMethodthis means that something gone wrong in the process. '(__error)' is the "pseudotype" for failed CTFE/instantiation.This error seems like it may be related some how: =20 enum index =3D __traits(getVirtualIndex, TestClass.instanceMethod); enum p =3D TestClass.classinfo.vtbl[index]; =20 The above code will produce this error: Error: typeid(main.TestClass).vtbl is not yet implemented at compile time =20 but if this is the problem, shouldn't Both of the test cases fail?why? everything is ok here. you can get method index in compile-time, but there is no built VMT in compile time. what do you want to achieve with second enum?
Oct 27 2014
this means that something gone wrong in the process. '(__error)' is the "pseudotype" for failed CTFE/instantiation.But why did the instantiation fail? and more importantly, why did it not have consistent behaviour between the two templates above?what do you want to achieve with second enum?I'm just trying to come up with a test case which may offer some insight into what's going wrong with the code in the OP. for example, I am wondering how far this issue extends; meaning, is it only classinfo.vtbl that's not available at compile time? or is it the address of member functions as a whole that are unavailable? The code below suggests the latter, although it doesn't explicitly state it: static addr = &TestClass.instanceMethod; Error: non-constant expression & instanceMethod I may be missing a subtle difference, but in C++, this code works: class TestAddr { public: virtual void test() { cout << "test" << endl; } }; int main(int argc, const char * argv[]) { TestAddr test; static auto ptr = &TestAddr::test; (test.*ptr)(); return 0; } so why would it not in D?
Oct 27 2014
Here is a better example, showing that virtual function pointers are available at compile time in C++. Essentially, I would expect my D code to function similarly, but it won't compile. class TestAddr { public: virtual void test() { cout << "test" << endl; } }; template<void (TestAddr::*FN)()> class PtrWrapper { public: void invoke(TestAddr *instance) { (instance->*FN)(); } }; int main(int argc, const char * argv[]) { TestAddr test; PtrWrapper<&TestAddr::test> wrapper; wrapper.invoke(&test); return 0; }quotes self
Oct 27 2014
On Mon, 27 Oct 2014 23:52:38 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:The code below suggests the latter, although it doesn't=20 explicitly state it: =20 static addr =3D &TestClass.instanceMethod; Error: non-constant expression & instanceMethod =20 I may be missing a subtle difference, but in C++, this code works: =20 class TestAddr { public: virtual void test() { cout << "test" << endl; } }; =20 int main(int argc, const char * argv[]) { TestAddr test; static auto ptr =3D &TestAddr::test; (test.*ptr)(); return 0; }C++ compiler does some trickery behind the curtains. besides, you aren't supposed to make such hackish things easily in D. yet you can: class TestClass { int n =3D 42; void test() { writeln("test: ", n); } } void main () { auto test =3D new TestClass(); void delegate () a; { a.ptr =3D *cast(void**)&test; // protection from opCast() // this can be done as `cast(void*)test;` too enum idx =3D __traits(getVirtualIndex, TestClass.test); a.funcptr =3D cast(void function())TestClass.classinfo.vtbl[idx]; a(); // outputs 'test: 42' } } you need to manually create initialization code that C++ compilers creates behind the curtains. if you aren't in urgent need of that code, may i suggest you to read D books and D specs? D is not C++, and D delegates aren't C++ member function pointers (yet they works nearly the same). this is kind of advanced topic, and i don't think that dumping working source code at you will help without you grasp the low-level mechanics first. besides, you can use CTFE to build wrapper code. Adam Ruppe has that in his jsvar.d, and i have that in my cong.d (cmdcon-ng) too. not that i'm not willing to help you, but i can't see what you understand and what not, so i don't know where i should start explaining.
Oct 27 2014
C++ compiler does some trickery behind the curtains. besides, you aren't supposed to make such hackish things easily in D. yet you can:There is nothing hackish in the above code. It's a non-type template parameter and a member function pointer. If I was trying to access the (implementation dependant)vtable pointer in C++ to call the function manually, I may concede to calling it a hack, but there is nothing non-standard about the above code.I was simply trying to show that what I wanted to do was possible in a similar language. I still believe it's a bug in the language. In the original code, both test cases should either both work, or both fail to compile. I have looked through the D docs online, and can't find anything supporting the argument that this is the intended behaviour of the compiler.D is not C++, and D delegates aren't C++ member function pointers (yet they works nearly the same).
Oct 27 2014
I have actually found a work around as well, which was to wrap the actual retrieval of the function address in a lambda, and pass the lambda by template parameter instead: auto getMethod = function() { return &__traits(getMember, SCOPE, member); }; this is later assigned to a delegate when called: delegate_type dg; dg.ptr = instance; dg.funcptr = getMethod(); dg(); The above works for global or static functions too. This seems to me like an unnecessary workaround though.. my current project can be viewed here: https://github.com/bitwise-github/D-Reflection/blob/master/reflection.d#L796 It's been implemented based on Andrei's suggestions here: http://forum.dlang.org/thread/juf7sk$16rl$1 digitalmars.com as far as I could google, no one has attempted this yet. My prototype works pretty much as he has described, less a feature here and there.besides, you can use CTFE to build wrapper code. Adam Ruppe has that in his jsvar.d, and i have that in my cong.d (cmdcon-ng) too.
Oct 27 2014
On Tue, 28 Oct 2014 01:36:01 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:I have actually found a work around as well, which was to wrap=20 the actual retrieval of the function address in a lambda, and=20 pass the lambda by template parameter instead:it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy. you can't get address of anything in compile time, 'cause there is no such address. but you can do it in runtime, of course. that's what you doing: postponing "&" to runtime.as far as I could google, no one has attempted this yet.no one published it yet, not "no one attempted". i desperately need runtime reflection for my (still private) component builder project, so i'm slowly writing that mechanics. and my "command console" using similar technique to allow user inspect variables and classes, and call function/methods in runtime. yet the thing is still in post-alpha stage (but it works). i also found some compiler bugs while writing it, but not yet filled bugreports (i know, i know, i shouldn't be so lazy).My prototype works pretty much as he has described, less a feature=20 here and there.ah, that's good. i'm not aiming for full reflection mechanics, but i want to have it. ;-)
Oct 27 2014
On Tuesday, 28 October 2014 at 02:34:14 UTC, ketmar via Digitalmars-d wrote:On Tue, 28 Oct 2014 01:36:01 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:I do something possibly similar for generating console help text using runtime reflection. It could have used compile-time reflection, but it gets annoying to use. Using https://shardsoft.com/stash/projects/SHARD/repos/dap/browse/sou ce/dap/Standalone.d (with https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/Shar Tools/CommandLine.d and https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/ShardTools/Reflection.d?until=45ded3019f3f05d7b68e57 6d34da1de7433ccf6), it generates something like: D Asset Pipeline Converts assets into an intermediate post-processed format more efficiently loaded at runtime. Commands: [-h|--help]: Displays the help string. [-a|--add]: Adds the given raw asset to the asset store using the default processor and default settings. [-r|--remove]: Removes the asset with the specified qualified name from the asset store. [-l|--list]: Lists all assets currently stored. [-b|--build]: Builds all dirty assets using current settings. [-i|--inspect]: Shows all properties of the given asset. [-m|--modify]: Modifies a property of a processor on an asset, or the processor used to build the asset. Options: [--input-folder]: The folder that assets should be read from and settings stored in. Default: Content\Input [--output-folder]: The folder that generated assets should be saved to. Default: Content\Output [--log-level]: The minimum severity for a message to be logged. Default: info The command lines then invoke methods registered through runtime reflection, which allow the user to change individual properties on the set processor using --modify, again using runtime reflection to find the setting that the user wants and converting their input to the appropriate type.I have actually found a work around as well, which was to wrap the actual retrieval of the function address in a lambda, and pass the lambda by template parameter instead:it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy. you can't get address of anything in compile time, 'cause there is no such address. but you can do it in runtime, of course. that's what you doing: postponing "&" to runtime.as far as I could google, no one has attempted this yet.no one published it yet, not "no one attempted". i desperately need runtime reflection for my (still private) component builder project, so i'm slowly writing that mechanics. and my "command console" using similar technique to allow user inspect variables and classes, and call function/methods in runtime.
Oct 28 2014
On Tuesday, 28 October 2014 at 02:34:14 UTC, ketmar via Digitalmars-d wrote:On Tue, 28 Oct 2014 01:36:01 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:Yea... I've tried several ways to clean this up.. the most favorable being this: MethodImpl!({ return &__traits(getMethod, SCOPE, m); })(...); But, although the above has worked for me in a trivial test case, it won't compile with my reflection implementation right now. Definitely on the list though.I have actually found a work around as well, which was to wrap the actual retrieval of the function address in a lambda, and pass the lambda by template parameter instead:it's not a "workaround", it's almost exactly what i did in my sample, just not that hairy.no one published it yet, not "no one attempted". i desperatelypublish or perish! =)yet the thing is still in post-alpha stage (but it works).Yeah.. mine too.. I have only really been testing the common cases. Not looking forward to unit-test time...i also found some compiler bugs while writing it, but not yet filled bugreportsI've hit a few myself... but the compiler messages have been a bit obscure.. so I'm not even sure how to classify some of the bugs I've hit. I think should classify the one in the OP as something like: "inconsistent availability of function pointers at compile time"
Oct 28 2014
On Wed, 29 Oct 2014 00:13:52 +0000 bitwise via Digitalmars-d <digitalmars-d puremagic.com> wrote:oh, i want it to be at least pre-beta before showing it to the world. ;-) what i'm really aiming at is a system like BlackBox Component Builder (it's alot of work, but at least it's fun).no one published it yet, not "no one attempted". i desperatelypublish or perish! =3D)lucky me, my system doesn't need full-featured runtime reflection, only some easy cases for classes, structs and free functions. and i'm not event started to mess with shared libraries yet, which are the next key element of my project.yet the thing is still in post-alpha stage (but it works).Yeah.. mine too.. I have only really been testing the common cases. Not looking forward to unit-test time...
Oct 28 2014
On 10/27/14 4:52 PM, bitwise wrote:That won't work in D because in D pointers to methods carry "this" with them, whereas in C++ they don't. -- AndreiHere is a better example, showing that virtual function pointers are available at compile time in C++. Essentially, I would expect my D code to function similarly, but it won't compile. class TestAddr { public: virtual void test() { cout << "test" << endl; } }; template<void (TestAddr::*FN)()> class PtrWrapper { public: void invoke(TestAddr *instance) { (instance->*FN)(); } }; int main(int argc, const char * argv[]) { TestAddr test; PtrWrapper<&TestAddr::test> wrapper; wrapper.invoke(&test); return 0; }quotes self
Oct 28 2014
On Wednesday, 29 October 2014 at 01:22:43 UTC, Andrei Alexandrescu wrote:That won't work in D because in D pointers to methods carry "this" with them, whereas in C++ they don't. -- AndreiI have an idea ! We should call them delegates so people won't make the confusion !
Oct 28 2014
Though, explicit reference to TestClass.instanceMethod is not a delegate. The delegate is constructed manually later in the code on attempt to call the method.
Oct 29 2014
I think I've got it figured out. In my original example, I added the following line inside the function-template, and the class-template: pragma(msg, typeof(T).stringof); in both cases, the (correct)result was a tuple of a function (void()) But, when it came time to retrieve the address of the function, the behaviour was not the same inside the scope of the function, and the class. this line behaved differently in both cases: dg.funcptr = &T[0]; inside the function-template, the above line worked correctly, and the function's address was taken. However, inside the class-template, the compiler attempted to call T[0] which yielded this error: Error: this for instanceMethod needs to be type TestClass not type main.FuncPtr!(instanceMethod).FuncPtr So, at this point, I saw that the compiler was not parsing things in a consistent way, and came up with this template: template addressOf(T...) { enum addressOf = &T[0]; } Finally, my example worked as expected after making this change: dg.funcptr = &T[0]; to dg.funcptr = addressOf!(T[0]); Onward.. Given that my current project is a reflection library, I proceeded to start swapping in __traits. All of these worked correctly (class methods): invokeFunction!(__traits(getMember, TestClass, "instanceMethod"))(cast(void*)testClass); invokeFunction!(__traits(getMember, TestClass, "staticMethod"))(null); auto fp1 = new FuncPtr!(__traits(getMember, TestClass, "instanceMethod")); auto fp2 = new FuncPtr!(__traits(getMember, TestClass, "staticMethod")); But these did not(global methods). invokeFunction!(__traits(getMember, thisModule, "GlobalMethod"))(null); auto fp3 = new FuncPtr!(__traits(getMember, thisModule, "GlobalMethod")); Both of the above lines failed with the following error: Error: function main.GlobalMethod () is not callable using argument types (void) So finally, I would describe these bugs as follows: 1) the addressOf(T...) template should not be needed inside the class-template. Both cases in the original post should behave consistently. The problem is inconsistent handling of template arguments between function templates and class templates. 2) the return of __traits(getMember) is not treated the same way as the symbol itself when parsed. In the case of GlobalMethod, the compiler tries to call the method when it's returned from a __trait, but does not when it's referenced directly. Does anyone concur with these diagnoses? And is there any way in D to explicitly request that a function not be called without parens?
Oct 29 2014
On Monday, 27 October 2014 at 22:17:25 UTC, bitwise wrote:This error seems like it may be related some how: enum index = __traits(getVirtualIndex, TestClass.instanceMethod); enum p = TestClass.classinfo.vtbl[index]; The above code will produce this error: Error: typeid(main.TestClass).vtbl is not yet implemented at compile time but if this is the problem, shouldn't Both of the test cases fail?You shouldn't need the vtbl at compile-time, only the index of the method. The method indices within the vtbl are computed, which is what getVirtualIndex gives you, but CTFE doesn't support directly accessing the vtbl yet. That being said, you only need to worry about any of this if you want to support virtual methods and have it invoke the actual overridden method, not the one you have saved through reflection. (For example, if Bar : Foo overrides foo, and you generated reflection info for Foo, it would call Foo.foo instead of Bar.foo even if passed in an instance of Bar.) It's hard to tell exactly what you're trying to do in your original code, but one thing to keep in mind is that something like Foo!(TestClass.TestInstance) may mean multiple things, as the method has overloads. At some point you may need to use __traits(getOverloads) (such as in https://shardsoft.com/stash/projects/SHARD/repos/shardtools/browse/source/ShardTools/Reflection.d?until=45ded3019f3f05d7b68e5746d3 to get the actual methods which you can then get a function pointer for (again though, requires more effort for properly handling virtual methods).
Oct 28 2014
That being said, you only need to worry about any of this if you want to support virtual methods and have it invoke the actual overridden method, not the one you have saved through reflection. (For example, if Bar : Foo overrides foo, and you generated reflection info for Foo, it would call Foo.foo instead of Bar.foo even if passed in an instance of Bar.)I understand what you mean, but I don't think I need to, or should support that based on my current design. Methods can be retrieved/invoked like this: getModule("test").getClass("Bar").getMethod("foo").invoke(cast(void*)new Bar); above, you would be explicitly asking for the version of 'foo' which is callable from an instance of 'Bar'. i.e. Each instance of MethodDeclImpl retrieved from my reflection system represents exactly one method. the type of the instance passed in will eventually be checked, and invoke() will throw an exception if the instance is of the wrong type. In the case of a base pointer, one could use the following code: Foo bar = new Bar; foreach(c; getModule("test").classes) { if(typeid(bar) == c.type) c.findMethod("foo").invoke(cast(void*)bar); } However, I would like to eventually have something more elegant than this.one thing to keep in mind is that something like Foo!(TestClass.TestInstance) may mean multiple things, as the method has overloads. At some point you may need to use __traits(getOverloads) (such as inThis was next on the list =) I just pushed support for overloaded functions, and added tests for virtual functions.
Oct 28 2014