digitalmars.D.learn - Passing C++ class to DLL for callbacks from D (Steam)
- cc (112/112) Jun 07 2018 Hello, I'm attempting to interface with the Steam API DLL in D
- Mike Parker (7/11) Jun 07 2018 I've not used any of the C++ interfacing features yet, but my
- cc (3/15) Jun 07 2018 Tried wrapping the CImpl class in extern(C++), no luck.. tried
- evilrat (2/22) Jun 08 2018 Can you upload reduced example somewhere to play with?
- cc (13/36) Jun 08 2018 I've put together a simplified test program here (124KB):
- cc (5/6) Jun 08 2018 Here is a pastebin of the D source file updated with some
- cc (10/10) Jun 08 2018 Sample output:
- evilrat (57/63) Jun 09 2018 The proper bare minimum signature for call result callback is
- cc (6/16) Jun 09 2018 I see, thank you for checking it out. Is it only the functions
- evilrat (5/23) Jun 09 2018 Only subsystems getters like SteamUser() or SteamInventory()
- cc (35/40) Jun 10 2018 Thanks to the information you supplied I was able to get it
- rikki cattermole (6/13) Jun 10 2018 Ugh what GetIntPtr? Unless of course this header file is wrong[0].
- cc (19/34) Jun 10 2018 Woops, that GetIntPtr came from the .cs header in the same folder
- rikki cattermole (3/6) Jun 10 2018 Stay with the c/c++ headers for c/c++ code. We don't generally don't go
- evilrat (12/45) Jun 07 2018 add extern(C++) to class as well to
Hello, I'm attempting to interface with the Steam API DLL in D and running into some trouble working with callbacks. I'm aware there's already a project here http://derelict-steamworks.dub.pm/ but it seems to have not yet addressed the same issue. Steam provides ways to poll for whether an asynchronous request has completed yet and retrieve the results, but what I am trying to implement is receiving actual callbacks from the API. Just for reference, I've gotten simple callbacks working in other APIs (passing a D function pointer to a C function and having it be called back), and the other basic Steam functions that don't deal with callbacks all work fine (defined in D as extern(C)). Here's where I'm running into trouble: Definitions in C++ headers provided by Steam: typedef uint64 SteamAPICall_t; S_API void S_CALLTYPE SteamAPI_RegisterCallResult( class CCallbackBase *pCallback, SteamAPICall_t hAPICall ); S_API void S_CALLTYPE SteamAPI_UnregisterCallResult( class CCallbackBase *pCallback, SteamAPICall_t hAPICall ); class CCallbackBase { public: CCallbackBase() { m_nCallbackFlags = 0; m_iCallback = 0; } // don't add a virtual destructor because we export this binary interface across dll's virtual void Run( void *pvParam ) = 0; virtual void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ) = 0; int GetICallback() { return m_iCallback; } virtual int GetCallbackSizeBytes() = 0; protected: enum { k_ECallbackFlagsRegistered = 0x01, k_ECallbackFlagsGameServer = 0x02 }; uint8 m_nCallbackFlags; int m_iCallback; friend class CCallbackMgr; private: CCallbackBase( const CCallbackBase& ); CCallbackBase& operator=( const CCallbackBase& ); }; It's actually more complex than that as it has templated classes that inherit from that base so that there is a different template for each expected struct that gets returned from various async requests. I'm trying to get the simplest possible scenario working first and hoping I'm not missing anything, but I don't actually understand how I'm supposed to work with the C++ objects enough really to be sure. Here's how I've defined it in D, as per https://dlang.org/spec/cpp_interface.html#classes : alias ulong SteamAPICall_t; extern(C) { void SteamAPI_RegisterCallResult(CCallbackBase pCallback, SteamAPICall_t hAPICall); void SteamAPI_UnregisterCallResult(CCallbackBase pCallback, SteamAPICall_t hAPICall); } extern(C++) { interface CCallbackBase { //this() { m_nCallbackFlags = 0; m_iCallback = 0; } void Run( void *pvParam ); void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ); int GetICallback(); int GetCallbackSizeBytes(); enum { k_ECallbackFlagsRegistered = 0x01, k_ECallbackFlagsGameServer = 0x02 } //uint8 m_nCallbackFlags; //int m_iCallback; //friend class CCallbackMgr; //CCallbackBase( const CCallbackBase& ); //CCallbackBase& operator=( const CCallbackBase& ); } } class CImpl : CCallbackBase { extern(C++) { this() { m_nCallbackFlags = 0; m_iCallback = 0; } void Run( void *pvParam ) { writeln("Run1"); } void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ) { writeln("Run2"); } int GetICallback() { return m_iCallback; } int GetCallbackSizeBytes() { return NumberOfCurrentPlayers_t.sizeof; } // ordinarily use templates to determine what type struct ptr to return } uint8 m_nCallbackFlags; int m_iCallback; } Which then gets called later on as: auto cbk = new CImpl(); cbk.m_iCallback = NumberOfCurrentPlayers_t.k_iCallback; auto hid = SteamAPI_ISteamUserStats_GetNumberOfCurrentPlayers(...); SteamAPI_RegisterCallResult(cbk, hid); And then every frame SteamAPI_RunCallbacks(); runs, which is supposed to fire off the callbacks. I've also tried using abstract class instead of interface. My expectation of what's supposed to happen is that one of those Run() functions gets called at some point. However, it never does (even though I can poll to determine the call has actually completed). I'm hoping the problem is I'm simply not passing the right expected type of data to the registration function and/or haven't defined the interface in D properly (I admit I have very little idea how the C++ is working anymore, including what those private methods are for), and not an issue with utilizing the API itself, since I doubt there's anyone I could ask on the Steam end of things about interfacing with D. My gut feeling is I'm doing something very incorrect/stupid on the definition end of things, I understand passing function pointers to C++ functions that intend to call back well enough but I'm confused about this concept of passing "class SomeClass *" to a function and expecting it to call a method on that class, I don't know where everything is supposed to "exist". If anyone has any insight to provide it would be greatly appreciated, thanks!
Jun 07 2018
On Friday, 8 June 2018 at 00:55:35 UTC, cc wrote:class CImpl : CCallbackBase { extern(C++) {If anyone has any insight to provide it would be greatly appreciated, thanks!I've not used any of the C++ interfacing features yet, but my understanding is the extern(C++) has to apply to the class declaration itself, not only the member functions: extern(C++) class CImpl : CCallbackBase { Does that make a difference?
Jun 07 2018
On Friday, 8 June 2018 at 02:52:10 UTC, Mike Parker wrote:On Friday, 8 June 2018 at 00:55:35 UTC, cc wrote:Tried wrapping the CImpl class in extern(C++), no luck.. tried final keyword on the methods too. Thanks though.class CImpl : CCallbackBase { extern(C++) {If anyone has any insight to provide it would be greatly appreciated, thanks!I've not used any of the C++ interfacing features yet, but my understanding is the extern(C++) has to apply to the class declaration itself, not only the member functions: extern(C++) class CImpl : CCallbackBase { Does that make a difference?
Jun 07 2018
On Friday, 8 June 2018 at 06:59:51 UTC, cc wrote:On Friday, 8 June 2018 at 02:52:10 UTC, Mike Parker wrote:Can you upload reduced example somewhere to play with?On Friday, 8 June 2018 at 00:55:35 UTC, cc wrote:Tried wrapping the CImpl class in extern(C++), no luck.. tried final keyword on the methods too. Thanks though.class CImpl : CCallbackBase { extern(C++) {If anyone has any insight to provide it would be greatly appreciated, thanks!I've not used any of the C++ interfacing features yet, but my understanding is the extern(C++) has to apply to the class declaration itself, not only the member functions: extern(C++) class CImpl : CCallbackBase { Does that make a difference?
Jun 08 2018
On Friday, 8 June 2018 at 07:32:54 UTC, evilrat wrote:On Friday, 8 June 2018 at 06:59:51 UTC, cc wrote:I've put together a simplified test program here (124KB): https://drive.google.com/uc?id=1VPxCcPShlYpGo8BggxniNlPH96iIRG8_&export=download The official SDK can be downloaded here (including the steam_api.dll if you don't want to use the one in the zip for security): https://partner.steamgames.com/doc/sdk Although it may require a developer account to download. Also I should note that Steam differentiates between "callbacks" (events that may be fired to the application that registers to listen for them at any time or multiple times) and "call results" (1:1 asynchronous requests, I believe). This test so far is using call results.On Friday, 8 June 2018 at 02:52:10 UTC, Mike Parker wrote:Can you upload reduced example somewhere to play with?On Friday, 8 June 2018 at 00:55:35 UTC, cc wrote:Tried wrapping the CImpl class in extern(C++), no luck.. tried final keyword on the methods too. Thanks though.class CImpl : CCallbackBase { extern(C++) {If anyone has any insight to provide it would be greatly appreciated, thanks!I've not used any of the C++ interfacing features yet, but my understanding is the extern(C++) has to apply to the class declaration itself, not only the member functions: extern(C++) class CImpl : CCallbackBase { Does that make a difference?
Jun 08 2018
On Saturday, 9 June 2018 at 03:07:39 UTC, cc wrote:I've put together a simplified test program here (124KB):Here is a pastebin of the D source file updated with some additional comments at the end with the callback class definitions from the original header files https://pastebin.com/M8hDXt6L
Jun 08 2018
Sample output: Initializing. User logged on: true Starting request. hid: 4838393704146785693 ..!!!!!!!! Request completed: NumberOfCurrentPlayers_t(1, 5828) Terminating. Not present: any indication that the registered callresult was executed.
Jun 08 2018
On Saturday, 9 June 2018 at 03:14:13 UTC, cc wrote:On Saturday, 9 June 2018 at 03:07:39 UTC, cc wrote:The proper bare minimum signature for call result callback is ------------------ extern(C++) abstract class CCallbackBase { abstract void Run( void *pvParam ) { writeln("Run()"); } abstract void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ); final int GetICallback() { return m_iCallback; } // this is actually optional abstract int GetCallbackSizeBytes(); protected: uint8 m_nCallbackFlags; // probably can be left for some specific use cases int m_iCallback; } ------------------ and the rest templates and stuff is purely for C++ convenience. Run(void*, bool, SteamAPICall_t) is what actually called for call result, then CallResult<> template overrides it to call appropriate pointer member in class with void (T::fun)(Data*, bool bIOFailure) signature. You can mimic it on D side, or just ignore and make own. There is one catch however, for it to be called flags should be equal 1, and iCallback must match struct enum, like NumberOfCurrentPlayers_t.k_iCallback for GetNumberOfCurrentPlayers() However steam devs decided to shield actual pointer and return pointer sized integer when C API is used(or they just screw up?). Anyway, the pointers for subsystems returned by context calls on C++ API and mirrored C API calls are different, but they also have some mechanism for filtering this stuff, that way both integer handle and pointer calls the same underlying implementation, but C API call again is shielded so setting up CallResult and CCallback are ignored. So my solution was just to make simple wrapper around C++ context calls and pass that real pointer to D side. ------------------- C++ code // build as static library & link with your project #ifdef _WIN32 // sorry, I just hardcoded it... #pragma comment (lib, "steam_api64.lib") #define _CRT_SECURE_NO_WARNINGS 1 #endif #include "public/steam/steam_api.h" #ifdef _WIN32 #define _CRT_SECURE_NO_WARNINGS 1 #endif extern "C" void* D_GetSteamUserStats() { return SteamUserStats(); } ------------------- D code // yes, I messed up with the signature to avoid casting, this may or may not be dangerous extern(C) ISteamUserStats D_GetSteamUserStats(); This of course involves making C++ wrapper, but I don't really see other ways because they inlined everything ...I've put together a simplified test program here (124KB):Here is a pastebin of the D source file updated with some additional comments at the end with the callback class definitions from the original header files https://pastebin.com/M8hDXt6L
Jun 09 2018
On Saturday, 9 June 2018 at 14:11:13 UTC, evilrat wrote:However steam devs decided to shield actual pointer and return pointer sized integer when C API is used(or they just screw up?). Anyway, the pointers for subsystems returned by context calls on C++ API and mirrored C API calls are different, but they also have some mechanism for filtering this stuff, that way both integer handle and pointer calls the same underlying implementation, but C API call again is shielded so setting up CallResult and CCallback are ignored. So my solution was just to make simple wrapper around C++ context calls and pass that real pointer to D side.I see, thank you for checking it out. Is it only the functions that return the interface pointers e.g. ISteamUserStats that need to be wrapped to use the class-based versions instead of the steam_api_flat versions? Or does the entire callback system need to be handled through the wrapper?
Jun 09 2018
On Sunday, 10 June 2018 at 01:35:40 UTC, cc wrote:On Saturday, 9 June 2018 at 14:11:13 UTC, evilrat wrote:Only subsystems getters like SteamUser() or SteamInventory() requires wrapping. I really can't understand why they ever choose to silently ignore registering callbacks received with C API systems handles...However steam devs decided to shield actual pointer and return pointer sized integer when C API is used(or they just screw up?). Anyway, the pointers for subsystems returned by context calls on C++ API and mirrored C API calls are different, but they also have some mechanism for filtering this stuff, that way both integer handle and pointer calls the same underlying implementation, but C API call again is shielded so setting up CallResult and CCallback are ignored. So my solution was just to make simple wrapper around C++ context calls and pass that real pointer to D side.I see, thank you for checking it out. Is it only the functions that return the interface pointers e.g. ISteamUserStats that need to be wrapped to use the class-based versions instead of the steam_api_flat versions? Or does the entire callback system need to be handled through the wrapper?
Jun 09 2018
On Sunday, 10 June 2018 at 02:57:34 UTC, evilrat wrote:Only subsystems getters like SteamUser() or SteamInventory() requires wrapping. I really can't understand why they ever choose to silently ignore registering callbacks received with C API systems handles...Thanks to the information you supplied I was able to get it working without a wrapper, like so: extern(C++) abstract class ISteamClient { //public abstract IntPtr GetIntPtr(); public abstract uint CreateSteamPipe(); public abstract bool BReleaseSteamPipe(uint hSteamPipe); public abstract uint ConnectToGlobalUser(uint hSteamPipe); public abstract uint CreateLocalUser(ref uint phSteamPipe,uint eAccountType); public abstract void ReleaseUser(uint hSteamPipe,uint hUser); public abstract ISteamUser GetISteamUser(uint hSteamUser,uint hSteamPipe,const(char)* pchVersion); ... public abstract ISteamUserStats GetISteamUserStats(uint hSteamUser,uint hSteamPipe,const(char)* pchVersion); ... } HSteamUser hSteamUser = SteamAPI_GetHSteamUser(); HSteamPipe hSteamPipe = SteamAPI_GetHSteamPipe(); auto steamClient = cast(ISteamClient) SteamInternal_CreateInterface(STEAMCLIENT_INTERFACE_VERSION); auto userStats = steamClient.GetISteamUserStats(hSteamUser, hSteamPipe, STEAMUSERSTATS_INTERFACE_VERSION); auto hid = SteamAPI_ISteamUserStats_GetNumberOfCurrentPlayers(userStats); auto cbk = new CallResult!NumberOfCurrentPlayers_t(); SteamAPI_RegisterCallResult(cbk, hid); And it successfully fires the 3-arg Run method of the callback object. However for some reason the function table of the ISteamClient seems to be off by one.. it kept calling the wrong methods until I commented one out, in this case GetIntPtr() as seen above, then everything seemed to line up. Not sure what the proper way to ensure it matches the C++ layout here, but at least it seems to be mostly working for now. Thanks again!
Jun 10 2018
On 10/06/2018 10:29 PM, cc wrote:And it successfully fires the 3-arg Run method of the callback object. However for some reason the function table of the ISteamClient seems to be off by one.. it kept calling the wrong methods until I commented one out, in this case GetIntPtr() as seen above, then everything seemed to line up. Not sure what the proper way to ensure it matches the C++ layout here, but at least it seems to be mostly working for now. Thanks again!Ugh what GetIntPtr? Unless of course this header file is wrong[0]. Make the members match exactly, order and everything and it should "just work". [0] https://github.com/ValveSoftware/source-sdk-2013/blob/master/mp/src/public/steam/isteamclient.h#L113
Jun 10 2018
On Sunday, 10 June 2018 at 10:47:58 UTC, rikki cattermole wrote:On 10/06/2018 10:29 PM, cc wrote:Woops, that GetIntPtr came from the .cs header in the same folder as the C++ headers distributed with the SDK, that'll teach me to ctrl+f "class ISteamClient" in all open files and copy/paste before reading. Anyway I played around with it some more and found the one single line that was causing the problem. It needs this: HSteamPipe SteamAPI_GetHSteamPipe(); rather than one of these: HSteamPipe SteamAPI_ISteamClient_CreateSteamPipe(ISteamClient instancePtr); virtual HSteamPipe CreateSteamPipe() = 0; It looks like the pipe is already created upon initializing and calling CreateSteamPipe creates a secondary one that doesn't receive callbacks. SteamAPI_GetHSteamPipe is called inline to retrieve the current pipe on initialization of the the classed version of the API, but the documentation doesn't mention it exists at all, leading to confusion when using the flat version. Well, mystery solved.And it successfully fires the 3-arg Run method of the callback object. However for some reason the function table of the ISteamClient seems to be off by one.. it kept calling the wrong methods until I commented one out, in this case GetIntPtr() as seen above, then everything seemed to line up. Not sure what the proper way to ensure it matches the C++ layout here, but at least it seems to be mostly working for now. Thanks again!Ugh what GetIntPtr? Unless of course this header file is wrong[0]. Make the members match exactly, order and everything and it should "just work". [0] https://github.com/ValveSoftware/source-sdk-2013/blob/master/mp/src/public/steam/isteamclient.h#L113
Jun 10 2018
On 10/06/2018 11:28 PM, cc wrote:Woops, that GetIntPtr came from the .cs header in the same folder as the C++ headers distributed with the SDK, that'll teach me to ctrl+f "class ISteamClient" in all open files and copy/paste before reading.Stay with the c/c++ headers for c/c++ code. We don't generally don't go testing the long way ;)
Jun 10 2018
On Friday, 8 June 2018 at 00:55:35 UTC, cc wrote:I've defined it in D, as per https://dlang.org/spec/cpp_interface.html#classes :change this to class, or even abstract class as shown in exampleextern(C++) { interface CCallbackBase { //this() { m_nCallbackFlags = 0; m_iCallback = 0; } void Run( void *pvParam ); void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ); int GetICallback(); int GetCallbackSizeBytes(); enum { k_ECallbackFlagsRegistered = 0x01, k_ECallbackFlagsGameServer = 0x02 } //uint8 m_nCallbackFlags; //int m_iCallback; //friend class CCallbackMgr; //CCallbackBase( const CCallbackBase& ); //CCallbackBase& operator=( const CCallbackBase& ); } }add extern(C++) to class as well toclass CImpl : CCallbackBase { extern(C++) { this() { m_nCallbackFlags = 0; m_iCallback = 0; } void Run( void *pvParam ) { writeln("Run1"); } void Run( void *pvParam, bool bIOFailure, SteamAPICall_t hSteamAPICall ) { writeln("Run2"); } int GetICallback() { return m_iCallback; } int GetCallbackSizeBytes() { return NumberOfCurrentPlayers_t.sizeof; } // ordinarily use templates to determine what type struct ptr to return } uint8 m_nCallbackFlags; int m_iCallback; }you also may or may not need to mark non-virtual C++ methods as final. Of course I haven't used D for quite some time so I can be mistaken. But the lesson I learned the hard way is that in D for extern(C++) you don't use interface(it simply has no mapping to C++ types?), and for defining COM-interfaces use interface or it will bite you. I hope it helps.
Jun 07 2018