digitalmars.D - The issue with libraries that use Windows API prototypes
- Andrej Mitrovic (66/66) Jul 09 2011 I've run into yet another incompatibility issue with D2 libraries that
- Jacob Carlborg (11/50) Jul 10 2011 We can probably add DWT to this list as well. This seems like quite a
- Andrej Mitrovic (9/10) Jul 10 2011 Yeah I would agree with this. For example, to use the winapi in Python
- Jacob Carlborg (4/14) Jul 10 2011 I'm already working on a package manager for D.
- Trass3r (1/6) Jul 10 2011 Long overdue indeed.
- Stewart Gordon (30/70) Jul 10 2011 I don't know Derelict at all. But if as I make out it's a cross-platfor...
I've run into yet another incompatibility issue with D2 libraries that use the Windows API. This time it's Derelict: testderelict.obj(testderelict) Error 42: Symbol Undefined _wglDeleteContext 4 testderelict.obj(testderelict) Error 42: Symbol Undefined _wglMakeCurrent 8 testderelict.obj(testderelict) Error 42: Symbol Undefined _wglCreateContext 4 --- errorlevel 3 The issue? Derelict uses its own Windows API header prototype in \Derelict2\DerelictUtil\derelict\util, which defines: alias void* HANDLE; alias HANDLE HGLRC; And the "_wgl" functions take HGLRC as the parameter. The problem is WindowsAPI also defines these, because well, it's a binding library so it has to. And it's prototypes are: typedef void* HANDLE; alias HANDLE HGLRC; When linking my project with WinsowsAPI and Derelict, the Derelict static library defines a function as "_wglDeleteContext", while the project that uses WindowsAPI sees the wglDeleteContext prototype from the import, but sees the HGLRC typedef in WindowsAPI, and DMD looks for the symbol "_wglDeleteContext 4". So now I can't really use Derelict and the WindowsAPI. Perhaps I can use it with std.c.windows.windows? Nope, guess again! std.c.windows.windows doesn't define NULL (small nuisance), and it doesn't use aliases to ANSI/WideChar versions of API functions like the WindowsAPI bindings, you have to explicitly use e.g. MessageBoxA or MessageBoxW. To add insult to injury, MessageBoxW isn't even defined in \druntime\src\core\sys\windows\windows.d (that's imported by std.c.windows.windows). It defines maybe 5% of anything in WindowsAPI, so using std.c.windows.windows is out of the question because of an enormous wall of errors due to missing symbols. But even so, std.c.windows.windows *also* defines HANDLE: typedef void *HANDLE; So far, we have 3 incompatible libraries. Add a couple of more libraries like DFL, DGui DCairo and now we're up to 6. *Every single* Windows-related library seems to define its own winapi prototypes, and because of issues like: 1. typedefs (which we're going to hopefully kill sometime soon) 2. small differences in how these prototypes were written these libraries become incompatible with each other. It's only going to get worse as each library author keeps growing their library and adds more prototypes. I can see two ways out of this situation: 1) D Library authors start using the WindowsAPI bindings from http://dsource.org/projects/bindings/wiki/WindowsApi, and expect the user of the their library to put WindowsAPI in the import path where the authors' libraries can find it to resolve symbols, and where the user's project can import it as well. This would mean the author's library's build process gets a bit more complicated as you would have to pass the path to the WindowsAPI bindings when building the author's library. 2) We devise some plan to incorporate WindowsAPI into Phobos itself. This way library users don't have to waste time fetching dependencies, and library authors don't have to waste time prototyping functions since the majority of this work will be already incorporated into Phobos. The DCairo author kindly suggested to use version(WindowsAPI) else(HisOwnPrototypes) as a workaround, but this doesn't scale. There could be numerous prototypes based on how many libraries you use. somewhere because the more time we wait, the worse this situation gets.
Jul 09 2011
On 2011-07-09 18:55, Andrej Mitrovic wrote:I've run into yet another incompatibility issue with D2 libraries that use the Windows API. This time it's Derelict: testderelict.obj(testderelict) Error 42: Symbol Undefined _wglDeleteContext 4 testderelict.obj(testderelict) Error 42: Symbol Undefined _wglMakeCurrent 8 testderelict.obj(testderelict) Error 42: Symbol Undefined _wglCreateContext 4 --- errorlevel 3 The issue? Derelict uses its own Windows API header prototype in \Derelict2\DerelictUtil\derelict\util, which defines: alias void* HANDLE; alias HANDLE HGLRC; And the "_wgl" functions take HGLRC as the parameter. The problem is WindowsAPI also defines these, because well, it's a binding library so it has to. And it's prototypes are: typedef void* HANDLE; alias HANDLE HGLRC; When linking my project with WinsowsAPI and Derelict, the Derelict static library defines a function as "_wglDeleteContext", while the project that uses WindowsAPI sees the wglDeleteContext prototype from the import, but sees the HGLRC typedef in WindowsAPI, and DMD looks for the symbol "_wglDeleteContext 4". So now I can't really use Derelict and the WindowsAPI. Perhaps I can use it with std.c.windows.windows? Nope, guess again! std.c.windows.windows doesn't define NULL (small nuisance), and it doesn't use aliases to ANSI/WideChar versions of API functions like the WindowsAPI bindings, you have to explicitly use e.g. MessageBoxA or MessageBoxW. To add insult to injury, MessageBoxW isn't even defined in \druntime\src\core\sys\windows\windows.d (that's imported by std.c.windows.windows). It defines maybe 5% of anything in WindowsAPI, so using std.c.windows.windows is out of the question because of an enormous wall of errors due to missing symbols. But even so, std.c.windows.windows *also* defines HANDLE: typedef void *HANDLE; So far, we have 3 incompatible libraries. Add a couple of more libraries like DFL, DGui DCairo and now we're up to 6.We can probably add DWT to this list as well. This seems like quite a mess that needs to be sorted out. It's hard to tell what the best solution would be. There need to be some kind of bindings in druntime, since it depends on the Windows APIs. On the other hand it seems to be unnecessary to have DirectX bindings and similar in druntime. Perhaps the best solution is to have an absolute minimum set of bindings in druntime and then merge all other bindings in the Windows API bindings project. This would be less of an issue when we get a package manager for D. -- /Jacob Carlborg
Jul 10 2011
On 7/10/11, Jacob Carlborg <doob me.com> wrote:This would be less of an issue when we get a package manager for D.Yeah I would agree with this. For example, to use the winapi in Python you would have to install http://sourceforge.net/projects/pywin32/ . Although for Windows these python packages come in a installation setup form, it's still damn easy to install one and even pick which Python version to install it for. As soon as you've got that package installed, you're pretty much covered for any python code that uses winapi. DSSS seemed like a good idea although its codebase scares me a little. :>
Jul 10 2011
On 2011-07-10 15:50, Andrej Mitrovic wrote:On 7/10/11, Jacob Carlborg<doob me.com> wrote:I'm already working on a package manager for D. -- /Jacob CarlborgThis would be less of an issue when we get a package manager for D.Yeah I would agree with this. For example, to use the winapi in Python you would have to install http://sourceforge.net/projects/pywin32/ . Although for Windows these python packages come in a installation setup form, it's still damn easy to install one and even pick which Python version to install it for. As soon as you've got that package installed, you're pretty much covered for any python code that uses winapi. DSSS seemed like a good idea although its codebase scares me a little. :>
Jul 10 2011
2) We devise some plan to incorporate WindowsAPI into Phobos itself. This way library users don't have to waste time fetching dependencies, and library authors don't have to waste time prototyping functions since the majority of this work will be already incorporated into Phobos.Long overdue indeed.
Jul 10 2011
On 09/07/2011 17:55, Andrej Mitrovic wrote: <snip>alias void* HANDLE; alias HANDLE HGLRC; And the "_wgl" functions take HGLRC as the parameter. The problem is WindowsAPI also defines these, because well, it's a binding library so it has to. And it's prototypes are: typedef void* HANDLE; alias HANDLE HGLRC;I don't know Derelict at all. But if as I make out it's a cross-platform library, I'm surprised if many functions in its API receive or return Windows API types directly, which should be the only case in which this matters. But for passing a HANDLE to a Derelict function, it still shouldn't be a problem as a typedef can be implicitly converted to its underlying type. derelict.util.wintypes.HANDLE and win32.basetsd.HANDLE are distinct symbols. If your code uses WindowsAPI, Derelict can still use its own bindings internally. Yes, it's wasteful, but the code should still compile.When linking my project with WinsowsAPI and Derelict, the Derelict static library defines a function as "_wglDeleteContext", while the project that uses WindowsAPI sees the wglDeleteContext prototype from the import, but sees the HGLRC typedef in WindowsAPI, and DMD looks for the symbol "_wglDeleteContext 4".And does it find that symbol? If not, have you checked that you're linking in all the right .lib files?So now I can't really use Derelict and the WindowsAPI. Perhaps I can use it with std.c.windows.windows? Nope, guess again! std.c.windows.windows doesn't define NULL (small nuisance),Why is it a nuisance at all? D code should use null, which is built in, not NULL.and it doesn't use aliases to ANSI/WideChar versions of API functions like the WindowsAPI bindings, you have to explicitly use e.g. MessageBoxA or MessageBoxW.std.c.windows is very deficient (last time I knew, Walter just added stuff as and when he needed it), so why single this omission out?*Every single* Windows-related library seems to define its own winapi prototypes, and because of issues like: 1. typedefs (which we're going to hopefully kill sometime soon) 2. small differences in how these prototypes were written these libraries become incompatible with each other. It's only going to get worse as each library author keeps growing their library and adds more prototypes.It's inefficient, but since the symbols have distinct fully qualified names, I can't see it causing clashes generally. Trouble occurs if: (a) A library API module (as opposed to a module for the library's internal use) publicly defines one of these symbols or publicly imports a module that does, thereby forcing you to use fully qualified names. (b) You need to pass data of Windows API types directly between your app and the library. Then you end up needing to use casts. You might also need to import a library module that defines the types, thereby making (a) an issue here too.I can see two ways out of this situation: 1) D Library authors start using the WindowsAPI bindings from http://dsource.org/projects/bindings/wiki/WindowsApi, and expect the user of the their library to put WindowsAPI in the import path where the authors' libraries can find it to resolve symbols, and where the user's project can import it as well. This would mean the author's library's build process gets a bit more complicated as you would have to pass the path to the WindowsAPI bindings when building the author's library.Then tell D programmers to install library code under a common base path, so that at most one extra import path in sc.ini is needed however many third party libraries you use.2) We devise some plan to incorporate WindowsAPI into Phobos itself. This way library users don't have to waste time fetching dependencies, and library authors don't have to waste time prototyping functions since the majority of this work will be already incorporated into Phobos.<snip> http://d.puremagic.com/issues/show_bug.cgi?id=317 Moreover, it's about time we had a plan for getting these bindings completed. Unfortunately now that I have a job.... Stewart.
Jul 10 2011