www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - extern(Windows, "user32.dll")

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
The Delphi programming language allows specifying the DLL name 
right on the declaration of extern symbols:

function GetVersion: DWORD; stdcall; external 'kernel32.dll';

I would like to suggest adding something similar to D:

extern(Windows, "kernel32.dll") DWORD GetVersion();

Rationale:

This absolves the need for import libraries.

* Import libraries are a major and unnecessary pain in the neck 
when using DLLs of any kind.

* Import libraries may need conversion from COFF format to OMF 
format with a tool which is not included with D (coffimplib).

* Import libraries can *sometimes* be created from the DLL using 
implib, but this does not always work, depending on the calling 
convention and its mangling.

* Name mangling mismatches are difficult to diagnose.

* Import libraries cannot be used with tools such as "rdmd", 
except via pragma(lib).

Any thoughts? DIP or not?
Feb 01 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/02/2015 11:26 p.m., Vladimir Panteleev wrote:

 The Delphi programming language allows specifying the DLL name right on
 the declaration of extern symbols:

 function GetVersion: DWORD; stdcall; external 'kernel32.dll';

 I would like to suggest adding something similar to D:

 extern(Windows, "kernel32.dll") DWORD GetVersion();

 Rationale:

 This absolves the need for import libraries.

 * Import libraries are a major and unnecessary pain in the neck when
 using DLLs of any kind.

 * Import libraries may need conversion from COFF format to OMF format
 with a tool which is not included with D (coffimplib).

 * Import libraries can *sometimes* be created from the DLL using implib,
 but this does not always work, depending on the calling convention and
 its mangling.

 * Name mangling mismatches are difficult to diagnose.

 * Import libraries cannot be used with tools such as "rdmd", except via
 pragma(lib).

 Any thoughts? DIP or not?


How about:

extern(System, "mylibrary"):

That way it'll be a bit more cross platform'ish.
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 10:34:05 UTC, Rikki Cattermole 
wrote:

 How about:

 extern(System, "mylibrary"):

 That way it'll be a bit more cross platform'ish.


DLLs can have any extension. Many Windows components that don't 
have a .exe or .dll extension are actually PE files.

You should be able to define the library as a constant, e.g.:

enum user32 = "user32.dll";

extern(Windows, user32) DWORD MessageBox(...);

This is also what Delphi does.

Then you can put the enum in version(...) blocks.
Feb 01 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/02/2015 11:36 p.m., Vladimir Panteleev wrote:

 On Sunday, 1 February 2015 at 10:34:05 UTC, Rikki Cattermole wrote:

 How about:

 extern(System, "mylibrary"):

 That way it'll be a bit more cross platform'ish.


 DLLs can have any extension. Many Windows components that don't have a
 .exe or .dll extension are actually PE files.

 You should be able to define the library as a constant, e.g.:

 enum user32 = "user32.dll";

 extern(Windows, user32) DWORD MessageBox(...);

 This is also what Delphi does.

 Then you can put the enum in version(...) blocks.


I see, reasonable.
Feb 01 2015
prev sibling next sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Vladimir Panteleev"  wrote in message 
news:kksyjkgkwdfqfwhqwldb forum.dlang.org...


 extern(Windows, "kernel32.dll") DWORD GetVersion();


export would be a better fit than extern, or a pragma.  It's probably a good 
idea to check that all object formats can actually handle an impdef in a 
normal object file.  IIRC you also need a way to specify undecorated names 
for some windows DLLs.

Seems like a good idea.
Feb 01 2015
prev sibling next sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 01.02.2015 um 11:26 schrieb Vladimir Panteleev:

 The Delphi programming language allows specifying the DLL name right on
 the declaration of extern symbols:

 function GetVersion: DWORD; stdcall; external 'kernel32.dll';

 I would like to suggest adding something similar to D:

 extern(Windows, "kernel32.dll") DWORD GetVersion();

 Rationale:

 This absolves the need for import libraries.

 * Import libraries are a major and unnecessary pain in the neck when
 using DLLs of any kind.

 * Import libraries may need conversion from COFF format to OMF format
 with a tool which is not included with D (coffimplib).

 * Import libraries can *sometimes* be created from the DLL using implib,
 but this does not always work, depending on the calling convention and
 its mangling.

 * Name mangling mismatches are difficult to diagnose.

 * Import libraries cannot be used with tools such as "rdmd", except via
 pragma(lib).

 Any thoughts? DIP or not?


Please no. Import libraries are the way it was designed and it should be 
used that way. I'm currently implementing D-Dll support for Windows with 
dmd and without import libraries it will never work.

Also what you are proposing could easily be implemented by a library. E.g.

 dllimport("kernel32.dll") extern(Windows) DWORD GetVersion();
mixin DllImportFunctions;

Where DllImportFunctions would iterate over all symbols in the module, 
look for the UDA dllimport and then attemp to load the Dll and import 
the function.

Kind Regards
Benjamin Thaut
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 10:40:06 UTC, Benjamin Thaut wrote:

 Please no. Import libraries are the way it was designed


Delphi does without them just fine.


 and it should be used that way.


Why? It seems to me like they are a historical artifact. I don't 
see any merit in the design, none at all. Linux does away with 
them because ld can link against .so, but OPTLINK can't link 
against .dll files (although UniLink can).


 I'm currently implementing D-Dll support for Windows with dmd 
 and without import libraries it will never work.


Why? I'm not suggesting to remove import library support.


 Also what you are proposing could easily be implemented by a 
 library. E.g.

  dllimport("kernel32.dll") extern(Windows) DWORD GetVersion();
 mixin DllImportFunctions;

 Where DllImportFunctions would iterate over all symbols in the 
 module, look for the UDA dllimport and then attemp to load the 
 Dll and import the function.


Dynamic loading delays the error until execution time, instead of 
link time, and is slower.
Feb 01 2015
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 01.02.2015 um 11:45 schrieb Vladimir Panteleev:

 On Sunday, 1 February 2015 at 10:40:06 UTC, Benjamin Thaut wrote:

 Please no. Import libraries are the way it was designed


 Delphi does without them just fine.


C++ does with them just fine.


 I'm currently implementing D-Dll support for Windows with dmd and
 without import libraries it will never work.


 Why? I'm not suggesting to remove import library support.


I don't care what you are suggesting. If you want a implementation of 
Dll support without import libraries do it yourself. And lets just say 
it is required to link against druntime correctly. I don't want to give 
you a 4 page text explanation why.


 Dynamic loading delays the error until execution time, instead of link
 time, and is slower.


I hardly doubt that. If you link against a dll the windows binary loader 
is just doing the work for you. But the symbols are looked up in the dll 
by string either way.

Also, proof of concept: http://dpaste.dzfl.pl/efbd54314a69

The real issue here is, that dmd simply does not come with all 
neccessary import libraries when using optlink. As soon as you switch to 
the microsoft linkers this becomes a non issue.

Kind Regards
Benjamin Thaut
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 11:10:21 UTC, Benjamin Thaut wrote:

 Am 01.02.2015 um 11:45 schrieb Vladimir Panteleev:

 On Sunday, 1 February 2015 at 10:40:06 UTC, Benjamin Thaut 
 wrote:

 Please no. Import libraries are the way it was designed


 Delphi does without them just fine.


 C++ does with them just fine.


And this is a valid argument, how?


 I don't care what you are suggesting. If you want a 
 implementation of Dll support without import libraries do it 
 yourself.


Nice.


 And lets just say it is required to link against druntime 
 correctly. I don't want to give you a 4 page text explanation 
 why.


So you expect us to just trust you, then?

I also did not say that we should replace all usage of import 
libraries in Druntime.


 I hardly doubt that. If you link against a dll the windows 
 binary loader is just doing the work for you. But the symbols 
 are looked up in the dll by string either way.


I believe this hasn't been true for a few Windows versions now. 
AFAIK static imports are cached by the PE loader.

Either way, a heap allocation during initialization could force a 
rebase.


 Also, proof of concept: http://dpaste.dzfl.pl/efbd54314a69


This is nothing new. Derelict does this. I have it implemented 
myself in my library.

It won't work in DLLs. You can't call LoadLibrary in DllMain. The 
runtime is initialized in DllMain, so static constructors run 
there too. Even if you defer the LoadLibrary call until the 
function is first called, that still leaves you the problem that 
the functions are unusable in static constructors.

It also won't work with TLS (i.e. all D DLLs) except on recent 
Windows versions.


 The real issue here is, that dmd simply does not come with all 
 neccessary import libraries when using optlink. As soon as you 
 switch to the microsoft linkers this becomes a non issue.


It literally *can't* come with *all* necessary import libraries.
Feb 01 2015
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
 And lets just say it is required to link against druntime correctly. I
 don't want to give you a 4 page text explanation why.


 So you expect us to just trust you, then?


No, you just need to wait like everyone else until I do the PR for dmd / 
druntime / phobos for a full explanation. If you don't want to wait 
until then just read the source code: 
https://github.com/Ingrater/dmd/tree/DllSupport


 I also did not say that we should replace all usage of import libraries
 in Druntime.


Sorry, I did jump over the "not" in :


 Why? I'm not suggesting to remove import library support.



 It won't work in DLLs. You can't call LoadLibrary in DllMain. The
 runtime is initialized in DllMain, so static constructors run there too.
 Even if you defer the LoadLibrary call until the function is first
 called, that still leaves you the problem that the functions are
 unusable in static constructors.


Thats why its called a proof of concept, it doesn't mean its perfect ;-)


 It also won't work with TLS (i.e. all D DLLs) except on recent Windows
 versions.


But the TLS issues are going to remain no matter if the dll is loaded 
via LoadLibrary or not. Also there are TLS fixes in core.sys.windows.dll 
for Windows XP. And we officially don't support anything before XP.


 The real issue here is, that dmd simply does not come with all
 neccessary import libraries when using optlink. As soon as you switch
 to the microsoft linkers this becomes a non issue.


 It literally *can't* come with *all* necessary import libraries.


But still, adding a feature for the sole purpose to link against Windows 
System Dlls is just overkill.

But feel free to go ahead and try getting it past Andrei and Walter. 
Just don't expect my support. Recently even the int[$] = [1,2,3,4] 
feature was killed off because it could be implemneted in a library. So 
I highly doubt that they are OK with adding a feature for importing 
Windows System Dlls only.
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 11:50:21 UTC, Benjamin Thaut wrote:

 Sorry, I did jump over the "not" in :


OK, I'm glad that was just a misunderstanding.


 No, you just need to wait like everyone else until I do the PR 
 for dmd / druntime / phobos for a full explanation. If you 
 don't want to wait until then just read the source code: 
 https://github.com/Ingrater/dmd/tree/DllSupport


OK, the explanation is no longer relevant.

But I haven't heard of this project before. In a few words, what 
does this do? Or a link to a DIP or existing discussion?


 Thats why its called a proof of concept, it doesn't mean its 
 perfect ;-)


OK, but you can't solve these problems without using DLL imports, 
can you?


 But the TLS issues are going to remain no matter if the dll is 
 loaded via LoadLibrary or not.


That's not how I understood it.


 Also there are TLS fixes in core.sys.windows.dll for Windows XP.


Cool, didn't know that. That argument isn't relevant, then.


 But still, adding a feature for the sole purpose to link 
 against Windows System Dlls is just overkill.

 But feel free to go ahead and try getting it past Andrei and 
 Walter. Just don't expect my support. Recently even the int[$] 
 = [1,2,3,4] feature was killed off because it could be 
 implemneted in a library. So I highly doubt that they are OK 
 with adding a feature for importing Windows System Dlls only.


But it's not just system Windows libraries, but all DLLs with a C 
interface. You won't need to mess with import libraries to load 
curl, or OpenSSL, or SQLite...
Feb 01 2015
parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 01.02.2015 um 13:02 schrieb Vladimir Panteleev:


 But I haven't heard of this project before. In a few words, what does
 this do? Or a link to a DIP or existing discussion?


The last discussion is over a year old, the resulting DIP was DIP 45:
http://wiki.dlang.org/DIP45

The currently blocking issue for this implementation is, that export 
both means "public" and "dllexport" which is discussed here:
http://forum.dlang.org/thread/m9lhc3$1r1v$1 digitalmars.com


 Thats why its called a proof of concept, it doesn't mean its perfect ;-)


 OK, but you can't solve these problems without using DLL imports, can you?


most likely not.



 But it's not just system Windows libraries, but all DLLs with a C
 interface. You won't need to mess with import libraries to load curl, or
 OpenSSL, or SQLite...


Yeah, but the problem is, that as soon as you do that you are limiting 
yourself to DLL versions of that library. If you for example want to 
link curl statically this will no longer work. And in my opinion its bad 
to hardcode in the sourcecode if your expecting a static or shared 
version of the library.

Thats also the nice thing about import libraries in my opinion. If you 
ship a library you can simply create two folders .e.g. one called static 
and the other called shared. Both contain .lib files with the same name. 
So the only thing a user as to do when he wants to switch between 
statically and dynamically linking against your library is changing the 
search directory passed to the linker.
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 12:09:56 UTC, Benjamin Thaut wrote:

 http://wiki.dlang.org/DIP45


OK, so I understand it's mostly about D DLLs.

This bit though:

"In an import library the original symbol is redifined as 
trampoline that simply dereferences the _imp_ pointer to the DLL 
function. Thus calling an exported function will be compatible 
with both import libraries and static libraries, in the later 
case without indirection."

Is this different from how things are now?

Also, how does this conflict with my proposal? If the compiler 
knows the function to be in a DLL, it can elide generating a 
trampoline at all, and reference the __imp__ symbol directly.


 Yeah, but the problem is, that as soon as you do that you are 
 limiting yourself to DLL versions of that library. If you for 
 example want to link curl statically this will no longer work. 
 And in my opinion its bad to hardcode in the sourcecode if your 
 expecting a static or shared version of the library.


That's a good point. It would be possible to define my proposal 
so that it would be easy to switch between the two with a 
-version switch. On one hand, you trade one compiler switch 
(static.lib or import.lib) for another (-version=static). On the 
other hand, for the DLL case, you skip the headache of import 
libraries.
Feb 01 2015
parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 01.02.2015 um 14:27 schrieb Vladimir Panteleev:

 On Sunday, 1 February 2015 at 12:09:56 UTC, Benjamin Thaut wrote:

 http://wiki.dlang.org/DIP45


 OK, so I understand it's mostly about D DLLs.


Yes of course. Because they are still not supported.


 This bit though:

 "In an import library the original symbol is redifined as trampoline
 that simply dereferences the _imp_ pointer to the DLL function. Thus
 calling an exported function will be compatible with both import
 libraries and static libraries, in the later case without indirection."

 Is this different from how things are now?


No its not. I'm sticking with the additional indirection. But linking 
against a Dll directly would still be possbile. The linker has to 
generate the additional indirection anyway, no matter if a import 
library was used or not. This must be done because otherwise the code 
generated by the compiler, which assumes a additional indirection, would 
no longer be working (in case of data symbols only, function symbols 
would work without the indirection).


 Also, how does this conflict with my proposal?


It does not. The only conflict would be if we would ban import libraries 
completely, which I misunderstood.


 If the compiler knows the
 function to be in a DLL, it can elide generating a trampoline at all,
 and reference the __imp__ symbol directly.


Yes it could.


 That's a good point. It would be possible to define my proposal so that
 it would be easy to switch between the two with a -version switch. On
 one hand, you trade one compiler switch (static.lib or import.lib) for
 another (-version=static). On the other hand, for the DLL case, you skip
 the headache of import libraries.


Your proposal could also be translated to:

pragma(lib, "kernel32.dll");
extern(dll) DWORD GetVersion();

You just want a way to tell the compiler that that function is definitly 
located in a dll. The additional benefit of the above example would be, 
that you don't have to repeat which library the symbol is in for each 
symbol. The problem I see here is that walter didn't want to have a 
seperation between export and import. So he designed export to mean 
both. So I don't know how happy he will be if you want to add in a 
dllimport equivalent keyword. You could do however:


pragma(lib, "wininet.dll");
export extern(Windows) HINTERNET InternetOpen(
   LPCTSTR lpszAgent,
   DWORD dwAccessType,
   LPCTSTR lpszProxyName,
   LPCTSTR lpszProxyBypass,
   DWORD dwFlags
);

The fun thing is, this would work with the current dmd. If export is 
applied to a function declaration, it means "dllimport". That means 
during linktime it would look for the __imp_InternetOpen symbol.
The only thing that doesn't work yet is pragma(lib, "kernel32.dll");
So what you should actually request is pragma(lib, "kernel32.dll") to be 
implemented. Although the only way this could work is, that the compiler 
actually calls implib for you (or the msvc equivalent) and then actually 
links against that. The issue that ketmar originally had would remain 
though, this would only work with libraries that behave nicely and don't 
change function mangling in their import library. (like wininet does)
Feb 01 2015
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 10:26:25 UTC, Vladimir Panteleev 
wrote:

 Rationale:


Another data point, apparently MinGW doesn't need import 
libraries either, if you specify `--enable-auto-import 
--enable-stdcall-fixup`.
Feb 01 2015