• Andre Pany (37/38) Nov 03 2017 Hi,
• Andre Pany (27/66) Nov 03 2017 I have an idea which solves several problems.
• Rainer Schuetze (22/102) Nov 04 2017 Finding the wrong linker seems to happen quite often, but I wonder why:
• Andre Pany (16/75) Nov 04 2017 I would say, by executing vcvarsall the user showed his intention
• Rainer Schuetze (6/91) Nov 04 2017 That's what I wrote above, too: the settings from vcvarsall should take

Andre Pany <andre s-e-a-p.de> writes:

Hi,

Visual Studio has a batch file which sets all needed environment 
variables for the Microsoft linker (LIB environment variables 
points to all necessary folders). Even the folder containing the 
link.exe for the specified architecture (x86, amd64) is added to 
the path variable as first path.

 vsdevcmd -arch=amd64

As the sc.ini targets the windows development (visuals studio) 
after calling the developer command prompt, dmd should be able to 
compile -m32mscoff, -m64 out of the box without the need to 
change anything within sc.ini.

There is even a comment in sc.ini which points in this direction:
; Add the library subdirectories of all VC and Windows SDK 
versions so things
; just work for users using dmd from the VS Command Prompt

Unfortunately it does not work for various reasons:
- optlink has the filename link.exe which conflicts with the 
filename of the microsoft linker
- from my investigation I have the assumption that even if an 
environment variable LIB exists, the value is discarded by the 
sc.ini logic. I tried hard, but it seems existing environment 
variable LIB cannot be used in sc.ini

If you can use the windows installer, it will set the right 
values in sc.ini. But there are scenarios in which you cannot use 
the windows installer but have to use the dmd zip archive (build 
infrastructure scripts).

In my scenario it seems the only possibility is to adapt my build 
infrastructure scripts to:
- extract dmd archive
- delete link.exe (optlink)
- modify the sc.ini

That is a solution for my case, but my gut feeling is, it should 
be solved in general
to ease the usage of the microsoft linker.

I thought it was planned to rename the optlink link.exe to 
optlink.exe?

Kind regards
André

Andre Pany <andre s-e-a-p.de> writes:

On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 Hi,

 Visual Studio has a batch file which sets all needed 
 environment variables for the Microsoft linker (LIB environment 
 variables points to all necessary folders). Even the folder 
 containing the link.exe for the specified architecture (x86, 
 amd64) is added to the path variable as first path.

 vsdevcmd -arch=amd64

 As the sc.ini targets the windows development (visuals studio) 
 after calling the developer command prompt, dmd should be able 
 to compile -m32mscoff, -m64 out of the box without the need to 
 change anything within sc.ini.

 There is even a comment in sc.ini which points in this 
 direction:
 ; Add the library subdirectories of all VC and Windows SDK 
 versions so things
 ; just work for users using dmd from the VS Command Prompt

 Unfortunately it does not work for various reasons:
 - optlink has the filename link.exe which conflicts with the 
 filename of the microsoft linker
 - from my investigation I have the assumption that even if an 
 environment variable LIB exists, the value is discarded by the 
 sc.ini logic. I tried hard, but it seems existing environment 
 variable LIB cannot be used in sc.ini

 If you can use the windows installer, it will set the right 
 values in sc.ini. But there are scenarios in which you cannot 
 use the windows installer but have to use the dmd zip archive 
 (build infrastructure scripts).

 In my scenario it seems the only possibility is to adapt my 
 build infrastructure scripts to:
 - extract dmd archive
 - delete link.exe (optlink)
 - modify the sc.ini

 That is a solution for my case, but my gut feeling is, it 
 should be solved in general
 to ease the usage of the microsoft linker.

 I thought it was planned to rename the optlink link.exe to 
 optlink.exe?

 Kind regards
 André

I have an idea which solves several problems.

Current state:
The dmd windows archive has a folder "bin" with a 32 bit dmd 
compiler which is able to create x86 and x86_64 applications. By 
default the dmd compiler creates 32 bit applications using the 
optlink link.exe contained also in this folder.

Idea:
Within the folder "bin64" the 64 bit dmd compiler could be 
shipped within the dmd windows archive. I assume the 64 bit dmd 
compiler is able to create x86 and x86_64 applications. In folder 
"bin64" no optlink link.exe should be available.
The usage of either the microsoft link.exe or another linker 
(maybe the one from LDC) is anticipated. The 64 bit dmd compiler 
could have the option -m64 set as default.

The user can decide whether he wants to use the "stable" dmd 
compiler with usage of optlink or he want to use the 64 bit dmd 
compiler which is open for other linkers than optlink by simply 
setting his path variable to bin or to bin64.

Using this approach:
- DMD compiler is working out of the box for the VS Developer 
Command Prompt - no need to rename the optlink link.exe
- A 64 bit dmd compiler is shipped which solves issues on large D 
projects

What do you think?

Kind regards
André

Rainer Schuetze <r.sagitario gmx.de> writes:

On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 Hi,

 Visual Studio has a batch file which sets all needed environment 
 variables for the Microsoft linker (LIB environment variables points 
 to all necessary folders). Even the folder containing the link.exe for 
 the specified architecture (x86, amd64) is added to the path variable 
 as first path.

 vsdevcmd -arch=amd64

 As the sc.ini targets the windows development (visuals studio) after 
 calling the developer command prompt, dmd should be able to compile 
 -m32mscoff, -m64 out of the box without the need to change anything 
 within sc.ini.

 There is even a comment in sc.ini which points in this direction:
 ; Add the library subdirectories of all VC and Windows SDK versions so 
 things
 ; just work for users using dmd from the VS Command Prompt

 Unfortunately it does not work for various reasons:
 - optlink has the filename link.exe which conflicts with the filename 
 of the microsoft linker
 - from my investigation I have the assumption that even if an 
 environment variable LIB exists, the value is discarded by the sc.ini 
 logic. I tried hard, but it seems existing environment variable LIB 
 cannot be used in sc.ini

 If you can use the windows installer, it will set the right values in 
 sc.ini. But there are scenarios in which you cannot use the windows 
 installer but have to use the dmd zip archive (build infrastructure 
 scripts).

 In my scenario it seems the only possibility is to adapt my build 
 infrastructure scripts to:
 - extract dmd archive
 - delete link.exe (optlink)
 - modify the sc.ini

 That is a solution for my case, but my gut feeling is, it should be 
 solved in general
 to ease the usage of the microsoft linker.

 I thought it was planned to rename the optlink link.exe to optlink.exe?

 Kind regards
 André

 
 I have an idea which solves several problems.
 
 Current state:
 The dmd windows archive has a folder "bin" with a 32 bit dmd compiler 
 which is able to create x86 and x86_64 applications. By default the dmd 
 compiler creates 32 bit applications using the optlink link.exe 
 contained also in this folder.
 
 Idea:
 Within the folder "bin64" the 64 bit dmd compiler could be shipped 
 within the dmd windows archive. I assume the 64 bit dmd compiler is able 
 to create x86 and x86_64 applications. In folder "bin64" no optlink 
 link.exe should be available.
 The usage of either the microsoft link.exe or another linker (maybe the 
 one from LDC) is anticipated. The 64 bit dmd compiler could have the 
 option -m64 set as default.
 
 The user can decide whether he wants to use the "stable" dmd compiler 
 with usage of optlink or he want to use the 64 bit dmd compiler which is 
 open for other linkers than optlink by simply setting his path variable 
 to bin or to bin64.
 
 Using this approach:
 - DMD compiler is working out of the box for the VS Developer Command 
 Prompt - no need to rename the optlink link.exe
 - A 64 bit dmd compiler is shipped which solves issues on large D projects
 
 What do you think?
 
 Kind regards
 André

Finding the wrong linker seems to happen quite often, but I wonder why:

sc.ini usually has the absolute path to the linker set via the LINKCMD 
variable, so finding the correct executable should not be a problem if 
the installer has set things up correctly. If extracted from the zip 
there are also defaults that might not work but should not find the 
wrong linker.

Is it the result of trying to fix the sc.ini manually?

The problem with overriding the LIB environment variable is that optlink 
and the MS linker both use it, but stumble over the libraries for the 
other linker due to different file formats.

I think the best option forward is to not rely on the installer to find 
the correct setup at installation time, but detect the available linker 
and its environment before invoking it:

- if the environment is already set up via vcvarsall, i.e. VCINSTALLDIR 
set, use that to call the approriate linker

- if VCINSTALLDIR is not set, use the registry to find the best fit, 
i.e. highest available VS version

- some way to override the automatism should be available, too.

We might also be able to provide some fallback if no VS installation is 
found, based on the LLVM linker, but with limited C runtime/platform 
library support.

Andre Pany <andre s-e-a-p.de> writes:

On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze 
wrote:
 On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 [...]

 
 I have an idea which solves several problems.
 
 Current state:
 The dmd windows archive has a folder "bin" with a 32 bit dmd 
 compiler which is able to create x86 and x86_64 applications. 
 By default the dmd compiler creates 32 bit applications using 
 the optlink link.exe contained also in this folder.
 
 Idea:
 Within the folder "bin64" the 64 bit dmd compiler could be 
 shipped within the dmd windows archive. I assume the 64 bit 
 dmd compiler is able to create x86 and x86_64 applications. In 
 folder "bin64" no optlink link.exe should be available.
 The usage of either the microsoft link.exe or another linker 
 (maybe the one from LDC) is anticipated. The 64 bit dmd 
 compiler could have the option -m64 set as default.
 
 The user can decide whether he wants to use the "stable" dmd 
 compiler with usage of optlink or he want to use the 64 bit 
 dmd compiler which is open for other linkers than optlink by 
 simply setting his path variable to bin or to bin64.
 
 Using this approach:
 - DMD compiler is working out of the box for the VS Developer 
 Command Prompt - no need to rename the optlink link.exe
 - A 64 bit dmd compiler is shipped which solves issues on 
 large D projects
 
 What do you think?
 
 Kind regards
 André

 Finding the wrong linker seems to happen quite often, but I 
 wonder why:

 sc.ini usually has the absolute path to the linker set via the 
 LINKCMD variable, so finding the correct executable should not 
 be a problem if the installer has set things up correctly. If 
 extracted from the zip there are also defaults that might not 
 work but should not find the wrong linker.

 Is it the result of trying to fix the sc.ini manually?

 The problem with overriding the LIB environment variable is 
 that optlink and the MS linker both use it, but stumble over 
 the libraries for the other linker due to different file 
 formats.

 I think the best option forward is to not rely on the installer 
 to find the correct setup at installation time, but detect the 
 available linker and its environment before invoking it:

 - if the environment is already set up via vcvarsall, i.e. 
 VCINSTALLDIR set, use that to call the approriate linker

 - if VCINSTALLDIR is not set, use the registry to find the best 
 fit, i.e. highest available VS version

 - some way to override the automatism should be available, too.

 We might also be able to provide some fallback if no VS 
 installation is found, based on the LLVM linker, but with 
 limited C runtime/platform library support.

I would say, by executing vcvarsall the user showed his intention 
to use the coff libraries and the Microsoft linker for the actual 
console session instead of the omf libraries.

I see a disadvantage by detecting the vc installation directory / 
lib directories. All paths are specific for the vs version. If 
there is x new vs version we are always behind.

But Microsoft has a solution for this problem. Vcvarsall sets the 
linker as first element of the path variable and the lib env 
variable points to the right folder. This solution will work for 
old vs version and any upcoming version.

I set linkcmd to "link.exe". Unfortunately the link.exe in the 
dmd bin folder has precedence over the path env variable.

Kind regards
Andre

Rainer Schuetze <r.sagitario gmx.de> writes:

On 04.11.2017 12:16, Andre Pany wrote:
 On Saturday, 4 November 2017 at 09:39:02 UTC, Rainer Schuetze wrote:
 On 03.11.2017 21:51, Andre Pany wrote:
 On Friday, 3 November 2017 at 11:52:10 UTC, Andre Pany wrote:
 [...]

 I have an idea which solves several problems.

 Current state:
 The dmd windows archive has a folder "bin" with a 32 bit dmd compiler 
 which is able to create x86 and x86_64 applications. By default the 
 dmd compiler creates 32 bit applications using the optlink link.exe 
 contained also in this folder.

 Idea:
 Within the folder "bin64" the 64 bit dmd compiler could be shipped 
 within the dmd windows archive. I assume the 64 bit dmd compiler is 
 able to create x86 and x86_64 applications. In folder "bin64" no 
 optlink link.exe should be available.
 The usage of either the microsoft link.exe or another linker (maybe 
 the one from LDC) is anticipated. The 64 bit dmd compiler could have 
 the option -m64 set as default.

 The user can decide whether he wants to use the "stable" dmd compiler 
 with usage of optlink or he want to use the 64 bit dmd compiler which 
 is open for other linkers than optlink by simply setting his path 
 variable to bin or to bin64.

 Using this approach:
 - DMD compiler is working out of the box for the VS Developer Command 
 Prompt - no need to rename the optlink link.exe
 - A 64 bit dmd compiler is shipped which solves issues on large D 
 projects

 What do you think?

 Kind regards
 André

 Finding the wrong linker seems to happen quite often, but I wonder why:

 sc.ini usually has the absolute path to the linker set via the LINKCMD 
 variable, so finding the correct executable should not be a problem if 
 the installer has set things up correctly. If extracted from the zip 
 there are also defaults that might not work but should not find the 
 wrong linker.

 Is it the result of trying to fix the sc.ini manually?

 The problem with overriding the LIB environment variable is that 
 optlink and the MS linker both use it, but stumble over the libraries 
 for the other linker due to different file formats.

 I think the best option forward is to not rely on the installer to 
 find the correct setup at installation time, but detect the available 
 linker and its environment before invoking it:

 - if the environment is already set up via vcvarsall, i.e. 
 VCINSTALLDIR set, use that to call the approriate linker

 - if VCINSTALLDIR is not set, use the registry to find the best fit, 
 i.e. highest available VS version

 - some way to override the automatism should be available, too.

 We might also be able to provide some fallback if no VS installation 
 is found, based on the LLVM linker, but with limited C 
 runtime/platform library support.

 
 I would say, by executing vcvarsall the user showed his intention to use 
 the coff libraries and the Microsoft linker for the actual console 
 session instead of the omf libraries.
 
 I see a disadvantage by detecting the vc installation directory / lib 
 directories. All paths are specific for the vs version. If there is x 
 new vs version we are always behind.
 
 But Microsoft has a solution for this problem. Vcvarsall sets the linker 
 as first element of the path variable and the lib env variable points to 
 the right folder. This solution will work for old vs version and any 
 upcoming version.

That's what I wrote above, too: the settings from vcvarsall should take 
precedence. The library path for the phobos library must be added, though.

 
 I set linkcmd to "link.exe". Unfortunately the link.exe in the dmd bin 
 folder has precedence over the path env variable.

This is caused by CreateProcess looking in the directory of the calling 
process first. This could be avoided in dmd by searching the PATH 
folders explicitly before invoking the linker.