digitalmars.D - X86_mscoff / x86_64 as default for dub projects (windows)
- Andre Pany (11/11) Dec 06 2018 Hi,
- Bastiaan Veelo (5/9) Dec 06 2018 I am om favour of this. It would help prevent people running into
- kinke (11/18) Dec 06 2018 I also think this is long overdue - but I wouldn't change dub's
- Jonathan M Davis (6/15) Dec 06 2018 I'm inclined to argue that it's a bad idea given that it's not dmd's
- Basile B. (5/16) Dec 06 2018 I'd wish this too (i also wish DWARF info in mscoff).
- rikki cattermole (3/23) Dec 06 2018 Agreed we should hold off until mingw + lld stuff gain more maturity.
- Andre Pany (15/24) Dec 07 2018 Thanks a lot for the opinions. Maybe we could check that e.g. all
- Rubn (10/31) Dec 07 2018 This is part of the problem, in that there are more tests run on
- Jonathan M Davis (8/40) Dec 07 2018 The entire test suite for druntime, Phobos, and dmd has to pass on all
- Rubn (10/54) Dec 07 2018 https://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fdmd/detail/PR...
- Jonathan M Davis (17/50) Dec 07 2018 I think that they moved those tests to buildkite, but I don't know what ...
- Seb (16/71) Dec 19 2018 That's correct. Buildkite is a lot easier to work with then
- Guillaume Piolat (6/10) Dec 07 2018 The problem is that DMD -m32 always works, else you need a
- Andre Pany (7/19) Dec 07 2018 Visual studio is no longer needed as lld and x86mscoff/x86_64
- Guillaume Piolat (2/7) Dec 08 2018 Is it any faster than OPTLINK?
- Andre Pany (13/23) Dec 08 2018 LLD is advertised as a very fast linker. Whether that is true I
- Andre Pany (18/28) Dec 19 2018 I had some tries. LLD link is very fast in my applications. It
- Andre Pany (12/42) Dec 20 2018 As pointed out by Kinke in issue 19503, dmd ships with an old
- Guillaume Piolat (4/15) Dec 25 2018 It would be a cool thing to change the default if D_SIMD was
- Dennis (11/14) Dec 07 2018 My biggest gripe is this confusing matrix of compiler flags:
- kinke (7/11) Dec 07 2018 Some corrections for LDC:
- Atila Neves (3/14) Dec 19 2018 To me, it's obvious that the default on Windows 64-bit (i.e.
- Andre Pany (13/32) Dec 19 2018 I propose a smooth transition in 3 steps:
Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards Andre
Dec 06 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows?I am om favour of this. It would help prevent people running into optlink problems such as this one: https://github.com/MartinNowak/scod/issues/4 Bastiaan.
Dec 06 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default.I also think this is long overdue - but I wouldn't change dub's defaults, but DMD's instead (something like `-m32` on Windows meaning `-m32mscoff`, and a new `-m32omf` switch). It's a breaking change, but definitely worth it IMO. Besides optlink bugs and limitations, and the executable unfortunately being named `link.exe` and so conflicting with the MS one (and people running into problems when using LDC, depending on their PATH), I see the primary drawback in that probably > 99.9% of 3rd-party libraries out there are in COFF format (I haven't heard of that old OMF format from DOS times before playing with DMD).
Dec 06 2018
On Thursday, December 6, 2018 10:58:17 AM MST Andre Pany via Digitalmars-d wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries.I'm inclined to argue that it's a bad idea given that it's not dmd's default. It risks being very confusing, and anyone who actually cares to make the change can easily set their dub configuration accordingly. - Jonathan M Davis
Dec 06 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.
Dec 06 2018
On 07/12/2018 11:50 AM, Basile B. wrote:On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Agreed we should hold off until mingw + lld stuff gain more maturity. Then we can swap over fully with minimal pain.Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.
Dec 06 2018
On Friday, 7 December 2018 at 06:52:41 UTC, rikki cattermole wrote:On 07/12/2018 11:50 AM, Basile B. wrote:Thanks a lot for the opinions. Maybe we could check that e.g. all dub packages compiles fine (and unittests succeeds) with LLD to check the maturity. When we reached that level we can switch the default architecture. I also like the idea to switch the architecture in DMD itself to avoid surprises. And I really like to have a 64 bit DMD executable. Fortunately it is created by the nightly builds, but having it as release download would be really great. Kind regards AndréI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.Agreed we should hold off until mingw + lld stuff gain more maturity. Then we can swap over fully with minimal pain.
Dec 07 2018
On Thursday, 6 December 2018 at 22:50:25 UTC, Basile B. wrote:On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:This is part of the problem, in that there are more tests run on Linux than Windows. DMD doesn't even compile it self on Windows to test. It does for linux though. Also some of the community projects are all compiled and tested on linux but not windows. That's the state it is in, the only tests that are run are the most basic ones. Can't expect DMD on Windows to be stable at all, if ever that's the way it goes though I guess. No one seems to care or thinks running tests on linux only is good enough for windows.Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.
Dec 07 2018
On Friday, December 7, 2018 3:15:30 PM MST Rubn via Digitalmars-d wrote:On Thursday, 6 December 2018 at 22:50:25 UTC, Basile B. wrote:The entire test suite for druntime, Phobos, and dmd has to pass on all supported platforms on the auto-tester: https://auto-tester.puremagic.com/ Windows is not exempt from that. So, unless there are a bunch of tests in the test suite which are needlessly platform-specific, Windows is getting tested just as much as Linux is. Where do you get the idea that it's not? - Jonathan M DavisOn Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:This is part of the problem, in that there are more tests run on Linux than Windows. DMD doesn't even compile it self on Windows to test. It does for linux though. Also some of the community projects are all compiled and tested on linux but not windows. That's the state it is in, the only tests that are run are the most basic ones. Can't expect DMD on Windows to be stable at all, if ever that's the way it goes though I guess. No one seems to care or thinks running tests on linux only is good enough for windows.Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.
Dec 07 2018
On Saturday, 8 December 2018 at 02:08:14 UTC, Jonathan M Davis wrote:On Friday, December 7, 2018 3:15:30 PM MST Rubn via Digitalmars-d wrote:https://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fdmd/detail/PR-8519/3/pipeline I guess that's just another dead project, so I guess no platform is getting those tests anymore. Too bad, I guess that's another story for D, unless they moved to something else. What you linked are just those basic tests, those tests don't do anything complicated like building DMD itself with the newly created DMD and running the tests for both to ensure a correct binary is produced.On Thursday, 6 December 2018 at 22:50:25 UTC, Basile B. wrote:The entire test suite for druntime, Phobos, and dmd has to pass on all supported platforms on the auto-tester: https://auto-tester.puremagic.com/ Windows is not exempt from that. So, unless there are a bunch of tests in the test suite which are needlessly platform-specific, Windows is getting tested just as much as Linux is. Where do you get the idea that it's not? - Jonathan M DavisOn Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:This is part of the problem, in that there are more tests run on Linux than Windows. DMD doesn't even compile it self on Windows to test. It does for linux though. Also some of the community projects are all compiled and tested on linux but not windows. That's the state it is in, the only tests that are run are the most basic ones. Can't expect DMD on Windows to be stable at all, if ever that's the way it goes though I guess. No one seems to care or thinks running tests on linux only is good enough for windows.Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreI'd wish this too (i also wish DWARF info in mscoff). Unfortunately last time i tried to built DCD using -m64 (and MINGW stuff + LLD) the resulting PE crashed, so maybe there are problems to fix before switching.
Dec 07 2018
On Friday, December 7, 2018 8:09:37 PM MST Rubn via Digitalmars-d wrote:On Saturday, 8 December 2018 at 02:08:14 UTC, Jonathan M Davis wrote:I think that they moved those tests to buildkite, but I don't know what the exact situation with that is right now. While the main auto-tester has been the same for ages, they keep changing what they're doing with the other tests. But I believe that the tests for each project are run using travis and depend on how each project set that up, so which which platforms they test is going to vary from project to project. And I don't think that that includes Windows at all, because that would require something like appveyor. So, the testing of projects beyond dmd and the standard library is only on *nix systems AFAIK, but dmd itself and the standard library are definitely tested on all of the platforms that dmd supports.On Friday, December 7, 2018 3:15:30 PM MST Rubn viahttps://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fdmd/detail/PR-8 519/3/pipeline I guess that's just another dead project, so I guess no platform is getting those tests anymore. Too bad, I guess that's another story for D, unless they moved to something else.This is part of the problem, in that there are more tests run on Linux than Windows. DMD doesn't even compile it self on Windows to test. It does for linux though. Also some of the community projects are all compiled and tested on linux but not windows. That's the state it is in, the only tests that are run are the most basic ones. Can't expect DMD on Windows to be stable at all, if ever that's the way it goes though I guess. No one seems to care or thinks running tests on linux only is good enough for windows.The entire test suite for druntime, Phobos, and dmd has to pass on all supported platforms on the auto-tester: https://auto-tester.puremagic.com/ Windows is not exempt from that. So, unless there are a bunch of tests in the test suite which are needlessly platform-specific, Windows is getting tested just as much as Linux is. Where do you get the idea that it's not? - Jonathan M DavisWhat you linked are just those basic tests, those tests don't do anything complicated like building DMD itself with the newly created DMD and running the tests for both to ensure a correct binary is produced.The new compiler is built and used to run the druntime and Phobos test suites. Then it's used with the newly built druntime and Phobos to run the dmd test suite. So, the new dmd binary is definitely being tested, but no, the newly built dmd isn't then used to build dmd yet again as part of those tests. - Jonathan M Davis
Dec 07 2018
On Saturday, 8 December 2018 at 03:25:46 UTC, Jonathan M Davis wrote:On Friday, December 7, 2018 8:09:37 PM MST Rubn via Digitalmars-d wrote:That's correct. Buildkite is a lot easier to work with then Jenkins. Here's the overview page for the dlang pipelines: https://buildkite.com/dlangOn Saturday, 8 December 2018 at 02:08:14 UTC, Jonathan M Davis wrote:I think that they moved those tests to buildkite, but I don't know what the exact situation with that is right now.On Friday, December 7, 2018 3:15:30 PM MST Rubn viahttps://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fdmd/detail/PR-8 519/3/pipeline I guess that's just another dead project, so I guess no platform is getting those tests anymore. Too bad, I guess that's another story for D, unless they moved to something else.This is part of the problem, in that there are more tests run on Linux than Windows. DMD doesn't even compile it self on Windows to test. It does for linux though. Also some of the community projects are all compiled and tested on linux but not windows. That's the state it is in, the only tests that are run are the most basic ones. Can't expect DMD on Windows to be stable at all, if ever that's the way it goes though I guess. No one seems to care or thinks running tests on linux only is good enough for windows.The entire test suite for druntime, Phobos, and dmd has to pass on all supported platforms on the auto-tester: https://auto-tester.puremagic.com/ Windows is not exempt from that. So, unless there are a bunch of tests in the test suite which are needlessly platform-specific, Windows is getting tested just as much as Linux is. Where do you get the idea that it's not? - Jonathan M DavisWhile the main auto-tester has been the same for ages, they keep changing what they're doing with the other tests.Yeah, we use too many resources for any of the free CIs to handle it. Though it looks like our current move to Buildkite was a good one and we can unify a few services under it - though as long as no one actually has Windows or OSX machines, that will be hard.But I believe that the tests for each project are run using travis and depend on how each project set that up, so which which platforms they test is going to vary from project to project. And I don't think that that includes Windows at all, because that would require something like appveyor.We use Appveyor for DMD and druntime, though we keep running into the limit there too.So, the testing of projects beyond dmd and the standard library is only on *nix systems AFAIK, but dmd itself and the standard library are definitely tested on all of the platforms that dmd supports.Yup, as mentioned currently we just don't have any Windows nor OSX machines to hook up with Buildkite.This bootstrapping is tested (though not on the Auto-Tester). That's e.g. what we do with SemaphoreCI.What you linked are just those basic tests, those tests don't do anything complicated like building DMD itself with the newly created DMD and running the tests for both to ensure a correct binary is produced.The new compiler is built and used to run the druntime and Phobos test suites. Then it's used with the newly built druntime and Phobos to run the dmd test suite. So, the new dmd binary is definitely being tested, but no, the newly built dmd isn't then used to build dmd yet again as part of those tests.
Dec 19 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows?The problem is that DMD -m32 always works, else you need a _correct_ install of VS to build the other archs and it's not always _that_ easy to have when you did things in an order that wasn't exactly the one of the installer. please, no.
Dec 07 2018
On Friday, 7 December 2018 at 13:11:37 UTC, Guillaume Piolat wrote:On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Visual studio is no longer needed as lld and x86mscoff/x86_64 libraries now included in dmd and LDC:) We just need to get it mature. Kind regards AndreHi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows?The problem is that DMD -m32 always works, else you need a _correct_ install of VS to build the other archs and it's not always _that_ easy to have when you did things in an order that wasn't exactly the one of the installer. please, no.
Dec 07 2018
On Friday, 7 December 2018 at 13:23:54 UTC, Andre Pany wrote:Visual studio is no longer needed as lld and x86mscoff/x86_64 libraries now included in dmd and LDC:) We just need to get it mature. Kind regards AndreIs it any faster than OPTLINK?
Dec 08 2018
On Saturday, 8 December 2018 at 15:04:12 UTC, Guillaume Piolat wrote:On Friday, 7 December 2018 at 13:23:54 UTC, Andre Pany wrote:LLD is advertised as a very fast linker. Whether that is true I can't say, I didn't measure it. But there should be also other benefits: -it is actively developed by llvm community - you do not have to convert static libs from coff to omf (e.g. SQLite,...) Current situation is, you can't create documentation in format ddox because of an Optlink bug. This bug is known for a very long time. Kind regards AndreVisual studio is no longer needed as lld and x86mscoff/x86_64 libraries now included in dmd and LDC:) We just need to get it mature. Kind regards AndreIs it any faster than OPTLINK?
Dec 08 2018
On Saturday, 8 December 2018 at 15:04:12 UTC, Guillaume Piolat wrote:On Friday, 7 December 2018 at 13:23:54 UTC, Andre Pany wrote:I had some tries. LLD link is very fast in my applications. It feels even noticeable faster than OPTLINK. But also I facing some maturity issues. - While trying to run dub test in combination with dependency d-unit and a module command.d containing an std.datetime import there is always an linker error: Native PDB Error: The entry already exists. The specified module already exists I have a reduced test case (dustmite is great!) and will file an issue - From time to time there is a spurious linker error: Error: module `NOLOGO` is in file '\NOLOGO.d' which cannot be read I don't know how to reproduce this error but it disappears on next try. Kind regards AndréVisual studio is no longer needed as lld and x86mscoff/x86_64 libraries now included in dmd and LDC:) We just need to get it mature. Kind regards AndreIs it any faster than OPTLINK?
Dec 19 2018
On Wednesday, 19 December 2018 at 11:32:58 UTC, Andre Pany wrote:On Saturday, 8 December 2018 at 15:04:12 UTC, Guillaume Piolat wrote:As pointed out by Kinke in issue 19503, dmd ships with an old lld-link version (v5.0.1). Replacing it with v7.0 (already shipped with recent LDC) solves the Native PDB Error. (I have to update the issue https://issues.dlang.org/show_bug.cgi?id=19503) The spurious NOLOGO problem still occurs on every 3 or 4 try for exactly the same dmd linker command (generated by dub). Kind regards AndréOn Friday, 7 December 2018 at 13:23:54 UTC, Andre Pany wrote:I had some tries. LLD link is very fast in my applications. It feels even noticeable faster than OPTLINK. But also I facing some maturity issues. - While trying to run dub test in combination with dependency d-unit and a module command.d containing an std.datetime import there is always an linker error: Native PDB Error: The entry already exists. The specified module already exists I have a reduced test case (dustmite is great!) and will file an issue - From time to time there is a spurious linker error: Error: module `NOLOGO` is in file '\NOLOGO.d' which cannot be read I don't know how to reproduce this error but it disappears on next try. Kind regards AndréVisual studio is no longer needed as lld and x86mscoff/x86_64 libraries now included in dmd and LDC:) We just need to get it mature. Kind regards AndreIs it any faster than OPTLINK?
Dec 20 2018
On Thursday, 20 December 2018 at 09:24:33 UTC, Andre Pany wrote:As pointed out by Kinke in issue 19503, dmd ships with an old lld-link version (v5.0.1). Replacing it with v7.0 (already shipped with recent LDC) solves the Native PDB Error. (I have to update the issue https://issues.dlang.org/show_bug.cgi?id=19503) The spurious NOLOGO problem still occurs on every 3 or 4 try for exactly the same dmd linker command (generated by dub). Kind regards AndréIt would be a cool thing to change the default if D_SIMD was defined for -m32mscoff. Was it a OPTLINK problem that D_SIMD isn't there for 32-bit Windows target?
Dec 25 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows?My biggest gripe is this confusing matrix of compiler flags: default -m32 -m32mscoff -m64 dmd OMF OMF COFF-32 COFF-64 ldc COFF-64 COFF-32 Error COFF-64 The same applies to dub's --arch flag. Currently, the default works out of the box and I don't want to change that. I also don't like the idea of changing semantics and break code / expectations of users that way. Maybe we can make the current flags for dub/the compilers undocumented and define new, consistent flags.
Dec 07 2018
On Friday, 7 December 2018 at 13:58:12 UTC, Dennis wrote:My biggest gripe is this confusing matrix of compiler flags: default -m32 -m32mscoff -m64 dmd OMF OMF COFF-32 COFF-64 ldc COFF-64 COFF-32 Error COFF-64Some corrections for LDC: * The default depends on the host. If you're using a 64-bit LDC executable, it defaults to producing 64-bit code. * `-m32mscoff` is supported by LDMD, which is the DMD-compatible wrapper (and simply translates it to `-m32` for the LDC executable).
Dec 07 2018
On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:Hi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreTo me, it's obvious that the default on Windows 64-bit (i.e. "Windows" for 99.9% of the population) should be x86_64.
Dec 19 2018
On Wednesday, 19 December 2018 at 13:11:19 UTC, Atila Neves wrote:On Thursday, 6 December 2018 at 17:58:17 UTC, Andre Pany wrote:I propose a smooth transition in 3 steps: - 64 bit version of DMD should default to x86_64 instead of x86 (Optlink). I will check whether I am able to create the pr. - an additional Windows package should be created for each release containing the 64 bit dmd version. This package should be clearly marked as experimental while the current Windows package is marked as stable. - in some years, if 64 bit dmd and lld is proofed to be stable, either both packages can be combined into 1 or we can leave both packages but remove the experimental tag. Kind regards AndreHi, This is mostly interesting for windows developers. What is your opinion of switching to architecture x86_mscoff or x86_64 as default on windows? To be pretty clear, you still can compile to x86 OMF by setting the default architecture either in settings.json or as command line argument. The proposal is just to set another default. This is now possible as DMD and LDC comes with lld and the needed libraries. Kind regards AndreTo me, it's obvious that the default on Windows 64-bit (i.e. "Windows" for 99.9% of the population) should be x86_64.
Dec 19 2018