www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Faster linker

reply bennygui <bennygui nowhere.com> writes:
Hi,

I am compiling a small vibe.d project with dmd via dub. The 
linker takes most of the time when invoking "dub build" (total: 
11 seconds). I've tried compiling with gdc and it's the same (11 
seconds). With ldc, it's faster (6 seconds).

I did not time anything except "dub build" but I get the feeling 
that dmd compiles much faster but is slowed down by the linker 
and that ldc takes longer to compile but has a faster linker.

Is there any way to get faster link time? I'm on Linux (Gentoo).

Thanks!
Nov 08 2018
next sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Fri, 09 Nov 2018 01:04:51 +0000, bennygui wrote:

 Is there any way to get faster link time? I'm on Linux (Gentoo).


Use dynamic libraries.

Dub doesn't currently support them. Fortunately, the main project I've had 
to do this with is gtkd, which has no dependencies. The hack I used was:

* use git submodules instead of dub for the dependency
* update the dub script for gtkd to build dynamic libraries
* tell dub to use gtkd's library artifacts manually
* tell dub about the gtkd import paths manually
Nov 08 2018
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Nov 09, 2018 at 01:04:51AM +0000, bennygui via Digitalmars-d wrote:

 Hi,
 
 I am compiling a small vibe.d project with dmd via dub. The linker
 takes most of the time when invoking "dub build" (total: 11 seconds).
 I've tried compiling with gdc and it's the same (11 seconds). With
 ldc, it's faster (6 seconds).
 
 I did not time anything except "dub build" but I get the feeling that
 dmd compiles much faster but is slowed down by the linker and that ldc
 takes longer to compile but has a faster linker.
 
 Is there any way to get faster link time? I'm on Linux (Gentoo).


[...]

Are you sure it's the linker, and not the compiler, or dub itself?

If it's the linker, then you could look into using gold instead of the
BFD linker.

I suspect most of your build time is spent in dub, though.  You could
try various options to turn off network access and dependency resolution
when running dub, which should improve total running time. (Of course,
you need to run it at least once without those options so that it pulls
in required dependencies, but once those are downloaded, there's no need
to check dependencies every single time.)  You could also ditch dub and
use your own build system, by creating an empty dummy project in a
subdirectory and running dub once to pull in your dependencies, then
link to the artifacts yourself directly. (I did that in one of my own
projects, and it sped up build time by an order of magnitude.)

Another possible cause is that Diet templates are very template- and
CTFE-heavy, and will often soak up gobs of RAM and CPU every time they
are compiled.  A possible solution is to separate out the Diet templates
into their own source files, and use separate compilation to avoid
rebuilding those files unless there was an actual change in the Diet
code. (I've also run into this in my vibe.d project. I'm considering to
replace Diet templates with something lighter-weight, but haven't taken
that step yet.)


T

-- 
It is the quality rather than the quantity that matters. -- Lucius Annaeus
Seneca
Nov 08 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/8/18 9:20 PM, H. S. Teoh wrote:

 On Fri, Nov 09, 2018 at 01:04:51AM +0000, bennygui via Digitalmars-d wrote:

 Hi,

 I am compiling a small vibe.d project with dmd via dub. The linker
 takes most of the time when invoking "dub build" (total: 11 seconds).
 I've tried compiling with gdc and it's the same (11 seconds). With
 ldc, it's faster (6 seconds).

 I did not time anything except "dub build" but I get the feeling that
 dmd compiles much faster but is slowed down by the linker and that ldc
 takes longer to compile but has a faster linker.

 Is there any way to get faster link time? I'm on Linux (Gentoo).


 [...]
 
 Are you sure it's the linker, and not the compiler, or dub itself?


Yeah, a vibe.d project with diet templates takes most of its time in the 
linker, I've found. Basically a big hang after dub prints:

Linking...


 If it's the linker, then you could look into using gold instead of the
 BFD linker.
 
 I suspect most of your build time is spent in dub, though.  You could
 try various options to turn off network access and dependency resolution
 when running dub, which should improve total running time. (Of course,
 you need to run it at least once without those options so that it pulls
 in required dependencies, but once those are downloaded, there's no need
 to check dependencies every single time.)  You could also ditch dub and
 use your own build system, by creating an empty dummy project in a
 subdirectory and running dub once to pull in your dependencies, then
 link to the artifacts yourself directly. (I did that in one of my own
 projects, and it sped up build time by an order of magnitude.)
 
 Another possible cause is that Diet templates are very template- and
 CTFE-heavy, and will often soak up gobs of RAM and CPU every time they
 are compiled.  A possible solution is to separate out the Diet templates
 into their own source files, and use separate compilation to avoid
 rebuilding those files unless there was an actual change in the Diet
 code. (I've also run into this in my vibe.d project. I'm considering to
 replace Diet templates with something lighter-weight, but haven't taken
 that step yet.)


I suspect the issue is not exactly the linker, but the symbol name sizes 
again (probably related to the diet templates). I haven't looked into it 
enough to see where the problem is coming from, but one of either vibe.d 
using naming tricks to limit the name size, or D doing something clever 
with string template parameters should help.

-Steve
Nov 08 2018
parent reply bennygui <bennygui nowhere.com> writes:
On Friday, 9 November 2018 at 02:33:42 UTC, Steven Schveighoffer 
wrote:

 Yeah, a vibe.d project with diet templates takes most of its 
 time in the linker, I've found. Basically a big hang after dub 
 prints:

 Linking...


Yep, that's it.


 I suspect the issue is not exactly the linker, but the symbol 
 name sizes again (probably related to the diet templates). I 
 haven't looked into it enough to see where the problem is 
 coming from, but one of either vibe.d using naming tricks to 
 limit the name size, or D doing something clever with string 
 template parameters should help.


I've tried commenting out all "render" call so I have no diet 
templates and I save 1 second of total time (10.5 to 9.6) and 
it's still the linker that takes all the time.

I looked at the gold linker and it's already a possible 
configuration with Gentoo Linux! I've switched and it now 
compiles and link in 4 seconds! Yah!

But I don't want to switch my whole system to use gold, is there 
a way to configure dub or dmd to use usr/bin/ld.gold instead of 
/usr/bin/ld ?

Thanks!
Nov 08 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Nov 09, 2018 at 02:47:38AM +0000, bennygui via Digitalmars-d wrote:
[...]

 I looked at the gold linker and it's already a possible configuration
 with Gentoo Linux! I've switched and it now compiles and link in 4
 seconds! Yah!
 
 But I don't want to switch my whole system to use gold, is there a way
 to configure dub or dmd to use usr/bin/ld.gold instead of /usr/bin/ld
 ?


[...]

You just need to pass the `-L-fuse-ld=gold` flag to your dmd invocation,
or `-fuse-ld=gold` for LDC.  Not 100% sure how to configure this in dub,
though.


T

-- 
Computers aren't intelligent; they only think they are.
Nov 08 2018
parent reply kinke <kinke libero.it> writes:
On Friday, 9 November 2018 at 03:41:18 UTC, H. S. Teoh wrote:

 You just need to pass the `-L-fuse-ld=gold` flag to your dmd 
 invocation, or `-fuse-ld=gold` for LDC.


As Joakim said, the linker switch for LDC is `-linker=gold`, not 
`-fuse-ld` (that's the gcc/clang switch). If you guys read the 
release notes, you'd have seen that LDC v1.13 newly defaults to 
the gold linker on Linux (due to various issues with older bfd 
versions), which is apparently the sole reason the OP saw much 
better performance with LDC.
Nov 09 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Nov 09, 2018 at 12:49:07PM +0000, kinke via Digitalmars-d wrote:

 On Friday, 9 November 2018 at 03:41:18 UTC, H. S. Teoh wrote:

 You just need to pass the `-L-fuse-ld=gold` flag to your dmd
 invocation, or `-fuse-ld=gold` for LDC.


 
 As Joakim said, the linker switch for LDC is `-linker=gold`, not
 `-fuse-ld` (that's the gcc/clang switch). If you guys read the release
 notes, you'd have seen that LDC v1.13 newly defaults to the gold
 linker on Linux (due to various issues with older bfd versions), which
 is apparently the sole reason the OP saw much better performance with
 LDC.


Ah, I guess I got confused because when cross-compiling for Android,
I'm actually using the NDK clang backend for linking, even though the D
part is compiled by LDC.  Apologies.


T

-- 
I'm still trying to find a pun for "punishment"...
Nov 09 2018
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 On Friday, 9 November 2018 at 02:33:42 UTC, Steven 
 Schveighoffer wrote:

 [...]


 Yep, that's it.


[...]


 I've tried commenting out all "render" call so I have no diet 
 templates and I save 1 second of total time (10.5 to 9.6) and 
 it's still the linker that takes all the time.

 I looked at the gold linker and it's already a possible 
 configuration with Gentoo Linux! I've switched and it now 
 compiles and link in 4 seconds! Yah!

 But I don't want to switch my whole system to use gold, is 
 there a way to configure dub or dmd to use usr/bin/ld.gold 
 instead of /usr/bin/ld ?

 Thanks!


Yes, LDC has the -linker=gold flag, that you can use to pass a 
different linker to your C compiler. Try it with lld, it's the 
fastest of all the linkers.
Nov 08 2018
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 [...]


 Yes, LDC has the -linker=gold flag, that you can use to pass a 
 different linker to your C compiler. Try it with lld, it's the 
 fastest of all the linkers.


lld mostly produces crashing binaries for me on Linux.
Nov 09 2018
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 9 November 2018 at 12:19:43 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 [...]


 Yes, LDC has the -linker=gold flag, that you can use to pass a 
 different linker to your C compiler. Try it with lld, it's the 
 fastest of all the linkers.


 lld mostly produces crashing binaries for me on Linux.


I find that hard to believe, considering Android is switching to 
it:

"LLD is now available for testing. AOSP is in the process of 
switching to using LLD by default and the NDK will follow 
(timeline unknown). Test LLD in your app by passing -fuse-ld=lld 
when linking."
https://github.com/android-ndk/ndk/wiki/Changelog-r18

It may still have bugs though, maybe you can report your problems.

LDC comes with lld built in, available through the 
`-link-internally` flag that kinke added.
Nov 09 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 9 November 2018 at 13:28:42 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 12:19:43 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 [...]


 Yes, LDC has the -linker=gold flag, that you can use to pass 
 a different linker to your C compiler. Try it with lld, it's 
 the fastest of all the linkers.


 lld mostly produces crashing binaries for me on Linux.


 I find that hard to believe, considering Android is switching 
 to it:


To the extent of my knowledge, Android doesn't link D binaries. 
It may well work for C and C++.

I tried symlinking /usr/bin/ld to /usr/bin/ld.lld, and... it 
seems to work now. The last time I tried (6 to 12 months ago? I 
don't remember) it produced crashing binaries most of the time 
for D code.

I guess I should switch for good now then!


 "LLD is now available for testing. AOSP is in the process of 
 switching to using LLD by default and the NDK will follow 
 (timeline unknown). Test LLD in your app by passing 
 -fuse-ld=lld when linking."
 https://github.com/android-ndk/ndk/wiki/Changelog-r18

 It may still have bugs though, maybe you can report your 
 problems.

 LDC comes with lld built in, available through the 
 `-link-internally` flag that kinke added.


Only in the unreleased 1.13.0 version, and using it in the beta 
yields this:

% ldc2 -link-internally foo.d
lld: error: unknown argument: --no-warn-search-mismatch
lld: error: unable to find library -lrt
lld: error: unable to find library -ldl
lld: error: unable to find library -lpthread
lld: error: unable to find library -lm

Ok, let's look in /usr/lib explicitly then (which shouldn't be 
needed):

% ldc2 -link-internally -L-L/usr/lib foo.d
lld: error: unknown argument: --no-warn-search-mismatch

Not ready for production, then.
Nov 09 2018
next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 9 November 2018 at 15:08:19 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 13:28:42 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 12:19:43 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 [...]


 lld mostly produces crashing binaries for me on Linux.


 I find that hard to believe, considering Android is switching 
 to it:


 To the extent of my knowledge, Android doesn't link D binaries. 
 It may well work for C and C++.

 I tried symlinking /usr/bin/ld to /usr/bin/ld.lld, and... it 
 seems to work now. The last time I tried (6 to 12 months ago? I 
 don't remember) it produced crashing binaries most of the time 
 for D code.

 I guess I should switch for good now then!


I spoke too soon. The reggae unit tests crash if linked with lld. 
Every other project I tried worked though.
Nov 09 2018
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 9 November 2018 at 15:08:19 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 13:28:42 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 12:19:43 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 [...]


 Yes, LDC has the -linker=gold flag, that you can use to pass 
 a different linker to your C compiler. Try it with lld, it's 
 the fastest of all the linkers.


 lld mostly produces crashing binaries for me on Linux.


 I find that hard to believe, considering Android is switching 
 to it:


 To the extent of my knowledge, Android doesn't link D binaries. 
 It may well work for C and C++.


OK, your statement made no reference to D, so I thought you were 
talking generally.


 I tried symlinking /usr/bin/ld to /usr/bin/ld.lld, and... it 
 seems to work now. The last time I tried (6 to 12 months ago? I 
 don't remember) it produced crashing binaries most of the time 
 for D code.

 I guess I should switch for good now then!


Maybe you were using an old or buggy version of lld back then. 
I've just been trying lld for Android and I was stunned at how 
fast it ran, as I've been stuck with bfd for Android for some 
time now. After making some druntime tweaks so I can use any 
linker on Android, I got the exact same test results when running 
the dmd testsuite natively on Android/AArch64 with bfd, gold, and 
lld.


 "LLD is now available for testing. AOSP is in the process of 
 switching to using LLD by default and the NDK will follow 
 (timeline unknown). Test LLD in your app by passing 
 -fuse-ld=lld when linking."
 https://github.com/android-ndk/ndk/wiki/Changelog-r18

 It may still have bugs though, maybe you can report your 
 problems.

 LDC comes with lld built in, available through the 
 `-link-internally` flag that kinke added.


 Only in the unreleased 1.13.0 version, and using it in the beta


No, it was added with 1.3.0, more than a year ago:

https://github.com/ldc-developers/ldc/releases/tag/v1.3.0


 yields this:

 % ldc2 -link-internally foo.d
 lld: error: unknown argument: --no-warn-search-mismatch
 lld: error: unable to find library -lrt
 lld: error: unable to find library -ldl
 lld: error: unable to find library -lpthread
 lld: error: unable to find library -lm

 Ok, let's look in /usr/lib explicitly then (which shouldn't be 
 needed):

 % ldc2 -link-internally -L-L/usr/lib foo.d
 lld: error: unknown argument: --no-warn-search-mismatch

 Not ready for production, then.


No, the issue is that ldc pre-configuring it for every toolchain 
and distro would be a giant PITA:

https://github.com/ldc-developers/ldc/issues/2717

The way dmd and ldc work now is that they expect there to be a C 
compiler installed, and simply invoke the local C compiler to 
link everything properly, assuming rightly that it will pass all 
the right flags and libc object files to its linker.

`-link-internally` cannot possibly do that- we'd have to maintain 
a database of all possible toolchain configurations out there- so 
it depends on _you_ configuring it for your environment if you 
want to use it. Once you do, it's just a built-in lld, it should 
work just as lld does.

I plan on moving the native packages for the Termux app on 
Android to use -link-internally by default (they currently 
default to clang/bfd), since it should be straightforward to add 
the necessary linker flags and logic for that limited 
environment. Maybe we could do that for Mac and even Windows 
someday too, but I can't imagine we'll ever have an official ldc 
that works for every linux distro! ;)
Nov 09 2018
next sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 9 November 2018 at 15:30:24 UTC, Joakim wrote:

 I plan on moving the native packages for the Termux app on 
 Android to use -link-internally by default (they currently 
 default to clang/bfd), since it should be straightforward to 
 add the necessary linker flags and logic for that limited 
 environment. Maybe we could do that for Mac and even Windows 
 someday too, but I can't imagine we'll ever have an official 
 ldc that works for every linux distro! ;)


Can't Debian-based systems be covered? It may make sense to have 
a config section for internal linker arguments, so that if it 
doesn't work, cc can still be used.
Nov 10 2018
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 10 November 2018 at 13:38:09 UTC, Kagamin wrote:

 On Friday, 9 November 2018 at 15:30:24 UTC, Joakim wrote:

 I plan on moving the native packages for the Termux app on 
 Android to use -link-internally by default (they currently 
 default to clang/bfd), since it should be straightforward to 
 add the necessary linker flags and logic for that limited 
 environment. Maybe we could do that for Mac and even Windows 
 someday too, but I can't imagine we'll ever have an official 
 ldc that works for every linux distro! ;)


 Can't Debian-based systems be covered?


Why, because it's the most common distro? I've never really used 
Debian much- I've long preferred and used Arch- but I just looked 
up what would be needed on Ubuntu 18.04.

Building a simple sample D file with the latest ldc 1.3 beta 
shows the following linker command invoked by the system 
compiler, gcc:

/usr/lib/gcc/x86_64-linux-gnu/7/collect2 -plugin 
/usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so 
-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper 
-plugin-opt=-fresolution=/tmp/ccd77bzy.res 
-plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s 
-plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc 
-plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id 
--eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed 
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now 
-fuse-ld=gold -z relro -o sieve 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o 
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o 
-L/home/joakim/ldc2-1.13.0-beta1-linux-x86_64/bin/../lib 
-L/home/joakim/ldc2-1.13.0-beta1-linux-x86_64/bin/../lib32 
-L/usr/lib/gcc/x86_64-linux-gnu/7 
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu 
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib 
-L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu 
-L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. 
sieve.o --no-warn-search-mismatch -lphobos2-ldc -ldruntime-ldc 
--gc-sections -lrt -ldl -lpthread -lm -lgcc --push-state 
--as-needed -lgcc_s --pop-state -lc -lgcc --push-state 
--as-needed -lgcc_s --pop-state 
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o

Some of that isn't strictly necessary, but you will likely need 
those libc files that it automatically adds, which dpkg tells me 
come in the libc6-dev and libgcc-7-dev packages.

OK, but how do we know to look in those packages' paths without 
the pre-configured system gcc telling us? Try running `apt search 
libc`: you'll come up with a ton of alternative packages, with 
many other paths and libc files that could've been used.

I'm not saying it couldn't be done to some extent, but it seems a 
pain even on Ubuntu alone.


 It may make sense to have a config section for internal linker 
 arguments, so that if it doesn't work, cc can still be used.


If you mean a config file, I think you could use ldc's ldc2.conf 
for that. And yeah, you could have ldc switch back and forth 
between a config for its internal lld and the external C compiler.

This all depends on someone putting in the work to remove that 
current dependency on the C compiler on various platforms. I'll 
be looking into it for the ldc package on Termux for Android, 
you're free to do the same on any other platform.
Nov 10 2018
next sibling parent Kagamin <spam here.lot> writes:
On Saturday, 10 November 2018 at 15:15:30 UTC, Joakim wrote:

 Why, because it's the most common distro? I've never really 
 used Debian much- I've long preferred and used Arch


AFAIK, Arch updates its ldc package often.


 OK, but how do we know to look in those packages' paths without 
 the pre-configured system gcc telling us? Try running `apt 
 search libc`: you'll come up with a ton of alternative 
 packages, with many other paths and libc files that could've 
 been used.


Only default. (glibc?)
But yeah, maintaining them might be difficult for gcc updates.


 It may make sense to have a config section for internal linker 
 arguments, so that if it doesn't work, cc can still be used.


 If you mean a config file, I think you could use ldc's 
 ldc2.conf for that. And yeah, you could have ldc switch back 
 and forth between a config for its internal lld and the 
 external C compiler.

 This all depends on someone putting in the work to remove that 
 current dependency on the C compiler on various platforms. I'll 
 be looking into it for the ldc package on Termux for Android, 
 you're free to do the same on any other platform.


I mean, cc would be the default, and internal linker arguments 
would be used only if internal linker was requested.
Nov 10 2018
prev sibling parent reply Kagamin <spam here.lot> writes:
BTW what paths you get on Arch?
Nov 10 2018
parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 10 November 2018 at 16:03:05 UTC, Kagamin wrote:

 BTW what paths you get on Arch?


I wiped my Arch VPS by mistake last month, so can't tell you. :)
Nov 10 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 9 November 2018 at 15:30:24 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 15:08:19 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 13:28:42 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 12:19:43 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 [...]


 Yes, LDC has the -linker=gold flag, that you can use to 
 pass a different linker to your C compiler. Try it with 
 lld, it's the fastest of all the linkers.


 lld mostly produces crashing binaries for me on Linux.


 I find that hard to believe, considering Android is switching 
 to it:


 To the extent of my knowledge, Android doesn't link D 
 binaries. It may well work for C and C++.


 OK, your statement made no reference to D, so I thought you 
 were talking generally.


 I tried symlinking /usr/bin/ld to /usr/bin/ld.lld, and... it 
 seems to work now. The last time I tried (6 to 12 months ago? 
 I don't remember) it produced crashing binaries most of the 
 time for D code.

 I guess I should switch for good now then!


 Maybe you were using an old or buggy version of lld back then. 
 I've just been trying lld for Android and I was stunned at how 
 fast it ran, as I've been stuck with bfd for Android for some 
 time now. After making some druntime tweaks so I can use any 
 linker on Android, I got the exact same test results when 
 running the dmd testsuite natively on Android/AArch64 with bfd, 
 gold, and lld.


Older, yes. Old, no (I'm on Arch, nothing is old). I was also 
stunned by how fast it ran. Then I was more stunned by how crashy 
the resulting binaries were.


 Only in the unreleased 1.13.0 version, and using it in the beta


 No, it was added with 1.3.0, more than a year ago:

 https://github.com/ldc-developers/ldc/releases/tag/v1.3.0


Huh, I could've sworn I tried it with 1.12.0 and it didn't work, 
but I tried again and it recognises the flag. My bad.


 % ldc2 -link-internally -L-L/usr/lib foo.d
 lld: error: unknown argument: --no-warn-search-mismatch

 Not ready for production, then.


 No, the issue is that ldc pre-configuring it for every 
 toolchain and distro would be a giant PITA:

 https://github.com/ldc-developers/ldc/issues/2717


I sympathise with the difficulty, but as I user I don't care - it 
doesn't work.


 The way dmd and ldc work now is that they expect there to be a 
 C compiler installed, and simply invoke the local C compiler to 
 link everything properly, assuming rightly that it will pass 
 all the right flags and libc object files to its linker.

 `-link-internally` cannot possibly do that- we'd have to 
 maintain a database of all possible toolchain configurations 
 out there- so it depends on _you_ configuring it for your 
 environment if you want to use it. Once you do, it's just a 
 built-in lld, it should work just as lld does.


Or I could symlink /usr/bin/ld to /usr/bin/ld.lld and not have to 
worry about any of that. Except for letting me switch linker per 
project, `-link-internally` is useless to me.

There's also the `--no-warn-search-mismatch` problem.
Nov 11 2018
parent Joakim <dlang joakim.fea.st> writes:
On Sunday, 11 November 2018 at 13:05:22 UTC, Atila Neves wrote:

 On Friday, 9 November 2018 at 15:30:24 UTC, Joakim wrote:

 [...]


 Older, yes. Old, no (I'm on Arch, nothing is old). I was also 
 stunned by how fast it ran. Then I was more stunned by how 
 crashy the resulting binaries were.


 [...]


 Huh, I could've sworn I tried it with 1.12.0 and it didn't 
 work, but I tried again and it recognises the flag. My bad.


 [...]


 I sympathise with the difficulty, but as I user I don't care - 
 it doesn't work.


 [...]


 Or I could symlink /usr/bin/ld to /usr/bin/ld.lld and not have 
 to worry about any of that. Except for letting me switch linker 
 per project, `-link-internally` is useless to me.


Then we come full circle and you can simply supply the 
-linker=lld flag to ldc whenever you compile D code, and switch 
up that flag if you have any problems.


 There's also the `--no-warn-search-mismatch` problem.


Yes, its mentioned in the ldc issue I linked you above, an 
incompatibility between lld and bfd/gold right now.
Nov 11 2018
prev sibling parent bennygui <bennygui nowhere.com> writes:
On Friday, 9 November 2018 at 03:47:09 UTC, Joakim wrote:

 On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 But I don't want to switch my whole system to use gold, is 
 there a way to configure dub or dmd to use usr/bin/ld.gold 
 instead of /usr/bin/ld ?




For the record, I did not find a way to configure dub. Adding

"dflags-dmd": ["-L-fuse-ld=gold"]

to dub.json did not work.

But what did work was modifying dmd.conf to add -L-fuse-ld=gold 
at the end of DFLAGS=... in the [Environment64] section.
Nov 09 2018
prev sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Friday, 9 November 2018 at 02:47:38 UTC, bennygui wrote:

 On Friday, 9 November 2018 at 02:33:42 UTC, Steven 
 Schveighoffer wrote:

 [...]


 Yep, that's it.


[...]


 I've tried commenting out all "render" call so I have no diet 
 templates and I save 1 second of total time (10.5 to 9.6) and 
 it's still the linker that takes all the time.

 I looked at the gold linker and it's already a possible 
 configuration with Gentoo Linux! I've switched and it now 
 compiles and link in 4 seconds! Yah!

 But I don't want to switch my whole system to use gold, is 
 there a way to configure dub or dmd to use usr/bin/ld.gold 
 instead of /usr/bin/ld ?

 Thanks!


For dmd there is the dmd.conf file which has the DFLAGS attribute.
I am not an expert in this area but you may could set here the
default linker.

Kind regards
André
Nov 09 2018