www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A very fast linker to replace gcc's ld

reply "Asman01" <jckj33 gmail.com> writes:
Today I've found a good alternative to slow gcc's linker: gold. 
Written by  Google in C++ it's a linker writen for large code 
bases with C++ in mind. The author claims it's about 5 times fast 
than gcc's one. It does support ELF only and UNIX-like symtem I 
think it's enough to dmd on linux or am I missing something? I 
believe it can reduce a lot dmd's compilation speed on these 
system. It's under same GPL as gcc's ld. I wonder if we can use 
it to make dmd still fastest. :)
It seems same case as Facebook needed a new C++ preprocessor for 
its large code base.

http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf
Apr 05 2014
next sibling parent reply "Joakim" <joakim airpost.net> writes:
On Saturday, 5 April 2014 at 21:47:33 UTC, Asman01 wrote:

 Today I've found a good alternative to slow gcc's linker: gold. 
 Written by  Google in C++ it's a linker writen for large code 
 bases with C++ in mind. The author claims it's about 5 times 
 fast than gcc's one. It does support ELF only and UNIX-like 
 symtem I think it's enough to dmd on linux or am I missing 
 something? I believe it can reduce a lot dmd's compilation 
 speed on these system. It's under same GPL as gcc's ld. I 
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf


Dmd already uses whatever the system linker is and on Arch that's 
gold.  The Android NDK also uses gold by default, though they 
also provide the original bfd ld and a newer llvm-based linker 
started by MediaTek, mclinker:

https://code.google.com/p/mclinker/
Apr 05 2014
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 on Arch that's gold.


it is? Not for me, unless I'm missing something.
Apr 05 2014
parent "Joakim" <dlang joakim.airpost.net> writes:
On Saturday, 5 April 2014 at 23:56:49 UTC, John Colvin wrote:

 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 on Arch that's gold.


 it is? Not for me, unless I'm missing something.


Sorry, I made a mistake.  Dmd simply invokes the system C 
compiler and the linker that it uses.  I set CC to clang and 
build dmd trunk with it.  Clang uses gold, which is what led me 
to assume it was the default.  But I just checked and gcc, the 
default C compiler on Arch, uses bfd ld, so it depends on what 
you use as your C compiler.

On Sunday, 6 April 2014 at 00:15:21 UTC, Asman01 wrote:

 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, 
 though they also provide the original bfd ld and a newer 
 llvm-based linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/


 Well, as far I know, it does invoke gcc and gcc does use the 
 ld. I see this with -v flag. What am I missing? I'll check out 
 this llvm-based linker. If it's the linker which I've hear they 
 are planing to replace gcc ld it's a big step.


See above, I was wrong.

As for llvm-based linkers, there are a couple going around.  The 
one you're probably thinking of is lld:

http://lld.llvm.org/
Apr 06 2014
prev sibling next sibling parent "Asman01" <jckj33 gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 On Saturday, 5 April 2014 at 21:47:33 UTC, Asman01 wrote:

 Today I've found a good alternative to slow gcc's linker: 
 gold. Written by  Google in C++ it's a linker writen for large 
 code bases with C++ in mind. The author claims it's about 5 
 times fast than gcc's one. It does support ELF only and 
 UNIX-like symtem I think it's enough to dmd on linux or am I 
 missing something? I believe it can reduce a lot dmd's 
 compilation speed on these system. It's under same GPL as 
 gcc's ld. I wonder if we can use it to make dmd still fastest. 
 :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf


 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, though 
 they also provide the original bfd ld and a newer llvm-based 
 linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/


Well, as far I know, it does invoke gcc and gcc does use the ld. 
I see this with -v flag. What am I missing? I'll check out this 
llvm-based linker. If it's the linker which I've hear they are 
planing to replace gcc ld it's a big step.
Apr 05 2014
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, though 
 they also provide the original bfd ld and a newer llvm-based 
 linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/


DMD emit object file that, depending on the version of Gold:
  - Will produce broken binaries.
  - Will fail with cryptic internal error;

I've tried to patch Gold, to accept dmd's file, but eventually 
ran out of time. If one want to take over that work I can point 
some places in Gold's source code to look at. I'd also not be 
that surprised that DMD does emit somthing weird in its object 
files.
Apr 05 2014
parent "Asman01" <jckj33 gmail.com> writes:
On Sunday, 6 April 2014 at 00:26:12 UTC, deadalnix wrote:

 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:

 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, 
 though they also provide the original bfd ld and a newer 
 llvm-based linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/


 DMD emit object file that, depending on the version of Gold:
  - Will produce broken binaries.
  - Will fail with cryptic internal error;


Why that?


 I've tried to patch Gold, to accept dmd's file, but eventually 
 ran out of time. If one want to take over that work I can point 
 some places in Gold's source code to look at. I'd also not be 
 that surprised that DMD does emit somthing weird in its object 
 files.


By weird do you mean gcc-specific?
Apr 05 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/14, 2:47 PM, Asman01 wrote:

 Today I've found a good alternative to slow gcc's linker: gold. Written
 by  Google in C++ it's a linker writen for large code bases with C++ in
 mind. The author claims it's about 5 times fast than gcc's one. It does
 support ELF only and UNIX-like symtem I think it's enough to dmd on
 linux or am I missing something? I believe it can reduce a lot dmd's
 compilation speed on these system. It's under same GPL as gcc's ld. I
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor for its
 large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html

 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf


s/reduce/improve/

Yah, we've been using gold for a long time. Not on all projects though, 
on some it crashes.


Andrei
Apr 05 2014
parent "asman" <lol.themask gmail.com> writes:
On Sunday, 6 April 2014 at 00:50:43 UTC, Andrei Alexandrescu 
wrote:

 On 4/5/14, 2:47 PM, Asman01 wrote:

 Today I've found a good alternative to slow gcc's linker: 
 gold. Written
 by  Google in C++ it's a linker writen for large code bases 
 with C++ in
 mind. The author claims it's about 5 times fast than gcc's 
 one. It does
 support ELF only and UNIX-like symtem I think it's enough to 
 dmd on
 linux or am I missing something? I believe it can reduce a lot 
 dmd's
 compilation speed on these system. It's under same GPL as 
 gcc's ld. I
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its
 large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html

 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf


 s/reduce/improve/

 Yah, we've been using gold for a long time. Not on all projects 
 though, on some it crashes.


 Andrei


haha. It's great already being using it. So, it's not a full 
replacement to gcc's ld because it is not stable enough, right? 
if so, do you think it's valuable (for DMD community) to someone 
put some effort to try to fix as possible this crashes? I will do 
some test on my linux machine later to see how fast and stable it 
is.
Apr 07 2014