digitalmars.D.ldc - Windows --> arm-linux cross compiler and lld
- Mike Franklin (11/11) Apr 08 2018 I'm exploring the idea of creating an LDC Windows (host) to
- Mike Franklin (8/14) Apr 08 2018 Well, actually, I suppose libc, libm, libpthread, and whatever
- Jacob Carlborg (7/14) Apr 08 2018 You can link with Clang instead of GCC by setting the `CC`
- Jacob Carlborg (6/10) Apr 09 2018 You can also have a look at ELLCC [1], a cross-compiler built on
- Kagamin (3/10) Apr 11 2018 You can write a small program (in D) named "gcc", that will
- Jacob Carlborg (11/22) Apr 08 2018 You can specify `-link-internally`, which at least works when not
- kinke (16/23) Apr 09 2018 For non-Windows targets, we expect a gcc-compatible linker
- Mike Franklin (33/43) Apr 10 2018 Thanks Jacob and kinke for the advice.
- Mike Franklin (5/7) Apr 10 2018 Oops! forgot the most important part; the triple. It should be
- Joakim (11/13) Apr 12 2018 I don't think it does, it's probably hard-coded based on the
- Joakim (5/20) Apr 16 2018 Btw, you may be interested in this thread about using the llvm
- Mike Franklin (62/66) Apr 17 2018 Thanks, it was an interesting read.
- Jacob Carlborg (5/9) Apr 18 2018 I agree. That's one case where Go shines. It makes cross-compiling so
- kinke (22/29) Apr 18 2018 I don't see a huge problem for the compiler (except for quite a
- kinke (10/16) Apr 18 2018 It'd probably be quite cool to merge all prebuilt packages into a
- Jacob Carlborg (22/35) Apr 19 2018 Originally Tango, which druntime is based, only used syscalls and (as
I'm exploring the idea of creating an LDC Windows (host) to arm-linux-gnueabihf (target) cross-compiler. However, it appears LDC still expects GCC to be present for linking, even on Windows. Does LDC still have a hard dependency on GCC for linking? I suppose even without GCC, itself, LDC would still need the libraries that GCC pulls in automatically (libc, etc.). Is there any plan to remove the dependency on the GNU toolchain altogether sometime in the future? Does LLVM have its own implementations of libc or whatever else LDC requires from the GNU toolchain? Thanks, Mike
Apr 08 2018
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:I suppose even without GCC, itself, LDC would still need the libraries that GCC pulls in automatically (libc, etc.). Is there any plan to remove the dependency on the GNU toolchain altogether sometime in the future? Does LLVM have its own implementations of libc or whatever else LDC requires from the GNU toolchain?Well, actually, I suppose libc, libm, libpthread, and whatever else is needed to create a binary is actually provided by the target, and not by the GNU toolchain. I suppose I could copy the necessary .a/.o/.so files to the Windows host, but is there some way to have ldc invoke lld.exe on the host, or will I have to resort to -c (separate compile and link) to avoid having to provision a windows GCC toolchain?
Apr 08 2018
On Monday, 9 April 2018 at 01:45:38 UTC, Mike Franklin wrote:Well, actually, I suppose libc, libm, libpthread, and whatever else is needed to create a binary is actually provided by the target, and not by the GNU toolchain. I suppose I could copy the necessary .a/.o/.so files to the Windows host, but is there some way to have ldc invoke lld.exe on the host, or will I have to resort to -c (separate compile and link) to avoid having to provision a windows GCC toolchain?You can link with Clang instead of GCC by setting the `CC` environment variable. Clang is already a cross-compiler. Using the `-isysroot` you can specify the root directory where it tries to find libraries, includes and binaries. -- /Jacob Carlborg
Apr 08 2018
On Monday, 9 April 2018 at 06:59:43 UTC, Jacob Carlborg wrote:You can link with Clang instead of GCC by setting the `CC` environment variable. Clang is already a cross-compiler. Using the `-isysroot` you can specify the root directory where it tries to find libraries, includes and binaries.You can also have a look at ELLCC [1], a cross-compiler built on Clang. [1] http://ellcc.org -- /Jacob Carlborg
Apr 09 2018
On Monday, 9 April 2018 at 01:45:38 UTC, Mike Franklin wrote:Well, actually, I suppose libc, libm, libpthread, and whatever else is needed to create a binary is actually provided by the target, and not by the GNU toolchain. I suppose I could copy the necessary .a/.o/.so files to the Windows host, but is there some way to have ldc invoke lld.exe on the host, or will I have to resort to -c (separate compile and link) to avoid having to provision a windows GCC toolchain?You can write a small program (in D) named "gcc", that will forward arguments to lld.
Apr 11 2018
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:I'm exploring the idea of creating an LDC Windows (host) to arm-linux-gnueabihf (target) cross-compiler. However, it appears LDC still expects GCC to be present for linking, even on Windows.You can specify `-link-internally`, which at least works when not cross-compiling on Windows. There's also the `-linker` flag which allows you to specify which linker to use, but it still invokes GCC. You can use the `CC` environment variable to override the C compiler that is invoked for linking.Does LDC still have a hard dependency on GCC for linking? I suppose even without GCC, itself, LDC would still need the libraries that GCC pulls in automatically (libc, etc.). Is there any plan to remove the dependency on the GNU toolchain altogether sometime in the future? Does LLVM have its own implementations of libc or whatever else LDC requires from the GNU toolchain?LLVM contains replacements for most of the GNU toolchain, although it does not have its own implementation of libc. musl can be used as a replacement but only on Linux. -- /Jacob Carlborg
Apr 08 2018
On Monday, 9 April 2018 at 01:04:51 UTC, Mike Franklin wrote:I'm exploring the idea of creating an LDC Windows (host) to arm-linux-gnueabihf (target) cross-compiler. However, it appears LDC still expects GCC to be present for linking, even on Windows. Does LDC still have a hard dependency on GCC for linking?For non-Windows targets, we expect a gcc-compatible linker driver, i.e., gcc or clang, and as Jacob pointed out, you can choose between them via CC env variable (and a hidden -gcc option IIRC). A while ago, I integrated LLD into LDC itself. It's currently only able to link MSVC binaries, so `-link-internally` only works for MSVC targets. There's a PR in which I extended it to ELF and Mach-O targets too; it seems to be working after a quick test (if you have the required libs obviously), but the needed linker flags normally added by gcc makes the command-line explode: https://github.com/ldc-developers/ldc/pull/2203#issuecomment-339167131 The command-line flags could be added once to the config file though if your environment is stable.I suppose I could copy the necessary .a/.o/.so files to the Windows hostYep, that's what I'd do.
Apr 09 2018
Thanks Jacob and kinke for the advice. I've been able to piece together a Frankenstein toolchain using LLVM 6, ldc2-1.8.0-windows-x64, and ldc2-1.8.0-linux-armhf I compile using ldc2-1.8.0-windows-x64: ldc2 -c main.d I create a samba share on my ARM device use it as sysroot from the Windows host at \\192.168.0.26\root. I pull the `lib` folder out of ldc2-1.8.0-linux-armhf for the following linking task. I link with something like this (pieced together from my notes; may not be perfect): ld.lld.exe -static -m armelf_linux_eabi -o main -sysroot \\192.168.0.26\root \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crt1.o \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crti.o \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtbegin.o -L \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf -L \\192.168.0.26\root\usr\lib\gcc\arm-linux-gnueabihf\4.9.2 main.o \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtend.o \\192.168.0.26\root\usr\lib\arm-linux-gnueabihf\crtn.o -L C:\ldc2-1.8.0-linux-armhf\lib -lphobos2-ldc -ldruntime-ldc -lrt -ldl -lpthread -lm -lc -lgcc -lgcc_eh It works. I couldn't get linking with clang or separate dynamic linking to work; my binary ends up throwing a segmentation fault, but I suspect I could probably troubleshoot and work out what I need to do eventually.A while ago, I integrated LLD into LDC itself. It's currently only able to link MSVC binaries, so `-link-internally` only works for MSVC targets. There's a PR in which I extended it to ELF and Mach-O targets too; it seems to be working after a quick test (if you have the required libs obviously), but the needed linker flags normally added by gcc makes the command-line explode: https://github.com/ldc-developers/ldc/pull/2203#issuecomment-339167131 The command-line flags could be added once to the config file though if your environment is stable.That looks really nice. I see you had to patch lld; is that going upstream? How does clang know how to generate its linker command? Whatever it is, can ldc be made do the same? Mike
Apr 10 2018
On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin wrote:I compile using ldc2-1.8.0-windows-x64: ldc2 -c main.dOops! forgot the most important part; the triple. It should be something like this: ldc2 -triple=arm-linux-gnueabihf -c main.d Mike
Apr 10 2018
On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin wrote:How does clang know how to generate its linker command?I don't think it does, it's probably hard-coded based on the target, though you can usually override various elements with flags. Since every target has its own selection of flags and object files it needs, we delegate linking to the C (cross-)compiler for that target. You can always choose your own linker for ldc with the -linker= flag and do everything manually if you want though.Whatever it is, can ldc be made do the same?Yes, but it would be an impossible task to do it for any non-trivial set of platforms. It's _much_ easier to free-ride off the C compiler which has already been configured for the target.
Apr 12 2018
On Friday, 13 April 2018 at 03:56:24 UTC, Joakim wrote:On Wednesday, 11 April 2018 at 00:39:23 UTC, Mike Franklin wrote:Btw, you may be interested in this thread about using the llvm linker, lld, which delves into a lot of the issues involved with not delegating to the C compiler: https://github.com/ldc-developers/ldc/issues/2028How does clang know how to generate its linker command?I don't think it does, it's probably hard-coded based on the target, though you can usually override various elements with flags. Since every target has its own selection of flags and object files it needs, we delegate linking to the C (cross-)compiler for that target. You can always choose your own linker for ldc with the -linker= flag and do everything manually if you want though.Whatever it is, can ldc be made do the same?Yes, but it would be an impossible task to do it for any non-trivial set of platforms. It's _much_ easier to free-ride off the C compiler which has already been configured for the target.
Apr 16 2018
On Tuesday, 17 April 2018 at 06:53:59 UTC, Joakim wrote:Btw, you may be interested in this thread about using the llvm linker, lld, which delves into a lot of the issues involved with not delegating to the C compiler: https://github.com/ldc-developers/ldc/issues/2028Thanks, it was an interesting read. IMO, however, all of this dependency and C (C runtime, C standard library, C compiler, etc...) is all superficial. With a little more due diligence in the compiler and runtime implementations, none of it would be necessary. I illustrate with the following example. It is a gross oversimplification but demonstrates that D doesn't need C...any of it. ---object.d module object; alias immutable(char)[] string; private long __d_sys_write(long arg1, in void* arg2, long arg3) { long result; asm { mov RAX, 1; mov RDI, arg1; mov RSI, arg2; mov RDX, arg3; syscall; } return result; } void write(string text) { __d_sys_write(2, text.ptr, text.length); } private void __d_sys_exit(long arg1) { asm { mov RAX, 60; mov RDI, arg1; syscall; } } extern void main(); private extern(C) void _start() { main(); __d_sys_exit(0); } ---main.d module main; void main() { write("Hello, World\n"); } $dmd -c -lib -conf= object.d main.d -of=main.o $ld main.o -o main $size main text data bss dec hex filename 176 0 0 176 b0 main $main Hello, World I admit that without leveraging C (actually gcc, even llvm still needs the crt files from gcc) D would be assuming more work, but I think doing away with the C compiler and going directly to the linker is a step in the right direction. Mike
Apr 17 2018
On 2018-04-17 10:55, Mike Franklin wrote:IMO, however, all of this dependency and C (C runtime, C standard library, C compiler, etc...) is all superficial. With a little more due diligence in the compiler and runtime implementations, none of it would be necessary.I agree. That's one case where Go shines. It makes cross-compiling so much simpler. -- /Jacob Carlborg
Apr 18 2018
On Wednesday, 18 April 2018 at 19:08:50 UTC, Jacob Carlborg wrote:On 2018-04-17 10:55, Mike Franklin wrote:I don't see a huge problem for the compiler (except for quite a bunch of LLVM 'intrinsics' boiling down to a C function call, so LLVM expects some core C libs to be linked in as well - so these C symbols would need to be provided by a D lib). Basing druntime on the C runtime layer, abstracting away many OS differences, makes things considerably simpler though; with the current manpower, I see no way to get rid of it and re-implement everything in D, while still supporting all platforms (and new platforms to come with as little hassle as possible). OTOH, libunwind (or for MSVC, the Visual C++ EH personality function used by LDC), libpthread, the math functions (which I'd really like to see all in pure D), I/O, memory management... Considering that we'll be stuck with core.stdc anyway, getting rid of the C dependencies won't be feasible for a very long time IMO. A cross-compiler with integrated cross-linking ability (and a tool to cross-compile druntime and Phobos), shipping with ready-made C libs and a preconfigured config file in a Docker image, working just out of the box should be a great experience for the user (and the Docker image generation easily automatable for us devs). I've never used Go, so how can that experience be further improved?IMO, however, all of this dependency and C (C runtime, C standard library, C compiler, etc...) is all superficial. With a little more due diligence in the compiler and runtime implementations, none of it would be necessary.I agree. That's one case where Go shines. It makes cross-compiling so much simpler.
Apr 18 2018
On Wednesday, 18 April 2018 at 21:07:01 UTC, kinke wrote:A cross-compiler with integrated cross-linking ability (and a tool to cross-compile druntime and Phobos), shipping with ready-made C libs and a preconfigured config file in a Docker image, working just out of the box should be a great experience for the user (and the Docker image generation easily automatable for us devs).It'd probably be quite cool to merge all prebuilt packages into a single universal Docker image (augmented by each platform's C libs & object files and the ldc2.conf config file extended by a section for each target triple). That would currently allow to freely cross-compile to 32/64-bit Windows/Linux/macOS and 32-bit ARM, simply depending on the used `-mtriple=...`. Legal aspects would need to be clarified first though; just saying that something like this wouldn't actually be that hard to do & automate.
Apr 18 2018
On 2018-04-18 23:07, kinke wrote:I don't see a huge problem for the compiler (except for quite a bunch of LLVM 'intrinsics' boiling down to a C function call, so LLVM expects some core C libs to be linked in as well - so these C symbols would need to be provided by a D lib). Basing druntime on the C runtime layer, abstracting away many OS differences, makes things considerably simpler though; with the current manpower, I see no way to get rid of it and re-implement everything in D, while still supporting all platforms (and new platforms to come with as little hassle as possible). OTOH, libunwind (or for MSVC, the Visual C++ EH personality function used by LDC), libpthread, the math functions (which I'd really like to see all in pure D), I/O, memory management...Originally Tango, which druntime is based, only used syscalls and (as far as I know) a C main function and everything necessary for that. But druntime has evolved quite a lot since then with new features and more platforms.I've never used Go, so how can that experience be further improved?The standard Go compiler is a cross-compiler. They use a custom linker which is a cross-linker as well. In Go you cannot directly link to a C function in the same way as you can in D. Go also requires (unless you want to mess with the whole reason to use Go) that every call is non-blocking. That means they need to reimplement most things and cannot rely on the C standard library. Therefore they don't and instead only using syscalls as the most basic layer. Cross-compiling using Go is as simple as: GOOS=linux GOARCH=arm go build main.go Since the compiler and linker are cross-tools and they're not relaying on any platform SDK it just works if you have the standard Go package. No issues with trying to find a cross-linker, no issues of the SDK cannot be redistributed for macOS or Windows. It will even generate statically linked binaries on macOS which the C standard library doesn't even support. -- /Jacob Carlborg
Apr 19 2018