www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Cross compilers; GDC/LDC

reply Manu <turkeyman gmail.com> writes:
Hi people.

So I've been AWOL a while, and I'll still be intermittent for some weeks
yet, but I'm trying to work on std.simd and also my dconf talks in any gaps
I can.
However, in order to do a thorough job, I'm missing some tools. I wonder if
anyone can help me out...?

I need to do some rather strenuous testing of SIMD code gen in D, to make
sure I don't make mistakes and/or lie.

This requires testing against as many compilers+architectures as I can.
I have barely any internet access to speak of, I don't really know how, and
I also don't run linux, so I can't practically build myself a suite of
toolchains to work with >_<

Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple

I wonder if anyone with know-how building GDC or LDC for windows could help
me out by producing as many of these in binary as possible?

As a side note, I think it would be really valuable to keep a cache of such
windows binaries somewhere on the net anyway, so anyone working on foreign
platforms can get to work and actually focus on platform support, without
wasting all their time learning how to build these toolchains.

I think Daniel Green who usually builds the windows GDC toolchains actually
builds them from linux? (which seems weird to me)

Cheers people!
Mar 03 2013
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 3 Mar 2013 23:33:27 +1000
schrieb Manu <turkeyman gmail.com>:


 Hi people.
 
 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?
 
 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.
 
 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<
 
 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple
 
 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?
 
 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.
 
 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)
 
 Cheers people!
 


I could build the gdc toolchains, but I need some more information:

What OS should the toolchains target? Linux, windows or "bare-metal"?
(Especially the x86 toolchains)

Do you need libdruntime/libphobos or is a plain compiler enough? I'm
not sure if druntime compiles for SH4, MIPS and PPC.
Mar 03 2013
next sibling parent reply Manu <turkeyman gmail.com> writes:
x86 and x64 are already available. I've seen ARM builds kicking around too
which I've already done some work with. I'm not sure if they're up to date,
but it's probably fine for me to use.

Linux or bare metal + no library for the others will suit me now.
Whatever's easiest.

Google seem to have a build script for building their Android ARM and MIPS
toolchains:
http://developer.android.com/tools/sdk/ndk/index.html
I wonder if the GDC frontend can be dropped in to their scripts easily?

I guess the most important one in that case is PPC, since VMX is really
standard, it's a very common arch, and there's no existing toolchains for
it.

I remember talk of a build-server that automatically rebuilds binaries of
many toolchains each time a new release of D comes out some time back, did
that lead anywhere?

In the future, it'd be great to have a more complete set of offerings
linked from http://dlang.org/download.html


On 4 March 2013 03:20, Johannes Pfau <nospam example.com> wrote:


 Am Sun, 3 Mar 2013 23:33:27 +1000
 schrieb Manu <turkeyman gmail.com>:


 Hi people.

 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?

 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.

 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<

 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?

 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.

 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)

 Cheers people!


 I could build the gdc toolchains, but I need some more information:

 What OS should the toolchains target? Linux, windows or "bare-metal"?
 (Especially the x86 toolchains)

 Do you need libdruntime/libphobos or is a plain compiler enough? I'm
 not sure if druntime compiles for SH4, MIPS and PPC.
Mar 03 2013
parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 4 Mar 2013 10:36:45 +1000
schrieb Manu <turkeyman gmail.com>:


 x86 and x64 are already available. I've seen ARM builds kicking
 around too which I've already done some work with. I'm not sure if
 they're up to date, but it's probably fine for me to use.


I included a new ARM build. Unfortunately ARM druntime broke when
upgrading to 2.062. We already had working cross-compilers with phobos
for ARM (raspberry pi).


 
 Linux or bare metal + no library for the others will suit me now.
 Whatever's easiest.


OK, then linux


 
 Google seem to have a build script for building their Android ARM and
 MIPS toolchains:
 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts
 easily?


Unless you actually need android support building a crosstool-NG
toolchain is easier.


 I guess the most important one in that case is PPC, since VMX is
 really standard, it's a very common arch, and there's no existing
 toolchains for it.
 
 I remember talk of a build-server that automatically rebuilds
 binaries of many toolchains each time a new release of D comes out
 some time back, did that lead anywhere?
 
 In the future, it'd be great to have a more complete set of offerings
 linked from http://dlang.org/download.html


Once we have working crosstool-NG configurations updating the cross
compilers is trivial. I should set up a qemu image though so others can
easily build the packages as well. However I think a bare compiler
without druntime is only useful for very few people.


I've uploaded the toolchains now. I verified that they can compile a
simple program, but I don't have hardware to test if the result
actually works.

Also attached a simple hello world. I guess you know this already but
you have to compile with -nophoboslib and make sure that gdc can find
the object.di file. The included object.di was just copied from my
local installation. It's not really used but the frontend does some
sanity checks on object.di now so an empty file doesn't work anymore.
Also make sure _Dmodule_ref is defined as in the test.d example file.


Some information about the toolchains:
GCC 4.7.2
GDC b8f5c22b0e7afa7e68a287ed788597e783540063 (2.062 frontend)
Endianness: Whatever was the default in ct-NG

(All version identifiers from http://dlang.org/version.html should
work.)

It's sometimes possible to change the endianness without recompiling gcc
(e.g. for MIPS -EB and -EL flags.). I'm not sure if this works though
as the C libraries are compiled with default endianness.

All toolchains target 32bit processors. If you need 64bit targets just
tell me. Multilib is not enabled.

http://www.mediafire.com/?b8pgyvbotbgpd

And here's an old release for Raspbian with druntime & phobos:
http://www.mediafire.com/?00d05h1gox9c50h
Mar 04 2013
parent reply Manu <turkeyman gmail.com> writes:
Wowzer! That's awesome! :)

So, I don't understand this whole crosstools-NG thing, how did you do that
so easily?

So, aside from doing these tests I'm working on now, let's say I were
working on some games console projects, how hard is it to integrate the
console patches (like the ones linked in my prior email) into this build
system to make actual working compilers for those platforms?
This seems like an amazing tool! But it'd be really awesome if it
maintained targets for games consoles too...


On 4 March 2013 22:57, Johannes Pfau <nospam example.com> wrote:


 Am Mon, 4 Mar 2013 10:36:45 +1000
 schrieb Manu <turkeyman gmail.com>:


 x86 and x64 are already available. I've seen ARM builds kicking
 around too which I've already done some work with. I'm not sure if
 they're up to date, but it's probably fine for me to use.


 I included a new ARM build. Unfortunately ARM druntime broke when
 upgrading to 2.062. We already had working cross-compilers with phobos
 for ARM (raspberry pi).


 Linux or bare metal + no library for the others will suit me now.
 Whatever's easiest.


 OK, then linux


 Google seem to have a build script for building their Android ARM and
 MIPS toolchains:
 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts
 easily?


 Unless you actually need android support building a crosstool-NG
 toolchain is easier.


 I guess the most important one in that case is PPC, since VMX is
 really standard, it's a very common arch, and there's no existing
 toolchains for it.

 I remember talk of a build-server that automatically rebuilds
 binaries of many toolchains each time a new release of D comes out
 some time back, did that lead anywhere?

 In the future, it'd be great to have a more complete set of offerings
 linked from http://dlang.org/download.html


 Once we have working crosstool-NG configurations updating the cross
 compilers is trivial. I should set up a qemu image though so others can
 easily build the packages as well. However I think a bare compiler
 without druntime is only useful for very few people.


 I've uploaded the toolchains now. I verified that they can compile a
 simple program, but I don't have hardware to test if the result
 actually works.

 Also attached a simple hello world. I guess you know this already but
 you have to compile with -nophoboslib and make sure that gdc can find
 the object.di file. The included object.di was just copied from my
 local installation. It's not really used but the frontend does some
 sanity checks on object.di now so an empty file doesn't work anymore.
 Also make sure _Dmodule_ref is defined as in the test.d example file.


 Some information about the toolchains:
 GCC 4.7.2
 GDC b8f5c22b0e7afa7e68a287ed788597e783540063 (2.062 frontend)
 Endianness: Whatever was the default in ct-NG

 (All version identifiers from http://dlang.org/version.html should
 work.)

 It's sometimes possible to change the endianness without recompiling gcc
 (e.g. for MIPS -EB and -EL flags.). I'm not sure if this works though
 as the C libraries are compiled with default endianness.

 All toolchains target 32bit processors. If you need 64bit targets just
 tell me. Multilib is not enabled.

 http://www.mediafire.com/?b8pgyvbotbgpd

 And here's an old release for Raspbian with druntime & phobos:
 http://www.mediafire.com/?00d05h1gox9c50h
Mar 04 2013
parent Johannes Pfau <nospam example.com> writes:
Am Tue, 5 Mar 2013 00:16:08 +1000
schrieb Manu <turkeyman gmail.com>:


 Wowzer! That's awesome! :)
 
 So, I don't understand this whole crosstools-NG thing, how did you do
 that so easily?
 
 So, aside from doing these tests I'm working on now, let's say I were
 working on some games console projects, how hard is it to integrate
 the console patches (like the ones linked in my prior email) into
 this build system to make actual working compilers for those
 platforms? This seems like an amazing tool! But it'd be really
 awesome if it maintained targets for games consoles too...
 


Crosstool-NG is a set of scripts & patches to built gcc cross
compilers. It includes patches for common build problems, a nice GUI to
configure the build and allows to build complicated cross compiler
configurations (like building a toolchain which targets C (e.g. ARM),
runs on B (e.g. Windows) and is compiled on B (e.g. Linux)).

(The important part is that it's maintained and used by many people so
it has patches for most of the problems you can run into. For example I
ran into this problem when building devkitPro:
http://crosstool-ng.org/hg/crosstool-ng/rev/372245834071)

It can also build additional useful tools and libraries (like gdb,
strace, ...)

Integrating D is very easy, here are some instructions:
http://gdcproject.org/wiki/Cross%20Compiler/crosstool-NG


Adding console toolchains might be a little more difficult. Adding the
GCC/binutils patches is simple (just copy them into the right folder),
but you have to be careful making sure that gcc is built with exactly
the same options as the current console toolchain (certainly doable).
I'm not sure how good the newlib support is in crosstool-NG though,
adding rules for newlib might be some extra work. But if I was a
devkitPro developer I'd try to move the toolchain to crosstool-NG as it
really simplifies building cross compilers.
Mar 06 2013
prev sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On Mar 4, 2013 12:38 AM, "Manu" <turkeyman gmail.com> wrote:

 x86 and x64 are already available. I've seen ARM builds kicking around


too which I've already done some work with. I'm not sure if they're up to
date, but it's probably fine for me to use.

 Linux or bare metal + no library for the others will suit me now.


Whatever's easiest.

 Google seem to have a build script for building their Android ARM and


MIPS toolchains:

 http://developer.android.com/tools/sdk/ndk/index.html
 I wonder if the GDC frontend can be dropped in to their scripts easily?

 I guess the most important one in that case is PPC, since VMX is really


standard, it's a very common arch, and there's no existing toolchains for
it.

 I remember talk of a build-server that automatically rebuilds binaries of


many toolchains each time a new release of D comes out some time back, did
that lead anywhere?

 In the future, it'd be great to have a more complete set of offerings


linked from http://dlang.org/download.html

 On 4 March 2013 03:20, Johannes Pfau <nospam example.com> wrote:

 Am Sun, 3 Mar 2013 23:33:27 +1000
 schrieb Manu <turkeyman gmail.com>:


 Hi people.

 So I've been AWOL a while, and I'll still be intermittent for some
 weeks yet, but I'm trying to work on std.simd and also my dconf talks
 in any gaps I can.
 However, in order to do a thorough job, I'm missing some tools. I
 wonder if anyone can help me out...?

 I need to do some rather strenuous testing of SIMD code gen in D, to
 make sure I don't make mistakes and/or lie.

 This requires testing against as many compilers+architectures as I
 can. I have barely any internet access to speak of, I don't really
 know how, and I also don't run linux, so I can't practically build
 myself a suite of toolchains to work with >_<

 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for windows
 could help me out by producing as many of these in binary as possible?

 As a side note, I think it would be really valuable to keep a cache
 of such windows binaries somewhere on the net anyway, so anyone
 working on foreign platforms can get to work and actually focus on
 platform support, without wasting all their time learning how to
 build these toolchains.

 I think Daniel Green who usually builds the windows GDC toolchains
 actually builds them from linux? (which seems weird to me)

 Cheers people!


 I could build the gdc toolchains, but I need some more information:

 What OS should the toolchains target? Linux, windows or "bare-metal"?
 (Especially the x86 toolchains)

 Do you need libdruntime/libphobos or is a plain compiler enough? I'm
 not sure if druntime compiles for SH4, MIPS and PPC.




I did try testing this with QEMU, however the only reliable target that can
build gcc is ARM under emulation.

The best method of doing this would be to have physical kit either donated
or bought through donations that can set-up as port boxes for anyone who
wants to port their D application over to architecture X.

Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';
Mar 04 2013
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:

 Ideally, I would really like all these toolchains:
  GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
  LDC: x86, x64, ARM, PPC <- used by apple

 I wonder if anyone with know-how building GDC or LDC for 
 windows could help
 me out by producing as many of these in binary as possible?


My dev laptop is being repaired right now, but I hope I'll be 
able to upload a recent MinGW build of LDC by the end of the 
week. I could build LLVM with all targets enabled, so you could 
at least do some basic codegen tests by manually specifying the 
target triple.

See http://wiki.dlang.org/LDC for information on the ARM/PPC 
status. You might want to contact Kai directly, maybe he can 
easily provide you with a "real" PPC64 cross-compiler.



 As a side note, I think it would be really valuable to keep a 
 cache of such
 windows binaries somewhere on the net anyway, so anyone working 
 on foreign
 platforms can get to work and actually focus on platform 
 support, without
 wasting all their time learning how to build these toolchains.


In case of LDC, the main reason such binaries don't already 
exist, at least for the bare compiler, is that no Windows 
binaries exist at all (a MinGW preview will hopefully be 
available with the next release).

Keep in mind, though, that any serious porting effort will likely 
require changes to the compiler source (e.g. custom ABI 
transformations), so the benefit of binaries for this specific 
use case will probably be limited.

David
Mar 04 2013
parent reply "Kai Nacke" <kai redstar.de> writes:
On Monday, 4 March 2013 at 13:29:34 UTC, David Nadlinger wrote:

 On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:

 Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple


 See http://wiki.dlang.org/LDC for information on the ARM/PPC 
 status. You might want to contact Kai directly, maybe he can 
 easily provide you with a "real" PPC64 cross-compiler.


Hi Manu,

I uploaded a Win64-version of LDC to 
http://www.redstar.de/ldc/LDC.zip (md5: 
5ae2c1c4b2c6e94e81b930b668a02cc4). It contains this version and 
targets:

LDC - the LLVM D compiler (trunk):
   based on DMD v2.061 and LLVM 3.3svn
   Default target: x86_64-pc-win32
   Host CPU: corei7
   http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     arm      - ARM
     mips     - Mips
     mips64   - Mips64 [experimental]
     mips64el - Mips64el [experimental]
     mipsel   - Mipsel
     ppc32    - PowerPC 32
     ppc64    - PowerPC 64
     thumb    - Thumb
     x86      - 32-bit X86: Pentium-Pro and above
     x86-64   - 64-bit X86: EM64T and AMD64

Maybe this is useful for you.

Kai
Mar 09 2013
parent Manu <turkeyman gmail.com> writes:
On 10 March 2013 03:58, Kai Nacke <kai redstar.de> wrote:


 On Monday, 4 March 2013 at 13:29:34 UTC, David Nadlinger wrote:


 On Sunday, 3 March 2013 at 13:57:15 UTC, Manu wrote:


 Ideally, I would really like all these toolchains:
 GDC: x86, x64, ARM, PPC, MIPS, SH4 <- common toolchains
 LDC: x86, x64, ARM, PPC <- used by apple


 See http://wiki.dlang.org/LDC for information on the ARM/PPC status. You
 might want to contact Kai directly, maybe he can easily provide you with a
 "real" PPC64 cross-compiler.


 Hi Manu,

 I uploaded a Win64-version of LDC to
http://www.redstar.de/ldc/LDC.**zip<http://www.redstar.de/ldc/LDC.zip>(md5:
5ae2c1c4b2c6e94e81b930b668a02c
 **c4). It contains this version and targets:

 LDC - the LLVM D compiler (trunk):
   based on DMD v2.061 and LLVM 3.3svn
   Default target: x86_64-pc-win32
   Host CPU: corei7
   http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     arm      - ARM
     mips     - Mips
     mips64   - Mips64 [experimental]
     mips64el - Mips64el [experimental]
     mipsel   - Mipsel
     ppc32    - PowerPC 32
     ppc64    - PowerPC 64
     thumb    - Thumb
     x86      - 32-bit X86: Pentium-Pro and above
     x86-64   - 64-bit X86: EM64T and AMD64

 Maybe this is useful for you.

 Kai


Awesome Kai! Thanks so much!
I think that covers the tools I need.
Now I just need to find some down time to put it all together >_<
Mar 10 2013