www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Windows/Android/Mac/RaspberryPi/Linux/ARM/FPGA/ASIC/DSP/uC

reply FoxyBrown <Foxy Brown.IPT> writes:
Hi,

I will be developing some hardware and software that will need to 
be portable across a multitude of systems.

At first, I will develop some of the non-specific infrastructural 
code base on windows for convenience. These involve the GUI, 
overall structure, etc.

I think will need to move in to more mobile aspects such as the 
android and Raspberry Pi which will allow me to start to 
integrate and work with the future hardware based parts of this 
system.

At this point I am in the setup stages that will organize and 
formulate a proper plan of action. I am aware that D works for 
Windows/Linux/Mac and has some Raspberry Pi and android/arm 
support with very little FPGA/ASIC/DSP/uC support.

I hope, by the time that I get in to the hardware layers that D 
will have advanced further down the line on them.

My understanding is that DMD is only applicable for the 3 main 
platforms, and this is fine for now as that is where I'll start. 
LDC is starting to progress on ARM/android and I have seen 
someone write a to use D on FPGA through a transformation 
process. Some work has been done with some lower level uC's such 
as the PIC and ARM processors.

Now that GDC is in the works, I assume that will open up even 
more doors. Because this is a long term project(10+ years) and 
involves many different systems, subsystems, architectures, and 
so forth, organization is key for success.  The problem I have 
with the choice of D over C/C++, at the moment, is in it's 
knowledge base. Most of the advancements for the different 
platforms are done by individuals with their own unique goals and 
motivations in mind. There is very little organized structure to 
the overall development. Finding information for specific aspects 
and requirements involves hunting down the appropriate 
information. Usually it is forum posts and link fishing, usually 
outdated. Structured documentation is key for efficient and 
successful progress, which I find D to be lacking in. While 
library functionality is documented and adequate, the 
architectural issues and development is not.

Is there any hope of seeing a unified structured interface to the 
different platforms that D is starting to be developed on rather 
than having to hunt and peck for the answers?

If not, may I suggest, at a minimum, some type of hierarchy based 
solution that involves all the major design applications of D 
with the ability to cross-reference, date, comment, and modify? A 
sort of wiki but with a bit more hierarchical structure.

D
   Cross-Platform
      Compilers
         ...
         GDC
         ...
      Library
         ...
      ...
   Windows
      Compiler
         ...
         Compiling
             ...
         ...
      Library
         ...
         Compiling
             ...
         Documentation
             ...
             Wi32 API
                 ...
                 Problems
                 ...
                 Functions
                 ...
                 Misc
                 ...
                 Uncategorized
                 ...
             ...
         ...
   ...
   DSP
      ...
      Progress
        ...
      ...


The idea is that one can delve in to the hierarchy and find 
whatever info they want or add to it if it is not there so that 
someone else can find it. At some point, most of what D can do is 
found in the hierarchy, which, as we all know, only requires 
around O(log2(n)) search time(rather than O(n^m) for googling 
topics and link fishing).  At some point, this hierarchy becomes 
very efficient at representing the information we all are 
searching for rather than a flat random access type of list. The 
point is to design such an interface and let users like me fill 
it out and as time goes on,  finding relevant information becomes 
quicker and quicker. For proper design, meta-information must be 
used to it's fullest. (Tags, date stamping, ratings, relevancy, 
cross-references, etc)

Or, perhaps, we already have something this powerful in the works?
Jun 23 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 23 June 2017 at 19:03:01 UTC, FoxyBrown wrote:

 Hi,

 I will be developing some hardware and software that will need 
 to be portable across a multitude of systems.

 At first, I will develop some of the non-specific 
 infrastructural code base on windows for convenience. These 
 involve the GUI, overall structure, etc.

 I think will need to move in to more mobile aspects such as the 
 android and Raspberry Pi which will allow me to start to 
 integrate and work with the future hardware based parts of this 
 system.

 At this point I am in the setup stages that will organize and 
 formulate a proper plan of action. I am aware that D works for 
 Windows/Linux/Mac and has some Raspberry Pi and android/arm 
 support with very little FPGA/ASIC/DSP/uC support.

 I hope, by the time that I get in to the hardware layers that D 
 will have advanced further down the line on them.

 My understanding is that DMD is only applicable for the 3 main 
 platforms, and this is fine for now as that is where I'll 
 start. LDC is starting to progress on ARM/android and I have 
 seen someone write a to use D on FPGA through a transformation 
 process. Some work has been done with some lower level uC's 
 such as the PIC and ARM processors.

 Now that GDC is in the works, I assume that will open up even 
 more doors. Because this is a long term project(10+ years) and 
 involves many different systems, subsystems, architectures, and 
 so forth, organization is key for success.  The problem I have 
 with the choice of D over C/C++, at the moment, is in it's 
 knowledge base. Most of the advancements for the different 
 platforms are done by individuals with their own unique goals 
 and motivations in mind. There is very little organized 
 structure to the overall development. Finding information for 
 specific aspects and requirements involves hunting down the 
 appropriate information. Usually it is forum posts and link 
 fishing, usually outdated. Structured documentation is key for 
 efficient and successful progress, which I find D to be lacking 
 in. While library functionality is documented and adequate, the 
 architectural issues and development is not.

 Is there any hope of seeing a unified structured interface to 
 the different platforms that D is starting to be developed on 
 rather than having to hunt and peck for the answers?

 If not, may I suggest, at a minimum, some type of hierarchy 
 based solution that involves all the major design applications 
 of D with the ability to cross-reference, date, comment, and 
 modify? A sort of wiki but with a bit more hierarchical 
 structure.

 D
   Cross-Platform
      Compilers
         ...
         GDC
         ...
      Library
         ...
      ...
   Windows
      Compiler
         ...
         Compiling
             ...
         ...
      Library
         ...
         Compiling
             ...
         Documentation
             ...
             Wi32 API
                 ...
                 Problems
                 ...
                 Functions
                 ...
                 Misc
                 ...
                 Uncategorized
                 ...
             ...
         ...
   ...
   DSP
      ...
      Progress
        ...
      ...


 The idea is that one can delve in to the hierarchy and find 
 whatever info they want or add to it if it is not there so that 
 someone else can find it. At some point, most of what D can do 
 is found in the hierarchy, which, as we all know, only requires 
 around O(log2(n)) search time(rather than O(n^m) for googling 
 topics and link fishing).  At some point, this hierarchy 
 becomes very efficient at representing the information we all 
 are searching for rather than a flat random access type of 
 list. The point is to design such an interface and let users 
 like me fill it out and as time goes on,  finding relevant 
 information becomes quicker and quicker. For proper design, 
 meta-information must be used to it's fullest. (Tags, date 
 stamping, ratings, relevancy, cross-references, etc)

 Or, perhaps, we already have something this powerful in the 
 works?


The ldc wiki page has a rudimentary version of what you're 
looking for, listing all platforms worked on and some idea of how 
well they're supported:

https://wiki.dlang.org/LDC

Dmd will likely never target the vast majority of the platforms 
you list, so it will depend on the gdc and ldc teams doing it.

Also, I'm looking at submitting a pull with my last ldc CMake 
patch to cross-compile the stdlib for Android/ARM and 
generalizing it for other platforms too:

https://gist.github.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636

Once that's in, building a cross-compiled stdlib for several 
platforms should be as simple as setting some flags with the 
location of a C cross-compiler and any C/linker flags you need, 
then running the same cmake/make combo that you use to build a 
native stdlib today.

No modifications are needed to ldc itself for well-supported 
platforms, as ldc is always a cross-compiler by default.  For 
example, kinke used the new llvm linker, lld, to cross-compile 
Windows/x64 binaries from linux/x64 by using the stock ldc 
compiler.  The instructions for how he did it are linked from the 
changelog for the latest ldc 1.3 beta2:

https://github.com/ldc-developers/ldc/pull/2142#issuecomment-304472412

Finally, the latest ldc 1.3 betas for the first time include the 
llvm code-generation target backends for various other CPUs, like 
ARM or PowerPC, whereas before ldc only bothered to compile in 
the x86/x64 backends, as those were the only stdlibs it 
distributed:

[joakim localhost ~]$ ./ldc2-1.3.0-beta2-linux-x86_64/bin/ldc2 
--version
LDC - the LLVM D compiler (1.3.0-beta2):
   based on DMD v2.073.2 and LLVM 4.0.0
   built with LDC - the LLVM D compiler (1.3.0-beta2)
   Default target: x86_64-unknown-linux-gnu
   Host CPU: broadwell                                             
            http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     aarch64    - AArch64 (little endian)
     aarch64_be - AArch64 (big endian)
     arm        - ARM
     arm64      - ARM64 (little endian)
     armeb      - ARM (big endian)
     ppc32      - PowerPC 32
     ppc64      - PowerPC 64
     ppc64le    - PowerPC 64 LE
     thumb      - Thumb
     thumbeb    - Thumb (big endian)
     x86        - 32-bit X86: Pentium-Pro and above
     x86-64     - 64-bit X86: EM64T and AMD64

Not very useful right now unless you're doing bare-metal 
programming on those newly added architectures, because the 
cross-compiled stdlibs are not included and there's no easy way 
to generate them, but we could also put out separate tarfiles 
with the cross-compiled stdlibs (separated only to save space in 
the compiler download for those who don't want them), just as 
kinke used the ldc stdlib compiled for Windows/x64 on linux/x64 
when cross-compiling.

Just last night, I was thinking about some easy way to let ldc 
users cross-compile the stdlib themselves from the regular ldc 
download, and I thought of doing it with dub, the D package 
manager, since we now distribute dub alongside ldc.  However, I 
don't know how feasible this is with dub, needs to be looked into.
Jun 23 2017
next sibling parent Johan Engelen <j j.nl> writes:
On Friday, 23 June 2017 at 19:56:17 UTC, Joakim wrote:

 
 Just last night, I was thinking about some easy way to let ldc 
 users cross-compile the stdlib themselves from the regular ldc 
 download, and I thought of doing it with dub, the D package 
 manager, since we now distribute dub alongside ldc.  However, I 
 don't know how feasible this is with dub, needs to be looked 
 into.


Being able to build druntime/Phobos locally on the user's machine 
would also help with generating LTO versions of the stdlibs.
I think the first step is making sure that all source is 
available in our packages. At the moment, druntime/src/rt is not 
present in the install (for example).

Cheers,
  Johan
Jun 24 2017
prev sibling parent reply FoxyBrown <Foxy Brown.IPT> writes:
On Friday, 23 June 2017 at 19:56:17 UTC, Joakim wrote:

 On Friday, 23 June 2017 at 19:03:01 UTC, FoxyBrown wrote:

 [...]


 The ldc wiki page has a rudimentary version of what you're 
 looking for, listing all platforms worked on and some idea of 
 how well they're supported:

 https://wiki.dlang.org/LDC


Thanks, it's a start.


 Dmd will likely never target the vast majority of the platforms 
 you list, so it will depend on the gdc and ldc teams doing it.


I'm not too worried about dmd, I am mainly interested in simply a 
way. D is a nice language, so being able to use it, regardless of 
which compiler, is the goal.


 Also, I'm looking at submitting a pull with my last ldc CMake 
 patch to cross-compile the stdlib for Android/ARM and 
 generalizing it for other platforms too:

 https://gist.github.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636

 Once that's in, building a cross-compiled stdlib for several 
 platforms should be as simple as setting some flags with the 
 location of a C cross-compiler and any C/linker flags you need, 
 then running the same cmake/make combo that you use to build a 
 native stdlib today.

 No modifications are needed to ldc itself for well-supported 
 platforms, as ldc is always a cross-compiler by default.  For 
 example, kinke used the new llvm linker, lld, to cross-compile 
 Windows/x64 binaries from linux/x64 by using the stock ldc 
 compiler.  The instructions for how he did it are linked from 
 the changelog for the latest ldc 1.3 beta2:

 https://github.com/ldc-developers/ldc/pull/2142#issuecomment-304472412

 Finally, the latest ldc 1.3 betas for the first time include 
 the llvm code-generation target backends for various other 
 CPUs, like ARM or PowerPC, whereas before ldc only bothered to 
 compile in the x86/x64 backends, as those were the only stdlibs 
 it distributed:

 [joakim localhost ~]$ ./ldc2-1.3.0-beta2-linux-x86_64/bin/ldc2 
 --version
 LDC - the LLVM D compiler (1.3.0-beta2):
   based on DMD v2.073.2 and LLVM 4.0.0
   built with LDC - the LLVM D compiler (1.3.0-beta2)
   Default target: x86_64-unknown-linux-gnu
   Host CPU: broadwell
            http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     aarch64    - AArch64 (little endian)
     aarch64_be - AArch64 (big endian)
     arm        - ARM
     arm64      - ARM64 (little endian)
     armeb      - ARM (big endian)
     ppc32      - PowerPC 32
     ppc64      - PowerPC 64
     ppc64le    - PowerPC 64 LE
     thumb      - Thumb
     thumbeb    - Thumb (big endian)
     x86        - 32-bit X86: Pentium-Pro and above
     x86-64     - 64-bit X86: EM64T and AMD64

 Not very useful right now unless you're doing bare-metal 
 programming on those newly added architectures, because the 
 cross-compiled stdlibs are not included and there's no easy way 
 to generate them, but we could also put out separate tarfiles 
 with the cross-compiled stdlibs (separated only to save space 
 in the compiler download for those who don't want them), just 
 as kinke used the ldc stdlib compiled for Windows/x64 on 
 linux/x64 when cross-compiling.

 Just last night, I was thinking about some easy way to let ldc 
 users cross-compile the stdlib themselves from the regular ldc 
 download, and I thought of doing it with dub, the D package 
 manager, since we now distribute dub alongside ldc.  However, I 
 don't know how feasible this is with dub, needs to be looked 
 into.


Nice to know it's being worked on. I am not really familiar with 
llvm but it seems like it can be quite useful. I'm I correct that 
basically llvm is an intermediate representation(sorta like 


This means that one simply needs a front end that converts the 
source code to llvm's IR and then a back end that converts the IR 
to targets binary?


For example, a quick search for llvm and 8051 gives this:

https://github.com/AndreaCorallo/llvm-i8051

Since, I assume(since there is no documentation describing what 
it does) is a backend compiler for llvm's IR to 8051, we can go 
from LDC to LLVM to 8051? In theory, of course.

This is all bare metal, of course.

Is GCC effectively the same(although the IR is skipped?)?


Thanks.
Jun 24 2017
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:

 Nice to know it's being worked on. I am not really familiar 
 with llvm but it seems like it can be quite useful. I'm I 
 correct that basically llvm is an intermediate 


 This means that one simply needs a front end that converts the 
 source code to llvm's IR and then a back end that converts the 
 IR to targets binary?


Yes indeed. LDC will output the IR when you use the `-output-ll` 
(human readable) or `-output-bc` (binary representation) compile 
switches.


 For example, a quick search for llvm and 8051 gives this:

 https://github.com/AndreaCorallo/llvm-i8051

 Since, I assume(since there is no documentation describing what 
 it does) is a backend compiler for llvm's IR to 8051, we can go 
 from LDC to LLVM to 8051? In theory, of course.


In theory, yes indeed. What may be important is that the LLVM 
versions match (the LLVM version used to build LDC, and the LLVM 
version of that 8051 fork). With every new LLVM version, the IR 
may change in subtle ways...


 Is GCC effectively the same(although the IR is skipped?)?


As far as I know, GCC also uses an IR (it's used for LTO).

-Johan
Jun 24 2017
parent reply FoxyBrown <Foxy Brown.IPT> writes:
On Saturday, 24 June 2017 at 18:51:44 UTC, Johan Engelen wrote:

 On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:

 [...]


 Yes indeed. LDC will output the IR when you use the 
 `-output-ll` (human readable) or `-output-bc` (binary 
 representation) compile switches.


 [...]


 In theory, yes indeed. What may be important is that the LLVM 
 versions match (the LLVM version used to build LDC, and the 
 LLVM version of that 8051 fork). With every new LLVM version, 
 the IR may change in subtle ways...


 [...]


 As far as I know, GCC also uses an IR (it's used for LTO).

 -Johan


Why, then, doesn't anything that targets llvm simply target 
everything else? D is not just for the ones listed but also for 
many other platforms? is it simply because no one has implemented 
the transformations to prove it works?
Jun 24 2017
parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 24 June 2017 at 19:18:24 UTC, FoxyBrown wrote:

 On Saturday, 24 June 2017 at 18:51:44 UTC, Johan Engelen wrote:

 On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:

 [...]


 Yes indeed. LDC will output the IR when you use the 
 `-output-ll` (human readable) or `-output-bc` (binary 
 representation) compile switches.


 [...]


 In theory, yes indeed. What may be important is that the LLVM 
 versions match (the LLVM version used to build LDC, and the 
 LLVM version of that 8051 fork). With every new LLVM version, 
 the IR may change in subtle ways...


 [...]


 As far as I know, GCC also uses an IR (it's used for LTO).

 -Johan


 Why, then, doesn't anything that targets llvm simply target 
 everything else? D is not just for the ones listed but also for 
 many other platforms? is it simply because no one has 
 implemented the transformations to prove it works?


In theory, once you can produce llvm IR, you get every CPU that 
llvm targets, which grows by the day.  However, being able to 
target CPUs is not the only aspect of porting, you also have to 
integrate with the OS.  D's runtime and phobos rely on the C 
runtime for some basic functions, though there has been some talk 
of re-implementing them in pure D.

So if you're running on an OS, you also need to wrap several C 
APIs from that OS, see my pulls for bionic/Android, mostly the 
bottom five:

https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ajoakim-noah+is%3Aclosed

If you're running on bare metal, that's obviously not an issue, 
but conversely those who've tried it say D doesn't really support 
stripping out the druntime for those targets.  There has been a 
renewed focus on this recently, with work on the -betterC flag, 
about which you can see another forum thread.
Jun 24 2017