www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - I have this game engine...

reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
So, I just wanted to put this idea out there, and see what other
people make of it.

I have this game engine (https://github.com/TurkeyMan/fuji), it's
lived for about 12 years now (first commit in 2004, and it existed
prior before source control). I called it 'Fuji' (a modest, yet
pleasing and attractive mountain). It supports (or has supported)
shit-loads of platforms; I'm a game-engine dev for life, and I have a
fetish for portability, and niche platform support.
Needless to say, it has had a LOT of time and energy put into it, and
I would say it's infrastructurally better than most proprietary
commercial game-engines I've worked with (although there are some
missing features, I just implement what I need), mainly in that I have
the luxury to aggressively refactor when design decisions turned out
to be mistakes, and no deadlines to meet.
It is a very good example of what we use in real-world AAA gamedev.

It's written in C, obviously. It has comprehensive bindings for D,
which I'm pretty sure I'm the only one that's ever used, and I use it
in all my modern D projects.

Anyway. I'm spending less time on hobby game-dev these days, obviously
I'm not going to make the worlds next big game engine... Hobbyists
will just use Unity. I'm not quite sure what to do with it.

I have been thinking about full-scale porting to D, and it would serve
there is a massive-scale long-term codebase, with portability as it's
primary objective, and I don't know of another project quite like this
in the D ecosystem? Also, perhaps D gamedevs might be interested in an
all-D game engine they can use and hack on.

So, when I think on that, I consider what would be lost... and the
answer is; almost all platforms. But this is something that can be
addressed.

If people find this to be an interesting project, and I do take some
time to do this port, what I'd really love is to work together with
some of the niche platform support guys and use it to stress test
toolchains for various platforms. I'd also like help from guys like
Johannes to make regularly released builds of some console
cross-compilers available so that it can keep on building.
The CI of this project for those platforms would more-or-less verify
that the toolchains and druntime+friends are working.

It seems like a pretty big project, but I can't think of a better use
for this codebase right now.

Does this generally interest anyone? The thought of it makes me
nervous, but I also think this might be the most valuable thing I can
do with this code today.
Alternative would be to port to Rust ;)
Oct 31 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 01/11/15 2:33 PM, Manu via Digitalmars-d wrote:
 So, I just wanted to put this idea out there, and see what other
 people make of it.

 I have this game engine (https://github.com/TurkeyMan/fuji), it's
 lived for about 12 years now (first commit in 2004, and it existed
 prior before source control). I called it 'Fuji' (a modest, yet
 pleasing and attractive mountain). It supports (or has supported)
 shit-loads of platforms; I'm a game-engine dev for life, and I have a
 fetish for portability, and niche platform support.
 Needless to say, it has had a LOT of time and energy put into it, and
 I would say it's infrastructurally better than most proprietary
 commercial game-engines I've worked with (although there are some
 missing features, I just implement what I need), mainly in that I have
 the luxury to aggressively refactor when design decisions turned out
 to be mistakes, and no deadlines to meet.
 It is a very good example of what we use in real-world AAA gamedev.

 It's written in C, obviously. It has comprehensive bindings for D,
 which I'm pretty sure I'm the only one that's ever used, and I use it
 in all my modern D projects.

 Anyway. I'm spending less time on hobby game-dev these days, obviously
 I'm not going to make the worlds next big game engine... Hobbyists
 will just use Unity. I'm not quite sure what to do with it.

 I have been thinking about full-scale porting to D, and it would serve
 there is a massive-scale long-term codebase, with portability as it's
 primary objective, and I don't know of another project quite like this
 in the D ecosystem? Also, perhaps D gamedevs might be interested in an
 all-D game engine they can use and hack on.

 So, when I think on that, I consider what would be lost... and the
 answer is; almost all platforms. But this is something that can be
 addressed.

 If people find this to be an interesting project, and I do take some
 time to do this port, what I'd really love is to work together with
 some of the niche platform support guys and use it to stress test
 toolchains for various platforms. I'd also like help from guys like
 Johannes to make regularly released builds of some console
 cross-compilers available so that it can keep on building.
 The CI of this project for those platforms would more-or-less verify
 that the toolchains and druntime+friends are working.

 It seems like a pretty big project, but I can't think of a better use
 for this codebase right now.

 Does this generally interest anyone? The thought of it makes me
 nervous, but I also think this might be the most valuable thing I can
 do with this code today.
 Alternative would be to port to Rust ;)
After completing std.experimental.color, would you mind doing the same sort of thing for audio? As you know I'm slowing building up code for game dev: https://github.com/rikkimax/alphaPhobos You have a lot of good code there, it would be nice to rebuild most abstractions that are commonly used out. Ok, I have another agenda then what you want, but still :p
Oct 31 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
O_o .. what's your agenda?

I think what I'm talking about here is something slightly different.
Also I think the goals of my engine are fairly specific too; run fast,
on anything (ie, heavily resource limited hardware), no
non-operating-system dependencies, small code. I think this is a
different angle than most modern games type projects. Obviously, the
point however is to make over a decade of development available for
people to use. There's a lot in there.

On 1 November 2015 at 11:43, Rikki Cattermole via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 01/11/15 2:33 PM, Manu via Digitalmars-d wrote:
 So, I just wanted to put this idea out there, and see what other
 people make of it.

 I have this game engine (https://github.com/TurkeyMan/fuji), it's
 lived for about 12 years now (first commit in 2004, and it existed
 prior before source control). I called it 'Fuji' (a modest, yet
 pleasing and attractive mountain). It supports (or has supported)
 shit-loads of platforms; I'm a game-engine dev for life, and I have a
 fetish for portability, and niche platform support.
 Needless to say, it has had a LOT of time and energy put into it, and
 I would say it's infrastructurally better than most proprietary
 commercial game-engines I've worked with (although there are some
 missing features, I just implement what I need), mainly in that I have
 the luxury to aggressively refactor when design decisions turned out
 to be mistakes, and no deadlines to meet.
 It is a very good example of what we use in real-world AAA gamedev.

 It's written in C, obviously. It has comprehensive bindings for D,
 which I'm pretty sure I'm the only one that's ever used, and I use it
 in all my modern D projects.

 Anyway. I'm spending less time on hobby game-dev these days, obviously
 I'm not going to make the worlds next big game engine... Hobbyists
 will just use Unity. I'm not quite sure what to do with it.

 I have been thinking about full-scale porting to D, and it would serve
 there is a massive-scale long-term codebase, with portability as it's
 primary objective, and I don't know of another project quite like this
 in the D ecosystem? Also, perhaps D gamedevs might be interested in an
 all-D game engine they can use and hack on.

 So, when I think on that, I consider what would be lost... and the
 answer is; almost all platforms. But this is something that can be
 addressed.

 If people find this to be an interesting project, and I do take some
 time to do this port, what I'd really love is to work together with
 some of the niche platform support guys and use it to stress test
 toolchains for various platforms. I'd also like help from guys like
 Johannes to make regularly released builds of some console
 cross-compilers available so that it can keep on building.
 The CI of this project for those platforms would more-or-less verify
 that the toolchains and druntime+friends are working.

 It seems like a pretty big project, but I can't think of a better use
 for this codebase right now.

 Does this generally interest anyone? The thought of it makes me
 nervous, but I also think this might be the most valuable thing I can
 do with this code today.
 Alternative would be to port to Rust ;)
After completing std.experimental.color, would you mind doing the same sort of thing for audio? As you know I'm slowing building up code for game dev: https://github.com/rikkimax/alphaPhobos You have a lot of good code there, it would be nice to rebuild most abstractions that are commonly used out. Ok, I have another agenda then what you want, but still :p
Oct 31 2015
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 01/11/15 2:56 PM, Manu via Digitalmars-d wrote:
 O_o .. what's your agenda?
My agenda is simple. To make D have the best usability/libraries when it comes to GUI/game development.
 I think what I'm talking about here is something slightly different.
 Also I think the goals of my engine are fairly specific too; run fast,
 on anything (ie, heavily resource limited hardware), no
 non-operating-system dependencies, small code. I think this is a
 different angle than most modern games type projects. Obviously, the
 point however is to make over a decade of development available for
 people to use. There's a lot in there.
Of course. What I'm thinking is port + refactor + make more complete/generic. There is no reason it can't be fast and usable on limited hardware. Your desire is to port it as is. Mine would be to port some in context of a greater library, refactor and make it more complete. Then repeat until complete. When a new platform is ready, add the platformy specific aspects to existing parts. It's a much slower process. But it will create a much more idiomatic D environment to work with.
Oct 31 2015
prev sibling next sibling parent reply Nerve <nervecenter7 gmail.com> writes:
On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 I have this game engine (https://github.com/TurkeyMan/fuji), 
 it's
 lived for about 12 years now (first commit in 2004, and it 
 existed
 prior before source control). I called it 'Fuji' (a modest, yet
 pleasing and attractive mountain). It supports (or has 
 supported)
 shit-loads of platforms; I'm a game-engine dev for life, and I 
 have a
 fetish for portability, and niche platform support.
 Needless to say, it has had a LOT of time and energy put into 
 it, and
 I would say it's infrastructurally better than most proprietary
 commercial game-engines I've worked with (although there are 
 some
 missing features, I just implement what I need), mainly in that 
 I have
 the luxury to aggressively refactor when design decisions 
 turned out
 to be mistakes, and no deadlines to meet.
 It is a very good example of what we use in real-world AAA 
 gamedev.
I would be very, very interested in this. I've been yearning for something in D that supports some lighter 3D with texturing, animated sprites, and lighting on both. Something flexible and light that can also have modules pulled independently for drawing and sound if possible. Unfortunately, DGame really only supports 2D, and Dash has all its weight behind being a full-featured 3D engine a la Unity/Unreal. Would Fuji fit the bill?
Oct 31 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 November 2015 at 12:10, Nerve via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 I have this game engine (https://github.com/TurkeyMan/fuji), it's
 lived for about 12 years now (first commit in 2004, and it existed
 prior before source control). I called it 'Fuji' (a modest, yet
 pleasing and attractive mountain). It supports (or has supported)
 shit-loads of platforms; I'm a game-engine dev for life, and I have a
 fetish for portability, and niche platform support.
 Needless to say, it has had a LOT of time and energy put into it, and
 I would say it's infrastructurally better than most proprietary
 commercial game-engines I've worked with (although there are some
 missing features, I just implement what I need), mainly in that I have
 the luxury to aggressively refactor when design decisions turned out
 to be mistakes, and no deadlines to meet.
 It is a very good example of what we use in real-world AAA gamedev.
I would be very, very interested in this. I've been yearning for something in D that supports some lighter 3D with texturing, animated sprites, and lighting on both. Something flexible and light that can also have modules pulled independently for drawing and sound if possible. Unfortunately, DGame really only supports 2D, and Dash has all its weight behind being a full-featured 3D engine a la Unity/Unreal. Would Fuji fit the bill?
In terms of what I've used commercially, Fuji is the platform abstraction and core concept implementation that lives below the layer that the high-level interacts with. Editors and tooling (I feel this is what you're talking about when you start using words like 'Unity' or 'Unreal') typically impose particular design decisions wrt scene-graph, physics implementations, etc. The goal of Fuji is not to be Unity, it's intended to be the platform which you could build Unity above, and all commercial engines I've had contact with do have such a layer. I'm not sure if that answers your question. For what it's intended to be, Fuji is quite comprehensive. As a full-game-engine a-la Unity/Unreal, it needs all the high-level stuff built on top. The reason I didn't touch that, is because that layer is extremely subjective, and there are no right/wrong answers there. I also change my mind on that stuff every year or 2. Whereas the lower level is a lot less subjective, and it's been more-or-less constant since I started Fuji in 2003. I still wouldn't do it differently today, although I have a lot more experience and console generations to draw wisdom from.
Oct 31 2015
parent reply Nerve <nervecenter7 gmail.com> writes:
On Sunday, 1 November 2015 at 02:35:49 UTC, Manu wrote:
 In terms of what I've used commercially, Fuji is the platform 
 abstraction and core concept implementation that lives below 
 the layer that the high-level interacts with. Editors and 
 tooling (I feel this is what you're talking about when you 
 start using words like 'Unity' or 'Unreal') typically impose 
 particular design decisions wrt scene-graph, physics 
 implementations, etc. The goal of Fuji is not to be Unity, it's 
 intended to be the platform which you could build Unity above, 
 and all commercial engines I've had contact with do have such a 
 layer.

 I'm not sure if that answers your question.
 For what it's intended to be, Fuji is quite comprehensive. As a
 full-game-engine a-la Unity/Unreal, it needs all the high-level 
 stuff
 built on top. The reason I didn't touch that, is because that 
 layer is
 extremely subjective, and there are no right/wrong answers 
 there. I
 also change my mind on that stuff every year or 2. Whereas the 
 lower
 level is a lot less subjective, and it's been more-or-less 
 constant
 since I started Fuji in 2003. I still wouldn't do it differently
 today, although I have a lot more experience and console 
 generations
 to draw wisdom from.
Nonono, you're fine, what I'm looking for is a lightweight engine that ISN'T like Unity or Unreal and the sophisticated nature of their tools. Often, their engine design decisions back me into a corner when it comes to how I want to organize my own project, and they're oftentimes overkill. About the only thing I would want to keep from those two monster engines is some live compilation feature where changes in source, assets, or UI scripting are immediately apparent in-game. But that takes a massive amount of work, I understand if it's not a priority. You have to port the thing to D in the first place, a significant undertaking in and of itself. What you have seems to be great, so I'll follow it eagerly.
Oct 31 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 November 2015 at 16:45, Nerve via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 About the only thing I would want to keep from those two monster engines is
 some live compilation feature where changes in source, assets, or UI
 scripting are immediately apparent in-game. But that takes a massive amount
 of work, I understand if it's not a priority. You have to port the thing to
 D in the first place, a significant undertaking in and of itself.
I actually wrote that exact thing at Remedy (for Quantum Break) which runtime compiles D code, and hot-swaps the new code into the live data... if only I could liberate the source >_<
Nov 01 2015
next sibling parent reply Nerve <nervecenter7 gmail.com> writes:
On Sunday, 1 November 2015 at 07:30:57 UTC, Manu wrote:
 I actually wrote that exact thing at Remedy (for Quantum Break) 
 which runtime compiles D code, and hot-swaps the new code into 
 the live data... if only I could liberate the source >_<
Wow, I had no idea D was being used for such a massive project, especially on a current-gen console. If I might ask before taking the thread too far off-topic, what tools did you guys use to maintain productivity and organization in such a huge project? (Debuggers, editors, IDEs, etc.)
Nov 01 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 November 2015 at 17:43, Nerve via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 07:30:57 UTC, Manu wrote:
 I actually wrote that exact thing at Remedy (for Quantum Break) which
 runtime compiles D code, and hot-swaps the new code into the live data... if
 only I could liberate the source >_<
Wow, I had no idea D was being used for such a massive project, especially on a current-gen console. If I might ask before taking the thread too far off-topic, what tools did you guys use to maintain productivity and organization in such a huge project? (Debuggers, editors, IDEs, etc.)
Visual Studio. I'm not aware of any practical alternatives on any platform >_< I really, really wish there was competition in the IDE space. We're all sick to death of MS and Visual Studio, we'd all jump ship in an instant, but there's just nothing that comes close.
Nov 01 2015
prev sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 1 November 2015 at 07:30:57 UTC, Manu wrote:
 I actually wrote that exact thing at Remedy (for Quantum Break) 
 which runtime compiles D code, and hot-swaps the new code into 
 the live data... if only I could liberate the source >_<
Well, Remedy owns the source, but they can't own your knowledge. You could write something up explaining how it works and let someone else write something from that.
Nov 01 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 November 2015 at 01:50, Jack Stouffer via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 07:30:57 UTC, Manu wrote:
 I actually wrote that exact thing at Remedy (for Quantum Break) which
 runtime compiles D code, and hot-swaps the new code into the live data... if
 only I could liberate the source >_<
Well, Remedy owns the source, but they can't own your knowledge. You could write something up explaining how it works and let someone else write something from that.
It's not that. We all felt happy to release the tech open-source (at the time I left), which may have changed since, but I have no reason to believe it has. It's just taking the time to separate the tech from any proprietary stuff.
Nov 01 2015
prev sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 01/11/15 08:30, Manu via Digitalmars-d wrote:
 I actually wrote that exact thing at Remedy (for Quantum Break) which
 runtime compiles D code, and hot-swaps the new code into the live
 data... if only I could liberate the source >_<
Wow. That sounds like it could have uses far beyond games -- I'm thinking Erlang-style hot-swapping of components of super-high-uptime (web) services ...
Nov 01 2015
parent Kagamin <spam here.lot> writes:
On Sunday, 1 November 2015 at 17:07:17 UTC, Joseph Rushton 
Wakeling wrote:
 Wow.  That sounds like it could have uses far beyond games -- 
 I'm thinking Erlang-style hot-swapping of components of 
 super-high-uptime (web) services ...
For D1 there was DDL http://www.dsource.org/projects/ddl/wiki/AboutDDL that could load and relocate object files as dynamic libraries.
Nov 02 2015
prev sibling next sibling parent reply rsw0x <anonymous anonymous.com> writes:
On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 ...
speaking of portability, I ran across this game written in D on github that claims to work on Android via NDK bindings, I thought it might interest you. https://github.com/mathstuf/abagames-gunroar Bye.
Oct 31 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 November 2015 at 12:20, rsw0x via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 ...
speaking of portability, I ran across this game written in D on github that claims to work on Android via NDK bindings, I thought it might interest you. https://github.com/mathstuf/abagames-gunroar Bye.
Does it support Dreamcast? :P
Oct 31 2015
next sibling parent reply rsw0x <anonymous anonymous.com> writes:
On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 On 1 November 2015 at 12:20, rsw0x via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 ...
speaking of portability, I ran across this game written in D on github that claims to work on Android via NDK bindings, I thought it might interest you. https://github.com/mathstuf/abagames-gunroar Bye.
Does it support Dreamcast? :P
Not even Sega supported the Dreamcast.
Oct 31 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 November 2015 at 12:45, rsw0x via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 On 1 November 2015 at 12:20, rsw0x via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 ...
speaking of portability, I ran across this game written in D on github that claims to work on Android via NDK bindings, I thought it might interest you. https://github.com/mathstuf/abagames-gunroar Bye.
Does it support Dreamcast? :P
Not even Sega supported the Dreamcast.
Ow, burn! :P
Oct 31 2015
prev sibling parent reply mattcoder <stop spam.com> writes:
On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 ...
 Does it support Dreamcast? :P
I don't know if you are being serious here but if yes... what's the catch with Dreamcast? I mean this is a Fan thing or what? PS: I'm asking this because I used to code for GBA a long time ago but just for fun and hobby! Matheus.
Nov 02 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 November 2015 at 11:07, mattcoder via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 ...
 Does it support Dreamcast? :P
I don't know if you are being serious here but if yes... what's the catch with Dreamcast? I mean this is a Fan thing or what?
I'm just a massive nerd, and the Dreamcast was a great console! :P
Nov 02 2015
parent reply Adrian Matoga <epi atari8.info> writes:
On Tuesday, 3 November 2015 at 04:10:57 UTC, Manu wrote:
 On 3 November 2015 at 11:07, mattcoder via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 ...
 Does it support Dreamcast? :P
I don't know if you are being serious here but if yes... what's the catch with Dreamcast? I mean this is a Fan thing or what?
I'm just a massive nerd, and the Dreamcast was a great console! :P
By the way, since it's called fuji, I assume it runs on Atari. ;)
Nov 04 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 4 November 2015 at 18:34, Adrian Matoga via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 3 November 2015 at 04:10:57 UTC, Manu wrote:
 On 3 November 2015 at 11:07, mattcoder via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 ...
 Does it support Dreamcast? :P
I don't know if you are being serious here but if yes... what's the catch with Dreamcast? I mean this is a Fan thing or what?
I'm just a massive nerd, and the Dreamcast was a great console! :P
By the way, since it's called fuji, I assume it runs on Atari. ;)
Hah! That's really pushing the limits! I did write some little Atari2600 games once... we even did a 2600 demo comp once ;) Terrible machine!
Nov 04 2015
parent Adrian Matoga <epi atari8.info> writes:
On Wednesday, 4 November 2015 at 09:14:12 UTC, Manu wrote:
 On 4 November 2015 at 18:34, Adrian Matoga via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Tuesday, 3 November 2015 at 04:10:57 UTC, Manu wrote:
 On 3 November 2015 at 11:07, mattcoder via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 02:36:16 UTC, Manu wrote:
 ...
 Does it support Dreamcast? :P
I don't know if you are being serious here but if yes... what's the catch with Dreamcast? I mean this is a Fan thing or what?
I'm just a massive nerd, and the Dreamcast was a great console! :P
By the way, since it's called fuji, I assume it runs on Atari. ;)
Hah! That's really pushing the limits! I did write some little Atari2600 games once... we even did a 2600 demo comp once ;) Terrible machine!
There's a chance I've seen that demo. what's its title? I've tried (with success) compiling C for STe with pre-built gcc, I only gave up porting D because I couldn't get my head around these messy patches the Atari gcc port is full of.
Nov 04 2015
prev sibling next sibling parent Joakim <dlang joakim.fea.st> writes:
On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 If people find this to be an interesting project, and I do take 
 some
 time to do this port, what I'd really love is to work together 
 with
 some of the niche platform support guys and use it to stress 
 test
 toolchains for various platforms. I'd also like help from guys 
 like
 Johannes to make regularly released builds of some console
 cross-compilers available so that it can keep on building.
 The CI of this project for those platforms would more-or-less 
 verify
 that the toolchains and druntime+friends are working.
Sounds great, would make a good stress test for Android and maybe even a base for D games for Android. See my post in the announce forum earlier today about Android progress.
Nov 01 2015
prev sibling next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 1 Nov 2015 11:33:21 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 So, I just wanted to put this idea out there, and see what other
 people make of it.
 
 I have this game engine (https://github.com/TurkeyMan/fuji), it's
 lived for about 12 years now (first commit in 2004, and it existed
 prior before source control). I called it 'Fuji' (a modest, yet
 pleasing and attractive mountain). It supports (or has supported)
 shit-loads of platforms; I'm a game-engine dev for life, and I have a
 fetish for portability, and niche platform support.
 Needless to say, it has had a LOT of time and energy put into it, and
 I would say it's infrastructurally better than most proprietary
 commercial game-engines I've worked with (although there are some
 missing features, I just implement what I need), mainly in that I have
 the luxury to aggressively refactor when design decisions turned out
 to be mistakes, and no deadlines to meet.
 It is a very good example of what we use in real-world AAA gamedev.
 
 It's written in C, obviously. It has comprehensive bindings for D,
 which I'm pretty sure I'm the only one that's ever used, and I use it
 in all my modern D projects.
 
 Anyway. I'm spending less time on hobby game-dev these days, obviously
 I'm not going to make the worlds next big game engine... Hobbyists
 will just use Unity. I'm not quite sure what to do with it.
 
 I have been thinking about full-scale porting to D, and it would serve
 there is a massive-scale long-term codebase, with portability as it's
 primary objective, and I don't know of another project quite like this
 in the D ecosystem? Also, perhaps D gamedevs might be interested in an
 all-D game engine they can use and hack on.
 
 So, when I think on that, I consider what would be lost... and the
 answer is; almost all platforms. But this is something that can be
 addressed.
 
 If people find this to be an interesting project, and I do take some
 time to do this port, what I'd really love is to work together with
 some of the niche platform support guys and use it to stress test
 toolchains for various platforms. I'd also like help from guys like
 Johannes to make regularly released builds of some console
 cross-compilers available so that it can keep on building.
 The CI of this project for those platforms would more-or-less verify
 that the toolchains and druntime+friends are working.
Sounds like a great idea! I can certainly build the GDC toolchains*. But I guess the first thing you'll need if you want to use Dreamcast, NDS and other low memory systems is a nogc druntime port? I had a look at this some time ago and there's some work to be done until we can build a druntime completely without GC. * or better: also provide build scripts. We now use docker.io containers to build the binary GDC toolchains. Building a toolchain with the latest GDC version is as simple as running one docker command: docker run -v ~/shared-dir/:/home/build/shared #results will be saved here -t jpf91/build-gdc /usr/bin/build-gdc build --toolchain=x86_64-w64-mingw32/gcc-5/x86_64-w64-mingw32 It should be easy enough to add support for console toolchain build scripts. Docs: http://build-gdc.readthedocs.org/en/latest/ Sources: https://github.com/D-Programming-GDC/build-gdc https://github.com/D-Programming-GDC/build-gdc-config https://github.com/D-Programming-GDC/build-gdc-docker
Nov 01 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 November 2015 at 04:17, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Sun, 1 Nov 2015 11:33:21 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 If people find this to be an interesting project, and I do take some
 time to do this port, what I'd really love is to work together with
 some of the niche platform support guys and use it to stress test
 toolchains for various platforms. I'd also like help from guys like
 Johannes to make regularly released builds of some console
 cross-compilers available so that it can keep on building.
 The CI of this project for those platforms would more-or-less verify
 that the toolchains and druntime+friends are working.
Sounds like a great idea! I can certainly build the GDC toolchains*. But I guess the first thing you'll need if you want to use Dreamcast, NDS and other low memory systems is a nogc druntime port? I had a look at this some time ago and there's some work to be done until we can build a druntime completely without GC.
Awesome! Your support on this is really encouraging. Almost all consoles are typically targeted by GCC. PS4 uses Clang officially, but I'm sure GCC works just as well. Easiest to start with some of the bigger systems. I think another benefit of this project may be that building for the smaller systems will start to reveal binary size issues, and perhaps we can use it as a vessel to drive some improvements in that area. Binary size is something that's been really irk-ing me for years. What implicitly linked parts of druntime are not nogc? I have no good reason to say that nogc should be an absolute requirement... it is almost certainly preferable to many gamedevs so it would be nice to support it, but GC should be available if the developer wants it... I'm not sure what cost that comes at in terms of binary size. Runtime memory is a matter for the developer, they know the compromise they are making if they opt to use the GC.
 or better: also provide build scripts. We now use docker.io containers
 to build the binary GDC toolchains. Building a toolchain with the latest
 GDC version is as simple as running one docker command:

 docker run
 -v ~/shared-dir/:/home/build/shared #results will be saved here
 -t jpf91/build-gdc /usr/bin/build-gdc build
 --toolchain=x86_64-w64-mingw32/gcc-5/x86_64-w64-mingw32

 It should be easy enough to add support for console toolchain build
 scripts.
Awesome, and that would just be run along with the rest of the CI when producing toolchains? Reliable binary downloads available in known places? Even the presence of these toolchains (whether people use them or not) will inspire some confidence in gamedev's that D means business in gamedev. They are more likely to give D a shot if they know that toolchains are available should ports need to be made at some later stage.
Nov 01 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 2 Nov 2015 14:22:33 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 2 November 2015 at 04:17, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Sounds like a great idea!

 I can certainly build the GDC toolchains*. But I guess the first
 thing you'll need if you want to use Dreamcast, NDS and other low
 memory systems is a  nogc druntime port? I had a look at this some
 time ago and there's some work to be done until we can build a
 druntime completely without GC.  
Awesome! Your support on this is really encouraging. Almost all consoles are typically targeted by GCC. PS4 uses Clang officially, but I'm sure GCC works just as well. Easiest to start with some of the bigger systems. I think another benefit of this project may be that building for the smaller systems will start to reveal binary size issues, and perhaps we can use it as a vessel to drive some improvements in that area. Binary size is something that's been really irk-ing me for years. What implicitly linked parts of druntime are not nogc?
The usual stuff: AA handling, array concatenation etc. It's not a problem as long as you don't use these. But to build a druntime without the GC we'd have to find all these cases and place them in a version(GCAvailable) block. If we build the GC stub or the GC we don't have this problem. Then there's some Exception handling related stuff: IIRC the backtrace data is allocated using the GC and I think core.demangle uses the GC as well. Probably not a big issue. Some higher level stuff such as Threads and Fibers also have special GC integration. And then there are probably some 'toString' methods every so often.
 I have no good reason to say that  nogc should be an absolute
 requirement... it is almost certainly preferable to many gamedevs so
 it would be nice to support it, but GC should be available if the
 developer wants it... I'm not sure what cost that comes at in terms of
 binary size. Runtime memory is a matter for the developer, they know
 the compromise they are making if they opt to use the GC.
 
Maybe I'm a little to pessimistic about the GC ;-) I'm just not sure if it can deal with low-memory situations such as if you've only got 16MB memory total. And IIRC the GC always preallocates some blocks. So even if we never use the GC it might steal some memory. OTOH it would likely be not much work to adjust this allocation size.
 
 or better: also provide build scripts. We now use docker.io
 containers to build the binary GDC toolchains. Building a toolchain
 with the latest GDC version is as simple as running one docker
 command:

 docker run
 -v ~/shared-dir/:/home/build/shared #results will be saved here
 -t jpf91/build-gdc /usr/bin/build-gdc build
 --toolchain=x86_64-w64-mingw32/gcc-5/x86_64-w64-mingw32

 It should be easy enough to add support for console toolchain build
 scripts.  
Awesome, and that would just be run along with the rest of the CI when producing toolchains? Reliable binary downloads available in known places?
These toolchains would be built like all other toolchains on gdcproject.org/downloads and would also be listed on that page. We currently do not have a CI setup for all these toolchains. Building the windows variant of a toolchain can take up to an hour (on my i5 notebook). Building all combinations for every commit on CI would be very costly. I hope to integrate all linux hosted toolchains with Travis-CI (as a bonus we can probably extract nightly builds). It's a bit of a PITA as building requires quite some disk space (10GB+ required, semaphoreCI offers only 5GB). And I have to avoid these stupid 10 minute command timeout on travis-ci. Let's see if this works: https://travis-ci.org/jpf91/GDC/builds/88839377 (seems to be quite slow though...) OTOH even if we manage to integrate CI this tests only that the code builds. I don't think runtime testing is possible in some easy way.
 Even the presence of these toolchains (whether people use them
 or not) will inspire some confidence in gamedev's that D means
 business in gamedev. They are more likely to give D a shot if they
 know that toolchains are available should ports need to be made at
 some later stage.
Nov 02 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 November 2015 at 04:38, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Mon, 2 Nov 2015 14:22:33 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 Maybe I'm a little to pessimistic about the GC ;-) I'm just not sure if
 it can deal with low-memory situations such as if you've only got 16MB
 memory total. And IIRC the GC always preallocates some blocks. So even
 if we never use the GC it might steal some memory. OTOH it would
 likely be not much work to adjust this allocation size.
Perhaps the pre-allocation could become an API which the user might prefer not to call?
 Let's see if this works:
 https://travis-ci.org/jpf91/GDC/builds/88839377
 (seems to be quite slow though...)

 OTOH even if we manage to integrate CI this tests only that the code
 builds. I don't think runtime testing is possible in some easy way.
I have a samples directory which it would be theoretically possible to run and see that they don't crash as part of a test run. Also, I'd like to annotate my whole engine quite comprehensively with unittests. It's something that I'm keen to work on, and then it further helps to assure those toolchains remain working.
Nov 02 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 3 Nov 2015 09:16:47 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically possible to
 run and see that they don't crash as part of a test run. Also, I'd
 like to annotate my whole engine quite comprehensively with unittests.
 It's something that I'm keen to work on, and then it further helps to
 assure those toolchains remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
Nov 02 2015
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 November 2015 at 07:30:44 UTC, Johannes Pfau wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically 
 possible to run and see that they don't crash as part of a 
 test run. Also, I'd like to annotate my whole engine quite 
 comprehensively with unittests. It's something that I'm keen 
 to work on, and then it further helps to assure those 
 toolchains remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
There's a Dreamcast emulator for Android/ARM: https://github.com/reicast/reicast-emulator You could run it inside the Android emulator on Travis: :) http://docs.travis-ci.com/user/languages/android/ I'm sure their servers can handle an emulator of a 200 MHz MIPS core with 16 MB of RAM running inside an ARM emulator. ;)
Nov 03 2015
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 3 November 2015 at 08:36:46 UTC, Joakim wrote:
 There's a Dreamcast emulator for Android/ARM:

 https://github.com/reicast/reicast-emulator

 You could run it inside the Android emulator on Travis: :)

 http://docs.travis-ci.com/user/languages/android/

 I'm sure their servers can handle an emulator of a 200 MHz MIPS 
 core with 16 MB of RAM running inside an ARM emulator. ;)
Whoops, it runs on linux/x64 also, no need for the Android intermediary.
Nov 03 2015
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 November 2015 at 18:36, Joakim via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 3 November 2015 at 07:30:44 UTC, Johannes Pfau wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically possible to
 run and see that they don't crash as part of a test run. Also, I'd like to
 annotate my whole engine quite comprehensively with unittests. It's
 something that I'm keen to work on, and then it further helps to assure
 those toolchains remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
There's a Dreamcast emulator for Android/ARM: https://github.com/reicast/reicast-emulator You could run it inside the Android emulator on Travis: :) http://docs.travis-ci.com/user/languages/android/ I'm sure their servers can handle an emulator of a 200 MHz MIPS core with 16 MB of RAM running inside an ARM emulator. ;)
For the record, I was mostly joking about Dreamcast ;) ... while I did support it actively some years back, I haven't built that code in a while. It would be a lot of fun to get it working again though :P Incidentally, there's a GCC dev that's been committing SIMD optimisations for the SH4 (Dreamcast) backend recently. He's obviously having some fun making all the vector intrinsics work with the DC hardware. Latest GCC is the best Dreamcast compiler we've ever had!
Nov 03 2015
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 4 Nov 2015 09:40:42 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 3 November 2015 at 18:36, Joakim via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Tuesday, 3 November 2015 at 07:30:44 UTC, Johannes Pfau wrote:  
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:
  
 I have a samples directory which it would be theoretically
 possible to run and see that they don't crash as part of a test
 run. Also, I'd like to annotate my whole engine quite
 comprehensively with unittests. It's something that I'm keen to
 work on, and then it further helps to assure those toolchains
 remain working.  
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
There's a Dreamcast emulator for Android/ARM: https://github.com/reicast/reicast-emulator You could run it inside the Android emulator on Travis: :) http://docs.travis-ci.com/user/languages/android/ I'm sure their servers can handle an emulator of a 200 MHz MIPS core with 16 MB of RAM running inside an ARM emulator. ;)
For the record, I was mostly joking about Dreamcast ;) ... while I did support it actively some years back, I haven't built that code in a while. It would be a lot of fun to get it working again though :P Incidentally, there's a GCC dev that's been committing SIMD optimisations for the SH4 (Dreamcast) backend recently. He's obviously having some fun making all the vector intrinsics work with the DC hardware. Latest GCC is the best Dreamcast compiler we've ever had!
The Dreamcast is SH4 based? IIRC there are some regressions in recent GCC builds for SH4 but it's being resurrected as the Open Processor Foundation now develops open source SH processors: http://lwn.net/Articles/647636/ http://0pf.org/about-ocf.html
Nov 04 2015
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 4 November 2015 at 00:40, Manu via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 3 November 2015 at 18:36, Joakim via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Tuesday, 3 November 2015 at 07:30:44 UTC, Johannes Pfau wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically possible to
 run and see that they don't crash as part of a test run. Also, I'd
like to
 annotate my whole engine quite comprehensively with unittests. It's
 something that I'm keen to work on, and then it further helps to assure
 those toolchains remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed).
If
 you're talking about running tests on the x86_64 architecture that
should be
 easy.
There's a Dreamcast emulator for Android/ARM: https://github.com/reicast/reicast-emulator You could run it inside the Android emulator on Travis: :) http://docs.travis-ci.com/user/languages/android/ I'm sure their servers can handle an emulator of a 200 MHz MIPS core
with 16
 MB of RAM running inside an ARM emulator. ;)
For the record, I was mostly joking about Dreamcast ;) ... while I did support it actively some years back, I haven't built that code in a while. It would be a lot of fun to get it working again though :P Incidentally, there's a GCC dev that's been committing SIMD optimisations for the SH4 (Dreamcast) backend recently. He's obviously having some fun making all the vector intrinsics work with the DC hardware. Latest GCC is the best Dreamcast compiler we've ever had!
I'm aware of this, not because I take an interest, but because I was cc'd into discussion when they discovered a C++ regression that was seen by comparing the md5sum of (D frontend) interpret.c sources between 2nd and 3rd generation bootstrapped builds. ;-) If you don't understand why, that's because this file contains all frontend const folding routines for every operation on every basic type supported in D, and then some. So itself becomes a good stress test of a compiler's codegen ability (or in this case, the ability to produce consistent code).
Nov 04 2015
parent Daniel Murphy <yebbliesnospam gmail.com> writes:
On 4/11/2015 7:09 PM, Iain Buclaw via Digitalmars-d wrote:
 I'm aware of this, not because I take an interest, but because I was cc'd
 into discussion when they discovered a C++ regression that was seen by
 comparing the md5sum of (D frontend) interpret.c sources between 2nd and
 3rd generation bootstrapped builds.  ;-)
That's awesome!
Nov 20 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 November 2015 at 17:30, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically possible to
 run and see that they don't crash as part of a test run. Also, I'd
 like to annotate my whole engine quite comprehensively with unittests.
 It's something that I'm keen to work on, and then it further helps to
 assure those toolchains remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
Emulators are the typical approach. Even Android + iOS have emulators that can be used for CI. Most emulators have such features required to return runtime errors back to the host environment for this sort of testing.
Nov 03 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 4 Nov 2015 09:37:46 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 3 November 2015 at 17:30, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:
  
 I have a samples directory which it would be theoretically
 possible to run and see that they don't crash as part of a test
 run. Also, I'd like to annotate my whole engine quite
 comprehensively with unittests. It's something that I'm keen to
 work on, and then it further helps to assure those toolchains
 remain working.  
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
Emulators are the typical approach. Even Android + iOS have emulators that can be used for CI. Most emulators have such features required to return runtime errors back to the host environment for this sort of testing.
OK, that should work. I think the main problem is still the resource usage when building GCC cross compilers. On Travis-CI we hit the 50 Minute execution limit, on SemaphoreCI we don't have enough temporary disk space. CircleCI seems to work but I'm not sure if the "build minutes per month" limit applies to OSS projects. (Using prebuilt GDC toolchains to do CI for _fuji_ should work fine). I've added support for devkitPro to our build scripts (I already knew how to build these compilers so this was easy). We can build devkitARM (GBA, DS, 3DS, N3DS) and devkitPPC (GC, Wii, WiiU) compilers. devkitPSP can't be supported as it uses GCC 4.6. The oldest supported GCC version for GDC is 4.8. Here is an build of all these toolchains running on CircleCI: https://circleci.com/gh/jpf91/GDC/11 In theory this builds ready to use packages and could deploy these to gdcproject.org as nightly builds. If you want to build manually, use docker.io: docker run -v ~/result-dir:/home/build/shared -t jpf91/build-gdc /usr/bin/build-gdc build --toolchain=i686-w64-mingw32/gcc-4.8/devkitPPC --toolchain=i686-w64-mingw32/gcc-4.9/devkitARM I've also uploaded initial toolchain builds here (untested): ftp://ftp.gdcproject.org/binaries/devkitpro/ I still need to integrate devkitPro builds into the download page, write some documentation and think about a directory layout which isn't confusing when we have multiple devkitpro releases. I'll probably have a look at this next weekend.
Nov 04 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 November 2015 at 06:31, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Wed, 4 Nov 2015 09:37:46 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 3 November 2015 at 17:30, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Am Tue, 3 Nov 2015 09:16:47 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 I have a samples directory which it would be theoretically
 possible to run and see that they don't crash as part of a test
 run. Also, I'd like to annotate my whole engine quite
 comprehensively with unittests. It's something that I'm keen to
 work on, and then it further helps to assure those toolchains
 remain working.
But how exactly would you run these? All CI machines are x86_64. I guess emulators could be a possibility as long as they run headless. We'd need some way to get feedback from the emulator though (test passed/failed). If you're talking about running tests on the x86_64 architecture that should be easy.
Emulators are the typical approach. Even Android + iOS have emulators that can be used for CI. Most emulators have such features required to return runtime errors back to the host environment for this sort of testing.
OK, that should work. I think the main problem is still the resource usage when building GCC cross compilers. On Travis-CI we hit the 50 Minute execution limit, on SemaphoreCI we don't have enough temporary disk space. CircleCI seems to work but I'm not sure if the "build minutes per month" limit applies to OSS projects. (Using prebuilt GDC toolchains to do CI for _fuji_ should work fine).
Travis-CI lets you install things prior to the build, this should work well assuming they cache these sorts of downloads ;)
 devkitPSP can't be supported as it uses GCC 4.6. The oldest supported GCC
 version for GDC  is 4.8.
Aww! It's one of the platforms I actively support, and works well. It's also the only MIPS platform, which is a good test bed. Do you know what's holding it back? Is it possible it may be updated, or has it just been left to die? I miss the PS2, but it hasn't been maintained at all :(
 I've also uploaded initial toolchain builds here (untested):
 ftp://ftp.gdcproject.org/binaries/devkitpro/
Awesome, downloading now...
 I still need to integrate devkitPro builds into the download page,
 write some documentation and think about a directory layout which isn't
 confusing when we have multiple devkitpro releases. I'll probably have
 a look at this next weekend.
This is great! Thanks! Is there instructions anywhere for attempting to build druntime with these toolchains so I can have a crack at filling in the gaps there?
Nov 05 2015
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 5 Nov 2015 21:37:50 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 5 November 2015 at 06:31, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 devkitPSP can't be supported as it uses GCC 4.6. The oldest
 supported GCC version for GDC  is 4.8.  
Aww! It's one of the platforms I actively support, and works well. It's also the only MIPS platform, which is a good test bed. Do you know what's holding it back? Is it possible it may be updated, or has it just been left to die?
The main problem is that these toolchains use some GCC patches. These need to be ported to newer GCC releases. This is usually the only problem. I'm not sure if the PSP toolchain is still supported. It seems like the sources weren't modified in the last 4 years. All other PSP buildscripts seem to use the same GCC 4.6 patches, so there's not much we can do. You could probably ask for some more information in the devkitPro forum.
 I miss the PS2, but it hasn't been maintained at all :(
 
 I've also uploaded initial toolchain builds here (untested):
 ftp://ftp.gdcproject.org/binaries/devkitpro/  
Awesome, downloading now...
 I still need to integrate devkitPro builds into the download page,
 write some documentation and think about a directory layout which
 isn't confusing when we have multiple devkitpro releases. I'll
 probably have a look at this next weekend.  
This is great! Thanks! Is there instructions anywhere for attempting to build druntime with these toolchains so I can have a crack at filling in the gaps there?
There's some very basic information here: http://wiki.dlang.org/GDC/Installation/Generic#Building_only_Druntime.2FPhobos Only tested on linux though. It should work on windows if you use a msys2 installation and install the base-devel package. Note that you need the GCC sources as well (unpatched is OK). These won't be compiled but the druntime Makefile/configure scripts depend on some macros from these sources. It might be clever to use the same druntime and GCC sources that were used when building these compilers. This means GCC4.8 + gdc-4.8 branch for devkitPPC and GCC4.9 + gdc-4.9 branch for devkitARM. (I'm not sure if you have to pass some additional devkitPro specifc flags to GCC? If think these could be set using the LDFLAGS/DFLAGS environment variable, but I'm not sure)
Nov 05 2015
parent Johannes Pfau <nospam example.com> writes:
Am Thu, 5 Nov 2015 14:32:47 +0100
schrieb Johannes Pfau <nospam example.com>:

 This is great! Thanks!
 Is there instructions anywhere for attempting to build druntime with
 these toolchains so I can have a crack at filling in the gaps
 there?  
There's some very basic information here: http://wiki.dlang.org/GDC/Installation/Generic#Building_only_Druntime.2FPhobos Only tested on linux though. It should work on windows if you use a msys2 installation and install the base-devel package.
Have you tried this already? For some reason configure seems to pick up the wrong gdc if I simply do: ./../gcc-4.9.2/libphobos/configure --prefix=/opt/gdc --host=arm-none-eabi I'll probably have to debug this, but here's a workaround: GDC=arm-none-eabi-gdc ../../gcc-4.9.2/libphobos/configure --prefix=/opt/gdc --host=arm-none-eabi Of course compilling druntime fails very early ;-)
Nov 09 2015
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 5 November 2015 at 11:37:59 UTC, Manu wrote:
 Is there instructions anywhere for attempting to build druntime 
 with
 these toolchains so I can have a crack at filling in the gaps 
 there?
Maybe not what you were asking, but if you want an idea of what needs to be added to druntime when porting to a new OS, take a look at the first 3-4 Android/x86 PRs from a couple years ago: https://github.com/D-Programming-Language/druntime/pulls?q=is%3Apr+android+is%3Aclosed+sort%3Acreated-asc Of course, you'll also need to add architecture-specific declarations, you can find those by searching for an existing arch, like "version(ARM)", and modifying the 19 files which refer to it already.
Nov 05 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 November 2015 at 21:37, Manu <turkeyman gmail.com> wrote:
 On 5 November 2015 at 06:31, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 I've also uploaded initial toolchain builds here (untested):
 ftp://ftp.gdcproject.org/binaries/devkitpro/
Awesome, downloading now...
I see these archives are minimal, only the gdc binaries, but there is also a gcc binary... why is that? The filesize is very different than the ones bundled in devkitPro; is it intended that arm-none-eabi-gcc.exe (and friends) overwrite the ones in the devkitPro distro?
Nov 05 2015
parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 5 Nov 2015 21:59:39 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 5 November 2015 at 21:37, Manu <turkeyman gmail.com> wrote:
 On 5 November 2015 at 06:31, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:  
 I've also uploaded initial toolchain builds here (untested):
 ftp://ftp.gdcproject.org/binaries/devkitpro/  
Awesome, downloading now...
I see these archives are minimal, only the gdc binaries, but there is also a gcc binary... why is that? The filesize is very different than the ones bundled in devkitPro; is it intended that arm-none-eabi-gcc.exe (and friends) overwrite the ones in the devkitPro distro?
That's the reason I still need to write a README ;-) I'm not sure why the file size could be very different, I'll have a look at that later. But yes, the idea is to overwrite the *gcc.exe files. In theory it doesn't really matter, but it's necessary to make the gcc command work with d files (arm-none-eabi-gcc test.d). If you always use arm-none-eabi-gdc you won't need to overwrite the gcc files. IIRC such a setup (gcc/gdc from different builds, this setup is also used in ArchLinux) can cause problems with LTO. As long as GDC doesn't support LTO this doesn't matter. We can always start shipping complete GCC toolchains but I'd like to avoid building GDB and all the devkitARM utils (these have some dependencies such as libusb which make building more difficult). So in that case you'd have to overwrite many more files. Maybe I can make full builds work as well, but this will also increase build times a lot.
Nov 05 2015
next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 November 2015 at 23:42, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Thu, 5 Nov 2015 21:59:39 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 5 November 2015 at 21:37, Manu <turkeyman gmail.com> wrote:
 On 5 November 2015 at 06:31, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 I've also uploaded initial toolchain builds here (untested):
 ftp://ftp.gdcproject.org/binaries/devkitpro/
Awesome, downloading now...
I see these archives are minimal, only the gdc binaries, but there is also a gcc binary... why is that? The filesize is very different than the ones bundled in devkitPro; is it intended that arm-none-eabi-gcc.exe (and friends) overwrite the ones in the devkitPro distro?
That's the reason I still need to write a README ;-) I'm not sure why the file size could be very different, I'll have a look at that later. But yes, the idea is to overwrite the *gcc.exe files. In theory it doesn't really matter, but it's necessary to make the gcc command work with d files (arm-none-eabi-gcc test.d). If you always use arm-none-eabi-gdc you won't need to overwrite the gcc files.
Right, this is exactly what I suspected.
 We can always start shipping complete GCC toolchains but I'd like to
 avoid building GDB and all the devkitARM utils (these have some
 dependencies such as libusb which make building more difficult). So in
 that case you'd have to overwrite many more files. Maybe I can make
 full builds work as well, but this will also increase build times a lot.
Cool, I think this should be fine for now.
Nov 05 2015
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 November 2015 at 23:42, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Thu, 5 Nov 2015 21:59:39 +1000

 We can always start shipping complete GCC toolchains but I'd like to
 avoid building GDB and all the devkitARM utils (these have some
 dependencies such as libusb which make building more difficult). So in
 that case you'd have to overwrite many more files. Maybe I can make
 full builds work as well, but this will also increase build times a lot.
Will not building GDB mean that the regular devkitPro GDB has no knowledge of D symbol demangling? I thought that support was included in GCC proper some time back?
Nov 05 2015
parent Johannes Pfau <nospam example.com> writes:
Am Fri, 6 Nov 2015 00:25:00 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 5 November 2015 at 23:42, Johannes Pfau via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Am Thu, 5 Nov 2015 21:59:39 +1000

 We can always start shipping complete GCC toolchains but I'd like to
 avoid building GDB and all the devkitARM utils (these have some
 dependencies such as libusb which make building more difficult). So
 in that case you'd have to overwrite many more files. Maybe I can
 make full builds work as well, but this will also increase build
 times a lot.  
Will not building GDB mean that the regular devkitPro GDB has no knowledge of D symbol demangling? I thought that support was included in GCC proper some time back?
IIRC the D support in GDB is always enabled. So the standard devkitPro builds should work. The GDB version is more interesting. 7.8 was the first version with extended D support and we're now at 7.10. devkitARM uses 7.8, devkitPPC 7.7. The devkitPro patches for gdb are trivial and only modify the gdb build scripts. It should be easy to port these patches and compile newer gdb versions if necessary. BTW: If you find some GDB with support for the arm-none-eabi and powerpc-eabi targets these should work as well. The devkitPro GDBs are unmodified standard GDB builds. Other builds might even provide some additional features (python scripting, ...) I can only highly recommend installing ArchLinux ;-) Then you can simply use this for devkitARM: https://www.archlinux.org/packages/community/x86_64/arm-none-eabi-gdb/ And building a powerpc-eabi GDB should be simple as well.
Nov 05 2015
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 5 Nov 2015 14:42:52 +0100
schrieb Johannes Pfau <nospam example.com>:

 I see these archives are minimal, only the gdc binaries, but there
 is also a gcc binary... why is that?
 The filesize is very different than the ones bundled in devkitPro;
 is it intended that arm-none-eabi-gcc.exe (and friends) overwrite
 the ones in the devkitPro distro?  
That's the reason I still need to write a README ;-) I'm not sure why the file size could be very different, I'll have a look at that later.
I've uploaded some new binaries. Changes: * Changed the download paths to to their final values: ftp://ftp.gdcproject.org/binaries/devkitARM/r44/ ftp://ftp.gdcproject.org/binaries/devkitPPC/r27/ * Integration into the homepage is finished and looks like this: http://www.pixhoster.info/f/2015-11/c2b4e71c57df7c718d6ab0f134d957b3.png I'm waiting for some feedback from the devkitpro maintainers as devkitPro/ARM/PPC are apparently trademarked. * The packages now contain README files * The packages now additionally contain builds of gdb 7.10 * The -gcc files have been removed for now * gdc -v now prints the DMD FE version and the used GDC commit * bugurl was changed to bugzilla.gdcproject.org I hope these were the last toolchain/build-setup related changes. Building toolchains with updated GDC versions is now trivial.
Nov 10 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
Thanks Johannes! I tried a couple of builds with the prev binaries,
but haven't been able to get to this stuff in any depth yet; I've been
doing really long hours at work. I've had about half an hour a day to
myself lately. Spread way too thin >_<
Hopefully it calms down a bit soon, I'm really keen to dive in to this stuff.
Thanks for setting this stuff up so quick! It's nice that these are
getting regular builds; hopefully they make it much easier for people
to dive in and get moving.

On 11 November 2015 at 03:51, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Thu, 5 Nov 2015 14:42:52 +0100
 schrieb Johannes Pfau <nospam example.com>:

 I see these archives are minimal, only the gdc binaries, but there
 is also a gcc binary... why is that?
 The filesize is very different than the ones bundled in devkitPro;
 is it intended that arm-none-eabi-gcc.exe (and friends) overwrite
 the ones in the devkitPro distro?
That's the reason I still need to write a README ;-) I'm not sure why the file size could be very different, I'll have a look at that later.
I've uploaded some new binaries. Changes: * Changed the download paths to to their final values: ftp://ftp.gdcproject.org/binaries/devkitARM/r44/ ftp://ftp.gdcproject.org/binaries/devkitPPC/r27/ * Integration into the homepage is finished and looks like this: http://www.pixhoster.info/f/2015-11/c2b4e71c57df7c718d6ab0f134d957b3.png I'm waiting for some feedback from the devkitpro maintainers as devkitPro/ARM/PPC are apparently trademarked. * The packages now contain README files * The packages now additionally contain builds of gdb 7.10 * The -gcc files have been removed for now * gdc -v now prints the DMD FE version and the used GDC commit * bugurl was changed to bugzilla.gdcproject.org I hope these were the last toolchain/build-setup related changes. Building toolchains with updated GDC versions is now trivial.
Nov 11 2015
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 11 Nov 2015 21:32:54 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 Thanks Johannes! I tried a couple of builds with the prev binaries,
 but haven't been able to get to this stuff in any depth yet; I've been
 doing really long hours at work. I've had about half an hour a day to
 myself lately. Spread way too thin >_<
 Hopefully it calms down a bit soon, I'm really keen to dive in to
 this stuff.
No need to hurry. It's just the opposite for me: Since I finished my bachelor thesis two weeks ago I've got much more free time at my disposal. And there are still more than enough GDC related projects to be done: Proper cross-compiler support for the test suite, proper MinGW support, shared libraries :-) BTW: One of the first problems you might encounter is missing version(*) support. AFAICS we can not easily set version(Gamecube) or version(Wii) from GDC. For now I recommend simply setting versions manually with -fversion. But this also means you always have to pass -fversion when compilling druntime and even when compilling user applications or libraries ! I'd propose version(CRuntime_Newlib) for C-library related stuff and a newlibVersion variable in core.[stdc.]config, version(DevkitPro) for the vendor and version(Gamecube, Wii, WiiU, GBA, DS, DSi, 3DS) for the system. Usage would be like this: version CRuntime_Newlib { //Newlib only supports this function with version 2.0 static if(newlibVersion > NewlibVersion(2, 0)) { extern(C) void foo(); } } else //Ansi C { extern(C) void foo(); } version(Wii) { version(DevkitPro) else version(Nintendo) } or version(DevkitPro) { version(Gamecube) else version(Wii) }
 Thanks for setting this stuff up so quick! It's nice that
 these are getting regular builds; hopefully they make it much easier
 for people to dive in and get moving.
I actually wanted to add support for these toolchains for some time now. Your post motivated me to finally do it ;-)
Nov 11 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-11-01 02:33, Manu via Digitalmars-d wrote:

 I have been thinking about full-scale porting to D
You could enhance DStep [1] to handle complete source code and not only headers. [1] https://github.com/jacob-carlborg/dstep -- /Jacob Carlborg
Nov 01 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 November 2015 at 05:27, Jacob Carlborg via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 2015-11-01 02:33, Manu via Digitalmars-d wrote:

 I have been thinking about full-scale porting to D
You could enhance DStep [1] to handle complete source code and not only headers.
I'll give it a shot... see how close it gets me. It's almost C code (C with light internal use of classes), so it should port really easily.
Nov 01 2015
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-11-02 05:24, Manu via Digitalmars-d wrote:

 I'll give it a shot... see how close it gets me.
 It's almost C code (C with light internal use of classes), so it
 should port really easily.
DStep currently only supports C code, not almost C code :). -- /Jacob Carlborg
Nov 01 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 2 November 2015 at 17:55, Jacob Carlborg via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 2015-11-02 05:24, Manu via Digitalmars-d wrote:

 I'll give it a shot... see how close it gets me.
 It's almost C code (C with light internal use of classes), so it
 should port really easily.
DStep currently only supports C code, not almost C code :).
It'll probably get through 95% of the files... actually, I use enum aswell, is that supported?
Nov 02 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-11-02 09:47, Manu via Digitalmars-d wrote:

 It'll probably get through 95% of the files... actually, I use enum
 aswell, is that supported?
Yes. You can get a rough idea of what's supported by looking in the "test_files" directory. -- /Jacob Carlborg
Nov 02 2015
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Mon, 2 Nov 2015 14:24:18 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:

 On 2 November 2015 at 05:27, Jacob Carlborg via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 2015-11-01 02:33, Manu via Digitalmars-d wrote:
  
 I have been thinking about full-scale porting to D  
You could enhance DStep [1] to handle complete source code and not only headers.
I'll give it a shot... see how close it gets me. It's almost C code (C with light internal use of classes), so it should port really easily.
And if dstep doesn't work maybe the DDMD magicport tool could be useful.
Nov 02 2015
prev sibling parent reply Poyeyo <poyeyo arcadechaser.com> writes:
On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 So, I just wanted to put this idea out there, and see what 
 other people make of it.

 [...]
I'm interested in learning what the commercial license is. Right now it is unclear and secretive, in contrast with Unity and Unreal engines.
Nov 29 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 30 November 2015 at 00:54, Poyeyo via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 1 November 2015 at 01:33:29 UTC, Manu wrote:
 So, I just wanted to put this idea out there, and see what other people
 make of it.

 [...]
I'm interested in learning what the commercial license is. Right now it is unclear and secretive, in contrast with Unity and Unreal engines.
Well, nobody has ever asked about it before. I wrote that because I wanted to reserve the right to grant permission to a developer wanting to use it commercially. Permission would almost certainly be granted, free of charge for a certain class of project (small-medium scale dev). But the reason I put it there was to avoid any situation where someone used my code, made something insanely successful, and gave the original developer nothing. Angry Birds is the prime example, the guy who wrote Box2D was 90% responsible for Angry Birds; which I expect is no more than 200 lines of code on top of Box2D and a bit of art. Rovio made *billions* of dollars, and never gave him a cent. I'm all for commercial use, but I want to know about commercial projects, grant permission, and/or agree on a reasonable cut if someone becomes a billionaire off of my work. Most likely case is that I'd just be a valuable resource of free dev support ;)
Nov 30 2015