digitalmars.D.announce - Hipreme Engine is fully ported to WebAssembly
- Hipreme (92/92) Feb 03 2023 This has been finished for quite a time but I was polishing some
- ryuukk_ (11/11) Feb 03 2023 Congrats!
- Mike Shah (4/7) Feb 03 2023 Very cool, thanks for sharing and congratulations on this
- Guillaume Piolat (2/4) Feb 04 2023 This custom runtime is a most welcome development.
- ryuukk_ (4/8) Feb 04 2023 I agree, but this is also unfortunate, one shouldn't have to do
- Hipreme (15/24) Feb 04 2023 I would like to say that with my experience doing this kind of
- Johan (10/21) Feb 04 2023 Can't you just `version`/`static if` out all the things in
- Hipreme (37/58) Feb 05 2023 Exactly because Sebastian did that and that never got upstreamed
- Richard (Rikki) Andrew Cattermole (3/5) Feb 05 2023 It never got upstreamed because it was never attempted.
- Dennis (3/5) Feb 05 2023 It's on LDC: https://github.com/ldc-developers/ldc/pull/3345
- H. S. Teoh (14/34) Feb 07 2023 This is awesome! Now I definitely have to look into this when I get
This has been finished for quite a time but I was polishing some features. I'll save this post to 4 things: **Advertise that [Hipreme Engine](https://github.com/MrcSnm/HipremeEngine) is now supporting 5 platforms**, being those: - Xbox Series - Windows - Linux - Android - Browser **1. D development workflow for web**: - The server: The server being employed is **arsd:cgi**, made by Adam D. Ruppe. CGI.d doesn't bring much bloat as vibe.d, specially for its requirements, it was chosen over python for not needing to install anything beyond `dub`. - Debugging: The debug is currently done by using another dub project: **wasm-sourcemaps**, made by Sebastian Koppe. It currently works really well and now it is correctly configured with D. Using the sourcemaps, we can get strack traces when assertions fails, when you access a null pointer, or when some exception is thrown. - Additional tooling? My engine employed some additional tooling for working with filesystem directories as it would greatly simplify the user experience, it is a simple code which generates a JSON representation of your game assets folder. - "Can my PC without any configuration build it?": Yes. Actually the browser is the less strict platform as you don't need any specific SDK. If a full runtime is done one day, it will be one of the easiest platforms to support along windows and linux. **2. D performance on web**: Incredible. This game runs with 5% CPU usage on web, and it still has many optimization opportunities that I didn't run for, such as: - I'm using WebGL 1.0 instead of WebGL 2. You can ask me "but why": more support. The engine itself is prepared enough to support WebGL 2 out of the box, and it could increase the performance by another set of batched calls on the GL site. - What is the slowest part of D? The JS bridge. The bloat is all over there, executing code from the bridge is by a magnitude of 10x slower than a single side code. Thankfully my engine has already dealt with that by every OpenGL call being batched. - Is there collection happening? Nope. The custom runtime in an effort to extend Arsd Webassembly doesn't implements the collection aspect. Although that happens, Hipreme Engine doesn't allocate anything inside the game loop. Which means 0 allocations is actually happening during the game. Although there is still the GC calls made by the game you're doing. But futurely the engine will support scene graphs which can greatly enhance the engine memory control. That means any simple game can be built on it and there are some ways to implement a temporary solution against any kind out of memory for bigger games such as doing timely game saves into browser's local storage and game reloading after some time. **3. How it was to port the game engine**: - If you're checking those errors on the right of the console, they were implemented before web was a thing for my engine. They were using synchronous functions and checking their return value. For Web, my take wasn't using D decoding libraries such as audioformats or arsd:image. For reducing bloat and porting code need, I'm using directly the browser decoding capabilities, which means it is impossible to use sync image decoding with my engine on web. - "How did you solve that then"? I sent to JS code D delegates which would be executed when the web promise was fulfilled. The way I done async loading in my engine was via Threads executing sync code and then returning. So, the user facing API basically didn't change anything, only the engine internals. - "Will I need to change my code between platforms?": Depends. It depends on your game assets loading strategy. The `await` for loading asset functionality does not work on web and it will fail on an assertion. But the game that I show here, uses the same code to run on all platforms. By using the AssetLoadStrategy.loadAll. The Game Engine will use reflection on all modules used by your game to know which assets needs to be loaded before starting the game's entry point. So, you won't need to worry about anything. **4. I want to make a web project without Hipreme Engine, what its custom runtime does not support?**: - Collection - Try/Catch: partial support. Undefined behavior if a throw code is being catched. - static this/~this Excluding static this (which I found no usage within my project), those 2 features are the hardest to support so I didn't get this far. Try/Catch is not being fully supported, but you don't need to remove that syntax from your code. Which means the entire D is being supported to build a project! Here's a sample image of the latest game done by Hipreme Engine running on web: 
Feb 03 2023
ryuukk_ <ryuukk.dev gmail.com> Congrats! This shows D capabilities, system language that is able to scale from as little as micro controllers to full blown cross platform games including the Web with WASM! That's why it's always important to keep runtime as simple as possible, and the std as minimal as possible, lets you target new platforms with ease! I also ported my engine to WASM, i'll have something interesting to show in the coming months, stay tuned! D playing nice with WASM, that's a strength not many languages have! Let's do more with it!
Feb 03 2023
On Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:This has been finished for quite a time but I was polishing some features. [...]Very cool, thanks for sharing and congratulations on this achievement! Would be curious to learn more about the challenges in supporting Mobile, Desktop, and XBox and now the web.
Feb 03 2023
On Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:This custom runtime is a most welcome development.
Feb 04 2023
On Saturday, 4 February 2023 at 12:47:30 UTC, Guillaume Piolat wrote:On Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:I agree, but this is also unfortunate, one shouldn't have to do that to target WASM, the druntime needs to be shrinked asapThis custom runtime is a most welcome development.
Feb 04 2023
On Saturday, 4 February 2023 at 19:04:19 UTC, ryuukk_ wrote:On Saturday, 4 February 2023 at 12:47:30 UTC, Guillaume Piolat wrote:I would like to say that with my experience doing this kind of work, it is a matter of: 1. Adding runtime hook functions for doing memory free and allocation. This way, it won't depends on runtime having implemented support for every OS/Arch. The same could be applied to other more complex fields, by looking into our GC code, it doesn't even need threading (which could be also optional) support for being executed. 2. I implemented almost entire support for the druntime for like nearly all use cases D code have with 10% of the druntime code. Maybe 90% of the other things are really hidden away? I don't know but there is an unseen part the runtime which I don't really understand. If the entire runtime don't hard depend on libc, it would be hugeOn Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:I agree, but this is also unfortunate, one shouldn't have to do that to target WASM, the druntime needs to be shrinked asapThis custom runtime is a most welcome development.
Feb 04 2023
On Saturday, 4 February 2023 at 20:03:04 UTC, Hipreme wrote:On Saturday, 4 February 2023 at 19:04:19 UTC, ryuukk_ wrote:Can't you just `version`/`static if` out all the things in druntime that WASM does not need? It is one of the strengths of D to be able to do that quite easily; shrinking the druntime is just a matter of writing `version() {}`. It'd be nice if you can spend some time in making standard druntime more compatible with WASM. Sebastian has worked on it, but it was never upstreamed. Just disable most/all, and only enable things that do work. We can always enable more / fix more later. -JohanOn Saturday, 4 February 2023 at 12:47:30 UTC, Guillaume Piolat wrote:On Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:I agree, but this is also unfortunate, one shouldn't have to do that to target WASM, the druntime needs to be shrinked asapThis custom runtime is a most welcome development.
Feb 04 2023
On Saturday, 4 February 2023 at 22:27:31 UTC, Johan wrote:On Saturday, 4 February 2023 at 20:03:04 UTC, Hipreme wrote:Exactly because Sebastian did that and that never got upstreamed that I didn't work on it. It would be pretty much plenty of time lost for nothing. I wasn't able to test his druntime, never worked on my PC. Working with a custom runtime is a lot easier to setup, build and run and that is what's important for me. You had to setup plenty of other things by using the standard runtime such as having cmake, make (which almost never works on windows), you need to know the correct build command, guarantee the version needed, then you needed to try and build WASI (which I always failed at that part) to be linked with the runtime. The custom runtime is just include and go. The custom runtime was literally taking of code from upstream druntime and removing parts that were related to GC. I have tried doing on the runtime a port from Newlib. But it is so much work and reading that kind of code is so hard (I'm talking about the C headers for that [I have already ported several other C code, the C std is complex to read]). I always prefer easy setup and integration over anything because I just want to get things finished. "But what about your code having collections" well duh, when someone does that, I'll just update my patch and get that performance upgrade for free because I already have a real product out. I have tried so many times to help out on the runtime thing, but there was just so much friction that I wasn't able to do so. After Dkorpel documenting the runtime, I was finally able to understand what the heck was going on so I started to use that to incrementally build a product which I could see working. Unfortunately you can't do that with the runtime we have today (in my experience), it is 8 or 80. Maybe the custom runtime I extended with Adam could even be a beginner's gateway to understand the druntime, as most of the code used to implement class allocation and dynamic arrays becomes a lot easier to understand (specially as it doesn't have those bizarre named variables from upstream) and that it uses familiar concepts such as malloc/free/realloc instead of a lot of the runtime specific nomenclature.On Saturday, 4 February 2023 at 19:04:19 UTC, ryuukk_ wrote:Can't you just `version`/`static if` out all the things in druntime that WASM does not need? It is one of the strengths of D to be able to do that quite easily; shrinking the druntime is just a matter of writing `version() {}`. It'd be nice if you can spend some time in making standard druntime more compatible with WASM. Sebastian has worked on it, but it was never upstreamed. Just disable most/all, and only enable things that do work. We can always enable more / fix more later. -JohanOn Saturday, 4 February 2023 at 12:47:30 UTC, Guillaume Piolat wrote:On Friday, 3 February 2023 at 13:41:35 UTC, Hipreme wrote:I agree, but this is also unfortunate, one shouldn't have to do that to target WASM, the druntime needs to be shrinked asapThis custom runtime is a most welcome development.
Feb 05 2023
On 06/02/2023 12:36 AM, Hipreme wrote:Exactly because Sebastian did that and that never got upstreamed that I didn't work on it.It never got upstreamed because it was never attempted. I checked, no PR's for druntime.
Feb 05 2023
On Sunday, 5 February 2023 at 18:48:05 UTC, Richard (Rikki) Andrew Cattermole wrote:It never got upstreamed because it was never attempted. I checked, no PR's for druntime.It's on LDC: https://github.com/ldc-developers/ldc/pull/3345
Feb 05 2023
On Fri, Feb 03, 2023 at 01:41:35PM +0000, Hipreme via Digitalmars-d-announce wrote:This has been finished for quite a time but I was polishing some features. I'll save this post to 4 things: **Advertise that [Hipreme Engine](https://github.com/MrcSnm/HipremeEngine) is now supporting 5 platforms**, being those: - Xbox Series - Windows - Linux - Android - BrowserThis is awesome! Now I definitely have to look into this when I get around to my web project again. [...]**2. D performance on web**: Incredible. This game runs with 5% CPU usage on web, and it still has many optimization opportunities that I didn't run for, such as:[...] Nice! [...]- What is the slowest part of D? The JS bridge. The bloat is all over there, executing code from the bridge is by a magnitude of 10x slower than a single side code. Thankfully my engine has already dealt with that by every OpenGL call being batched.[...] Yeah, just as I expected, the JS bridge is a bottleneck. But hopefully as WASM matures more, we may be able to bypass JS, probably partially at first, eventually completely, hopefully. T -- Spaghetti code may be tangly, but lasagna code is just cheesy.
Feb 07 2023