digitalmars.D - Proposed strategy to support WASM in D
- AnimusPEXUS (54/54) Nov 17 2022 0. probably two different approaches to supporting wasm is needed
- AnimusPEXUS (57/57) Nov 17 2022 also I've collected some useful links for various WASM and JS
- Adam D Ruppe (11/18) Nov 17 2022 I took Sebastiaan's thing from two years ago and applied it to
- AnimusPEXUS (12/16) Nov 24 2022 Have question(s) on Sebastiaan's wasm.
- Sebastiaan Koppe (14/23) Nov 25 2022 DRuntime depends on libc. So if you want to use DRuntime in WASM
- Johan (5/9) Nov 17 2022 Can you give an example of what goes wrong? `i32` is LDC/LLVM's
- AnimusPEXUS (5/11) Nov 17 2022 for example, if I export function uint fname(uint p1, uint p2),
- Paulo Pinto (6/24) Nov 18 2022 It looks like that if you look at the number, but in reality WASM
- Pierce Ng (54/57) Nov 18 2022 WASI is like a libc interface for Wasm and [WASI
- AnimusPEXUS (4/6) Nov 18 2022 probably you can try playing with
- Adam D Ruppe (8/13) Nov 18 2022 All D symbols are namespaced by their module name. You can expose
- Pierce Ng (48/55) Nov 18 2022 Thanks. I misremembered, the trouble isn't with time() but with
0. probably two different approaches to supporting wasm is needed simultatiously: 0.1: [wasm module] + [js glue] -- this is current approach, similar to golang's 0.2: wasi support - this approach means what wasi platforms provide interface for wasi application to interact with the system I'm talking here about [wasm module] + [js glue] approach 1. this approach mean the task have to be separated to two subtasks: 1.1 js glue code, which allows wasm code to interact with js environment. 1.2 druntime have to be written, so Dlang code would be compilable with ldc (or yet better with dmd it self) with wasm triplet, without --betterC option, like any usual D program. not so long ago I've started new project https://github.com/AnimusPEXUS/dwasmjs and I've wrote some basic js code to work with js values from inside D ( https://github.com/AnimusPEXUS/dwasmjs/blob/master/source/dwasmjs/dwasmjsi.js ) it's far from being complete. but the general Idea - is to store js values on js side of js<>d glue code. (also I think reflect module of js could be used to improve js interactions, to avoid eval() function use, as eval would slow everything down) also, to address memory transfers from js to d (wasm) - I've figured out what js side can ask D side to allocate memory arrays as needed and js should write bytes back to pointer passed by D side. this js side of glue is working with identifiers (randomly generated unique values). if D side needs to get some string - D side calls function on JS side with id of value from which D side wants to get string, and D side also passes pointer to preallocated array (D side gets size needed for array by calling another function). in my code I've stacked with need to allocate memory inside of wasm module instance. I've seen ADR's mini-druntime for wasm and seen how he used ldc's special fields to calculate addressing for memory allocation and used special ldc function to grow wasm blob (I think, those function can be called from standard WebAssembly module of JS, avoiding need in special ldc functions) vvv so basically, the main problem now in literally is to write WebAssembly version of druntime. gluing it to js is not so problematic. ^^^ -------------- as for WASI support, I've didn't read documentation yet, but I imagine it provides some binary interface so it can be used to interact with the system, so WASI, probably should be separate version of druntime - separate from WebAssembly(64) so currently I'm reading druntime and trying to figureout how to modify it to be compilable in wasm -------------- additionally: 1. LDC have to support unsigned integers in wasm function parameters, as currently wasm generated by ldc puts i32 type on all parameters. 2. we have to answer question: does stdc parts of druntime also should be ported to WebAssembly, as I've discovered what, looks like druntime requires struct tm from sys.posix.time module
Nov 17 2022
AnimusPEXUS <animuspexus protonmail.com> also I've collected some useful links for various WASM and JS resources related to WASM<>JS integration for ppl less to Google. here * https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions * https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API * https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Browser_support_for_JavaScript_APIs * https://developer.mozilla.org/en-US/docs/Web * https://developer.mozilla.org/en-US/docs/Web/API * https://developer.mozilla.org/en-US/docs/WebAssembly * https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface * https://developer.mozilla.org/en-US/docs/WebAssembly/JavaScript_interface/Memory/grow * https://developer.mozilla.org/en-US/docs/WebAssembly/Using_the_JavaScript_API * https://developer.mozilla.org/en-US/docs/Web/JavaScript * https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference * https://github.com/adamdruppe/webassembly * https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-documents.md * https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-intro.md * https://github.com/ldc-developers/ldc * https://github.com/WebAssembly/proposals * https://github.com/WebAssembly/tool-conventions/blob/main/BasicCABI.md * https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md * https://github.com/WebAssembly/WASI/blob/main/phases/snapshot/docs.md * https://lld.llvm.org/WebAssembly.html * https://llvm.org/doxygen/namespacellvm_1_1wasm.html * https://llvm.org/doxygen/namespaces.html * https://rob-blackbourn.github.io/blog/webassembly/wasm/strings/javascript/c/libc/wasm-libc/clang/2020/06/20/wasm-string-passing.html * https://stackoverflow.com/questions/41353389/how-can-i-return-a-javascript-string-from-a-webassembly-function * https://wasi.dev/ * https://webassembly.github.io/spec/core/ * https://webassembly.github.io/spec/core/exec/index.html * https://webassembly.github.io/spec/core/exec/instructions.html#memory-instructions * https://webassembly.github.io/spec/js-api/index.html * https://webassembly.github.io/spec/js-api/#memories * https://webassembly.github.io/spec/web-api/index.html * https://webassembly.org/specs/ * https://wiki.dlang.org/Generating_WebAssembly_with_LDC#Calling_external_functions * https://wiki.dlang.org/LDC-specific_language_changes * https://wiki.dlang.org/Runtime_internals * https://wiki.dlang.org/Using_LDC * https://www.llvm.org/docs/ExtendingLLVM.html * https://www.w3.org/TR/wasm-core/
Nov 17 2022
On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote:vvv so basically, the main problem now in literally is to write WebAssembly version of druntime. gluing it to js is not so problematic. ^^^I took Sebastiaan's thing from two years ago and applied it to new druntime here: https://github.com/adamdruppe/dmd/tree/wasm I haven't tried actually compiling it yet though, I've been pretty swamped with work lately. When the new ldc comes out, I'll get Sebastiaan's documentation together and give it a try.interact with the system, so WASI, probably should be separate version of druntime - separate from WebAssembly(64)WebAssembly is a kind of CPU and WASI is a kind of C library. So it should be two separate versions kinda like how there's version(X86) and version(Windows) and version(CRuntime_Microsoft) all independent (though often used together you don't have to).
Nov 17 2022
On Friday, 18 November 2022 at 02:00:40 UTC, Adam D Ruppe wrote:On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote: I took Sebastiaan's thing from two years ago and applied it to new druntime here: https://github.com/adamdruppe/dmd/tree/wasmHave question(s) on Sebastiaan's wasm. 1. Sebastiaan, does your wasm realization concentrates primarily on wasi interfacing, or will it allows wasm usage in [wasm]+[js glue] fashion? 2. if [wasm]+[js glue] fashion is not supported by Sebastiaan's wasm realization, then I, I think need to concentrate on [wasm]+[js glue] variant. so if this is the case, then how to separate [wasm wasi] version from [wasm]+[js glue] version in druntime. 3. if [wasm]+[js glue] doesn't have place in druntime, then how to properly organize forking/patching of druntime?
Nov 24 2022
On Thursday, 24 November 2022 at 20:22:43 UTC, AnimusPEXUS wrote:Have question(s) on Sebastiaan's wasm. 1. Sebastiaan, does your wasm realization concentrates primarily on wasi interfacing, or will it allows wasm usage in [wasm]+[js glue] fashion?DRuntime depends on libc. So if you want to use DRuntime in WASM you need a WASM implementation of libc. That is what WASI-libc provides. Why WASI? Well, it is the interface many WASM runtimes use. That is all my DRuntime 'port' does.2. if [wasm]+[js glue] fashion is not supported by Sebastiaan's wasm realization, then I, I think need to concentrate on [wasm]+[js glue] variant. so if this is the case, then how to separate [wasm wasi] version from [wasm]+[js glue] version in druntime.Interfacing with JS is out of the scope of the DRuntime WASM/WASI port. In the Spasm library I have a bindgen tool that spits out D/JS glue code based on web IDL files. It allows you to call pretty much every web api out there. Before my current job I briefly worked on one that creates D/JS glue code for typescript libraries. It isn't finished, but it shows its viable.
Nov 25 2022
On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote:1. LDC have to support unsigned integers in wasm function parameters, as currently wasm generated by ldc puts i32 type on all parameters.Can you give an example of what goes wrong? `i32` is LDC/LLVM's type of signed _and_ unsigned integers, there is no distinction on LLVM IR level. -Johan
Nov 17 2022
On Friday, 18 November 2022 at 07:06:08 UTC, Johan wrote:On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote:for example, if I export function uint fname(uint p1, uint p2), the resulting wasm code treats values as signed. but actually I don't know if it possible for wasm to really know I wast to pass unsigned value.Can you give an example of what goes wrong? `i32` is LDC/LLVM's type of signed _and_ unsigned integers, there is no distinction on LLVM IR level. -Johan
Nov 17 2022
On Friday, 18 November 2022 at 07:28:33 UTC, AnimusPEXUS wrote:On Friday, 18 November 2022 at 07:06:08 UTC, Johan wrote:It looks like that if you look at the number, but in reality WASM makes no distiction, it is up for the application themselves to use the appropriate set of opcodes.On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote:for example, if I export function uint fname(uint p1, uint p2), the resulting wasm code treats values as signed. but actually I don't know if it possible for wasm to really know I wast to pass unsigned value.Can you give an example of what goes wrong? `i32` is LDC/LLVM's type of signed _and_ unsigned integers, there is no distinction on LLVM IR level. -JohanThe class defines uninterpreted integers, whose signedness interpretation can vary depending on context. In the abstract syntax, they are represented as unsigned values. However, some operations convert them to signed based on a two’s complement interpretation.https://webassembly.github.io/spec/core/syntax/values.html#integers https://webassembly.github.io/spec/core/syntax/instructions.html#syntax-sx
Nov 18 2022
On Friday, 18 November 2022 at 00:55:57 UTC, AnimusPEXUS wrote:0.2: wasi support - this approach means what wasi platforms provide interface for wasi application to interact with the systemWASI is like a libc interface for Wasm and [WASI libc](https://github.com/WebAssembly/wasi-libc) is an implementation. A Wasm program calls WASI to access libc/OS-like functionality provided by the Wasm runtime, which could be the web browser or other runtimes. Earlier this year, for the fantasy game console [TIC-80](https://github.com/nesbox/TIC-80), I implemented the [D-Wasm template](https://github.com/nesbox/TIC-80/tree/main/templates/d). However, I'm stuck actually implementing games in D, (IHIRC) because of a symbol clash between TIC-80's time() API function and WASI libc's time(). For comparison and contrast, with the Zig TIC-80 Wasm interface, Zig source code calls tic.time() for the TIC-80 time() function and thus no symbol conflict. Here's the code snippet: ``` // D void TIC() { cls(0); for (int x=0; x<240; x+=28) { for (int y=0; y<136; y+=28) { cx = 12*sinf(time()/30000*(x+y+1)); // <========== time() symbol conflict cy = 12*cosf(time()/30000*(x+y+1)); // <========== ditto Pir(x, y, 25, 25, x+cx, y+cy); } } } // Zig export fn TIC() void { // some var initialization removed tic.cls(3); while (ax <= 240) { fx = intToFloat(f32, ax); ay = 0; while (ay <= 136) { fy = intToFloat(f32, ay); ca = 12*std.math.sin(tic.time()/30000*(fx+fy+1)); // tic.time() cb = 12*std.math.cos(tic.time()/30000*(fx+fy+1)); // no conflict Pir(fx, fy, 25, 25, fx+ca, fy+cb); ay += 28; } ax += 28; } } ``` Without modifying TIC-80 source code, is there a way to do some 'namespace' thingie for D like how Zig does it? More generally, as a D newbie, the confusion for me was (still is): Should I be writing D programs (whether for TIC-80 or more mundane platforms like Linux) in D, or in BetterC?
Nov 18 2022
On Saturday, 19 November 2022 at 01:16:00 UTC, Pierce Ng wrote:Without modifying TIC-80 source code, is there a way to do some 'namespace' thingie for D like how Zig does it?probably you can try playing with [pragma(mangle)](https://dlang.org/spec/pragma.html#mangle) for this.
Nov 18 2022
On Saturday, 19 November 2022 at 01:16:00 UTC, Pierce Ng wrote:Without modifying TIC-80 source code, is there a way to do some 'namespace' thingie for D like how Zig does it?All D symbols are namespaced by their module name. You can expose the JS functions in different modules. I don't know your specific problem but it ought to just be a case of adjusting the imports on JS and using them through different modules in D.More generally, as a D newbie, the confusion for me was (still is): Should I be writing D programs (whether for TIC-80 or more mundane platforms like Linux) in D, or in BetterC?Always use real D unless it is impractical to do so. No point going through suffering when you don't have to.
Nov 18 2022
On Saturday, 19 November 2022 at 01:57:21 UTC, Adam D Ruppe wrote:On Saturday, 19 November 2022 at 01:16:00 UTC, Pierce Ng wrote:Thanks. I misremembered, the trouble isn't with time() but with trying to use WASI libc's sinf() and cosf(). If I try to use D's std.math instead, the code doesn't compile in BetterC mode: ``` % make dub build --quiet --build release --compiler ldc2 --arch wasm32-unknown-unknown-wasm /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4433,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4435,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4437,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4537,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4539,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4541,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4558,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4560,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/math.d(4562,13): Error: undefined identifier `c_long` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(875,5): Error: undefined identifier `fenv_t` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(878,5): Error: undefined identifier `fexcept_t` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(880,5): Error: undefined identifier `fexcept_t` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(888,5): Error: undefined identifier `fenv_t` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(890,5): Error: undefined identifier `fenv_t` /home/pierce/pkg/ldc/include/d/core/stdc/fenv.d(939,9): Error: undefined identifier `fenv_t` ldc2 failed with exit code 1. make: *** [Makefile:7: build] Error 2 ``` In TIC-80's case there is no JS. TIC-80 embeds the [wasm3](https://github.com/wasm3/wasm3) standalone runtime and makes its (TIC-80's) APIs available to Wasm programs (Well, TIC-80 also embeds Duktape, thus making it programmable with JS, but that's not relevant here.)Without modifying TIC-80 source code, is there a way to do some 'namespace' thingie for D like how Zig does it?All D symbols are namespaced by their module name. You can expose the JS functions in different modules.Always use real D unless it is impractical to do so. No point going through suffering when you don't have to.Good point. Let me update D, TIC-80, etc. and try again.
Nov 18 2022