www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Surfing on wave.. WASM

reply KnightMare <black80 bk.ru> writes:
In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
wave.
This means not only compile to WASM/WASI by compilers but bring 
to it Dlang-RT (bytecode it not useful without some RT).
When Dlang can generate same program for WASM as for terminal 
(with same RT not only "u can use templates only") it will lead a 
lot of people to Dlang, it will lead business too with money 
cause they want it, they ready to pay for it.
No?
May 27
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:
 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?
Do you know about e.g. Spasm? https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org Also, this year there are two GSoC students working on WASM related topics (runtime hook conversion + independence of D from the C standard library). See also: https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19
May 27
next sibling parent KnightMare <black80 bk.ru> writes:
On Monday, 27 May 2019 at 12:53:20 UTC, Seb wrote:
 Do you know about e.g. Spasm?
 https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org
I read about Spasm now. I dont ready to talk about, need time to understand and to try it.
 Also, this year there are two GSoC students working on WASM 
 related topics (runtime hook conversion + independence of D 
 from the C standard library).
Yes, GSoC is great idea: people will meet D and they will do useful work.
 See also: 
 https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19
May 27
prev sibling parent reply JN <666total wp.pl> writes:
On Monday, 27 May 2019 at 12:53:20 UTC, Seb wrote:
 Do you know about e.g. Spasm?

 https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org

 Also, this year there are two GSoC students working on WASM 
 related topics (runtime hook conversion + independence of D 
 from the C standard library).

 See also: 
 https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19
It seems that Spasm and similar are actually workable solutions, assuming that we stick to BetterC subset. Do you know how plausible it would be to ever enable WASM for non BetterC code? *cough* *cough* sometimes I feel like D should have been BetterC from the start *cough*
May 27
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:
 Do you know how plausible it would be to ever enable WASM
 for non BetterC code?
Certainly possible. dscripten (and dscripten-tools) is proof of that. The hard part is the GC.
May 27
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:
 Do you know how plausible it would be to ever enable
 WASM for non BetterC code?
It is more tedious than it is difficult, but I think real usefulness is kinda blocked on stuff from the wasm side (interop with the real world) and on the D compiler side (efficient small code).
 *cough* *cough* sometimes I feel like D should have been 
 BetterC from the start *cough*
Crippled D would have been a mistake. This stuff is extremely niche.
May 27
parent reply KnightMare <black80 bk.ru> writes:
On Monday, 27 May 2019 at 14:38:00 UTC, Adam D. Ruppe wrote:
 On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:
 Do you know how plausible it would be to ever enable
 WASM for non BetterC code?
It is more tedious than it is difficult, but I think real usefulness is kinda blocked on stuff from the wasm side (interop with the real world) and on the D compiler side (efficient small code).
proxies of outer objects in wasm space will solve interop from inside. in case of nonmoveable GC in WASM host can invoke inside objects directly (we had ref to it somehow). in both cases JIT/dynamicCompilation(as in LDC) will be very helpful. GC will appear in WASM in some future. most probably with compact/move ability. so access to object need to pin object first. its not native Dlang behavior. in case Dlang community will write GC/RT for WASM interop can be 10x faster and whole bundle will be most independent for development. and Dlang progs can host WASM as VM/script engine. so next things are desirable: - platform independent RT. most probably in some bytecode style (LLVM-IR?). versioned (as I described in few hours ago). - dynamic codegeneration and compilation, JIT (LLVM will helps too) with 2 such things Dlang will rise to heaven - scripting, protocol adapters, dynamically types interop, serialization from and to different formats, so formatters, ORM in runtime with schemas from DB-connection, GUI scripts as QML...
May 27
next sibling parent KnightMare <black80 bk.ru> writes:
 interop can be 10x faster
except JavaScript interop. I meant D<->WASM
May 27
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 15:30:46 UTC, KnightMare wrote:
 proxies of outer objects in wasm space will solve interop from 
 inside.
Yes, I know. In fact, I just wrote that for my blog thing this week: export extern(C) void magic() { document.insertAdjacentHTML("beforeend", "cool!<span id=\"beans\">beans</span>"); auto e = document.querySelector("span"); e.insertAdjacentHTML("beforeend", " man"); } That D runs as wasm and yields this page: http://arsdnet.net/wasm/test.html (see source for the JS glue there too) It is doable and not too bloated, but, I don't love it still and it used a couple filthy hacks regardless. idk though I might spend more time on it. Of course, you could use the emscripten thing now to do a bunch of in-browser web app stuff.
 with 2 such things Dlang will rise to heaven - scripting, 
 protocol adapters, dynamically types interop, serialization 
 from and to different formats, so formatters, ORM in runtime 
 with schemas from DB-connection, GUI scripts as QML...

We already have all that stuff fyi.
May 27
next sibling parent reply KnightMare <black80 bk.ru> writes:
 with 2 such things Dlang will rise to heaven - scripting, 
 protocol adapters, dynamically types interop, serialization 
 from and to different formats, so formatters, ORM in runtime 
 with schemas from DB-connection, GUI scripts as QML...

We already have all that stuff fyi.
The fact is that you cannot run this material in WASM. Speaking figuratively: We guessed all the letters, but we cannot read the word in the “wheel of fortune”. We are standing on a platform with people and want to get into the country of WASM. Which train comes first (rust, D, C++, Kotlin) and will be comfortable enough for these people (and we just hope/expect D-train on this forum), most people will board it, and when these people reach and call to others, that they drove well, and in this country everything is fine, the rest of the people (and new ones that will appear later) also gather on the trains of this company. Writing CLI-tools on D is enough for now. Improvements to this can wait a year. We must look around and fight for people with their wallets and fresh blood for the future. Building this train can bring a lot of gold to D. And, probably, you will have 6-12 months to be on the first line.
May 27
parent reply KnightMare <black80 bk.ru> writes:
 Writing CLI-tools on D is enough for now.
*is good enough continue: thousands of people crave this stuff. there is demand. you just need something to offer them something working, and benefits will flow like a river
May 27
parent KnightMare <black80 bk.ru> writes:
any compiler based on LLVM can compile wasm-code.
we should provide the environment and runtime.
I understand that easy to speak and hard to do, but party leaders 
must indicate direction/target for nation. "Gondor calls for aid! 
And Rohan will answer!"
May 27
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 15:42:16 UTC, Adam D. Ruppe wrote:
 I just wrote that for my blog thing this week:
here's my little blog entry http://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_27.html the code is miserable but it works, there's some potential in this kind of interop for small amounts of glue code to enable access to the browser.
May 27
parent reply KnightMare <black80 bk.ru> writes:
 here's my little blog entry
 http://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_27.html
cool! are u sure that need something like retain/release? imo wasm can work with data only in wasm region and can invoke exported functions. I find another blog with strings in wasm interop. author used another technique: https://blog.usejournal.com/string-manipulation-webassembly-5011e64c6b76
May 27
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 21:41:24 UTC, KnightMare wrote:
 are u sure that need something like retain/release?
The idea there is to use objects created in javascript without leaking the resources forever. Not sure if it really works but it at least has a good chance of working well.
 I find another blog with strings in wasm interop.
strings are pretty easy, i was more interested in native objects from the JS side like dom elements.
May 27
parent reply KnightMare <black80 bk.ru> writes:
 strings are pretty easy, i was more interested in native 
 objects from the JS side like dom elements.
https://github.com/skoppe/spasm/blob/v0.1.13/README.md#spa-framework here Koppe went deep enough in dom
May 28
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 07:47:33 UTC, KnightMare wrote:
 strings are pretty easy, i was more interested in native 
 objects from the JS side like dom elements.
https://github.com/skoppe/spasm/blob/v0.1.13/README.md#spa-framework here Koppe went deep enough in dom
That is the spa framework part of spasm. I also have bindings to many web apis. All generated from WebIDL files. https://github.com/skoppe/spasm/tree/master/source/spasm/bindings Example: https://github.com/skoppe/spasm/blob/master/examples/dom/source/app.d https://github.com/skoppe/spasm/blob/master/examples/fetch/source/app.d
May 28
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 28 May 2019 at 11:19:47 UTC, Sebastiaan Koppe wrote:
 I also have bindings to many web apis. All generated from 
 WebIDL files.
Indeed, and then, as I understand it, you generate javascript for the extern(C) things actually referenced after the linker does its thing, with one extern(C) function corresponding to one Javascript method, right? I see int handles into a node array, just like I was thinking... but do you have a solution to memory management? What I wanna do here is see if I can manage storing things fairly effectively with a minimal amount of code. I think a refcounting scheme, similar to how objective-c does it for the native handles would work and is reasonably easy to use.
May 28
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 13:22:57 UTC, Adam D. Ruppe wrote:
 On Tuesday, 28 May 2019 at 11:19:47 UTC, Sebastiaan Koppe wrote:
 I also have bindings to many web apis. All generated from 
 WebIDL files.
Indeed, and then, as I understand it, you generate javascript for the extern(C) things actually referenced after the linker does its thing, with one extern(C) function corresponding to one Javascript method, right?
Yep. There is a bindgen utility that generates only the js glue code you need. Last time I looked Go takes a similar approach to yours, i.e. dynamically resolving calls based on strings. It does result in less js glue code, at the cost of having to decode strings. Both approaches are valid, but I wanted to focus on calling js function with little overhead.
 I see int handles into a node array, just like I was thinking...
 but do you have a solution to memory management?
I tried doing Unique+move, but failed with optional and sumtypes (don't like disable this(this)) So it will probably be refcounts (but on the D side).
 What I wanna do here is see if I can manage storing things 
 fairly effectively with a minimal amount of code. I think a 
 refcounting scheme, similar to how objective-c does it for the 
 native handles would work and is reasonably easy to use.
Nice. Good to see more people working on this. btw. for the strings you can have a look at https://github.com/skoppe/spasm/blob/master/examples/fetch/spasm/modules/spasm.js#L42
May 28
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 28 May 2019 at 14:30:00 UTC, Sebastiaan Koppe wrote:
 I wanted to focus on calling js function with little overhead.
Yeah, and I like how you have a pretty good amount of type safety. One of the things I like about strings though is working with existing code. Like if I can just, on the D side, do `window.my_own_object` I think that will be pretty cool.
 btw. for the strings you can have a look at
Indeed, that is next on my list. Actually, I kinda think I could pass D slices in general pretty directly as an i64, something like ptr << 32 | length... and a reinterpret cast on the D side could just work to do that. But that might be too clever for my own good. (actually it briefly crossed my mind to define a communication bytecode thing to build calls but lolol, defining some kind of stack-based bytecode in javascript so our stack-based bytecode from javascript can communicated to javascript. absurd.)
May 28
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 14:49:48 UTC, Adam D. Ruppe wrote:
 Yeah, and I like how you have a pretty good amount of type 
 safety.
As well as auto-completion. One serious downside is that there is so much binding code that a full recompile of the library is painstakingly slow. It could be because of the many uses of Sumtype and Optional, or that I am public importing everything through a package.d, idk.
 One of the things I like about strings though is working with 
 existing code. Like if I can just, on the D side, do 
 `window.my_own_object` I think that will be pretty cool.
Yeah, whichever way you go you cant go wrong. D has so much better tools to write bindings than Rust or Go. ```go func main() { document := js.Global().Get("document") p := document.Call("createElement", "p") p.Set("innerHTML", "Hello WASM from Go!") document.Get("body").Call("appendChild", p) } ``` ```rust pub fn run() -> Result<(), JsValue> { let window = web_sys::window().expect("no global `window` exists"); let document = window.document().expect("should have a document on window"); let body = document.body().expect("document should have a body"); let val = document.create_element("p")?; val.set_inner_html("Hello from Rust!"); body.append_child(&val)?; Ok(()) } ```
 Indeed, that is next on my list. Actually, I kinda think I 
 could pass D slices in general pretty directly as an i64, 
 something like ptr << 32 | length... and a reinterpret cast on 
 the D side could just work to do that. But that might be too 
 clever for my own good.
You can just put string/array types in the extern(C) definition. Somehow LLVM expands array arguments into two u32's. Dunno why. And for return types it adds an u32 as first argument (on the js side) that points to the return value on the stack. Pretty rad. See: https://github.com/skoppe/spasm/blob/master/webidl/source/webidl/binding/test.d#L177,L192
May 28
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 28 May 2019 at 18:19:14 UTC, Sebastiaan Koppe wrote:
 Somehow LLVM expands array arguments into two u32's. Dunno why.
It's analogous to the other target ABIs - pointer-sized length and pointer are passed as 2 separate arguments, usually in 2 registers.
 And for return types it adds an u32 as first argument (on the 
 js side) that points to the return value on the stack. Pretty 
 rad.
If the return type is a struct or static array, yes. This is the behaviour of LDC's UnknownTargetABI [1], which is currently selected for wasm targets. This could of course all be tweaked, e.g., packing a slice arg/return value into a i64 (=> single JS number, although probably limited to 53 bits). [1]: https://github.com/ldc-developers/ldc/blob/9de916b8f63653d6836273d7336615f2e5e9a85e/gen/abi.cpp#L304-L325
May 28
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 18:54:24 UTC, kinke wrote:
 On Tuesday, 28 May 2019 at 18:19:14 UTC, Sebastiaan Koppe wrote:
 Somehow LLVM expands array arguments into two u32's. Dunno why.
It's analogous to the other target ABIs - pointer-sized length and pointer are passed as 2 separate arguments, usually in 2 registers.
Makes sense.
 And for return types it adds an u32 as first argument (on the 
 js side) that points to the return value on the stack. Pretty 
 rad.
If the return type is a struct or static array, yes. This is the behaviour of LDC's UnknownTargetABI [1], which is currently selected for wasm targets.
Ok, cool. I like it. I suppose it'll stay unknown for now?
May 28
prev sibling next sibling parent KnightMare <black80 bk.ru> writes:
I understand that maybe I looks as troll and raise next holy war, 
maybe I still have adolescent maximalism..
But
How much can you say "It is not for such things"?
Dlang not for WEB. Its not for GUI. Its not for mobiles.
Its not for DB-ORM/network/FPGA/AI. (We have wrappers to some 
libs, use it boy)
Well, main area is terminal tools only.
With best 
UFCS/CTFE/GC/Exceptions/mixins/OOP/FP/InternalAsm/SSEtypes.
(Sounds like "mountain gave birth to a mouse").
And we want to add bare-metal RT to this without GC.
Well, how many developers need bare metal RT from all possible 
programming tasks in the world? 0.001%? 0.01%? Maybe 0.1% not 
more. Do they have another tools for such thing? C/rust? Probably 
has.
What the future of D? Replacement C/C++ in bare metal/terminal 
tools? It won't happen. It will be used but will not replace.
Do we hear some excuse "Yes, I know but I like do such job" or 
roadmap to heaven/communism/orAnotherSomethingGreat?
What will happened with Dlang after.. hmm.. bus factor with 
leads? Does Dlang has another lovely dictators who will continue 
to lead the flock to a bright future?
(Will we remain a small sect of lovers of Mars and not Snickers?)

3 men can write collections lib (like STL) that will be enough 
for bare metal without using GC in one year. So this is not 
future big target.

D can bypass Kotlin/Native, rust, C++, Go, Swift in many fields 
when you set goals correctly.

WASM is one of great target. (Yes, I still remember VM/Flash is 
bad, VM/Silverlight is bad, pure HTML5/Js are Rulez! Oh, we need 
some VM for WEB)
LDC can generate WASM code, give to it RT, port Dlang-RT/GC to 
WASM/VM that can be embedded in any app, that can execute logic 
in browsers with their WASM/VM.
Yes, is big task and some big corp with big wallet will write GC 
for WASM.
Will Dlang be ready run it program in WASM enironment from the 
box?

LLVM can generate platform code at runtime. Is it time to move 
the DMD to LLVM? Then Dlang can be used for writting VMs.

Can this/new runtime run on mobile devices? This is another great 
target: run Dlang on mobiles. ("LDC can generate code for 
Android" sounds like "LDC can generate WASM code" - it doesn't 
helps to run this code on VM or device, code is useless without 
RT or OS integration, its just pure obj file).
May 27
prev sibling next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:
 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?
With WASI that is definitely possible. LDC can already target WASM and once WASI becomes standard I am sure it will be supported by LLVM.
May 27
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:
 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?
I had some spare hours a couple of weeks ago, so I started porting druntime to WASI. It is far from complete and at some point I definitely need some druntime expert to help me out when I invariably get stuck. Or you can just use betterC and write D with WASI right now. https://github.com/skoppe/ldc/tree/wasi https://github.com/skoppe/druntime/tree/wasi https://github.com/skoppe/phobos/tree/wasi
May 27