www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Surfing on wave.. WASM

reply KnightMare <black80 bk.ru> writes:
In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
wave.
This means not only compile to WASM/WASI by compilers but bring 
to it Dlang-RT (bytecode it not useful without some RT).
When Dlang can generate same program for WASM as for terminal 
(with same RT not only "u can use templates only") it will lead a 
lot of people to Dlang, it will lead business too with money 
cause they want it, they ready to pay for it.
No?
May 27 2019
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:

 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?


Do you know about e.g. Spasm?

https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org

Also, this year there are two GSoC students working on WASM 
related topics (runtime hook conversion + independence of D from 
the C standard library).

See also: 
https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19
May 27 2019
next sibling parent KnightMare <black80 bk.ru> writes:
On Monday, 27 May 2019 at 12:53:20 UTC, Seb wrote:

 Do you know about e.g. Spasm?
 https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org


I read about Spasm now. I dont ready to talk about, need time to 
understand and to try it.


 Also, this year there are two GSoC students working on WASM 
 related topics (runtime hook conversion + independence of D 
 from the C standard library).


Yes, GSoC is great idea: people will meet D and they will do 
useful work.


 See also: 
 https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19
May 27 2019
prev sibling parent reply JN <666total wp.pl> writes:
On Monday, 27 May 2019 at 12:53:20 UTC, Seb wrote:

 Do you know about e.g. Spasm?

 https://forum.dlang.org/thread/lhkhbtpkbliddonfwliv forum.dlang.org

 Also, this year there are two GSoC students working on WASM 
 related topics (runtime hook conversion + independence of D 
 from the C standard library).

 See also: 
 https://github.com/dlang/projects/issues?q=is%3Aissue+is%3Aopen+label%3Agsoc19


It seems that Spasm and similar are actually workable solutions, 
assuming that we stick to BetterC subset. Do you know how 
plausible it would be to ever enable WASM for non BetterC code?


*cough* *cough* sometimes I feel like D should have been BetterC 
from the start *cough*
May 27 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:

 Do you know how plausible it would be to ever enable WASM
 for non BetterC code?


Certainly possible. dscripten (and dscripten-tools) is proof of 
that. The hard part is the GC.
May 27 2019
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:

 Do you know how plausible it would be to ever enable
 WASM for non BetterC code?


It is more tedious than it is difficult, but I think real 
usefulness is kinda blocked on stuff from the wasm side (interop 
with the real world) and on the D compiler side (efficient small 
code).


 *cough* *cough* sometimes I feel like D should have been 
 BetterC from the start *cough*


Crippled D would have been a mistake. This stuff is extremely 
niche.
May 27 2019
parent reply KnightMare <black80 bk.ru> writes:
On Monday, 27 May 2019 at 14:38:00 UTC, Adam D. Ruppe wrote:

 On Monday, 27 May 2019 at 14:01:46 UTC, JN wrote:

 Do you know how plausible it would be to ever enable
 WASM for non BetterC code?


 It is more tedious than it is difficult, but I think real 
 usefulness is kinda blocked on stuff from the wasm side 
 (interop with the real world) and on the D compiler side 
 (efficient small code).


proxies of outer objects in wasm space will solve interop from 
inside.
in case of nonmoveable GC in WASM host can invoke inside objects 
directly (we had ref to it somehow).
in both cases JIT/dynamicCompilation(as in LDC) will be very 
helpful.

GC will appear in WASM in some future. most probably with 
compact/move ability.
so access to object need to pin object first. its not native 
Dlang behavior.

in case Dlang community will write GC/RT for WASM interop can be 
10x faster and whole bundle will be most independent for 
development. and Dlang progs can host WASM as VM/script engine.

so next things are desirable:
- platform independent RT. most probably in some bytecode style 
(LLVM-IR?). versioned (as I described in few hours ago).
- dynamic codegeneration and compilation, JIT (LLVM will helps 
too)
with 2 such things Dlang will rise to heaven - scripting, 
protocol adapters, dynamically types interop, serialization from 
and to different formats, so formatters, ORM in runtime with 
schemas from DB-connection, GUI scripts as QML...
May 27 2019
next sibling parent KnightMare <black80 bk.ru> writes:
 interop can be 10x faster


except JavaScript interop. I meant D<->WASM
May 27 2019
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 15:30:46 UTC, KnightMare wrote:

 proxies of outer objects in wasm space will solve interop from 
 inside.


Yes, I know. In fact, I just wrote that for my blog thing this 
week:


export extern(C) void magic() {
         document.insertAdjacentHTML("beforeend", "cool!<span 
id=\"beans\">beans</span>");

         auto e = document.querySelector("span");
         e.insertAdjacentHTML("beforeend", " man");
}

That D runs as wasm and yields this page: 
http://arsdnet.net/wasm/test.html (see source for the JS glue 
there too)


It is doable and not too bloated, but, I don't love it still and 
it used a couple filthy hacks regardless. idk though I might 
spend more time on it.

Of course, you could use the emscripten thing now to do a bunch 
of in-browser web app stuff.


 with 2 such things Dlang will rise to heaven - scripting, 
 protocol adapters, dynamically types interop, serialization 
 from and to different formats, so formatters, ORM in runtime 
 with schemas from DB-connection, GUI scripts as QML...



We already have all that stuff fyi.
May 27 2019
next sibling parent reply KnightMare <black80 bk.ru> writes:
 with 2 such things Dlang will rise to heaven - scripting, 
 protocol adapters, dynamically types interop, serialization 
 from and to different formats, so formatters, ORM in runtime 
 with schemas from DB-connection, GUI scripts as QML...



 We already have all that stuff fyi.


The fact is that you cannot run this material in WASM.
Speaking figuratively:
We guessed all the letters, but we cannot read the word in the 
“wheel of fortune”.
We are standing on a platform with people and want to get into 
the country of WASM. Which train comes first (rust, D, C++, 
Kotlin) and will be comfortable enough for these people (and we 
just hope/expect D-train on this forum), most people will board 
it, and when these people reach and call to others, that they 
drove well, and in this country everything is fine, the rest of 
the people (and new ones that will appear later) also gather on 
the trains of this company.

Writing CLI-tools on D is enough for now. Improvements to this 
can wait a year. We must look around and fight for people with 
their wallets and fresh blood for the future. Building this train 
can bring a lot of gold to D. And, probably, you will have 6-12 
months to be on the first line.
May 27 2019
parent reply KnightMare <black80 bk.ru> writes:
 Writing CLI-tools on D is enough for now.


*is good enough

continue:
thousands of people crave this stuff. there is demand. you just 
need something to offer them something working, and benefits will 
flow like a river
May 27 2019
parent KnightMare <black80 bk.ru> writes:
any compiler based on LLVM can compile wasm-code.
we should provide the environment and runtime.
I understand that easy to speak and hard to do, but party leaders 
must indicate direction/target for nation. "Gondor calls for aid! 
And Rohan will answer!"
May 27 2019
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 15:42:16 UTC, Adam D. Ruppe wrote:

 I just wrote that for my blog thing this week:


here's my little blog entry

http://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_27.html

the code is miserable but it works, there's some potential in 
this kind of interop for small amounts of glue code to enable 
access to the browser.
May 27 2019
parent reply KnightMare <black80 bk.ru> writes:
 here's my little blog entry
 http://dpldocs.info/this-week-in-d/Blog.Posted_2019_05_27.html


cool!
are u sure that need something like retain/release?
imo wasm can work with data only in wasm region and can invoke 
exported functions.

I find another blog with strings in wasm interop. author used 
another technique:
https://blog.usejournal.com/string-manipulation-webassembly-5011e64c6b76
May 27 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 27 May 2019 at 21:41:24 UTC, KnightMare wrote:

 are u sure that need something like retain/release?


The idea there is to use objects created in javascript without 
leaking the resources forever. Not sure if it really works but it 
at least has a good chance of working well.


 I find another blog with strings in wasm interop.


strings are pretty easy, i was more interested in native objects 
from the JS side like dom elements.
May 27 2019
parent reply KnightMare <black80 bk.ru> writes:
 strings are pretty easy, i was more interested in native 
 objects from the JS side like dom elements.


https://github.com/skoppe/spasm/blob/v0.1.13/README.md#spa-framework
here Koppe went deep enough in dom
May 28 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 07:47:33 UTC, KnightMare wrote:

 strings are pretty easy, i was more interested in native 
 objects from the JS side like dom elements.


 https://github.com/skoppe/spasm/blob/v0.1.13/README.md#spa-framework
 here Koppe went deep enough in dom


That is the spa framework part of spasm.

I also have bindings to many web apis. All generated from WebIDL 
files.

https://github.com/skoppe/spasm/tree/master/source/spasm/bindings

Example:

https://github.com/skoppe/spasm/blob/master/examples/dom/source/app.d
https://github.com/skoppe/spasm/blob/master/examples/fetch/source/app.d
May 28 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 28 May 2019 at 11:19:47 UTC, Sebastiaan Koppe wrote:

 I also have bindings to many web apis. All generated from 
 WebIDL files.


Indeed, and then, as I understand it, you generate javascript for 
the extern(C) things actually referenced after the linker does 
its thing, with one extern(C) function corresponding to one 
Javascript method, right? I see int handles into a node array, 
just like I was thinking... but do you have a solution to memory 
management?

What I wanna do here is see if I can manage storing things fairly 
effectively with a minimal amount of code. I think a refcounting 
scheme, similar to how objective-c does it for the native handles 
would work and is reasonably easy to use.
May 28 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 13:22:57 UTC, Adam D. Ruppe wrote:

 On Tuesday, 28 May 2019 at 11:19:47 UTC, Sebastiaan Koppe wrote:

 I also have bindings to many web apis. All generated from 
 WebIDL files.


 Indeed, and then, as I understand it, you generate javascript 
 for the extern(C) things actually referenced after the linker 
 does its thing, with one extern(C) function corresponding to 
 one Javascript method, right?


Yep. There is a bindgen utility that generates only the js glue 
code you need.

Last time I looked Go takes a similar approach to yours, i.e. 
dynamically resolving calls based on strings. It does result in 
less js glue code, at the cost of having to decode strings. Both 
approaches are valid, but I wanted to focus on calling js 
function with little overhead.


 I see int handles into a node array, just like I was thinking...
 but do you have a solution to memory management?


I tried doing Unique+move, but failed with optional and sumtypes 
(don't like  disable this(this))

So it will probably be refcounts (but on the D side).


 What I wanna do here is see if I can manage storing things 
 fairly effectively with a minimal amount of code. I think a 
 refcounting scheme, similar to how objective-c does it for the 
 native handles would work and is reasonably easy to use.


Nice. Good to see more people working on this.

btw. for the strings you can have a look at 
https://github.com/skoppe/spasm/blob/master/examples/fetch/spasm/modules/spasm.js#L42
May 28 2019
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 28 May 2019 at 14:30:00 UTC, Sebastiaan Koppe wrote:

 I wanted to focus on calling js function with little overhead.


Yeah, and I like how you have a pretty good amount of type safety.

One of the things I like about strings though is working with 
existing code. Like if I can just, on the D side, do 
`window.my_own_object` I think that will be pretty cool.


 btw. for the strings you can have a look at


Indeed, that is next on my list. Actually, I kinda think I could 
pass D slices in general pretty directly as an i64, something 
like ptr << 32 | length... and a reinterpret cast on the D side 
could just work to do that. But that might be too clever for my 
own good.

(actually it briefly crossed my mind to define a communication 
bytecode thing to build calls but lolol, defining some kind of 
stack-based bytecode in javascript so our stack-based bytecode 
from javascript can communicated to javascript. absurd.)
May 28 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 14:49:48 UTC, Adam D. Ruppe wrote:

 Yeah, and I like how you have a pretty good amount of type 
 safety.


As well as auto-completion. One serious downside is that there is 
so much binding code that a full recompile of the library is 
painstakingly slow. It could be because of the many uses of 
Sumtype and Optional, or that I am public importing everything 
through a package.d, idk.


 One of the things I like about strings though is working with 
 existing code. Like if I can just, on the D side, do 
 `window.my_own_object` I think that will be pretty cool.


Yeah, whichever way you go you cant go wrong. D has so much 
better tools to write bindings than Rust or Go.

```go
func main() {
     document := js.Global().Get("document")
     p := document.Call("createElement", "p")
     p.Set("innerHTML", "Hello WASM from Go!")
     document.Get("body").Call("appendChild", p)
}
```

```rust

pub fn run() -> Result<(), JsValue> {
     let window = web_sys::window().expect("no global `window` 
exists");
     let document = window.document().expect("should have a 
document on window");
     let body = document.body().expect("document should have a 
body");
     let val = document.create_element("p")?;
     val.set_inner_html("Hello from Rust!");
     body.append_child(&val)?;
     Ok(())
}
```


 Indeed, that is next on my list. Actually, I kinda think I 
 could pass D slices in general pretty directly as an i64, 
 something like ptr << 32 | length... and a reinterpret cast on 
 the D side could just work to do that. But that might be too 
 clever for my own good.


You can just put string/array types in the extern(C) definition. 
Somehow LLVM expands array arguments into two u32's. Dunno why. 
And for return types it adds an u32 as first argument (on the js 
side) that points to the return value on the stack. Pretty rad.

See:
https://github.com/skoppe/spasm/blob/master/webidl/source/webidl/binding/test.d#L177,L192
May 28 2019
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 28 May 2019 at 18:19:14 UTC, Sebastiaan Koppe wrote:

 Somehow LLVM expands array arguments into two u32's. Dunno why.


It's analogous to the other target ABIs - pointer-sized length 
and pointer are passed as 2 separate arguments, usually in 2 
registers.


 And for return types it adds an u32 as first argument (on the 
 js side) that points to the return value on the stack. Pretty 
 rad.


If the return type is a struct or static array, yes. This is the 
behaviour of LDC's UnknownTargetABI [1], which is currently 
selected for wasm targets.

This could of course all be tweaked, e.g., packing a slice 
arg/return value into a i64 (=> single JS number, although 
probably limited to 53 bits).

[1]: 
https://github.com/ldc-developers/ldc/blob/9de916b8f63653d6836273d7336615f2e5e9a85e/gen/abi.cpp#L304-L325
May 28 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 28 May 2019 at 18:54:24 UTC, kinke wrote:

 On Tuesday, 28 May 2019 at 18:19:14 UTC, Sebastiaan Koppe wrote:

 Somehow LLVM expands array arguments into two u32's. Dunno why.


 It's analogous to the other target ABIs - pointer-sized length 
 and pointer are passed as 2 separate arguments, usually in 2 
 registers.


Makes sense.


 And for return types it adds an u32 as first argument (on the 
 js side) that points to the return value on the stack. Pretty 
 rad.


 If the return type is a struct or static array, yes. This is 
 the behaviour of LDC's UnknownTargetABI [1], which is currently 
 selected for wasm targets.


Ok, cool. I like it. I suppose it'll stay unknown for now?
May 28 2019
prev sibling next sibling parent KnightMare <black80 bk.ru> writes:
I understand that maybe I looks as troll and raise next holy war, 
maybe I still have adolescent maximalism..
But
How much can you say "It is not for such things"?
Dlang not for WEB. Its not for GUI. Its not for mobiles.
Its not for DB-ORM/network/FPGA/AI. (We have wrappers to some 
libs, use it boy)
Well, main area is terminal tools only.
With best 
UFCS/CTFE/GC/Exceptions/mixins/OOP/FP/InternalAsm/SSEtypes.
(Sounds like "mountain gave birth to a mouse").
And we want to add bare-metal RT to this without GC.
Well, how many developers need bare metal RT from all possible 
programming tasks in the world? 0.001%? 0.01%? Maybe 0.1% not 
more. Do they have another tools for such thing? C/rust? Probably 
has.
What the future of D? Replacement C/C++ in bare metal/terminal 
tools? It won't happen. It will be used but will not replace.
Do we hear some excuse "Yes, I know but I like do such job" or 
roadmap to heaven/communism/orAnotherSomethingGreat?
What will happened with Dlang after.. hmm.. bus factor with 
leads? Does Dlang has another lovely dictators who will continue 
to lead the flock to a bright future?
(Will we remain a small sect of lovers of Mars and not Snickers?)

3 men can write collections lib (like STL) that will be enough 
for bare metal without using GC in one year. So this is not 
future big target.

D can bypass Kotlin/Native, rust, C++, Go, Swift in many fields 
when you set goals correctly.

WASM is one of great target. (Yes, I still remember VM/Flash is 
bad, VM/Silverlight is bad, pure HTML5/Js are Rulez! Oh, we need 
some VM for WEB)
LDC can generate WASM code, give to it RT, port Dlang-RT/GC to 
WASM/VM that can be embedded in any app, that can execute logic 
in browsers with their WASM/VM.
Yes, is big task and some big corp with big wallet will write GC 
for WASM.
Will Dlang be ready run it program in WASM enironment from the 
box?

LLVM can generate platform code at runtime. Is it time to move 
the DMD to LLVM? Then Dlang can be used for writting VMs.

Can this/new runtime run on mobile devices? This is another great 
target: run Dlang on mobiles. ("LDC can generate code for 
Android" sounds like "LDC can generate WASM code" - it doesn't 
helps to run this code on VM or device, code is useless without 
RT or OS integration, its just pure obj file).
May 27 2019
prev sibling next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:

 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?


With WASI that is definitely possible. LDC can already target 
WASM and once WASI becomes standard I am sure it will be 
supported by LLVM.
May 27 2019
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 27 May 2019 at 12:33:16 UTC, KnightMare wrote:

 In case WASM is future of WEB/EmbeddedVM Dlang should ride the 
 wave.
 This means not only compile to WASM/WASI by compilers but bring 
 to it Dlang-RT (bytecode it not useful without some RT).
 When Dlang can generate same program for WASM as for terminal 
 (with same RT not only "u can use templates only") it will lead 
 a lot of people to Dlang, it will lead business too with money 
 cause they want it, they ready to pay for it.
 No?


I had some spare hours a couple of weeks ago, so I started 
porting druntime to WASI. It is far from complete and at some 
point I definitely need some druntime expert to help me out when 
I invariably get stuck.

Or you can just use betterC and write D with WASI right now.

https://github.com/skoppe/ldc/tree/wasi
https://github.com/skoppe/druntime/tree/wasi
https://github.com/skoppe/phobos/tree/wasi
May 27 2019