www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Spasm - webassembly libary for single page applications

reply Sebastiaan Koppe <mail skoppe.eu> writes:
I like to announce Spasm https://github.com/skoppe/spasm

It is a webassembly library to develop single page applications 
and builds on my previous work 
(https://forum.dlang.org/post/eqneqstmwfzugymfewqo forum.dlang.org).

It generates fast and small webassembly binaries. The example 
todo-mvc is only 5995 bytes (wasm) + 2199 bytes (html+js) when 
gzipped, and loads pretty fast 
(https://skoppe.github.io/spasm/examples/todo-mvc/)

Spasm can be used with dub and only requires a recent version of 
ldc. See the readme on github how to get started.

The library includes a webpack bootstrap project to build a 
production ready web page.

Spasm is written in betterC.

Next steps are:

- more js host bindings (xhr, localstorage, etc.)
- memory deallocation
- gather an angry mob to make phobos more betterC compatible ;)
Oct 12 2018
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/12/18 3:43 PM, Sebastiaan Koppe wrote:

 I like to announce Spasm https://github.com/skoppe/spasm
 
 It is a webassembly library to develop single page applications and 
 builds on my previous work 
 (https://forum.dlang.org/post/eqneqstmwfzugymfewqo forum.dlang.org).
 
 It generates fast and small webassembly binaries. The example todo-mvc 
 is only 5995 bytes (wasm) + 2199 bytes (html+js) when gzipped, and loads 
 pretty fast (https://skoppe.github.io/spasm/examples/todo-mvc/)
 
 Spasm can be used with dub and only requires a recent version of ldc. 
 See the readme on github how to get started.
 
 The library includes a webpack bootstrap project to build a production 
 ready web page.
 
 Spasm is written in betterC.
 
 Next steps are:
 
 - more js host bindings (xhr, localstorage, etc.)
 - memory deallocation
 - gather an angry mob to make phobos more betterC compatible ;)


OK, this is pretty cool. I need to start checking out webasm.

-Steve
Oct 12 2018
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
Nifty, I'll have to look into this. Any idea what it would take to get 
this doing some WebGL? (Or playing audio?) Or is this more for HTML-ish 
sorts of stuff?

What are the main current limitations?
Oct 12 2018
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 13 October 2018 at 04:34:28 UTC, Nick Sabalausky 
(Abscissa) wrote:

 Nifty, I'll have to look into this. Any idea what it would take 
 to get this doing some WebGL? (Or playing audio?) Or is this 
 more for HTML-ish sorts of stuff?


This is more focused on HTML rendering, but you can do anything 
that you can do from JS, you just have to write JS glue code.

WebGL is based on and follows the OpenGL ES (Embedded Systems) 
spec, which is a subset of OpenGL. So if you have the API 
definitions you are already half done. The other half is writing 
that in JS and calling the actual WebGL.

If you want to do it right now you can use vladimir's 
dscripten-tools project, which is based on emscripten.

I just don't like emscripten because it has a complex toolchain 
(D->LDC->(patched)LLVM->asm.js->binaryen->wasm) and results in 
bloated code.
Oct 13 2018
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 13 October 2018 at 04:34:28 UTC, Nick Sabalausky 
(Abscissa) wrote:

 What are the main current limitations?


Mainly that it is written in betterC, which is quite the 
limitation if you are used to freely call anything from phobos, 
or expect the GC to clean up.

Also I currently don't free memory.
Oct 13 2018
prev sibling next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Friday, 12 October 2018 at 19:43:25 UTC, Sebastiaan Koppe 
wrote:

 I like to announce Spasm https://github.com/skoppe/spasm

 It is a webassembly library to develop single page applications 
 and builds on my previous work 
 (https://forum.dlang.org/post/eqneqstmwfzugymfewqo forum.dlang.org).


I must say even at this stage this library is awesome! Good job! 
Keep up with the good work! :)
Oct 13 2018
prev sibling next sibling parent reply Bogdan <szabobogdan yahoo.com> writes:
On Friday, 12 October 2018 at 19:43:25 UTC, Sebastiaan Koppe 
wrote:

 I like to announce Spasm https://github.com/skoppe/spasm

 It is a webassembly library to develop single page applications 
 and builds on my previous work 
 (https://forum.dlang.org/post/eqneqstmwfzugymfewqo forum.dlang.org).

 It generates fast and small webassembly binaries. The example 
 todo-mvc is only 5995 bytes (wasm) + 2199 bytes (html+js) when 
 gzipped, and loads pretty fast 
 (https://skoppe.github.io/spasm/examples/todo-mvc/)

 Spasm can be used with dub and only requires a recent version 
 of ldc. See the readme on github how to get started.

 The library includes a webpack bootstrap project to build a 
 production ready web page.

 Spasm is written in betterC.

 Next steps are:

 - more js host bindings (xhr, localstorage, etc.)
 - memory deallocation
 - gather an angry mob to make phobos more betterC compatible ;)


Awesome work! I remember that, at some point the 
https://glimmerjs.com/ authors wanted to write their vm in rust 
for better performance. It looks like D is a new option for such 
projects.

Bogdan
Oct 13 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 14 October 2018 at 06:03:10 UTC, Bogdan wrote:

 Awesome work! I remember that, at some point the 
 https://glimmerjs.com/ authors wanted to write their vm in rust 
 for better performance. It looks like D is a new option for 
 such projects.

 Bogdan


Thanks, a lot of credits go to LDC for supporting the webassembly 
backend of LLVM.

I thought about doing a VM as well, specifically because I was 
afraid of the performance penalty of switching a lot between js 
and wasm. The idea was to emit all dom operations into one 
bytebuffer (possibly at compile-time) and then instruct js to 
execute that.

It turns out jumping between wasm and js isn't really a big deal 
(at least not anymore), so I ditched that idea to keep things 
simple.

Plus, there is a good chance that in the near future wasm will be 
able to call the browsers' api directly.
Oct 14 2018
next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Sunday, 14 October 2018 at 19:04:51 UTC, Sebastiaan Koppe 
wrote:

 It turns out jumping between wasm and js isn't really a big 
 deal (at least not anymore), so I ditched that idea to keep 
 things simple.

 Plus, there is a good chance that in the near future wasm will 
 be able to call the browsers' api directly.


Precisely, that is why I like this approach. Good job.
Oct 15 2018
prev sibling parent aberba <karabutaworld gmail.com> writes:
On Sunday, 14 October 2018 at 19:04:51 UTC, Sebastiaan Koppe 
wrote:

 On Sunday, 14 October 2018 at 06:03:10 UTC, Bogdan wrote:

 Awesome work! I remember that, at some point the 
 https://glimmerjs.com/ authors wanted to write their vm in 
 rust for better performance. It looks like D is a new option 
 for such projects.

 Bogdan


 Thanks, a lot of credits go to LDC for supporting the 
 webassembly backend of LLVM.

 I thought about doing a VM as well, specifically because I was 
 afraid of the performance penalty of switching a lot between js 
 and wasm. The idea was to emit all dom operations into one 
 bytebuffer (possibly at compile-time) and then instruct js to 
 execute that.

 It turns out jumping between wasm and js isn't really a big 
 deal (at least not anymore), so I ditched that idea to keep 
 things simple.


I saw a recent announcement by the Firefox devs on some massive 
improvements they've made in the js-wasm switching speed. Its 
negligible for most cases...unless probably something crazy. Its 
was very fast...especially for a benchmark of 100 million calls 
with Math.random()

https://hacks.mozilla.org/2018/10/calls-between-javascript-and-webassembly-are-finally-fast-%F0%9F%8E%89/


 Plus, there is a good chance that in the near future wasm will 
 be able to call the browsers' api directly.


Have you seen this? 
https://developer.mozilla.org/en-US/docs/WebAssembly/Using_the_JavaScript_API

I'm not sure how limited it is though.
Oct 17 2018
prev sibling next sibling parent reply Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 12 October 2018 at 19:43:25 UTC, Sebastiaan Koppe 
wrote:

 I like to announce Spasm https://github.com/skoppe/spasm

 It is a webassembly library to develop single page applications 
 and builds on my previous work


This is really interesting. I don't do web development myself and 
haven't been doing hobby programming, but I like the idea of of 
webasm.

It would be cool if D provided the easiest way to develop webasm 
first to see if it could claim that market.
Oct 15 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 16 October 2018 at 03:23:21 UTC, Jesse Phillips wrote:

 It would be cool if D provided the easiest way to develop 
 webasm first to see if it could claim that market.


If you have some minutes to spare it would be great if you could 
try it out. It should only take 10min to render your first divs, 
otherwise something is wrong.

The major hurdle with any wasm dom framework is that there are no 
standard components to build on (like dropdowns, drag-n-drop, 
input validations, notifications, etc.), nor any css frameworks 
(like material ui, bootstrap). So'll need to build everything 
from scratch, and no sane person is likely to do that.

What might be an option is to try to integrate with other wasm 
libraries out there, so at least we can use their components. But 
everyone does his own thing, so integrating is going to be hard 
as well.
Oct 16 2018
parent reply aberba <karabutaworld gmail.com> writes:
On Tuesday, 16 October 2018 at 07:57:12 UTC, Sebastiaan Koppe 
wrote:

 On Tuesday, 16 October 2018 at 03:23:21 UTC, Jesse Phillips 
 wrote:

 It would be cool if D provided the easiest way to develop 
 webasm first to see if it could claim that market.


 If you have some minutes to spare it would be great if you 
 could try it out. It should only take 10min to render your 
 first divs, otherwise something is wrong.

 The major hurdle with any wasm dom framework is that there are 
 no standard components to build on (like dropdowns, 
 drag-n-drop, input validations, notifications, etc.), nor any 
 css frameworks (like material ui, bootstrap). So'll need to 
 build everything from scratch, and no sane person is likely to 
 do that.

 What might be an option is to try to integrate with other wasm 
 libraries out there, so at least we can use their components. 
 But everyone does his own thing, so integrating is going to be 
 hard as well.


A common use case for wasm is to port C++ native apps to web. 
e.g. is the recent autoCAD web app which does almost everything 
the desktop app can. That's the only reason to IMO do stuff in 
wasm. Games, productivity software, etc...performance. Spasm 
might just be perfect for that kind of stuff
Oct 17 2018
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 17 October 2018 at 19:07:16 UTC, aberba wrote:

 A common use case for wasm is to port C++ native apps to web. 
 e.g. is the recent autoCAD web app which does almost everything 
 the desktop app can. That's the only reason to IMO do stuff in 
 wasm. Games, productivity software, etc...performance. Spasm 
 might just be perfect for that kind of stuff


There are issues getting the current GC ported to webassembly, so 
it is hard to port D code to wasm. That is one of the reasons why 
spasm has taken the betterC approach.

But remember, spasm is just a library to render and update html, 
and to respond to dom events. It won't help you in anyway to port 
something to wasm.

If you really want to port existing D code to wasm you either 
need to rewrite that in betterC or port druntime, which includes 
writing a precise GC. Dscripten-tools is a move in that direction.

The reason spasm exists is because I wanted to optimise web 
application's rendering code at compile-time, to reduce the 
runtime (setup) costs and to deliver high performant web 
applications.

I first tried to do that by writing a javascript optimiser that 
can take React code as input and spit out highly optimised js 
code. I got pretty far with that but at one point I realised that 
to do it well I needed advanced things like data-flow analysis 
and abstract interpretation. So I decided to ditched that and 
just use D's static introspection and LLVM's wasm target. A 
couple of weeks after that spasm was born.
Oct 18 2018
parent Suliman <evermind live.ru> writes:
https://github.com/kripken/emscripten/wiki/Pthreads-with-WebAssembly
Oct 18 2018
prev sibling parent reply a11e99z <black80 bk.ru> writes:
On Friday, 12 October 2018 at 19:43:25 UTC, Sebastiaan Koppe 
wrote:

 I like to announce Spasm https://github.com/skoppe/spasm


hi. can not compile for Windows

DUB used first time.
LDC ver 1.16.0.

 C:\temp\D\test_spasm>where dub.exe
 C:\programz\D\ldc2\bin\dub.exe
 C:\programz\D\dmd2\windows\bin\dub.exe


https://github.com/skoppe/spasm#how-to-start OK
https://github.com/skoppe/spasm#how-to-compile-your-application 
ERROR

 C:\temp\D\test_spasm>dub build --compiler=ldc2 --build=release
 Performing "release" build using ldc2 for x86_64.
 bolts 0.11.1: target for configuration "library" is up to date.
 optional 0.10.1: target for configuration "unittest" is up to 
 date.
 stdx-allocator 2.77.3: target for configuration "library" is up 
 to date.
 spasm 0.1.13: target for configuration "library" is up to date.
 test_spasm ~master: building configuration "application"...
 Error: unrecognized file extension lib               <<<<<<<<   
 ??????
 ldc2 failed with exit code 1.


and I have some questions:
- can DUB generate VisualD project?
- spasm contains some GC-allocator. can I use "hello" ~ 
123.to!string or something? or I can use string literals only 
without some kind of string builder?
Aug 06 2019
next sibling parent JN <666total wp.pl> writes:
On Tuesday, 6 August 2019 at 19:02:09 UTC, a11e99z wrote:

 hi. can not compile for Windows

 DUB used first time.
 LDC ver 1.16.0.

 C:\temp\D\test_spasm>where dub.exe
 C:\programz\D\ldc2\bin\dub.exe
 C:\programz\D\dmd2\windows\bin\dub.exe




Apparently some guy made it working on Windows by modifying some 
stdlib files:

https://github.com/skoppe/spasm/issues/15
Aug 06 2019
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 6 August 2019 at 19:02:09 UTC, a11e99z wrote:

 hi. can not compile for Windows
 LDC ver 1.16.0.


Currently ldc 1.16.0 isn't supported. You can downgrade to ldc 
1.15.0


 spasm 0.1.13: target for configuration "library" is up to date.
 test_spasm ~master: building configuration "application"...
 Error: unrecognized file extension lib               <<<<<<<<
  ??????
 ldc2 failed with exit code 1.




Have you tried the github issues? I remember dukc having the same 
issue on windows as well.


 and I have some questions:
 - spasm contains some GC-allocator. can I use "hello" ~ 
 123.to!string or something? or I can use string literals only 
 without some kind of string builder?


No you cannot use the concat operator (~). Neither can you use 
new, class, AA or dynamic arrays among others. This is because 
Spasm uses betterC, and a lot of D features aren't available in 
betterC. I am working on a PR for druntime to at least compile to 
wasm without betterC. This will open the way to incrementally 
support more and more D features.

In the meantime you need to get familiar with the betterC 
constraints. You can always look in the examples for workable 
code. Make sure to use the 0.1.13 tag, since master has some new 
unreleased stuff (which I hope to release in the coming month).

There is a string builder in spasm as well as a betterC version 
of the phobos `text` function.

The GC allocator is still unreleased and experimental.
Aug 06 2019
parent reply a11e99z <black80 bk.ru> writes:
On Tuesday, 6 August 2019 at 20:20:13 UTC, Sebastiaan Koppe wrote:

 On Tuesday, 6 August 2019 at 19:02:09 UTC, a11e99z wrote:

 hi. can not compile for Windows
 LDC ver 1.16.0.


 Currently ldc 1.16.0 isn't supported. You can downgrade to ldc 
 1.15.0


 spasm 0.1.13: target for configuration "library" is up to 
 date.
 test_spasm ~master: building configuration "application"...
 Error: unrecognized file extension lib               <<<<<<<<
  ??????
 ldc2 failed with exit code 1.




 Have you tried the github issues? I remember dukc having the 
 same issue on windows as well.


tried. --combined => got error about time.d. could not fix, idk 
how to fix.

but noticed (dub ... -verbose):

 LDC 1.16.0 is working (it compiles manually)
 next is all one command for ldc2:


------------------------------
ldc2 -march=wasm32 -mtriple=wasm32-unknown-unknown-wasm

(TRIED TO COMPILE AS EXE. STRANGE.)
-of.dub\build\application-release-windows-x86_64-ldc_2086-56E77341043F1C073D6FBEAF77ADA9DD\test_spasm.exe

-release -enable-inlining -Hkeep-all-bodies -O3 -w -betterC -oq 
-od=.dub/obj -d-version=Have_test_spasm -d-version=Have_spasm 
-d-version=Have_optional -d-version=Have_stdx_allocator 
-d-version=Have_silly -d-version=Have_bolts -Isource 
-IC:\Users\Administrator\AppData\Local\dub\packages\spasm-0.1.13\spasm\source
-IC:\Users\Administrator\AppData\Local\dub\packages\optional-0.
0.1\optional\source -IC:\Users\Administrator\AppData\Local\dub\packages\optional-0
10.1\optional\tests -IC:\Users\Administrator\AppData\Local\dub\packag
s\silly-0.8.2\silly -IC:\Users\Administrator\AppData\Local\dub\packages\bolts
0.11.1\bolts\source -IC:\Users\Administrator\AppData\Local\dub\packages\stdx-allocator-2.77.3\s
dx-allocator\source source\app.d

(THEN GOES LIBs)
C:\Users\Administrator\AppData\Local\dub\packages\spasm-0.1.13\spasm\.dub\build\library-release-windows-x86_64-ldc_2086-03DE6820B544BF9EAB97CE
5CB0DA4D7\spasm.lib C:\Users\Administrator\AppData\Local\dub\packages\optional-0.10.1\optional\.dub\build\unittest-release-windows-x86_64-ldc_2086-DEEA6FC19C1418B5171E828BA
E99DC8\optional.lib C:\Users\Administrator\AppData\Local\dub\packages\bolts-0.11.1\bolts\.dub\build\library-release-windows-x86_64-ldc_2086-E329422C7585D577B1C88E
E057669D3\bolts.lib C:\Users\Administrator\AppData\Local\dub\packages\stdx-allocator-2.77.3\stdx-allocator\.dub\build\library-release-windows-x86_64-ldc_2086-3B7150DF4B0522F5FB22F1145FECC671\stdx-allocator.lib

(THEN GOES OPTIONS AGAIN)
-L=-allow-undefined -L=-import-memory -L=--export-table 
-L=--export=domEvent -L=--export=allocString -vcolumns
------------------------------
Error: unrecognized file extension lib
------------------------------

so, I call ldc2 manually with almost same args but:
- all libs I put in quotes with prefix -L="././.lib" with 
Unix-slashes (not Windows \)
- moved all libs as last params
- removed -of option that request EXE

after such work LDC compiled my sample to valid WASM (not ran it 
yet, but see as WAT through 
https://webassembly.github.io/wabt/demo/wasm2wat/)
app.wasm 2,216,143 (size in bytes)

so, I think need to fix dub/sdl/json generating for wasm/spasm 
for Windows
- -of=..WASM not EXE
- libs as last with prefix -L="". probably all file names better 
put in quotes. probably with Unix-slashes
Aug 06 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 6 August 2019 at 22:57:52 UTC, a11e99z wrote:

 tried. --combined => got error about time.d. could not fix, idk 
 how to fix.


If calling ldc2 manually you need to use the -betterC flag, see 
https://wiki.dlang.org/Generating_WebAssembly_with_LDC for the 
minimal working example.


 so, I think need to fix dub/sdl/json generating for wasm/spasm 
 for Windows
 - -of=..WASM not EXE
 - libs as last with prefix -L="". probably all file names 
 better put in quotes. probably with Unix-slashes


I have opened a dub issue for the .lib .exe issue. 
https://github.com/dlang/dub/issues/1749
Aug 07 2019
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Tuesday, 6 August 2019 at 22:57:52 UTC, a11e99z wrote:

 On Tuesday, 6 August 2019 at 20:20:13 UTC, Sebastiaan Koppe 
 wrote:

 On Tuesday, 6 August 2019 at 19:02:09 UTC, a11e99z wrote:

 hi. can not compile for Windows
 LDC ver 1.16.0.


 Currently ldc 1.16.0 isn't supported. You can downgrade to ldc 
 1.15.0


 spasm 0.1.13: target for configuration "library" is up to 
 date.
 test_spasm ~master: building configuration "application"...
 Error: unrecognized file extension lib               <<<<<<<<
  ??????
 ldc2 failed with exit code 1.




 Have you tried the github issues? I remember dukc having the 
 same issue on windows as well.


 tried. --combined => got error about time.d. could not fix, idk 
 how to fix.


Read further down the github issue. You have to hack some 
included DUB libraries, unless you can come up with a cleaner 
solution than I did.


 LDC 1.16.0 is working (it compiles manually)


ldc2 1.16.0 will compile the code, but not link it. And if you 
want to run ldc2 manually, you will want to dry-run dub with 
verbose and copy the invocation from it. A `--combined` build is 
too complicated to get working otherwise.

Don't mind DUB trying to build an executable. In fact, it has to 
be. If it builds a "library", it will silently generate some code 
that's not WASM, or at least can't be used as WASM. The file 
extension will be incorrect, but you can change it. Sebastiaan 
just recently opened a LDC2 issue for that.
Aug 07 2019