www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Question about WebAssembly

reply materus <materus213 gmail.com> writes:
Are there any plans to support webassembly?
I know betterC is working but i mean full language.

what's preventing D to work?
Oct 02 2019
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 2 October 2019 at 09:26:56 UTC, materus wrote:

 Are there any plans to support webassembly?
 I know betterC is working but i mean full language.

 what's preventing D to work?


https://github.com/skoppe/spasm
Oct 02 2019
prev sibling next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 2 October 2019 at 09:26:56 UTC, materus wrote:

 Are there any plans to support webassembly?
 I know betterC is working but i mean full language.

 what's preventing D to work?


The short answer is manpower.

Long answer depends on what you mean by working. Go produces very 

browser. TinyGo is an option but doesn't support the full 
language [https://tinygo.org/lang-support/]. Rust is the most 
mature, mostly because it doesn't have a runtime to port, so it 
is a lot easier.

With D the main issue is that someone needs to port druntime to 
wasm. I am working on it in my spare time, but progress is slow.

Here are some examples of what I have got running:

https://skoppe.github.io/spasm/examples/todo-mvc/ -> famous 
todo-mvc
https://skoppe.github.io/spasm-imgui/ -> use c++ library
https://skoppe.github.io/spasm/examples/underrun/ -> ported js 
game
https://skoppe.github.io/spasm-tradingview-example/index.html ->  
calling typescript
Oct 02 2019
parent reply Max Haughton <maxhaton gmail.com> writes:
On Wednesday, 2 October 2019 at 10:23:45 UTC, Sebastiaan Koppe 
wrote:

 On Wednesday, 2 October 2019 at 09:26:56 UTC, materus wrote:

 [...]


 The short answer is manpower.

 Long answer depends on what you mean by working. Go produces 

 in your browser. TinyGo is an option but doesn't support the 
 full language [https://tinygo.org/lang-support/]. Rust is the 
 most mature, mostly because it doesn't have a runtime to port, 
 so it is a lot easier.

 With D the main issue is that someone needs to port druntime to 
 wasm. I am working on it in my spare time, but progress is slow.

 Here are some examples of what I have got running:

 https://skoppe.github.io/spasm/examples/todo-mvc/ -> famous 
 todo-mvc
 https://skoppe.github.io/spasm-imgui/ -> use c++ library
 https://skoppe.github.io/spasm/examples/underrun/ -> ported js 
 game
 https://skoppe.github.io/spasm-tradingview-example/index.html 
 ->  calling typescript


Is there still a barrier on having syscalls to use, e.g. being 
able to implement malloc/free? Assuming it's technically possible 
I would like to work on this but probably in 2020 :(
Oct 03 2019
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Thursday, 3 October 2019 at 15:33:54 UTC, Max Haughton wrote:

 Is there still a barrier on having syscalls to use, e.g. being 
 able to implement malloc/free? Assuming it's technically 
 possible I would like to work on this but probably in 2020 :(


Why would you need syscalls to implement malloc/free? Just 
allocate out of a static buffer. It is not even that hard to do, 
versions I use myself (feel free to use for any purpose, but bad 
code quality and high probability of bugs as it's fairly little 
used):

```
 nogc:

/+
0x0000_0000 = varaamaton
0x0000_0001 = varattu, yksi 4kt lohko
0x0000_0002 = varattu, yli 4kt
0x0000_0003 = varattu, osa varausta aikaisemmalla sivulla
0x0000_0004 = puoliksi varattu, yksi 2kt lohko
0x0000_0005 = kokonaan varattu, 2 kt lohkoissa
0x0000_0007 = jälkimmäinen 2kt puolisko varattu
0x0000_0010 = varattu, yksi 1kt lohko
0x0000_0011 = varattu, 2*1kt
0x0000_0013 = varattu kokonaan, 1kt lohkoissa
0x0000_0015 = varattu 2*1kt joista jompi kumpi vapautettu
0x0000_001B = varattu 4*1kt, kaksi vapautettu
0x0000_001F = varattu 4*1kt, kolme vapautettu
0x0000_004X = varattu 512t lohkoissa
0x0000_01XX = varattu 256t lohkoissa
0x0000_040X = varattu 128t lohkoissa
0x0000_1XXX = varattu 64t lohkoissa
0x0000_4XXX = varattu 32t lohkoissa
0x0001_XXXX = varattu 16t lohkoissa
0x0004_XXXX = varattu 8t lohkoissa
0x01XX_XXXX = varattu 1t lohkoissa
+/

enum memPageAmount = 0x200u;
enum memSizeClasses = 13u;
enum pageSize = 1u << memSizeClasses-1;
__gshared ushort[memSizeClasses] lastMemoryPages;
__gshared uint[memPageAmount] heapMap;
__gshared void[pageSize][memPageAmount] heap = void;

export extern(C) void* malloc(size_t amount)
{   if (amount == 0) return null;
     foreach(allocSize; memSizeClasses.iota)
         if (amount <= (1u << allocSize))
         foreach(pageI; 
lastMemoryPages[allocSize].iota(lastMemoryPages[allocSize] + 
memPageAmount).map!(i => cast(short)i))
     {   pageI %= memPageAmount;

         void* result;
         if
         (   heapMap[pageI] >= 0x0100_0000u >>> allocSize * 2
             &&  heapMap[pageI] <  0x0300_0000u >>> allocSize * 2
             && ~heapMap[pageI] &  0x0000_0FFFu >>> allocSize
         )
         {   result = &heap[pageI][0] + ((heapMap[pageI] & 0xFFF 

 allocSize) + 1) * (1u << allocSize);






             //merkitsee muistikarttaan lohkon käyttöönoton.
             heapMap[pageI]++;
         } else if (heapMap[pageI] == 0)
         // varaamaton sivu. Eiköhän oteta käyttöön.
         {   heapMap[pageI] = 0x0100_0000u >>> allocSize * 2;
             result = heap[pageI].ptr;
         }
         else continue;

         assert(result >= heap.ptr && result < heap.ptr + 
memPageAmount);
         lastMemoryPages[allocSize] = pageI;
         assert(heapMap[pageI] != 0);
         return result;
     }

     //yli muistisivun kokoisen alueen varaus
     auto pagesNeeded = (amount - 1) / pageSize + 1;
     size_t result = 0;
     foreach (canditate; heapMap[].slide(pagesNeeded))
     {   if (canditate.all!(page => page == 0u)) goto foundArea;
         result++;
     }
     //muisti loppu
     return null;

     foundArea:
     heapMap[result] = 2u;
     foreach(ref pageMark; heapMap[].drop(result + 
1).takeExactly(pagesNeeded-1))
     {   pageMark = 3u;
     }

     return heap[result].ptr;
}

export extern(C) void free(const(void)* cAllocation)
{   //Oletettavasti tälle funktiolle välitetty osoitin dumpataan 
jorpakkoon heti jälkeenpäin,
     //joten tämä tyyppimuunnos ei luultavasti aiheuta ongelmia.
     auto allocation = cast(void*) cAllocation;
     if (allocation == null) return;
     assert(allocation >= heap.ptr && allocation < heap.ptr + 
memPageAmount, "free() ei osoita kekoon!");

     ptrdiff_t allocAddr = allocation - &heap[0][0];
     size_t pageI = allocAddr / pageSize;
     size_t pageOffset = allocAddr % pageSize;

     assert (heapMap[pageI] != 0, "Yritetään vapauttaa jo 
valmiiksi varaamatonta muistia");
     assert (heapMap[pageI] != 3, "Muistin vapautus ei osoita 
varatun pätkän alkuun");

     foreach(allocSize; memSizeClasses.iota) if
     (   heapMap[pageI] >= 0x0100_0000u >>> allocSize * 2
         &&  heapMap[pageI] <  0x0300_0000u >>> allocSize * 2
     )
     {   /+assert
         (   heapMap[pageI] >= cast(ulong)pageOffset * pageSize + 
0x0100_0000u >>> allocSize * 2,
             "Yritetään vapauttaa jo valmiiksi varaamatonta 
muistia"
         );+/
         assert (pageOffset % 1u << allocSize == 0, "Muistin 
vapautus ei osoita varatun pätkän alkuun");

         //kirjoitetaan muistiin: yksi vapautettu.
         heapMap[pageI] += pageSize >>> allocSize;
         if (heapMap[pageI] > 0x0200_0000u >>> allocSize * 2) 
heapMap[pageI] = 0;
         return;
     }

     assert (heapMap[pageI] == 2u);
     heapMap[pageI] = 0;
     foreach(ref pageMark; heapMap[pageI+1 .. $].until!(a => 
a!=3u)) pageMark = 0;
}
```
Oct 03 2019
parent Max Haughton <maxhaton gmail.com> writes:
On Thursday, 3 October 2019 at 16:28:43 UTC, Dukc wrote:

 On Thursday, 3 October 2019 at 15:33:54 UTC, Max Haughton wrote:

 [...]


 Why would you need syscalls to implement malloc/free? Just 
 allocate out of a static buffer. It is not even that hard to 
 do, versions I use myself (feel free to use for any purpose, 
 but bad code quality and high probability of bugs as it's 
 fairly little used):

 [...]


I am aware, but I was more thinking about the other calls too. 
Obviously it can be done as you have described, but it also would 
involve completely reimplementing druntime
Oct 03 2019
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 3 October 2019 at 15:33:54 UTC, Max Haughton wrote:

 Is there still a barrier on having syscalls to use, e.g. being 
 able to implement malloc/free? Assuming it's technically 
 possible I would like to work on this but probably in 2020 :(


It all depends on what you want to do. If you want to write 
client side web applications, you likely have no need for 
syscalls, since you will just use the web apis, like I did in the 
examples I posted earlier.

If you want your wasm to run outside the browser, e.g. in the 
cloud, then your best bet would be to port druntime to WASI. Or 
use the libc provided by the WASI C sdk and write bindings for 
druntime for it.

On the other hand, I already have hello-world code running on 
cloudflare workers (http://skoppe.nl/hello) and have also had 
something on fastly as well.

In the next month or so I want to finish a document outlining my 
approach to getting more of D supported in WebAssembly. 
Ultimately that will require porting druntime, or at least its 
most essential features.

What do you want to build?
Oct 03 2019
parent reply Jani Hur <spam com.invalid> writes:
Is D's WebAssembly support usable in Screeps 
(https://screeps.com/) ?

I just yesterday run into Screeps so I have no experience about 
it but the following caught my eye:

"Use JavaScript or compile other languages via WebAssembly."

See https://docs.screeps.com/modules.html#Binary-modules about 
the Screeps' WASM support. As I know nothing about Screeps and 
WebAssembly that tells me nothing at the moment :(
Dec 10 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 11 December 2019 at 07:46:23 UTC, Jani Hur wrote:

 Is D's WebAssembly support usable in Screeps 
 (https://screeps.com/) ?

 I just yesterday run into Screeps so I have no experience about 
 it but the following caught my eye:

 "Use JavaScript or compile other languages via WebAssembly."

 See https://docs.screeps.com/modules.html#Binary-modules about 
 the Screeps' WASM support. As I know nothing about Screeps and 
 WebAssembly that tells me nothing at the moment :(


Of course. Follow the 'Building WebAssembly' section on 
https://wiki.dlang.org/Generating_WebAssembly_with_LDC to compile 
a wasm file, then call that using the instructions on the 
https://docs.screeps.com/modules.html#Binary-modules page. Just 
skip the part about emscripten since you already have a wasm file.

The only caveat is that it is betterC. But that should change 
soon.
Dec 11 2019
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 11 December 2019 at 07:46:23 UTC, Jani Hur wrote:

 Is D's WebAssembly support usable in Screeps 
 (https://screeps.com/) ?

 I just yesterday run into Screeps so I have no experience about 
 it but the following caught my eye:

 "Use JavaScript or compile other languages via WebAssembly."


Yes it is. Interfacing will be real hard, but it can be done if 
you somehow enjoy spending a large chunk of your freetime on that.
Dec 11 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 11 December 2019 at 08:40:13 UTC, Dukc wrote:

 On Wednesday, 11 December 2019 at 07:46:23 UTC, Jani Hur wrote:

 Is D's WebAssembly support usable in Screeps 
 (https://screeps.com/) ?

 I just yesterday run into Screeps so I have no experience 
 about it but the following caught my eye:

 "Use JavaScript or compile other languages via WebAssembly."


 Yes it is. Interfacing will be real hard, but it can be done if 
 you somehow enjoy spending a large chunk of your freetime on 
 that.


Interfacing is indeed the hardest part. The bindings in spasm can 
help you with calling webapi's (although you will need a custom 
entry for Screeps, but it is just JS). I have started a 
typescript binding generator that can wrap typescript libraries, 
but that isn't fully completed yet.

Hopefully the webassembly interface types proposal will stabilise 
next year, as this will ease calling other languages and dealing 
with strings/structs/arrays.
Dec 11 2019
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 2 October 2019 at 09:26:56 UTC, materus wrote:




If you mean Bridge.NET[1], well, nothing in theory. Bridge.NET 
essentially implements .NET runtime library in JavaScript, and 

compiler that does the same to D -indeed, it would likely be 
easier thanks to lighter runtime library. But still, you'd need a 
lot of manpower -Bridge.NET has a company behind it, not just 
freetime volunteers. Bridge.NET is the main tool behind my 
commercial project, with some parts written in Spasm.


useful, the challenge of doing a specialized JavaScript 
transpiler shows. It often has worse error messages than D 
compilers for errors of similar complexity, and you do run into a 
codegen bug every now and then. Also, the runtime library isn't 
completely implemented, and there are not very good docs about 
what's implemented and what's not. When you consider how mature 


1: https://bridge.net/
Oct 02 2019
parent Mark Rousell <mark.rousell signal100.com> writes:
On 02/10/2019 15:27, Dukc via Digitalmars-d wrote:

 On Wednesday, 2 October 2019 at 09:26:56 UTC, materus wrote:




 If you mean Bridge.NET[1], well, nothing in theory. Bridge.NET
 essentially implements .NET runtime library in JavaScript, and

 that does the same to D -indeed, it would likely be easier thanks to
 lighter runtime library. But still, you'd need a lot of manpower
 -Bridge.NET has a company behind it, not just freetime volunteers.
 Bridge.NET is the main tool behind my commercial project, with some
 parts written in Spasm.


 the challenge of doing a specialized JavaScript transpiler shows. It
 often has worse error messages than D compilers for errors of similar
 complexity, and you do run into a codegen bug every now and then.
 Also, the runtime library isn't completely implemented, and there are
 not very good docs about what's implemented and what's not. When you


 1: https://bridge.net/


https://blazor.net/ surely.



-- 
Mark Rousell
Oct 02 2019