www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Simple web server benchmark - vibe.d is slower than node.js and Go?

reply Vadim Lopatin <coolreader.org gmail.com> writes:
There is a simple set of simple web server apps written in 
several languages (Go, Rust, Scala, Node-js):

https://github.com/nuald/simple-web-benchmark

I've sent PR to include D benchmark (vibe.d).

I was hoping it could show performance at least not worse than 
other languages.
But it appears to be slower than Go and even Node.js

Are there any tips to achieve better performance in this test?

Under windows, I can get vibe.d configured to use libevent to 
show results comparable with Go. With other configurations, it 
works two times slower.

Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. The 
only advantage of D here is CPU load - 90% vs 120% in Go.

I'm using DMD. Probably, ldc could speed up it a bit.

Probably, it's caused by single threaded async implementation 
while other languages are using parallel handling of requests?
Sep 21 2017
next sibling parent reply Suliman <evermind live.ru> writes:
Can it be issue with regex speed?
auto reg = ctRegex!"^/greeting/([a-z]+)$";

Did you try without it?
Sep 21 2017
parent Vadim Lopatin <coolreader.org gmail.com> writes:
On Thursday, 21 September 2017 at 08:18:51 UTC, Suliman wrote:

 Can it be issue with regex speed?
 auto reg = ctRegex!"^/greeting/([a-z]+)$";

 Did you try without it?


Regex is used for request paths like "/greeting/username" and not 
used for path "/"

There is no big slowdown caused by regex.
45.6K requests/second with regex vs 46.8K requests/second w/o 
regex.
Sep 21 2017
prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 21.09.2017 um 10:01 schrieb Vadim Lopatin:

 There is a simple set of simple web server apps written in several 
 languages (Go, Rust, Scala, Node-js):
 
 https://github.com/nuald/simple-web-benchmark
 
 I've sent PR to include D benchmark (vibe.d).
 
 I was hoping it could show performance at least not worse than other 
 languages.
 But it appears to be slower than Go and even Node.js
 
 Are there any tips to achieve better performance in this test?
 
 Under windows, I can get vibe.d configured to use libevent to show 
 results comparable with Go. With other configurations, it works two 
 times slower.
 
 Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. The only 
 advantage of D here is CPU load - 90% vs 120% in Go.
 
 I'm using DMD. Probably, ldc could speed up it a bit.
 
 Probably, it's caused by single threaded async implementation while 
 other languages are using parallel handling of requests?


Multi-threading (or multi-process processing) is probably the main 
reason. The following initialization code should get this on par:

     shared static this()
     {
         runWorkerTaskDist({
             listenHTTP("0.0.0.0:3000", &handleRequest);
         });
     }

The other part is that currently the HTTP server code works rather 
inefficiently with the new vibe-core implementation, ironically due to 
the main feature that is meant to speed up vibe-core over the previous 
implementation: using statically typed structs instead of dynamic 
interfaces for streams. This currently requires using proxy objects in 
the HTTP server, which perform their own type of dynamic dispatch, with 
a higher overhead than using classes/interfaces directly.

But there is a pending redesign of the whole HTTP implementation, which 
will, among other things, get rid of this and will use statically typed 
streams throughout the code. It should then be considerably faster than 
the current code path that uses classes/interfaces.

Finally, there is also a considerable performance bug in vibe-core 
currently, which I can't fix due to an unresolved Optlink bug: 
https://github.com/vibe-d/vibe-core/pull/27 (I tried to reduce this with 
dustmite, took about a week, but of course it reduced to a piece of code 
that was actually broken - I'll have to redo this with using the MS 
linker in parallel as a counter test)
Sep 21 2017
parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
      shared static this()
      {
          (...)
      }


BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

instead and dropping the VibeDefaultMain version in the package recipe. 
The old approach is planned to be faded out slowly, because it requires 
some special DUB support that would be nice to see gone at some point.
Sep 21 2017
parent reply Vadim Lopatin <coolreader.org gmail.com> writes:
On Thursday, 21 September 2017 at 10:21:17 UTC, Sönke Ludwig 
wrote:

      shared static this()
      {
          (...)
      }


 BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

 instead and dropping the VibeDefaultMain version in the package 
 recipe. The old approach is planned to be faded out slowly, 
 because it requires some special DUB support that would be nice 
 to see gone at some point.


Thank you!
Trying to use multithreaded mode.

Under windows, best Vibe.d results are equal to results of Go 
server.
Under linux, Vibe.d is still slower even if there are 4 threads 
listening.

Results:

OS       lang:config   req/s    comments
=======  ==========    =======  ==========
Linux    go            53K
Linux    D:default     48K      1 thread
Linux    D:libevent    48K      1 thread
Linux    D:libasync    46.5K    4 threads

Windows  go            12K
Windows  D:default     12K      4 threads
Windows  D:libevent    12K      4 threads
Windows  D:libasync    7K       4 threads

Under Linux in default and libevent configurations, I see error 
messages from 3 of 4 threads - looks like only one thread can 
process connections.

Failed to listen on 0.0.0.0:3000
Task terminated with uncaught exception: Failed to listen for 
incoming HTTP connections on any of the supplied interfaces.
Sep 21 2017
next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:

 On Thursday, 21 September 2017 at 10:21:17 UTC, Sönke Ludwig wrote:

      shared static this()
      {
          (...)
      }


 BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

 instead and dropping the VibeDefaultMain version in the package 
 recipe. The old approach is planned to be faded out slowly, because it 
 requires some special DUB support that would be nice to see gone at 
 some point.


 
 Thank you!
 Trying to use multithreaded mode.
 
 Under windows, best Vibe.d results are equal to results of Go server.
 Under linux, Vibe.d is still slower even if there are 4 threads listening.
 
 Results:
 
 OS       lang:config   req/s    comments
 =======  ==========    =======  ==========
 Linux    go            53K
 Linux    D:default     48K      1 thread
 Linux    D:libevent    48K      1 thread
 Linux    D:libasync    46.5K    4 threads
 
 Windows  go            12K
 Windows  D:default     12K      4 threads
 Windows  D:libevent    12K      4 threads
 Windows  D:libasync    7K       4 threads
 
 Under Linux in default and libevent configurations, I see error messages 
 from 3 of 4 threads - looks like only one thread can process connections.
 
 Failed to listen on 0.0.0.0:3000
 Task terminated with uncaught exception: Failed to listen for incoming 
 HTTP connections on any of the supplied interfaces.
 
 


Oh, sorry, I forgot the reusePort option, so that multiple sockets can 
listen on the same port:

     auto settings = new HTTPServerSettings("0.0.0.0:3000");
     settings.options |= HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);
Sep 21 2017
next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

On Thu, Sep 21, 2017 at 2:50 PM, S=C3=B6nke Ludwig via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:


 On Thursday, 21 September 2017 at 10:21:17 UTC, S=C3=B6nke Ludwig wrote:


      shared static this()

      {
          (...)
      }


 BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

 instead and dropping the VibeDefaultMain version in the package recipe.
 The old approach is planned to be faded out slowly, because it requires
 some special DUB support that would be nice to see gone at some point.


 Thank you!
 Trying to use multithreaded mode.

 Under windows, best Vibe.d results are equal to results of Go server.
 Under linux, Vibe.d is still slower even if there are 4 threads listenin=




g.

 Results:

 OS       lang:config   req/s    comments
 =3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D    =3D=3D=3D=3D=3D=




=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 Linux    go            53K
 Linux    D:default     48K      1 thread
 Linux    D:libevent    48K      1 thread
 Linux    D:libasync    46.5K    4 threads

 Windows  go            12K
 Windows  D:default     12K      4 threads
 Windows  D:libevent    12K      4 threads
 Windows  D:libasync    7K       4 threads

 Under Linux in default and libevent configurations, I see error messages
 from 3 of 4 threads - looks like only one thread can process connections=




.

 Failed to listen on 0.0.0.0:3000
 Task terminated with uncaught exception: Failed to listen for incoming
 HTTP connections on any of the supplied interfaces.


 Oh, sorry, I forgot the reusePort option, so that multiple sockets can
 listen on the same port:

     auto settings =3D new HTTPServerSettings("0.0.0.0:3000");
     settings.options |=3D HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);
Sep 21 2017
prev sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
wrong version, this is my letest version:
https://paste.ofcode.org/qWsQikdhKiAywgBpKwANFR

On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

 On Thu, Sep 21, 2017 at 2:50 PM, S=C3=B6nke Ludwig via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:


 On Thursday, 21 September 2017 at 10:21:17 UTC, S=C3=B6nke Ludwig wrote=






:

      shared static this()

      {
          (...)
      }


 BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

 instead and dropping the VibeDefaultMain version in the package recipe=








.

 The old approach is planned to be faded out slowly, because it require=








s

 some special DUB support that would be nice to see gone at some point.


 Thank you!
 Trying to use multithreaded mode.

 Under windows, best Vibe.d results are equal to results of Go server.
 Under linux, Vibe.d is still slower even if there are 4 threads
 listening.

 Results:

 OS       lang:config   req/s    comments
 =3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D    =3D=3D=3D=3D=






=3D=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 Linux    go            53K
 Linux    D:default     48K      1 thread
 Linux    D:libevent    48K      1 thread
 Linux    D:libasync    46.5K    4 threads

 Windows  go            12K
 Windows  D:default     12K      4 threads
 Windows  D:libevent    12K      4 threads
 Windows  D:libasync    7K       4 threads

 Under Linux in default and libevent configurations, I see error message=






s

 from 3 of 4 threads - looks like only one thread can process connection=






s.

 Failed to listen on 0.0.0.0:3000
 Task terminated with uncaught exception: Failed to listen for incoming
 HTTP connections on any of the supplied interfaces.


 Oh, sorry, I forgot the reusePort option, so that multiple sockets can
 listen on the same port:

     auto settings =3D new HTTPServerSettings("0.0.0.0:3000");
     settings.options |=3D HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);
Sep 21 2017
next sibling parent reply Vadim Lopatin <coolreader.org gmail.com> writes:
On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak 
wrote:

 wrong version, this is my letest version: 
 https://paste.ofcode.org/qWsQikdhKiAywgBpKwANFR


Thank you!
Updated.
Sep 21 2017
next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
BTW: I have changed dub.json a little too:

{
    "name": "vibedtest",
    "targetName": "vibedtest",
    "targetType": "executable",
    "targetPath": "bin",
    "dependencies": {
        "vibe-d": "~>0.8.2-alpha.1"
    },
    "subConfigurations": {"vibe-d": "libevent"},
    "versions": [
        "VibeManualMemoryManagement"
    ],
    "buildTypes": {
        "release": { "buildOptions": ["releaseMode", "optimize", "inline"],
                     "dflags-posix-ldc": ["-flto=thin",
"-Xcc=-fuse-ld=gold"] }
    }
}

On Thu, Sep 21, 2017 at 3:32 PM, Vadim Lopatin via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:


 wrong version, this is my letest version: https://paste.ofcode.org/qWsQi
 kdhKiAywgBpKwANFR


 Thank you!
 Updated.
Sep 21 2017
prev sibling next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
And it seems it is faster with GC. So removing VibeManualMemoryManagement
helped in some cases too.

On Thu, Sep 21, 2017 at 3:43 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 BTW: I have changed dub.json a little too:

 {
     "name": "vibedtest",
     "targetName": "vibedtest",
     "targetType": "executable",
     "targetPath": "bin",
     "dependencies": {
         "vibe-d": "~>0.8.2-alpha.1"
     },
     "subConfigurations": {"vibe-d": "libevent"},
     "versions": [
         "VibeManualMemoryManagement"
     ],
     "buildTypes": {
         "release": { "buildOptions": ["releaseMode", "optimize", "inline"],
                      "dflags-posix-ldc": ["-flto=thin",
 "-Xcc=-fuse-ld=gold"] }
     }
 }

 On Thu, Sep 21, 2017 at 3:32 PM, Vadim Lopatin via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:


 wrong version, this is my letest version: https://paste.ofcode.org/qWsQi
 kdhKiAywgBpKwANFR


 Thank you!
 Updated.
Sep 21 2017
prev sibling next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
Ok, after some more testing it seems it is almost sema with or
without VibeManualMemoryManagement

On Thu, Sep 21, 2017 at 3:44 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 And it seems it is faster with GC. So removing VibeManualMemoryManagement
 helped in some cases too.

 On Thu, Sep 21, 2017 at 3:43 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 BTW: I have changed dub.json a little too:

 {
     "name": "vibedtest",
     "targetName": "vibedtest",
     "targetType": "executable",
     "targetPath": "bin",
     "dependencies": {
         "vibe-d": "~>0.8.2-alpha.1"
     },
     "subConfigurations": {"vibe-d": "libevent"},
     "versions": [
         "VibeManualMemoryManagement"
     ],
     "buildTypes": {
         "release": { "buildOptions": ["releaseMode", "optimize",
 "inline"],
                      "dflags-posix-ldc": ["-flto=thin",
 "-Xcc=-fuse-ld=gold"] }
     }
 }

 On Thu, Sep 21, 2017 at 3:32 PM, Vadim Lopatin via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:


 wrong version, this is my letest version:
 https://paste.ofcode.org/qWsQikdhKiAywgBpKwANFR


 Thank you!
 Updated.
Sep 21 2017
parent reply Vadim Lopatin <coolreader.org gmail.com> writes:
On Thursday, 21 September 2017 at 13:46:49 UTC, Daniel Kozak 
wrote:

 Ok, after some more testing it seems it is almost sema with or 
 without VibeManualMemoryManagement


Do you see any benefits from using LDC instead of DMD?
Sep 21 2017
parent Daniel Kozak <kozzi11 gmail.com> writes:
with dmd 26K - 28K
witlh ldc 27K - 29K

On Thu, Sep 21, 2017 at 3:53 PM, Vadim Lopatin via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:46:49 UTC, Daniel Kozak wrote:


 Ok, after some more testing it seems it is almost sema with or without
 VibeManualMemoryManagement


 Do you see any benefits from using LDC instead of DMD?
Sep 21 2017
prev sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
s'/sema/same/

On Thu, Sep 21, 2017 at 3:46 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 Ok, after some more testing it seems it is almost sema with or without
 VibeManualMemoryManagement

 On Thu, Sep 21, 2017 at 3:44 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 And it seems it is faster with GC. So removing VibeManualMemoryManagement
 helped in some cases too.

 On Thu, Sep 21, 2017 at 3:43 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 BTW: I have changed dub.json a little too:

 {
     "name": "vibedtest",
     "targetName": "vibedtest",
     "targetType": "executable",
     "targetPath": "bin",
     "dependencies": {
         "vibe-d": "~>0.8.2-alpha.1"
     },
     "subConfigurations": {"vibe-d": "libevent"},
     "versions": [
         "VibeManualMemoryManagement"
     ],
     "buildTypes": {
         "release": { "buildOptions": ["releaseMode", "optimize",
 "inline"],
                      "dflags-posix-ldc": ["-flto=thin",
 "-Xcc=-fuse-ld=gold"] }
     }
 }

 On Thu, Sep 21, 2017 at 3:32 PM, Vadim Lopatin via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:


 wrong version, this is my letest version:
 https://paste.ofcode.org/qWsQikdhKiAywgBpKwANFR


 Thank you!
 Updated.
Sep 21 2017
prev sibling parent reply ade90036 <andrea.rizzini gmail.com> writes:
On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak 
wrote:

 wrong version, this is my letest version: 
 https://paste.ofcode.org/qWsQikdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:


 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

 On Thu, Sep 21, 2017 at 2:50 PM, Sönke Ludwig via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:


 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:


[...]


 Oh, sorry, I forgot the reusePort option, so that multiple 
 sockets can listen on the same port:

     auto settings = new HTTPServerSettings("0.0.0.0:3000");
     settings.options |= HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);






Hi, would it be possible to re-share the example of vibe.d woth 
multithreaded support.

The pastebin link has expired and the pull request doesnt have 
the latest version.

Thanks

Ade
Oct 30 2017
parent reply Daniel Kozak <kozzi11 gmail.com> writes:
Maybe this one:

import vibe.d;
import std.regex;
import std.array : appender;

static reg =3D ctRegex!"^/greeting/([a-z]+)$";

void main()
{
    setupWorkerThreads(logicalProcessorCount);
    runWorkerTaskDist(&runServer);
    runApplication();
}

void runServer()
{
    auto settings =3D new HTTPServerSettings;
    settings.options |=3D HTTPServerOption.reusePort;
    settings.port =3D 3000;
    settings.serverString =3D null;
    listenHTTP(settings, &handleRequest);
}

void handleRequest(HTTPServerRequest req,
                    HTTPServerResponse res)
{
    switch(req.path)
    {
    case "/": res.writeBody("Hello World", "text/plain");
        break;
    default:
        auto m =3D matchFirst(req.path, reg);
        string message =3D "Hello, ";
        auto app =3D appender(message);
        app.reserve(32);
        app ~=3D m[1];
        res.writeBody(app.data, "text/plain");
    }
}

On Mon, Oct 30, 2017 at 5:41 PM, ade90036 via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:


 wrong version, this is my letest version: https://paste.ofcode.org/qWsQi
 kdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak <kozzi11 gmail.com> wrote:

 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

 On Thu, Sep 21, 2017 at 2:50 PM, S=C3=B6nke Ludwig via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:

 [...]


 Oh, sorry, I forgot the reusePort option, so that multiple sockets can
 listen on the same port:

     auto settings =3D new HTTPServerSettings("0.0.0.0:3000");
     settings.options |=3D HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);






 Hi, would it be possible to re-share the example of vibe.d woth
 multithreaded support.

 The pastebin link has expired and the pull request doesnt have the latest
 version.

 Thanks

 Ade





 Oct 30 2017



 next sibling parent reply Arun Chandrasekaran <aruncxy gmail.com>  writes:


On Monday, 30 October 2017 at 17:23:02 UTC, Daniel Kozak wrote:

 Maybe this one:

 import vibe.d;
 import std.regex;
 import std.array : appender;

 static reg = ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     setupWorkerThreads(logicalProcessorCount);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings = new HTTPServerSettings;
     settings.options |= HTTPServerOption.reusePort;
     settings.port = 3000;
     settings.serverString = null;
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     switch(req.path)
     {
     case "/": res.writeBody("Hello World", "text/plain");
         break;
     default:
         auto m = matchFirst(req.path, reg);
         string message = "Hello, ";
         auto app = appender(message);
         app.reserve(32);
         app ~= m[1];
         res.writeBody(app.data, "text/plain");
     }
 }

 On Mon, Oct 30, 2017 at 5:41 PM, ade90036 via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak 
 wrote:


 wrong version, this is my letest version: 
 https://paste.ofcode.org/qWsQi kdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:

 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

 On Thu, Sep 21, 2017 at 2:50 PM, Sönke Ludwig via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:

 [...]


 Oh, sorry, I forgot the reusePort option, so that multiple 
 sockets can listen on the same port:

     auto settings = new HTTPServerSettings("0.0.0.0:3000");
     settings.options |= HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);






 Hi, would it be possible to re-share the example of vibe.d 
 woth multithreaded support.

 The pastebin link has expired and the pull request doesnt have 
 the latest version.

 Thanks

 Ade




I tried to run the example, and I'm struck with a linker error 
DMD v2.076.1 and LDC 1.4.0.

03-11-2017 11:34:16 arun-desk-r7 
~/code/personal/d/simple-web-benchmark
$ dub run --root=d --compiler=dmd --build=release --config=dmd
Fetching libevent 2.0.2+2.0.16 (getting selected version)...
Fetching diet-ng 1.4.3 (getting selected version)...
Fetching taggedalgebraic 0.10.7 (getting selected version)...
Fetching openssl 1.1.5+1.0.1g (getting selected version)...
Fetching botan 1.12.9 (getting selected version)...
Fetching memutils 0.4.9 (getting selected version)...
Fetching vibe-d 0.8.2-alpha.2 (getting selected version)...
Fetching vibe-core 1.2.1-alpha.2 (getting selected version)...
Fetching libasync 0.8.3 (getting selected version)...
Fetching botan-math 1.0.3 (getting selected version)...
Fetching eventcore 0.8.20 (getting selected version)...
Package vibe-core can be upgraded from 1.2.1-alpha.2 to 
1.3.0-alpha.2.
Package eventcore can be upgraded from 0.8.20 to 0.8.21.
Use "dub upgrade" to perform those changes.
Performing "release" build using dmd for x86_64.
vibe-d:utils 0.8.2-alpha.2: building configuration "library"...
vibe-d:data 0.8.2-alpha.2: building configuration "library"...
immutable(ubyte)[]
vibe-d:core 0.8.2-alpha.2: building configuration "libevent"...
immutable(ubyte)[]
vibe-d:crypto 0.8.2-alpha.2: building configuration "library"...
diet-ng 1.4.3: building configuration "library"...
vibe-d:stream 0.8.2-alpha.2: building configuration "library"...
vibe-d:textfilter 0.8.2-alpha.2: building configuration 
"library"...
vibe-d:inet 0.8.2-alpha.2: building configuration "library"...
vibe-d:tls 0.8.2-alpha.2: building configuration "notls"...
vibe-d:http 0.8.2-alpha.2: building configuration "library"...
immutable(ubyte)[]
vibe-d:mail 0.8.2-alpha.2: building configuration "library"...
vibe-d:mongodb 0.8.2-alpha.2: building configuration "library"...
vibe-d:redis 0.8.2-alpha.2: building configuration "library"...
immutable(ubyte)[]
vibe-d:web 0.8.2-alpha.2: building configuration "library"...
vibe-d 0.8.2-alpha.2: building configuration "libevent"...
vibedtest ~master: building configuration "dmd"...
Linking...
/usr/bin/ld: cannot find -levent
/usr/bin/ld: cannot find -levent_pthreads
collect2: error: ld returned 1 exit status
Error: linker exited with status 1
dmd failed with exit code 1.
03-11-2017 11:35:10 arun-desk-r7 
~/code/personal/d/simple-web-benchmark
$



 Nov 03 2017



 next sibling parent  Neia Neutuladh <neia ikeran.org>  writes:


On Friday, 3 November 2017 at 18:44:30 UTC, Arun Chandrasekaran 
wrote:

 I tried to run the example, and I'm struck with a linker error 
 DMD v2.076.1 and LDC 1.4.0.

 /usr/bin/ld: cannot find -levent
 /usr/bin/ld: cannot find -levent_pthreads


Did you try installing libevent and libevent-pthreads?



 Nov 03 2017



prev sibling  parent  CRAIG DILLABAUGH <craig.dillabaugh gmail.com>  writes:


On Friday, 3 November 2017 at 18:44:30 UTC, Arun Chandrasekaran 
wrote:

 On Monday, 30 October 2017 at 17:23:02 UTC, Daniel Kozak wrote:

 Maybe this one:


   clip

 vibedtest ~master: building configuration "dmd"...
 Linking...
 /usr/bin/ld: cannot find -levent
 /usr/bin/ld: cannot find -levent_pthreads
 collect2: error: ld returned 1 exit status
 Error: linker exited with status 1
 dmd failed with exit code 1.
 03-11-2017 11:35:10 arun-desk-r7 
 ~/code/personal/d/simple-web-benchmark
 $


Perhaps you need to install the libevent development package on 
your system (libevent-dev or something similar).



 Nov 03 2017



prev sibling  parent reply Arun Chandrasekaran <aruncxy gmail.com>  writes:


On Monday, 30 October 2017 at 17:23:02 UTC, Daniel Kozak wrote:

 Maybe this one:

 import vibe.d;
 import std.regex;
 import std.array : appender;

 static reg = ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     setupWorkerThreads(logicalProcessorCount);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings = new HTTPServerSettings;
     settings.options |= HTTPServerOption.reusePort;
     settings.port = 3000;
     settings.serverString = null;
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     switch(req.path)
     {
     case "/": res.writeBody("Hello World", "text/plain");
         break;
     default:
         auto m = matchFirst(req.path, reg);
         string message = "Hello, ";
         auto app = appender(message);
         app.reserve(32);
         app ~= m[1];
         res.writeBody(app.data, "text/plain");
     }
 }

 On Mon, Oct 30, 2017 at 5:41 PM, ade90036 via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak 
 wrote:


 wrong version, this is my letest version: 
 https://paste.ofcode.org/qWsQi kdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:

 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj

 On Thu, Sep 21, 2017 at 2:50 PM, Sönke Ludwig via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:

 [...]


 Oh, sorry, I forgot the reusePort option, so that multiple 
 sockets can listen on the same port:

     auto settings = new HTTPServerSettings("0.0.0.0:3000");
     settings.options |= HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);






 Hi, would it be possible to re-share the example of vibe.d 
 woth multithreaded support.

 The pastebin link has expired and the pull request doesnt have 
 the latest version.

 Thanks

 Ade




With vibe.d 0.8.2, even when multiple worker threads are setup, 
only one thread handles the requests:

```
import core.thread;
import vibe.d;
import std.experimental.all;

auto reg = ctRegex!"^/greeting/([a-z]+)$";

void main()
{
     writefln("Master %d is running", getpid());
     setupWorkerThreads(logicalProcessorCount + 1);
     runWorkerTaskDist(&runServer);
     runApplication();
}

void runServer()
{
     auto settings = new HTTPServerSettings;
     settings.options |= HTTPServerOption.reusePort;
     settings.port = 8080;
     settings.bindAddresses = ["127.0.0.1"];
     listenHTTP(settings, &handleRequest);
}

void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
{
     writeln("My Thread Id: ", to!string(thisThreadID));
     // simulate long runnig task
     Thread.sleep(dur!("seconds")(3));
	
     if (req.path == "/")
         res.writeBody("Hello, World! from " ~ 
to!string(thisThreadID), "text/plain");
     else if (auto m = matchFirst(req.path, reg))
         res.writeBody("Hello, " ~ m[1] ~ " from " ~ 
to!string(thisThreadID), "text/plain");
}
```

That could be the reason for slowness.



 May 09 2018



  parent reply Daniel Kozak <kozzi11 gmail.com>  writes:


On which system? AFAIK HTTPServerOption.reusePort works on Linux but maybe
not on others OSes. Other question is what events driver is use (libasync,
libevent, vibe-core)

On Wed, May 9, 2018 at 9:12 PM, Arun Chandrasekaran via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Monday, 30 October 2017 at 17:23:02 UTC, Daniel Kozak wrote:


 Maybe this one:

 import vibe.d;
 import std.regex;
 import std.array : appender;

 static reg =3D ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     setupWorkerThreads(logicalProcessorCount);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings =3D new HTTPServerSettings;
     settings.options |=3D HTTPServerOption.reusePort;
     settings.port =3D 3000;
     settings.serverString =3D null;
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     switch(req.path)
     {
     case "/": res.writeBody("Hello World", "text/plain");
         break;
     default:
         auto m =3D matchFirst(req.path, reg);
         string message =3D "Hello, ";
         auto app =3D appender(message);
         app.reserve(32);
         app ~=3D m[1];
         res.writeBody(app.data, "text/plain");
     }
 }

 On Mon, Oct 30, 2017 at 5:41 PM, ade90036 via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak wrote:

 wrong version, this is my letest version: https://paste.ofcode.org/qWsQ=






i

 kdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak <kozzi11 gmail.com>
 wrote:

 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj


 On Thu, Sep 21, 2017 at 2:50 PM, S=C3=B6nke Ludwig via Digitalmars-d =










<

 digitalmars-d puremagic.com> wrote:

 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:


 [...]


 Oh, sorry, I forgot the reusePort option, so that multiple sockets


 can listen on the same port:

     auto settings =3D new HTTPServerSettings("0.0.0.0:3000");
     settings.options |=3D HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);


 Hi, would it be possible to re-share the example of vibe.d woth




 multithreaded support.

 The pastebin link has expired and the pull request doesnt have the
 latest version.

 Thanks

 Ade




 With vibe.d 0.8.2, even when multiple worker threads are setup, only one
 thread handles the requests:

 ```
 import core.thread;
 import vibe.d;
 import std.experimental.all;

 auto reg =3D ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     writefln("Master %d is running", getpid());
     setupWorkerThreads(logicalProcessorCount + 1);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings =3D new HTTPServerSettings;
     settings.options |=3D HTTPServerOption.reusePort;
     settings.port =3D 8080;
     settings.bindAddresses =3D ["127.0.0.1"];
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     writeln("My Thread Id: ", to!string(thisThreadID));
     // simulate long runnig task
     Thread.sleep(dur!("seconds")(3));

     if (req.path =3D=3D "/")
         res.writeBody("Hello, World! from " ~ to!string(thisThreadID),
 "text/plain");
     else if (auto m =3D matchFirst(req.path, reg))
         res.writeBody("Hello, " ~ m[1] ~ " from " ~
 to!string(thisThreadID), "text/plain");
 }
 ```

 That could be the reason for slowness.





 May 09 2018



  parent reply Arun Chandrasekaran <aruncxy gmail.com>  writes:


On Wednesday, 9 May 2018 at 21:55:15 UTC, Daniel Kozak wrote:

 On which system? AFAIK HTTPServerOption.reusePort works on 
 Linux but maybe not on others OSes. Other question is what 
 events driver is use (libasync, libevent, vibe-core)

 On Wed, May 9, 2018 at 9:12 PM, Arun Chandrasekaran via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:


 On Monday, 30 October 2017 at 17:23:02 UTC, Daniel Kozak wrote:


 Maybe this one:

 import vibe.d;
 import std.regex;
 import std.array : appender;

 static reg = ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     setupWorkerThreads(logicalProcessorCount);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings = new HTTPServerSettings;
     settings.options |= HTTPServerOption.reusePort;
     settings.port = 3000;
     settings.serverString = null;
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     switch(req.path)
     {
     case "/": res.writeBody("Hello World", "text/plain");
         break;
     default:
         auto m = matchFirst(req.path, reg);
         string message = "Hello, ";
         auto app = appender(message);
         app.reserve(32);
         app ~= m[1];
         res.writeBody(app.data, "text/plain");
     }
 }

 On Mon, Oct 30, 2017 at 5:41 PM, ade90036 via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:

 On Thursday, 21 September 2017 at 13:09:33 UTC, Daniel Kozak 
 wrote:

 wrong version, this is my letest version: 
 https://paste.ofcode.org/qWsQi

 kdhKiAywgBpKwANFR

 On Thu, Sep 21, 2017 at 3:01 PM, Daniel Kozak 
 <kozzi11 gmail.com> wrote:

 my version: https://paste.ofcode.org/RLX7GM6SHh3DjBBHd7wshj


 On Thu, Sep 21, 2017 at 2:50 PM, Sönke Ludwig via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 Am 21.09.2017 um 14:41 schrieb Vadim Lopatin:


 [...]


 Oh, sorry, I forgot the reusePort option, so that 
 multiple sockets


 can listen on the same port:

     auto settings = new 
 HTTPServerSettings("0.0.0.0:3000");
     settings.options |= HTTPServerOption.reusePort;
     listenHTTP(settings, &handleRequest);


 Hi, would it be possible to re-share the example of vibe.d 
 woth




 multithreaded support.

 The pastebin link has expired and the pull request doesnt 
 have the latest version.

 Thanks

 Ade




 With vibe.d 0.8.2, even when multiple worker threads are 
 setup, only one thread handles the requests:

 ```
 import core.thread;
 import vibe.d;
 import std.experimental.all;

 auto reg = ctRegex!"^/greeting/([a-z]+)$";

 void main()
 {
     writefln("Master %d is running", getpid());
     setupWorkerThreads(logicalProcessorCount + 1);
     runWorkerTaskDist(&runServer);
     runApplication();
 }

 void runServer()
 {
     auto settings = new HTTPServerSettings;
     settings.options |= HTTPServerOption.reusePort;
     settings.port = 8080;
     settings.bindAddresses = ["127.0.0.1"];
     listenHTTP(settings, &handleRequest);
 }

 void handleRequest(HTTPServerRequest req,
                     HTTPServerResponse res)
 {
     writeln("My Thread Id: ", to!string(thisThreadID));
     // simulate long runnig task
     Thread.sleep(dur!("seconds")(3));

     if (req.path == "/")
         res.writeBody("Hello, World! from " ~ 
 to!string(thisThreadID),
 "text/plain");
     else if (auto m = matchFirst(req.path, reg))
         res.writeBody("Hello, " ~ m[1] ~ " from " ~
 to!string(thisThreadID), "text/plain");
 }
 ```

 That could be the reason for slowness.




Ubuntu 17.10 64 bit, DMD v2.079.1, E7-4860, 8 core 32 GB RAM.

With slight modifcaition to capture the timestamp of the request 
on the server:

     import std.datetime.systime : Clock;
     auto tm = Clock.currTime().toISOExtString();
     writeln(tm, " My Thread Id: ", to!string(thisThreadID));
     // simulate long runnig task
     Thread.sleep(dur!("seconds")(3));

     if (req.path == "/")
         res.writeBody(tm ~ " Hello, World! from " ~ 
to!string(thisThreadID), "text/plain");


Launch two parallel curls.. and here is the server log..

Master 13284 is running
[vibe-6(5fQI) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-7(xljY) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-2(FVCk) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-3(peZP) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-8(c5pQ) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-4(T/oM) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-5(zc5i) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-1(Rdux) INF] Listening for requests on http://0.0.0.0:8080/
[vibe-0(PNMK) INF] Listening for requests on http://0.0.0.0:8080/
2018-05-09T15:32:41.5424275 My Thread Id: 140129463940864
2018-05-09T15:32:44.5450092 My Thread Id: 140129463940864
2018-05-09T15:32:56.3998322 My Thread Id: 140129463940864
2018-05-09T15:32:59.4022579 My Thread Id: 140129463940864
2018-05-09T15:33:12.4973215 My Thread Id: 140129463940864
2018-05-09T15:33:15.4996923 My Thread Id: 140129463940864


PS: Your top posting makes reading your replies difficult



 May 09 2018



  parent reply Daniel Kozak <kozzi11 gmail.com>  writes:


On Wednesday, 9 May 2018 at 22:37:22 UTC, Arun Chandrasekaran 
wrote:

 That could be the reason for slowness.




 Ubuntu 17.10 64 bit, DMD v2.079.1, E7-4860, 8 core 32 GB RAM.

 With slight modifcaition to capture the timestamp of the 
 request on the server:

     import std.datetime.systime : Clock;
     auto tm = Clock.currTime().toISOExtString();
     writeln(tm, " My Thread Id: ", to!string(thisThreadID));
     // simulate long runnig task
     Thread.sleep(dur!("seconds")(3));

     if (req.path == "/")
         res.writeBody(tm ~ " Hello, World! from " ~ 
 to!string(thisThreadID), "text/plain");


 Launch two parallel curls.. and here is the server log..

 Master 13284 is running
 [vibe-6(5fQI) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-7(xljY) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-2(FVCk) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-3(peZP) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-8(c5pQ) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-4(T/oM) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-5(zc5i) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-1(Rdux) INF] Listening for requests on 
 http://0.0.0.0:8080/
 [vibe-0(PNMK) INF] Listening for requests on 
 http://0.0.0.0:8080/
 2018-05-09T15:32:41.5424275 My Thread Id: 140129463940864
 2018-05-09T15:32:44.5450092 My Thread Id: 140129463940864
 2018-05-09T15:32:56.3998322 My Thread Id: 140129463940864
 2018-05-09T15:32:59.4022579 My Thread Id: 140129463940864
 2018-05-09T15:33:12.4973215 My Thread Id: 140129463940864
 2018-05-09T15:33:15.4996923 My Thread Id: 140129463940864


 PS: Your top posting makes reading your replies difficult


I have change my example a little:

case "/": res.writeBody("Hello World " ~ to!string(thisThreadID), 
"text/plain");

And I get this (siege -p -c15 0b -t 10s http://127.0.0.1:3000 | 
grep World | sort | uniq):

Hello World 140064214951680
Hello World 140064223344384
Hello World 140064231737088
Hello World 140064240129792
Hello World 140064248522496
Hello World 140064256915200

Si I get six different thread ids, which is OK because I have 6 
cores



 May 11 2018



 next sibling parent  Daniel Kozak <kozzi11 gmail.com>  writes:


On Friday, 11 May 2018 at 07:56:04 UTC, Daniel Kozak wrote:

 On Wednesday, 9 May 2018 at 22:37:22 UTC, Arun Chandrasekaran 
 wrote:

 [...]


 I have change my example a little:

 case "/": res.writeBody("Hello World " ~ 
 to!string(thisThreadID), "text/plain");

 And I get this (siege -p -c15 0b -t 10s http://127.0.0.1:3000 | 
 grep World | sort | uniq):

 Hello World 140064214951680
 Hello World 140064223344384
 Hello World 140064231737088
 Hello World 140064240129792
 Hello World 140064248522496
 Hello World 140064256915200

 Si I get six different thread ids, which is OK because I have 6 
 cores


my dub.json:

{
	"name": "sbench",
	"authors": [
		"Daniel Kozak"
	],
	"dependencies": {
		"vibe-d": "~>0.8.4-beta.1",
		"vibe-d:tls": "0.8.4-beta.1",
		"vibe-core": "1.4.1-beta.2"
	},
	"subConfigurations": {
		"vibe-d:core": "vibe-core",
		"vibe-d:tls": "notls"
	},
	"description": "A simple vibe.d server application.",
	"copyright": "Copyright © 2018, Daniel Kozak",
	"license": "proprietary"
}



 May 11 2018



prev sibling  parent reply Arun Chandrasekaran <aruncxy gmail.com>  writes:


On Friday, 11 May 2018 at 07:56:04 UTC, Daniel Kozak wrote:

 On Wednesday, 9 May 2018 at 22:37:22 UTC, Arun Chandrasekaran 
 wrote:

 [...]


 I have change my example a little:

 case "/": res.writeBody("Hello World " ~ 
 to!string(thisThreadID), "text/plain");

 And I get this (siege -p -c15 0b -t 10s http://127.0.0.1:3000 | 
 grep World | sort | uniq):

 Hello World 140064214951680
 Hello World 140064223344384
 Hello World 140064231737088
 Hello World 140064240129792
 Hello World 140064248522496
 Hello World 140064256915200

 Si I get six different thread ids, which is OK because I have 6 
 cores


siege makes a difference. Earlier I had two chrome windows open 
and I just tried a simple GET from each, with a considerable 
delay and I saw the same thread ID in both the responses. Thanks 
for your help! But couldn't understand how two chrome windows 
could consistently get the same thread ID. netstat shows one 
ESTABLISHED socket. May be Chrome multiplexes the connection? 
That doesn't sound right to me.



 May 11 2018



  parent  Joakim <dlang joakim.fea.st>  writes:


On Friday, 11 May 2018 at 23:24:32 UTC, Arun Chandrasekaran wrote:

 On Friday, 11 May 2018 at 07:56:04 UTC, Daniel Kozak wrote:

 [...]


 siege makes a difference. Earlier I had two chrome windows open 
 and I just tried a simple GET from each, with a considerable 
 delay and I saw the same thread ID in both the responses. 
 Thanks for your help! But couldn't understand how two chrome 
 windows could consistently get the same thread ID. netstat 
 shows one ESTABLISHED socket. May be Chrome multiplexes the 
 connection? That doesn't sound right to me.


Very likely:

https://www.igvita.com/posa/high-performance-networking-in-google-chrome/#tcp-pre-connect



 May 11 2018



prev sibling  parent  Daniel Kozak <kozzi11 gmail.com>  writes:


my results:

OS       lang:config   req/s    comments
=3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D    =3D=3D=3D=3D=3D=3D=
=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Linux    go                24K
Linux    D:libevent    27K        4 threads
Linux    D:libasync   26.5K     4 threads

On Thu, Sep 21, 2017 at 2:41 PM, Vadim Lopatin via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Thursday, 21 September 2017 at 10:21:17 UTC, S=C3=B6nke Ludwig wrote:


      shared static this()

      {
          (...)
      }


 BTW, I'd recommend using

     void main()
     {
         (...)
         runApplication();
     }

 instead and dropping the VibeDefaultMain version in the package recipe.
 The old approach is planned to be faded out slowly, because it requires
 some special DUB support that would be nice to see gone at some point.


 Thank you!
 Trying to use multithreaded mode.

 Under windows, best Vibe.d results are equal to results of Go server.
 Under linux, Vibe.d is still slower even if there are 4 threads listening=


.

 Results:

 OS       lang:config   req/s    comments
 =3D=3D=3D=3D=3D=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D    =3D=3D=3D=3D=3D=


=3D=3D  =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

 Linux    go            53K
 Linux    D:default     48K      1 thread
 Linux    D:libevent    48K      1 thread
 Linux    D:libasync    46.5K    4 threads

 Windows  go            12K
 Windows  D:default     12K      4 threads
 Windows  D:libevent    12K      4 threads
 Windows  D:libasync    7K       4 threads

 Under Linux in default and libevent configurations, I see error messages
 from 3 of 4 threads - looks like only one thread can process connections.

 Failed to listen on 0.0.0.0:3000
 Task terminated with uncaught exception: Failed to listen for incoming
 HTTP connections on any of the supplied interfaces.





 Sep 21 2017



prev sibling next sibling parent reply Kagamin <spam here.lot>  writes:


Other implementations use switch. 
https://dpaste.dzfl.pl/406dafbedbd9 is it slower?



 Sep 21 2017



 next sibling parent  Daniel Kozak <kozzi11 gmail.com>  writes:


in this situation I dont thing there will be difference

On Thu, Sep 21, 2017 at 6:20 PM, Kagamin via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 Other implementations use switch. https://dpaste.dzfl.pl/406dafbedbd9 is
 it slower?





 Sep 21 2017



prev sibling  parent reply Daniel Kozak <kozzi11 gmail.com>  writes:


Ok, maybe there is a some small improvments, but it is nothing what would
make this faster than rust and still undere 30K on my pc

On Thu, Sep 21, 2017 at 7:02 PM, Daniel Kozak <kozzi11 gmail.com> wrote:


 in this situation I dont thing there will be difference

 On Thu, Sep 21, 2017 at 6:20 PM, Kagamin via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:


 Other implementations use switch. https://dpaste.dzfl.pl/406dafbedbd9 is
 it slower?







 Sep 21 2017



  parent  Vadim Lopatin <coolreader.org gmail.com>  writes:


On Thursday, 21 September 2017 at 17:13:16 UTC, Daniel Kozak 
wrote:

 Ok, maybe there is a some small improvments, but it is nothing 
 what would make this faster than rust and still undere 30K on 
 my pc


More test results:
On my Win10 PC, DMD/x86/libevent:27-29K, Go:31-33K



 Sep 21 2017



prev sibling next sibling parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Thursday, 21 September 2017 at 08:01:23 UTC, Vadim Lopatin 
wrote:

 There is a simple set of simple web server apps written in 
 several languages (Go, Rust, Scala, Node-js):

 https://github.com/nuald/simple-web-benchmark

 I've sent PR to include D benchmark (vibe.d).

 I was hoping it could show performance at least not worse than 
 other languages.
 But it appears to be slower than Go and even Node.js

 Are there any tips to achieve better performance in this test?

 Under windows, I can get vibe.d configured to use libevent to 
 show results comparable with Go. With other configurations, it 
 works two times slower.

 Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. 
 The only advantage of D here is CPU load - 90% vs 120% in Go.

 I'm using DMD. Probably, ldc could speed up it a bit.

 Probably, it's caused by single threaded async implementation 
 while other languages are using parallel handling of requests?


Doesn't vibe-d use Fibers?

I tried to build a simple web server with a fiber-based approach 
once - it was horribly slow.


eventually come to D.



 Sep 21 2017



 next sibling parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Thursday, 21 September 2017 at 18:49:00 UTC, bitwise wrote:

 On Thursday, 21 September 2017 at 08:01:23 UTC, Vadim Lopatin 
 wrote:

 There is a simple set of simple web server apps written in 
 several languages (Go, Rust, Scala, Node-js):

 https://github.com/nuald/simple-web-benchmark

 I've sent PR to include D benchmark (vibe.d).

 I was hoping it could show performance at least not worse than 
 other languages.
 But it appears to be slower than Go and even Node.js

 Are there any tips to achieve better performance in this test?

 Under windows, I can get vibe.d configured to use libevent to 
 show results comparable with Go. With other configurations, it 
 works two times slower.

 Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. 
 The only advantage of D here is CPU load - 90% vs 120% in Go.

 I'm using DMD. Probably, ldc could speed up it a bit.

 Probably, it's caused by single threaded async implementation 
 while other languages are using parallel handling of requests?


 Doesn't vibe-d use Fibers?

 I tried to build a simple web server with a fiber-based 
 approach once - it was horribly slow.


 will eventually come to D.


It does. But Golang uses them, too. Goroutines.



 Sep 21 2017



  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Thursday, 21 September 2017 at 18:55:04 UTC, Vadim Lopatin 
wrote:

 On Thursday, 21 September 2017 at 18:49:00 UTC, bitwise wrote:

 On Thursday, 21 September 2017 at 08:01:23 UTC, Vadim Lopatin 
 wrote:

 [...]


 Doesn't vibe-d use Fibers?

 I tried to build a simple web server with a fiber-based 
 approach once - it was horribly slow.


 will eventually come to D.


 It does. But Golang uses them, too. Goroutines.


Indeed. I'm reading about them right now, and they seem to be 
"multiplexed". I wonder if Vibe.d does something similar.

The fact that you've observed lower CPU usage by the D version 
makes me think some kind of scheduling or thread-priority issue 
is the cause.

For example, on windows, the default timer frequency is very low. 
It would seem reasonable to get 1000 iterations per second in the 
example below, but you get ~64.

`
auto now = steady_clock::now();
auto done = now + milliseconds(10000);
int iterations = 0;

while(steady_clock::now() < done) {
     ++iterations;
     Sleep(1);
}

cout << (iterations / 10) << endl;
`

When I wrap the above code with timeBeginPeriod(1) and 
timeEndPeriod(1), I get ~550 on my machine.

IIRC, you get similar behavior on MacOS(maybe linux too?) unless 
you explicitly raise the thread priority.

https://msdn.microsoft.com/en-us/library/windows/desktop/dd757624(v=vs.85).aspx

So if you're benchmarking anything that sleeps regularly, like an 
event based framework, something like 
timeBeginPeriod/timeEndPeriod may help.



 Sep 21 2017



  parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Thursday, 21 September 2017 at 19:40:48 UTC, bitwise wrote:

 On Thursday, 21 September 2017 at 18:55:04 UTC, Vadim Lopatin

 It does. But Golang uses them, too. Goroutines.


 Indeed. I'm reading about them right now, and they seem to be 
 "multiplexed". I wonder if Vibe.d does something similar.

 The fact that you've observed lower CPU usage by the D version 
 makes me think some kind of scheduling or thread-priority issue 
 is the cause.


Fibers are being switched by waiting for signals/events.
Waiting blocks thread.
Timer should affect only non-blocked threads switching IMHO.



 Sep 22 2017



  parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org>  writes:


Am 22.09.2017 um 09:45 schrieb Vadim Lopatin:

 On Thursday, 21 September 2017 at 19:40:48 UTC, bitwise wrote:

 On Thursday, 21 September 2017 at 18:55:04 UTC, Vadim Lopatin

 It does. But Golang uses them, too. Goroutines.


 Indeed. I'm reading about them right now, and they seem to be 
 "multiplexed". I wonder if Vibe.d does something similar.

 The fact that you've observed lower CPU usage by the D version makes 
 me think some kind of scheduling or thread-priority issue is the cause.


 
 Fibers are being switched by waiting for signals/events.
 Waiting blocks thread.
 Timer should affect only non-blocked threads switching IMHO.


What's was the last status? Could you observe any meaningful thread scaling?

I tested on a 32-core machine a while back and could observe the 
performance rising almost linearly when increasing the number of cores 
(as it should). The effect is obviously smaller on a dual-core system 
where the benchmark application runs on the same system, but even there 
it was well visible.

If the multi-threaded version doesn't show 100% CPU usage, that would 
mean that some kind of thread-blocking is happening - GC collections or 
lock contention would be the likely candidates for that. The latter 
shouldn't happen anymore, as everything except for the logger should be 
thread-local in the latest version.

BTW, I ran Daniel's version on my dual-core notebook against wrk (Linux) 
and got 75kreq/s when using runWorkerTask and ~56kreq/s when using just 
a single thread, which is about what I would expect, considering that 
wrk ran on the same machine.



 Sep 22 2017



  parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Friday, 22 September 2017 at 09:40:00 UTC, Sönke Ludwig wrote:

 What's was the last status? Could you observe any meaningful 
 thread scaling?


It works for me - multithreading improves performance on my PC.


So far, test results on 
https://github.com/nuald/simple-web-benchmark
show that D is 2-3 times slower than any other language including 
node.js

nuald reverted change which enables multithreading since it's 
"unfair".



 Sep 24 2017



 next sibling parent  Arun Chandrasekaran <aruncxy gmail.com>  writes:


On Sunday, 24 September 2017 at 18:36:50 UTC, Vadim Lopatin wrote:

 On Friday, 22 September 2017 at 09:40:00 UTC, Sönke Ludwig 
 wrote:

 What's was the last status? Could you observe any meaningful 
 thread scaling?


 It works for me - multithreading improves performance on my PC.


 So far, test results on 
 https://github.com/nuald/simple-web-benchmark
 show that D is 2-3 times slower than any other language 
 including node.js

 nuald reverted change which enables multithreading since it's 
 "unfair".


Just cloned the repo and tried. With ldc 1.4.0 (release mode) I 
get Illegal instruction (core dumped). Debug mode works fine. 
Works fine with dmd v2.075.1 release mode.

```
24-09-2017 15:08:37 vaalaham ~/code/d/simple-web-benchmark/d
$ bin/vibedtest
Illegal instruction (core dumped)
24-09-2017 15:08:40 vaalaham ~/code/d/simple-web-benchmark/d
$
```



 Sep 24 2017



prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org>  writes:


Am 24.09.2017 um 20:36 schrieb Vadim Lopatin:

 On Friday, 22 September 2017 at 09:40:00 UTC, Sönke Ludwig wrote:

 What's was the last status? Could you observe any meaningful thread 
 scaling?


 It works for me - multithreading improves performance on my PC.
 
 
 So far, test results on https://github.com/nuald/simple-web-benchmark
 show that D is 2-3 times slower than any other language including node.js


The response times look very strange for the kind of load that appears 
to be generated. Unfortunately the testing methodology is so simple that 
it's difficult so judge anything without running it again locally.

Having said that, the Windows implementation does have performance 
issues and needs to be looked at. It has rather low priority for me 
though, because I neither run any servers with Windows, nor did I hear 
that from anyone else who uses vibe.d.


 
 nuald reverted change which enables multithreading since it's "unfair".


How on earth can that be unfair when the Go, node.js and Scala versions 
appear to use multi-threading, too?



 Sep 24 2017



  parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Sunday, 24 September 2017 at 22:54:11 UTC, Sönke Ludwig wrote:

 How on earth can that be unfair when the Go, node.js and Scala 
 versions appear to use multi-threading, too?


Looks like repo owner thinks they are single threaded.



 Sep 24 2017



  parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Monday, 25 September 2017 at 06:56:58 UTC, Vadim Lopatin wrote:

 On Sunday, 24 September 2017 at 22:54:11 UTC, Sönke Ludwig 
 wrote:

 How on earth can that be unfair when the Go, node.js and Scala 
 versions appear to use multi-threading, too?


 Looks like repo owner thinks they are single threaded.


Just checked threads of Go version. I see 7 'go' and 7 'main' 
threads.



 Sep 25 2017



  parent reply tchaloupka <chalucha gmail.com>  writes:


On Monday, 25 September 2017 at 07:05:57 UTC, Vadim Lopatin wrote:

 On Monday, 25 September 2017 at 06:56:58 UTC, Vadim Lopatin 
 wrote:

 On Sunday, 24 September 2017 at 22:54:11 UTC, Sönke Ludwig 
 wrote:

 How on earth can that be unfair when the Go, node.js and 
 Scala versions appear to use multi-threading, too?


 Looks like repo owner thinks they are single threaded.


 Just checked threads of Go version. I see 7 'go' and 7 'main' 
 threads.


I've just tried this on my linux box (only dmd as ldc2 fails with 
release build - 
https://github.com/ldc-developers/ldc/issues/2280).

rust:			Requests/sec:	38757.2625
vibe-d:core libevent:	Requests/sec:	27906.8119
vibe-d:core libasync:	Requests/sec:	20534.3057
vibe-core:		Requests/sec:	18042.4251

Didn't include the Go version as it's indeed using more threads.
Results are just for the base url to not include the regex 
matching there.



 Sep 25 2017



  parent reply Vadim Lopatin <coolreader.org gmail.com>  writes:


On Monday, 25 September 2017 at 08:01:02 UTC, tchaloupka wrote:

 On Monday, 25 September 2017 at 07:05:57 UTC, Vadim Lopatin 
 wrote:

 On Monday, 25 September 2017 at 06:56:58 UTC, Vadim Lopatin 
 wrote:

 On Sunday, 24 September 2017 at 22:54:11 UTC, Sönke Ludwig 
 wrote:

 How on earth can that be unfair when the Go, node.js and 
 Scala versions appear to use multi-threading, too?


 Looks like repo owner thinks they are single threaded.


 Just checked threads of Go version. I see 7 'go' and 7 'main' 
 threads.


 I've just tried this on my linux box (only dmd as ldc2 fails 
 with release build - 
 https://github.com/ldc-developers/ldc/issues/2280).

 rust:			Requests/sec:	38757.2625
 vibe-d:core libevent:	Requests/sec:	27906.8119
 vibe-d:core libasync:	Requests/sec:	20534.3057
 vibe-core:		Requests/sec:	18042.4251

 Didn't include the Go version as it's indeed using more threads.
 Results are just for the base url to not include the regex 
 matching there.


I've sent PR 
https://github.com/nuald/simple-web-benchmark/pull/11 to 
re-enable multithreading in D test app.

BTW, does Rust version use multithreading?



 Sep 25 2017



  parent reply tchaloupka <chalucha gmail.com>  writes:


On Monday, 25 September 2017 at 08:36:31 UTC, Vadim Lopatin wrote:

 On Monday, 25 September 2017 at 08:01:02 UTC, tchaloupka wrote:

 On Monday, 25 September 2017 at 07:05:57 UTC, Vadim Lopatin 
 wrote:

 On Monday, 25 September 2017 at 06:56:58 UTC, Vadim Lopatin 
 wrote:

 On Sunday, 24 September 2017 at 22:54:11 UTC, Sönke Ludwig 
 wrote:

 How on earth can that be unfair when the Go, node.js and 
 Scala versions appear to use multi-threading, too?


 Looks like repo owner thinks they are single threaded.


 Just checked threads of Go version. I see 7 'go' and 7 'main' 
 threads.


 I've just tried this on my linux box (only dmd as ldc2 fails 
 with release build - 
 https://github.com/ldc-developers/ldc/issues/2280).

 rust:			Requests/sec:	38757.2625
 vibe-d:core libevent:	Requests/sec:	27906.8119
 vibe-d:core libasync:	Requests/sec:	20534.3057
 vibe-core:		Requests/sec:	18042.4251

 Didn't include the Go version as it's indeed using more 
 threads.
 Results are just for the base url to not include the regex 
 matching there.


 I've sent PR 
 https://github.com/nuald/simple-web-benchmark/pull/11 to 
 re-enable multithreading in D test app.

 BTW, does Rust version use multithreading?


Rust doesn't seem to use multiple threads, it's just faster I'm 
afraid.

Tried the ldc release build with the suggested switch removal 
(https://github.com/ldc-developers/ldc/issues/2280#issuecomment-331823377), and
the results are:

ldc2 vibe-d:core libevent:	Requests/sec:	29782.9605
ldc2 vibe-core:			Requests/sec:	21396.2019



 Sep 25 2017



 next sibling parent reply Daniel Kozak <kozzi11 gmail.com>  writes:


Not at all. Rust test is fake.  Does not process headers,  does not write
headers. Does not send right output.  Does not work with browser.  Every
one or two request it will die.  Rust result shoud not be taken seriously.
Until fixed

Dne 25. 9. 2017 11:50 dopoledne napsal u=C5=BEivatel "tchaloupka via
Digitalmars-d" <digitalmars-d puremagic.com>:


 On Monday, 25 September 2017 at 08:36:31 UTC, Vadim Lopatin wrote:


 On Monday, 25 September 2017 at 08:01:02 UTC, tchaloupka wrote:


 On Monday, 25 September 2017 at 07:05:57 UTC, Vadim Lopatin wrote:


 On Monday, 25 September 2017 at 06:56:58 UTC, Vadim Lopatin wrote:


 On Sunday, 24 September 2017 at 22:54:11 UTC, S=C3=B6nke Ludwig wrote=










:

 How on earth can that be unfair when the Go, node.js and Scala
 versions appear to use multi-threading, too?


 Looks like repo owner thinks they are single threaded.


 Just checked threads of Go version. I see 7 'go' and 7 'main' threads.


 I've just tried this on my linux box (only dmd as ldc2 fails with
 release build - https://github.com/ldc-developers/ldc/issues/2280).

 rust:                   Requests/sec:   38757.2625
 vibe-d:core libevent:   Requests/sec:   27906.8119
 vibe-d:core libasync:   Requests/sec:   20534.3057
 vibe-core:              Requests/sec:   18042.4251

 Didn't include the Go version as it's indeed using more threads.
 Results are just for the base url to not include the regex matching
 there.


 I've sent PR https://github.com/nuald/simple-web-benchmark/pull/11 to
 re-enable multithreading in D test app.

 BTW, does Rust version use multithreading?


 Rust doesn't seem to use multiple threads, it's just faster I'm afraid.

 Tried the ldc release build with the suggested switch removal (
 https://github.com/ldc-developers/ldc/issues/2280#issuecomment-331823377)=


,

 and the results are:

 ldc2 vibe-d:core libevent:      Requests/sec:   29782.9605
 ldc2 vibe-core:                 Requests/sec:   21396.2019





 Sep 25 2017



  parent  tchaloupka <chalucha gmail.com>  writes:


On Monday, 25 September 2017 at 13:18:45 UTC, Daniel Kozak wrote:

 Not at all. Rust test is fake.  Does not process headers,  does 
 not write headers. Does not send right output.  Does not work 
 with browser.  Every one or two request it will die.  Rust 
 result shoud not be taken seriously. Until fixed


Well I didn't expected that it would be this way and didn't check 
myself.. Thanks for testing and clarification :)



 Sep 25 2017



prev sibling  parent  Atila Neves <atila.neves gmail.com>  writes:


On Monday, 25 September 2017 at 09:45:13 UTC, tchaloupka wrote:

 On Monday, 25 September 2017 at 08:36:31 UTC, Vadim Lopatin 
 wrote:

 On Monday, 25 September 2017 at 08:01:02 UTC, tchaloupka wrote:

 On Monday, 25 September 2017 at 07:05:57 UTC, Vadim Lopatin 
 wrote:

 [...]


 I've just tried this on my linux box (only dmd as ldc2 fails 
 with release build - 
 https://github.com/ldc-developers/ldc/issues/2280).

 rust:			Requests/sec:	38757.2625
 vibe-d:core libevent:	Requests/sec:	27906.8119
 vibe-d:core libasync:	Requests/sec:	20534.3057
 vibe-core:		Requests/sec:	18042.4251

 Didn't include the Go version as it's indeed using more 
 threads.
 Results are just for the base url to not include the regex 
 matching there.


 I've sent PR 
 https://github.com/nuald/simple-web-benchmark/pull/11 to 
 re-enable multithreading in D test app.

 BTW, does Rust version use multithreading?


 Rust doesn't seem to use multiple threads, it's just faster I'm 
 afraid.

 Tried the ldc release build with the suggested switch removal 
 (https://github.com/ldc-developers/ldc/issues/2280#issuecomment-331823377),
and the results are:

 ldc2 vibe-d:core libevent:	Requests/sec:	29782.9605
 ldc2 vibe-core:			Requests/sec:	21396.2019


 From my benchmarks comparing D to Rust using an MQTT broker I 
wrote, Rust wasn't faster, mio (a Rust async IO library) was. 
i.e. Rust/mio and D/mio performed similarly, D/vibe.d was slower. 
That was over a year ago, I haven't tried it again since.

Atila



 Sep 26 2017



prev sibling  parent  Jacob Carlborg <doob me.com>  writes:


On 2017-09-24 20:36, Vadim Lopatin wrote:


 nuald reverted change which enables multithreading since it's "unfair".


That's kind of stupid. In a real world scenario one would do everything 
possible to get the best performance. If one of the frameworks doesn't 
support mutlithreading, too bad for that framework.

-- 
/Jacob Carlborg



 Sep 25 2017



prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org>  writes:


Am 21.09.2017 um 20:49 schrieb bitwise:

 
 Doesn't vibe-d use Fibers?
 
 I tried to build a simple web server with a fiber-based approach once - 
 it was horribly slow.
 

 eventually come to D.


It uses them and the overhead actually diminishes once the application 
does anything meaningful. To test this, I created two low-level tests 
for eventcore that mimic a minimal HTTP server. AFAIR, I got around 
300kreq/s on a single core without fibers and around 290kreq/s with 
fibers, which amounts to an overhead of about 0.1µs per request.

https://github.com/vibe-d/eventcore/tree/master/examples

Stackless fibers would be really nice to have because of the merged 
stacks and the lower amount of reserved memory required (even though 
this is not a really big issue on 64-bit systems), but for pure 
performance I don't think they would be a critical addition.



 Sep 22 2017



  parent reply bitwise <bitwise.pvt gmail.com>  writes:


On Friday, 22 September 2017 at 09:48:47 UTC, Sönke Ludwig wrote:

 Am 21.09.2017 um 20:49 schrieb bitwise:

 
 Doesn't vibe-d use Fibers?
 
 I tried to build a simple web server with a fiber-based 
 approach once - it was horribly slow.
 

 will eventually come to D.


 It uses them and the overhead actually diminishes once the 
 application does anything meaningful. To test this, I created 
 two low-level tests for eventcore that mimic a minimal HTTP 
 server. AFAIR, I got around 300kreq/s on a single core without 
 fibers and around 290kreq/s with fibers, which amounts to an 
 overhead of about 0.1µs per request.


Interesting - I thought the cost would be higher.

Of the few different architectures I tried, the fiber based 
approach was much slower. It's possible that my implementation 
did too many unnecessary context switches.


 Stackless fibers would be really nice to have because of the 
 merged stacks and the lower amount of reserved memory required 
 (even though this is not a really big issue on 64-bit systems), 
 but for pure performance I don't think they would be a critical 
 addition.


I suppose this is off topic, but for games, or any realtime 
application where things need to run intermittently, but at high 
frequency, and in high numbers, stackless resumable functions are 
a big win. A lot of AI I've been writing lately (including some 




 Sep 23 2017



  parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com>  writes:


On Saturday, 23 September 2017 at 22:07:58 UTC, bitwise wrote:

 Of the few different architectures I tried, the fiber based 
 approach was much slower. It's possible that my implementation 
 did too many unnecessary context switches.


Can you give a bit more details? What kind of architectures do 
you mean (hardware, software, ..)?
What was your use case? IO-multiplexing, coarse-grained 
high-level cooperative multi-tasking, or range-like data 
processing state-machines?


 I suppose this is off topic, but for games, or any realtime 
 application where things need to run intermittently, but at 
 high frequency, and in high numbers, stackless resumable 
 functions are a big win. A lot of AI I've been writing lately 
 (including some flocking behaviors) have been built on top of 



Very interesting, I would like to hear more about your approach. 

synchronous pull-style code I constantly miss the power of D's 

IEnumerable<T> extension methods, expression trees and run-time 
reflection.
About push-style asynchronous code, I find async/await unusable 
for anything more than the absolute basic stuff. For 95% of my 
code in this area I use RX.NET (http://reactivex.io/). I'm sure 
they probably use async/await somewhere down in their 
implementation, but I just find that it scales very poorly when 
complexity increases.
My time "lost" by going from Task<T> to IObservable<T> (yes, even 
for single items) is immediately regained by using a few powerful 
operators like Buffer, CombineLatest and Switch.



 Sep 24 2017



  parent  bitwise <bitwise.pvt gmail.com>  writes:


On Sunday, 24 September 2017 at 08:08:35 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Saturday, 23 September 2017 at 22:07:58 UTC, bitwise wrote:

 [...]


 Can you give a bit more details? What kind of architectures do 
 you mean (hardware, software, ..)?
 What was your use case? IO-multiplexing, coarse-grained 
 high-level cooperative multi-tasking, or range-like data 
 processing state-machines?


Probably not without embarrassing myself ;)
Even now, server tech isn't really my domain, and it was a long 
time ago.

The effort was mainly a learning experience to better understand 
what's going on under the hood when working on multiplayer games, 
web-apps, etc..

All of my implementations were built around the same code though, 
except request scheduling, where I first tried some kind of state 
machine, and then switched to fibers afterward, which totally 
destroyed it's already mediocre performance.


 Very interesting, I would like to hear more about your 

 When writing synchronous pull-style code I constantly miss the 
 power of D's ranges


I'm not talking about using IEnumerators as generators. I mean 
using them for user-level threads. So basically, an IEnumerator 
method being "ticked" regularly by a scheduler/engine could 
return a scalar or predicate that specified when execution should 
resume, or it could return another IEnumerator method that the 
scheduler would run to completion before returning to the first 
one.

Simplified pseudo-example:

`
IEnumerator Attack(enemy) {
     // attack until enemy dead or escaped
     // maybe the enemy parameter should be 'scope' XD
}

IEnumerator Idle() {
     enemy = null;

     while(true) {
         position += wanderOffset();

         if(enemy = getEnemiesInRange()) {
             yield Attack(enemy);
             enemy = null;
             yield 2.seconds;
         }

         yield;
     }
}

engine.runCoroutine(Idle());

while(true) { // 60 FPS+
     engine.tickCoroutines();
     // ...
     engine.draw();
}
`

I think it's easy to imagine how using fibers would be much more 
expensive in the above example as it scales in number of AI 
characters and complexity of behavior.


 About push-style asynchronous code, I find async/await unusable 
 for anything more than the absolute basic stuff. For 95% of my 
 code in this area I use RX.NET (http://reactivex.io/). I'm sure 
 they probably use async/await somewhere down in their 
 implementation, but I just find that it scales very poorly when 
 complexity increases.
 My time "lost" by going from Task<T> to IObservable<T> (yes, 
 even for single items) is immediately regained by using a few 
 powerful operators like Buffer, CombineLatest and Switch.


I'm not sure I understand exactly what you're talking about here, 
but I know that basic applications get A LOT easier with 
async/await. For GUI-based applications with network 
connectivity, the whole concept of threading basically 
disappears. No need for locks, no need to dispatch UI updates to 
the main thread. Just code naturally and expect functions that 
make network calls to just take as long as they need without ever 
blocking the UI thread.


standard library before than the underlying concept itself. I 
don't think there's anything inherently slow about an async/await 
type framework, but if you look at microsoft's reference source 

in mind, and that performance takes a back seat. I'm sure the 
situation in C++ will be very different if stackless resumables 
are accepted.



 Sep 24 2017



prev sibling  parent  Adam Wilson <flyboynw gmail.com>  writes:


On 9/21/17 11:49, bitwise wrote:

 On Thursday, 21 September 2017 at 08:01:23 UTC, Vadim Lopatin wrote:

 There is a simple set of simple web server apps written in several
 languages (Go, Rust, Scala, Node-js):

 https://github.com/nuald/simple-web-benchmark

 I've sent PR to include D benchmark (vibe.d).

 I was hoping it could show performance at least not worse than other
 languages.
 But it appears to be slower than Go and even Node.js

 Are there any tips to achieve better performance in this test?

 Under windows, I can get vibe.d configured to use libevent to show
 results comparable with Go. With other configurations, it works two
 times slower.

 Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. The only
 advantage of D here is CPU load - 90% vs 120% in Go.

 I'm using DMD. Probably, ldc could speed up it a bit.

 Probably, it's caused by single threaded async implementation while
 other languages are using parallel handling of requests?


 Doesn't vibe-d use Fibers?

 I tried to build a simple web server with a fiber-based approach once -
 it was horribly slow.


 eventually come to D.



free up the thread while some long-running IO operation is taking place 

ASP.NET/Core which allows the server to process many times the number of 
incoming requests(threads) than there are physical cores on the device. 
This works because another request is often doing some other work behind 
the scenes (DB query, HTTP call to remote service, etc.)

In fact MSFT says that Async/Await will decrease performance of a single 
instance of execution and are not to be used in situations where the 
delay is less than about 50ms (in 2011, i've heard that it could be even 
less with newer versions of the compiler) as it can actually take more 
time dehydrate/rehydrate the thread than the blocking operation would've 
taken.

-- 
Adam Wilson
IRC: LightBender
import quiet.dlang.dev;



 Sep 23 2017



prev sibling  parent  Regio <Regio regio.com>  writes:


On Thursday, 21 September 2017 at 08:01:23 UTC, Vadim Lopatin 
wrote:

 There is a simple set of simple web server apps written in 
 several languages (Go, Rust, Scala, Node-js):

 https://github.com/nuald/simple-web-benchmark

 I've sent PR to include D benchmark (vibe.d).

 I was hoping it could show performance at least not worse than 
 other languages.
 But it appears to be slower than Go and even Node.js

 Are there any tips to achieve better performance in this test?

 Under windows, I can get vibe.d configured to use libevent to 
 show results comparable with Go. With other configurations, it 
 works two times slower.

 Under linux, vibe.d shows 45K requests/seconds, and Go - 50K. 
 The only advantage of D here is CPU load - 90% vs 120% in Go.

 I'm using DMD. Probably, ldc could speed up it a bit.

 Probably, it's caused by single threaded async implementation 
 while other languages are using parallel handling of requests?


Its a bit uneven benchmark as you are testing default Go vs 
default D + Vibe.D.

One can use a more faster framework like Go's Gin

https://github.com/gin-gonic/gin

In my tests in the past with Vibe.D 0.8, Go was faster with the 
alternative frameworks.



 Sep 22 2017