www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Vibe.d - very low performance

reply Marek <siloamx gmail.com> writes:
https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext

C++, Java and Go frameworks have very high performance. Vibe.d is 
supposed to have similar performance, but in fact vibe.d 
performance is very low. Why?
Jul 06 2017
next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 06.07.2017 um 09:27 schrieb Marek:

 https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext 
 
 
 C++, Java and Go frameworks have very high performance. Vibe.d is 
 supposed to have similar performance, but in fact vibe.d performance is 
 very low. Why?


This is a scalability issue, which should hopefully be fixed with 0.8.0. 
I'll open a PR once that is out. Basically with the version that was 
used in the last benchmark round, it didn't scale at all, and they use a 
server with many cores (40 + hyperthreading).
Jul 06 2017
next sibling parent reply Marek <siloamx gmail.com> writes:
On Thursday, 6 July 2017 at 10:57:31 UTC, Sönke Ludwig wrote:

 This is a scalability issue, which should hopefully be fixed 
 with 0.8.0. I'll open a PR once that is out. Basically with the 
 version that was used in the last benchmark round, it didn't 
 scale at all, and they use a server with many cores (40 + 
 hyperthreading).


What do you mean by 'scalability'? Raw tornado or bottle 
frameworks have much better results than vibe.d. Python and Ruby 
have GIL so they can't use threads in their standard 
implementations. They have much better results anyway.
Jul 07 2017
parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-07 20:22, Marek wrote:


 What do you mean by 'scalability'? Raw tornado or bottle frameworks have
 much better results than vibe.d. Python and Ruby have GIL so they can't
 use threads in their standard implementations. They have much better
 results anyway.


I think that vibe.d didn't take full advantage of multi core, even when 
enabling threading support. Ruby, or rather Rails, applications are 
usually run using multiple processes, which allows to scale on a multi 
core CPU. You can do the same with vibe.d as well.

-- 
/Jacob Carlborg
Jul 07 2017
parent reply Marek <siloamx gmail.com> writes:
On Friday, 7 July 2017 at 19:03:52 UTC, Jacob Carlborg wrote:

 I think that vibe.d didn't take full advantage of multi core, 
 even when enabling threading support. Ruby, or rather Rails, 
 applications are usually run using multiple processes, which 
 allows to scale on a multi core CPU. You can do the same with 
 vibe.d as well.


So why Ruby or Python frameworks are much faster in this 
benchmark?
Jul 14 2017
next sibling parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 14.07.2017 um 11:55 schrieb Marek:

 On Friday, 7 July 2017 at 19:03:52 UTC, Jacob Carlborg wrote:

 I think that vibe.d didn't take full advantage of multi core, even 
 when enabling threading support. Ruby, or rather Rails, applications 
 are usually run using multiple processes, which allows to scale on a 
 multi core CPU. You can do the same with vibe.d as well.


 
 So why Ruby or Python frameworks are much faster in this benchmark?


- The vibe.d benchmark didn't scale across cores plus overhead
- The Ruby/Python benchmarks scaled >= 40x using multiple processes

You could do the same for the vibe.d benchmark, too (and it's generally 
a good idea). The current version should have the in-process scalability 
issue fixed, though, so presumably that will also work correctly now.
Jul 14 2017
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-07-14 11:55, Marek wrote:


 So why Ruby or Python frameworks are much faster in this benchmark?


They scale better since, at least Ruby on Rails applications, are run 
using multiple processes.

-- 
/Jacob Carlborg
Jul 15 2017
prev sibling parent reply FoxyBrown <Foxy Brown.IPT> writes:
On Thursday, 6 July 2017 at 10:57:31 UTC, Sönke Ludwig wrote:

 Am 06.07.2017 um 09:27 schrieb Marek:

 https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext
 
 
 C++, Java and Go frameworks have very high performance. Vibe.d 
 is supposed to have similar performance, but in fact vibe.d 
 performance is very low. Why?


 This is a scalability issue, which should hopefully be fixed 
 with 0.8.0. I'll open a PR once that is out. Basically with the 
 version that was used in the last benchmark round, it didn't 
 scale at all, and they use a server with many cores (40 + 
 hyperthreading).


So you are saying that will solve the nearly 200x factor from the 
top? At least get it in the top 20?
Jul 07 2017
parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 07.07.2017 um 21:27 schrieb FoxyBrown:

 On Thursday, 6 July 2017 at 10:57:31 UTC, Sönke Ludwig wrote:

 Am 06.07.2017 um 09:27 schrieb Marek:

 https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext



 C++, Java and Go frameworks have very high performance. Vibe.d is
 supposed to have similar performance, but in fact vibe.d performance
 is very low. Why?


 This is a scalability issue, which should hopefully be fixed with
 0.8.0. I'll open a PR once that is out. Basically with the version
 that was used in the last benchmark round, it didn't scale at all, and
 they use a server with many cores (40 + hyperthreading).


 So you are saying that will solve the nearly 200x factor from the top?
 At least get it in the top 20?


There are more factors involved, but just based on this I would expect a 
factor of around x50 (~>800k/s). The 15k/s is very low even for a single 
core, though, and it's very possible that the factor will be 
considerably larger. However, this is hard to predict, as I currently 
don't have access to similar hardware to test this on.
Jul 07 2017
parent tetyys <tetyys tetyys.com> writes:
these tests are pretty flawed considering that top places are 
taken by event loop libraries that have no other features other 
than 'respond to request'
Jul 08 2017
prev sibling parent yawniek <yawniek srtnwz.com> writes:
On Thursday, 6 July 2017 at 07:27:24 UTC, Marek wrote:

 https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext

 C++, Java and Go frameworks have very high performance. Vibe.d 
 is supposed to have similar performance, but in fact vibe.d 
 performance is very low. Why?


these tests are completely flawed, especially also in regards to 
todays world where you deploy microservices.


having benchmarked vibe about 1.5 years ago i can confirm that 
with a little bit of tuning and choosing the right libs and 
compilers you can become very competitive, especially once the 
app gets more complex.

write your own benchmark that operates on 1 core and test it.
its mostly a test of how fast your  http parser / json lib or db 
lib is.

none the less, for D it would be good to be much higher in this 
ranking.
Jul 14 2017