www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DigitalWhip

reply artemalive <artemalive gmail.com> writes:
Dear Community,

I've prepared a valentine for you;)
It's a project I've been working for the last few months in my 
free time.

DigitalWhip is a performance benchmark of statically typed 
programming languages that
compile to native code: https://github.com/artemalive/DigitalWhip

D is fast;)

Thanks,
Artem
Feb 13 2016
next sibling parent reply Johan Engelen <j j.nl> writes:
On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 DigitalWhip is a performance benchmark of statically typed 
 programming languages that
 compile to native code: 
 https://github.com/artemalive/DigitalWhip
Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)
Feb 13 2016
next sibling parent reply artemalive <artemalive gmail.com> writes:
On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen 
wrote:
 On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 DigitalWhip is a performance benchmark of statically typed 
 programming languages that
 compile to native code: 
 https://github.com/artemalive/DigitalWhip
Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)
These files are just for demonstration of expected output. The benchmark should work fine with the latest versions of D compilers. No guarantee for older versions;) I tested with dmd v2.069.2.
Feb 13 2016
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 13 February 2016 at 19:26:39 UTC, artemalive wrote:
 On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen 
 wrote:
 Could you add the compiler versions to the outputted .txt 
 file, e.g. `dmd --version`? (the example output files don't 
 have it)
These files are just for demonstration of expected output. The benchmark should work fine with the latest versions of D compilers. No guarantee for older versions;) I tested with dmd v2.069.2.
What I meant to say is that the framework should output the compiler version, instead of "READY". At least for the D compilers, the performance depends so much on which version is used, that the output is unusable without version numbers, I feel.
Feb 14 2016
parent reply artemalive <artemalive gmail.com> writes:
On Sunday, 14 February 2016 at 17:49:10 UTC, Johan Engelen wrote:
 On Saturday, 13 February 2016 at 19:26:39 UTC, artemalive wrote:
 On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen 
 wrote:
 Could you add the compiler versions to the outputted .txt 
 file, e.g. `dmd --version`? (the example output files don't 
 have it)
These files are just for demonstration of expected output. The benchmark should work fine with the latest versions of D compilers. No guarantee for older versions;) I tested with dmd v2.069.2.
What I meant to say is that the framework should output the compiler version, instead of "READY". At least for the D compilers, the performance depends so much on which version is used, that the output is unusable without version numbers, I feel.
Thanks. Good suggestion. I'll check if the version information can be retrieved automatically for all compilers. If that's the case then version information will be added soon. The thing I don't want to do is to ask user manually to specify compiler version in config.py, since it's easy to forget to updated it and also I like to keep config.py as simple as possible. At the moment DigitalWhip is just for personal usage, the user knows which compilers are installed. But version information will help with archiving/sharing results.
Feb 14 2016
parent Johan Engelen <j j.nl> writes:
On Sunday, 14 February 2016 at 18:07:00 UTC, artemalive wrote:
 Thanks. Good suggestion. I'll check if the version information 
 can be retrieved automatically for all compilers. If that's the 
 case then version information will be added soon.
I would simply print the output of "--version" for all compilers.
  But version information will help with archiving/sharing 
 results.
Exactly.
Feb 14 2016
prev sibling parent artemalive <artemalive gmail.com> writes:
On Saturday, 13 February 2016 at 19:19:46 UTC, Johan Engelen 
wrote:
 On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 DigitalWhip is a performance benchmark of statically typed 
 programming languages that
 compile to native code: 
 https://github.com/artemalive/DigitalWhip
Could you add the compiler versions to the outputted .txt file, e.g. `dmd --version`? (the example output files don't have it)
DigitalWhip 1.2.0 has been released: https://github.com/artemalive/DigitalWhip/releases/tag/v1.2.0 Compiler version information is reported now.
Mar 03 2016
prev sibling next sibling parent reply David Nadlinger <code klickverbot.at> writes:
Hi Artem,

On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 https://github.com/artemalive/DigitalWhip
Your scripts had bounds checking enabled for LDC but not the other two D compilers. I posted a pull request with the fix. LDC isn't unreasonably slow any longer on a random EC2 box I used for a quick test: --- Compiler relative times: clang 1.00 gcc 1.02 ldc 1.07 dmd 2.07 --- — David
Feb 13 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 13 February 2016 at 20:45:41 UTC, David Nadlinger 
wrote:
 Your scripts had bounds checking enabled for LDC but not the 
 other two D compilers.
I strongly recommend people to always keep bounds checking enabled in real world programs because it is so useful in keeping programs sane. Specific parts of the code might disable it via `.ptr`, but the command line switch is dangerous and I can't recommend anyone to use it... ever. We should run benchmarks with bounds checking enabled to better reflect real world results. Yes, it might "lose" to C, but we'll be the ones laughing when the C program crashes with yet another buffer overrun or when it's milliseconds in execution time lead to days of debug time and a multi-million dollar business loss when an unchecked pointer leads to a security breech.
Feb 13 2016
next sibling parent reply artemalive <artemalive gmail.com> writes:
On Saturday, 13 February 2016 at 21:10:11 UTC, Adam D. Ruppe 
wrote:
 On Saturday, 13 February 2016 at 20:45:41 UTC, David Nadlinger 
 wrote:
 Your scripts had bounds checking enabled for LDC but not the 
 other two D compilers.
I strongly recommend people to always keep bounds checking enabled in real world programs because it is so useful in keeping programs sane. Specific parts of the code might disable it via `.ptr`, but the command line switch is dangerous and I can't recommend anyone to use it... ever. We should run benchmarks with bounds checking enabled to better reflect real world results. Yes, it might "lose" to C, but we'll be the ones laughing when the C program crashes with yet another buffer overrun or when it's milliseconds in execution time lead to days of debug time and a multi-million dollar business loss when an unchecked pointer leads to a security breech.
Hi Adam, I'll check the influence of enabled bounds check on benchmark result. Did not try this before. Nevertheless, I have to admit I'm still on the dark side. Benchmark has some algorithms related to raytracing techniques and in this area I can't resist temptation to max performance at any cost.
Feb 14 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 14 February 2016 at 17:38:54 UTC, artemalive wrote:
 Hi Adam, I'll check the influence of enabled bounds check on 
 benchmark result. Did not try this before.
If you do, then you should use bounds checks in C++ too. (STL container.at(index) )
Feb 14 2016
parent artemalive <artemalive gmail.com> writes:
On Sunday, 14 February 2016 at 17:43:01 UTC, Ola Fosheim Grøstad 
wrote:
 On Sunday, 14 February 2016 at 17:38:54 UTC, artemalive wrote:
 Hi Adam, I'll check the influence of enabled bounds check on 
 benchmark result. Did not try this before.
If you do, then you should use bounds checks in C++ too. (STL container.at(index) )
Thanks, I'll take this into account. At the moment I don't think bounds checking will be enabled. I believe that real world usage when you have a render farm is to have the fastest code possible, because any performance penalty is too costly. When new rendering algorithm is developed sometimes we run infinite rendering with all debug/logic check enabled to find the bugs. There could be even dedicated computer for this. But for production when thousands of cores do the same ray processing the fastest code is needed.
Feb 14 2016
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Saturday, 13 February 2016 at 21:10:11 UTC, Adam D. Ruppe 
wrote:
 On Saturday, 13 February 2016 at 20:45:41 UTC, David Nadlinger 
 wrote:
 Your scripts had bounds checking enabled for LDC but not the 
 other two D compilers.
I strongly recommend people to always keep bounds checking enabled in real world programs because it is so useful in keeping programs sane.
I agree - but not for all situations. Sometimes it's fine to let programs crash, if performance is important, and the environment is configured to handle that gracefully, or where there is no risk of data loss, such as video games. I think that in the context of a render farm, disabling bounds checking is completely reasonable. Bugs will manifest as crashes or rendering artifacts, and there is no risk of code execution exploits.
Feb 14 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 14 February 2016 at 19:29:54 UTC, Vladimir Panteleev 
wrote:
 I think that in the context of a render farm, disabling bounds 
 checking is completely reasonable. Bugs will manifest as 
 crashes or rendering artifacts, and there is no risk of code 
 execution exploits.
But nobody would write a render-engine in Go. Most likely you would write it for GPU support and certainly use SIMD heavily. That's what current engines do. So if it is a benchmark, then it makes perfect sense to use the same feature set for all languages. But optimization should be set to the highest possible, including whole program optimization where available. It's a benchmark...
Feb 14 2016
prev sibling parent ixid <nuaccount gmail.com> writes:
On Saturday, 13 February 2016 at 21:10:11 UTC, Adam D. Ruppe 
wrote:
 We should run benchmarks with bounds checking enabled to better 
 reflect real world results. Yes, it might "lose" to C
Like for like comparisons are the best approach, making it clear what a given result is for. The most effective story for D is that it's fast.
Feb 18 2016
prev sibling parent reply artemalive <artemalive gmail.com> writes:
On Saturday, 13 February 2016 at 20:45:41 UTC, David Nadlinger 
wrote:
 Hi Artem,

 On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 https://github.com/artemalive/DigitalWhip
Your scripts had bounds checking enabled for LDC but not the other two D compilers. I posted a pull request with the fix. LDC isn't unreasonably slow any longer on a random EC2 box I used for a quick test: --- Compiler relative times: clang 1.00 gcc 1.02 ldc 1.07 dmd 2.07 --- — David
Hi David, I use -release option. It seems no need to disable bounds check explicitly. From ldc output: "-release - Disables asserts, invariants, contracts and boundscheck". -singleobj really does the magic. I'll commit only this option. Thanks for the suggestion and for the pull request. It's my first pull request;)
Feb 14 2016
parent reply David Nadlinger <code klickverbot.at> writes:
On Sunday, 14 February 2016 at 17:31:37 UTC, artemalive wrote:
 From ldc output:
 "-release - Disables asserts, invariants, contracts and 
 boundscheck".
We (LDC team) should clarify this description. In D2, -release does not disable bounds-checking for safe code anymore.
 -singleobj really does the magic. I'll commit only this option. 
 Thanks for the suggestion and for the pull request. It's my 
 first pull request;)
I'm not convinced that it makes for a good comparison to use different optimization settings for one of the compilers… — David
Feb 14 2016
parent artemalive <artemalive gmail.com> writes:
On Sunday, 14 February 2016 at 18:12:03 UTC, David Nadlinger 
wrote:
 On Sunday, 14 February 2016 at 17:31:37 UTC, artemalive wrote:
 From ldc output:
 "-release - Disables asserts, invariants, contracts and 
 boundscheck".
We (LDC team) should clarify this description. In D2, -release does not disable bounds-checking for safe code anymore.
 -singleobj really does the magic. I'll commit only this 
 option. Thanks for the suggestion and for the pull request. 
 It's my first pull request;)
I'm not convinced that it makes for a good comparison to use different optimization settings for one of the compilers… — David
Sure, we need the same optimization for all compilers if there is direct correspondence. Your pull request has been merged.
Feb 14 2016
prev sibling parent rsw0x <anonymous anonymous.com> writes:
On Saturday, 13 February 2016 at 18:48:12 UTC, artemalive wrote:
 Dear Community,

 I've prepared a valentine for you;)
 It's a project I've been working for the last few months in my 
 free time.

 DigitalWhip is a performance benchmark of statically typed 
 programming languages that
 compile to native code: 
 https://github.com/artemalive/DigitalWhip

 D is fast;)

 Thanks,
 Artem
latest LDC beta + -singleobj flag puts LDC back into a competitive spot. Compiler relative times: gcc 1.00 ldc 1.10 btw, LDC beta sped up the ray intersection by 100%+, wonder what commit caused that.
Feb 13 2016