www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Benchmarks - LDC vs GDC

reply Cecil Ward <cecil cecilward.com> writes:
I just read an interesting article about a benchmark comparison 
between GDC and LDC. The code being tested did a lot of (32-bit?) 
integer divisions and unfirtunately LDC rather bent the 
comparison by using an algorithmic replacement for the x86 idiv 
instruction which may or may not be preferable depending on the 
vintage of CPU. It also biases the comparison in favour of the 
detailed handing if that one single topic.

I wonder if anyone else is interested in doing some more LDC vs 
GDC comparisons ? With a common benchmark driver if possible.

GDC: -O3 -frelease -mtune=native -march=native
LDC: -O3 -release -boundscheck=off -mcpu=native

If using floating point, then we should add in the fast-math 
switches - can’t remember the exact details, so perhaps someone 
would append them?

I would especially like a meat-and-potatoes one that does not 
depend on maths in tight loops and on the power of loop-specific 
optimisations. By meat and potatoes I mean: integer arithmetic, 
lots of test and branches, some memory accesses. Something like 
parsing or string processing. The maths-oriented ones and 
floating point are welcome too, and I’d really like to see D 
80-bit real tested in a compiler benchmark. Complex numbers in D 
too.

But most of all, a real-world piece of code, run n times and 
turned into a benchmark. Run time ought to be > 1 min so as to 
make sure that timing inaccuracies do not drown the results in 
noise.

Anyone up for it?
Jun 26 2023
next sibling parent reply Sergey <kornburn yandex.ru> writes:
On Monday, 26 June 2023 at 20:22:24 UTC, Cecil Ward wrote:

 Anyone up for it?


Hi. Every problem from this set of benchmarks has D solution 
https://programming-language-benchmarks.vercel.app/
It is open-sourced on GitHub. Currently the benchmark run it with 
DMD and LDC (it is lacking GDC because some problems with GitHub 
Actions).

But you can run it locally with very little effort.
For time measurement I can recommend to use hyperfine tool.

Also GDC and LDC have their special things for vectorizations. 
And maybe Bruce Carneal could share some examples/benchmarks of 
them.
Jun 27 2023
next sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Tuesday, 27 June 2023 at 09:31:50 UTC, Sergey wrote:

 On Monday, 26 June 2023 at 20:22:24 UTC, Cecil Ward wrote:

 Anyone up for it?


 Hi. Every problem from this set of benchmarks has D solution 
 https://programming-language-benchmarks.vercel.app/
 ...


Nice link.


 Also GDC and LDC have their special things for vectorizations. 
 And maybe Bruce Carneal could share some examples/benchmarks of 
 them.


The TL;DR from my auto vectorization explorations is that GDC and 
LDC are very close in performance with GDC having a slight edge 
when doing some patterned "gathers" (like pulling elements from 
an RGGB Bayer pattern) and in exploiting the per-lane 
capabilities of later AVX-512 although I expect LDC will match it 
there pretty soon. Anecdotally it appears that they'll both do 
very well on SVE2 and RVV platforms.

Once you get things to unit stride form LDC and GDC are both very 
good. They both handle ternary expression style conditionals 
well, for example.  You may wish to use  restrict pointers to 
signal independence.

Also, if you're serious about programmer friendly data 
parallelism on CPUs definitely take a look at mir.  dcompute (LDC 
only) is worth a hard look for CUDA/OpenCL deployments.

Godbolt is your friend here.  Have fun!
Jun 27 2023
prev sibling parent reply partypooper <pythonproof gmail.com> writes:
On Tuesday, 27 June 2023 at 09:31:50 UTC, Sergey wrote:

 On Monday, 26 June 2023 at 20:22:24 UTC, Cecil Ward wrote:

 Anyone up for it?


 Hi. Every problem from this set of benchmarks has D solution 
 https://programming-language-benchmarks.vercel.app/
 It is open-sourced on GitHub. Currently the benchmark run it 
 with DMD and LDC (it is lacking GDC because some problems with 
 GitHub Actions).

 But you can run it locally with very little effort.
 For time measurement I can recommend to use hyperfine tool.

 Also GDC and LDC have their special things for vectorizations. 
 And maybe Bruce Carneal could share some examples/benchmarks of 
 them.


This benchmarks are not trustworthy. A lot of languages modified 
by contributors, dlang on the other case are just some meh 
translations from other languages by the repository author.

There is https://github.com/kostya/benchmarks which has ldc, gdc 
and dmd results and are more or less provided by the community at 
least languages like d, rust, zig, c, c++.
Jun 29 2023
parent reply Sergey <kornburn yandex.ru> writes:
On Thursday, 29 June 2023 at 07:47:17 UTC, partypooper wrote:

 On Tuesday, 27 June 2023 at 09:31:50 UTC, Sergey wrote:
 This benchmarks are not trustworthy. A lot of languages 
 modified by contributors, dlang on the other case are just some 
 meh translations from other languages by the repository author.

 There is https://github.com/kostya/benchmarks which has ldc, 
 gdc and dmd results and are more or less provided by the 
 community at least languages like d, rust, zig, c, c++.


If you can provide some improvements to solutions - you are 
welcome :)
Author of the repo is accepting PRs (at least he did it in the 
past, but recently it seems he doesn't have time to support it)
Jun 29 2023
parent reply partypooper <pythonproof gmail.com> writes:
On Thursday, 29 June 2023 at 09:23:53 UTC, Sergey wrote:

 If you can provide some improvements to solutions - you are 
 welcome :)
 Author of the repo is accepting PRs (at least he did it in the 
 past, but recently it seems he doesn't have time to support it)


You are reading behind the lines. I never said that i want to 
contribute to such benchmarks, but do not know how. Or never 
asked if author of this becnhmarks accepting PRs (i knew he does).

What I wanted to say is: "do not trust becnhmarks from Sergey 
user link, because they are not so good". It was the message more 
to OP, that to you. Benchmarks from you link are lame at least 
for D language (and not only) and represent in bright light only 
some big community languages or some hacker individual who 
polished specific solutions for his language.

Because of that I just provided the link with much better 
benchmarks (at least for larger part of languages), which 
actually has benchmarks of ldc and gdc. And which shows that in 
most cases gdc lags behind (in most cases by not so much, but 
still does).
Jun 29 2023
parent Sergey <kornburn yandex.ru> writes:
On Thursday, 29 June 2023 at 10:18:04 UTC, partypooper wrote:

 On Thursday, 29 June 2023 at 09:23:53 UTC, Sergey wrote:
 more to OP, that to you. Benchmarks from you link are lame at 
 least for D language (and not only) and represent in bright 
 light only some big community languages or some hacker 
 individual who polished specific solutions for his language.


The rule of the benchmark is to use the same approach for all 
languages (sometimes it is not always the case unfortunately), 
but I definitely can’t see confirmation of your words for this 
benchmark :) like everyone is free to submit a solution with some 
restrictions. I haven’t seen any “big community language” 
superiority there. Moreover it is just the source of some code 
that can be benchmarked for timing. No matter of different 
languages.

Kostya’s is good and also much older and mature.
Jun 29 2023
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Be leery of using fastmath switches unless you are very knowledgeable about fp 
math and how fastmath changes it.

DMD doesn't have a fastmath switch for good reason.
Jun 27 2023
next sibling parent reply Cecil Ward <cecil cecilward.com> writes:
On Tuesday, 27 June 2023 at 19:58:57 UTC, Walter Bright wrote:

 Be leery of using fastmath switches unless you are very 
 knowledgeable about fp math and how fastmath changes it.

 DMD doesn't have a fastmath switch for good reason.


Walter, your warning is appreciated. I read some of the GCC 
documentation, but I’ve no intention of going near the fast math 
thing in a production program as indeed I haven’t thought it 
through enough.

Since DMD doesn’t have such a thing, then fast-math needs to be 
banned in benchmarks that include DMD otherwise it’s not a level 
playing field.

Isn’t the main thing ignoring infinities and NaNs? I don’t think 
that’s all there is to it though. I would say, in my extremely 
limited exposure to floating point, that if you have debugged 
your code then infinities and NaNs should not be encountered 
anyway unless you have possible special cases to deal with 
relating to input that you’re not in control of.
Jun 28 2023
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/28/2023 4:29 PM, Cecil Ward wrote:

 Isn’t the main thing ignoring infinities and NaNs? I don’t think that’s
all 
 there is to it though. I would say, in my extremely limited exposure to
floating 
 point, that 


I forgot precisely what it did, but it takes shortcuts. I just concluded "nope" 
to that.


 if you have debugged your code then infinities and NaNs should not
 be encountered anyway


Famous last words.
Jun 28 2023
next sibling parent reply Cecil Ward <cecil cecilward.com> writes:
On Thursday, 29 June 2023 at 04:41:54 UTC, Walter Bright wrote:

 On 6/28/2023 4:29 PM, Cecil Ward wrote:

 Isn’t the main thing ignoring infinities and NaNs? I don’t 
 think that’s all there is to it though. I would say, in my 
 extremely limited exposure to floating point, that


 I forgot precisely what it did, but it takes shortcuts. I just 
 concluded "nope" to that.


 if you have debugged your code then infinities and NaNs


 should not

 be encountered anyway


 Famous last words.


I truly hear you. :-) It rather depends on the particular 
situation, but then what do I know, seeing as floating point is 
not my thing at all. I will educate myself if I ever go down that 
particular road and take on board what you have said.
Jun 28 2023
parent bachmeier <no spam.net> writes:
On Thursday, 29 June 2023 at 05:23:29 UTC, Cecil Ward wrote:

 On Thursday, 29 June 2023 at 04:41:54 UTC, Walter Bright wrote:

 On 6/28/2023 4:29 PM, Cecil Ward wrote:

 Isn’t the main thing ignoring infinities and NaNs? I don’t 
 think that’s all there is to it though. I would say, in my 
 extremely limited exposure to floating point, that


 I forgot precisely what it did, but it takes shortcuts. I just 
 concluded "nope" to that.


 if you have debugged your code then infinities and NaNs


 should not

 be encountered anyway


 Famous last words.


 I truly hear you. :-) It rather depends on the particular 
 situation, but then what do I know, seeing as floating point is 
 not my thing at all. I will educate myself if I ever go down 
 that particular road and take on board what you have said.


-ffast-math is a good idea until you think about it. This is [a 
good blog post on 
-ffast-math](https://kristerw.github.io/2021/10/19/fast-math/). 
There are probably valid use cases for it, but we have high-level 
languages for a reason.
Jun 29 2023
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 29 June 2023 at 04:41:54 UTC, Walter Bright wrote:

 On 6/28/2023 4:29 PM, Cecil Ward wrote:

 Isn’t the main thing ignoring infinities and NaNs? I don’t 
 think that’s all there is to it though. I would say, in my 
 extremely limited exposure to floating point, that


 I forgot precisely what it did, but it takes shortcuts. I just 
 concluded "nope" to that.


 if you have debugged your code then infinities and NaNs


 should not

 be encountered anyway


 Famous last words.


Hey, that's what `feenableexcept(FE_ALL_EXCEPT)` is for.

```
#define _GNU_SOURCE

#include <fenv.h>
#include <stdio.h>

void main() {
   printf("infinity = %f\n", 1.0f / 0.0f);
   feenableexcept(FE_ALL_EXCEPT);
   // will crash with a float error.
   printf("infinity = %f\n", 1.0f / 0.0f);
}
```

(I thought there was a way to make it never generate 
infinities/nans in the first place, but I can't find it.)
Jun 30 2023
prev sibling next sibling parent bachmeier <no spam.net> writes:
On Tuesday, 27 June 2023 at 19:58:57 UTC, Walter Bright wrote:

 Be leery of using fastmath switches unless you are very 
 knowledgeable about fp math and how fastmath changes it.

 DMD doesn't have a fastmath switch for good reason.


https://mastodon.gamedev.place/ aeva/110617204223014295
Jun 28 2023
prev sibling parent Rune Morling <ermo serpentos.com> writes:
On Tuesday, 27 June 2023 at 19:58:57 UTC, Walter Bright wrote:

 Be leery of using fastmath switches unless you are very 
 knowledgeable about fp math and how fastmath changes it.

 DMD doesn't have a fastmath switch for good reason.


Walter, have you (or the foundation) ever considered contributing 
idiomatic DLang solutions to (the more well-known) benchmark 
sites and linking to them in the DLang intro (= 
"marketing/funneling") materials?

Using this to showcase DLang might be a boon, particularly if the 
goal is to market the language to people re. how straightforward 
it is to write performant code in DLang compared to, say, 
C/C++/Java/Python/Go/Rust etc.?

I might be overly naïve here, but I honestly see this as an easy 
way for prospective adopters to check whether DLang actually does 
what it says on the tin so to speak? Particularly if the the 
person or set of persons contributing the examples are part of 
the core dlang set of contributors?

The other bonus is that it will help showcase areas of 
improvements if said benchmarks are revisited regularly, perhaps 
even as part of the compiler release test suite(s)?
Jun 29 2023